codefuse-ai/CodeFuse-DevOps-Model-7B-Base

1---2license: other3language:4- zh5tags:6- Text Generation7- LLM8---9<div align="center">10<h1>11 DevOps-Model-7B-Base12</h1>13</div>14 15<p align="center">16🤗 <a href="https://huggingface.co/codefuse-ai" target="_blank">Hugging Face</a> • 17🤖 <a href="https://modelscope.cn/organization/codefuse-ai" target="_blank">ModelScope</a> 18</p>19 20DevOps-Model is a Chinese **DevOps large model**, mainly dedicated to exerting practical value in the field of DevOps. Currently, DevOps-Model can help engineers answer questions encountered in the all DevOps life cycle.21 22Based on the Qwen series of models, we output the **Base** model after additional training with high-quality Chinese DevOps corpus, and then output the **Chat** model after alignment with DevOps QA data. Our Base model and Chat model can achieve the best results among models of the same scale based on evaluation data related to the DevOps fields.23 24<br>25At the same time, we are also building an evaluation benchmark [DevOpsEval](https://github.com/codefuse-ai/codefuse-devops-eval) exclusive to the DevOps field to better evaluate the effect of the DevOps field model.26<br>27<br>28 29# Evaluation30We first selected a total of six exams related to DevOps in the two evaluation data sets of CMMLU and CEval. There are a total of 574 multiple-choice questions. The specific information is as follows:31 32| Evaluation dataset | Exam subjects | Number of questions |33|:-------:|:-------:|:-------:|34|   CMMLU  | Computer science | 204 |35|   CMMLU  | Computer security | 171 |36|   CMMLU  | Machine learning | 122 |37| CEval   | College programming | 37 |38| CEval   | Computer architecture | 21 |39| CEval   | Computernetwork | 19 |40 41 42We tested the results of Zero-shot and Five-shot respectively. Our 7B and 14B series models can achieve the best results among the tested models. More tests will be released later.43 44|Model|Size|Zero-shot Score|Five-shot Score|45|--|--|--|--|46|**DevOps-Model-7B-Base**|**7B**|**62.72**|**62.02**|47|Qwen-7B-Base|7B|55.75|56.0|48|Baichuan2-7B-Base|7B|49.30|55.4|49|Internlm-7B-Base|7B|47.56|52.6|50 51 52 53<br>54 55# Quickstart56We provide simple examples to illustrate how to quickly use Devops-Model-Chat models with 🤗 Transformers.57 58## Requirement59```bash60cd path_to_download_model61pip install -r requirements.txt62```63 64## Model Example65 66```python67from transformers import AutoModelForCausalLM, AutoTokenizer68from transformers.generation import GenerationConfig69 70tokenizer = AutoTokenizer.from_pretrained("path_to_DevOps-Model", trust_remote_code=True)71 72model = AutoModelForCausalLM.from_pretrained("path_to_DevOps-Model", device_map="auto", trust_remote_code=True, bf16=True).eval()73 74model.generation_config = GenerationConfig.from_pretrained("path_to_DevOps-Model", trust_remote_code=True)75 76inputs = '''The implementation principle of HashMap in Java is'''77input_ids = tokenizer(inputs, return_tensors='pt')78input_ids = input_ids.to(model.device)79pred = model.generate(**input_ids)80```81 82 83 84# Disclaimer85Due to the characteristics of language models, the content generated by the model may contain hallucinations or discriminatory remarks. Please use the content generated by the DevOps-Model family of models with caution.86If you want to use this model service publicly or commercially, please note that the service provider needs to bear the responsibility for the adverse effects or harmful remarks caused by it. The developer of this project does not assume any responsibility for any consequences caused by the use of this project (including but not limited to data, models, codes, etc.) ) resulting in harm or loss.87 88 89 90# Acknowledgments91This project refers to the following open source projects, and I would like to express my gratitude to the relevant projects and research and development personnel.92- [LLaMA-Efficient-Tuning](https://github.com/hiyouga/LLaMA-Efficient-Tuning)93- [QwenLM](https://github.com/QwenLM)