softhell/code_docstring_model

1---2library_name: transformers3license: apache-2.04base_model: Salesforce/codet5-base5tags:6- generated_from_trainer7datasets:8- code_search_net9metrics:10- bleu11model-index:12- name: code_docstring_model13  results:14  - task:15      name: Sequence-to-sequence Language Modeling16      type: text2text-generation17    dataset:18      name: code_search_net19      type: code_search_net20      config: python21      split: validation22      args: python23    metrics:24    - name: Bleu25      type: bleu26      value: 0.0186592984855665827---28 29<!-- This model card has been generated automatically according to the information the Trainer had access to. You30should probably proofread and complete it, then remove this comment. -->31 32# code_docstring_model33 34This model is a fine-tuned version of [Salesforce/codet5-base](https://huggingface.co/Salesforce/codet5-base) on the code_search_net dataset.35It achieves the following results on the evaluation set:36- Loss: 0.905137- Bleu: 0.018738 39## Model description40 41More information needed42 43## Intended uses & limitations44 45More information needed46 47## Training and evaluation data48 49More information needed50 51## Training procedure52 53### Training hyperparameters54 55The following hyperparameters were used during training:56- learning_rate: 5e-0557- train_batch_size: 1658- eval_batch_size: 859- seed: 4260- gradient_accumulation_steps: 461- total_train_batch_size: 6462- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments63- lr_scheduler_type: linear64- lr_scheduler_warmup_ratio: 0.165- num_epochs: 766- mixed_precision_training: Native AMP67 68### Training results69 70| Training Loss | Epoch  | Step | Validation Loss | Bleu   |71|:-------------:|:------:|:----:|:---------------:|:------:|72| 1.111         | 1.0    | 1002 | 0.9781          | 0.0108 |73| 0.998         | 2.0    | 2004 | 0.9397          | 0.0109 |74| 0.9295        | 3.0    | 3006 | 0.9204          | 0.0120 |75| 0.8814        | 4.0    | 4008 | 0.9088          | 0.0159 |76| 0.8557        | 5.0    | 5010 | 0.9064          | 0.0171 |77| 0.8364        | 6.0    | 6012 | 0.9055          | 0.0180 |78| 0.8184        | 6.9933 | 7007 | 0.9051          | 0.0187 |79 80 81### Framework versions82 83- Transformers 4.47.084- Pytorch 2.5.1+cu12185- Datasets 3.3.186- Tokenizers 0.21.087