Modelpublic

Salesforce/codegen25-7b-multi_P

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face

138likes387downloads

tokenizer_config.json13 linesDownload Raw Back to root

1{2  "add_eos_token": false,3  "add_special_tokens": true,4  "clean_up_tokenization_spaces": true,5  "eos_token": "<|endoftext|>",6  "model_max_length": 1000000000000000019884624838656,7  "pad_token": null,8  "tokenizer_class": "CodeGen25Tokenizer",9  "auto_map": {10    "AutoTokenizer": ["tokenization_codegen25.CodeGen25Tokenizer", null]11  }12}13