Modelpublic

SattwikAyyagari/codegen-350M-NL-Java

sourceHugging Faceupdated 4mo agoView on Hugging Face

0likes38downloads

tokenizer_config.json11 linesDownload Raw Back to root

1{2  "add_prefix_space": false,3  "backend": "tokenizers",4  "bos_token": "<|endoftext|>",5  "eos_token": "<|endoftext|>",6  "is_local": false,7  "model_max_length": 2048,8  "tokenizer_class": "TokenizersBackend",9  "unk_token": "<|endoftext|>"10}11