Modelpublic

optimum-intel-internal-testing/tiny-random-ltx2.5

sourceHugging Faceupdated 10d agoView on Hugging Face

0likes282downloads

tokenizer_config.json13 linesDownload Raw Back to tokenizer

1{2  "backend": "tokenizers",3  "bos_token": "<bos>",4  "eos_token": "<eos>",5  "is_local": true,6  "local_files_only": false,7  "mask_token": "<mask>",8  "model_max_length": 1024,9  "pad_token": "<pad>",10  "tokenizer_class": "GemmaTokenizer",11  "unk_token": "<unk>"12}13