Modelpublic

Compactbot/ldt-10m

sourceHugging Faceapache-2.0updated 6d agoView on Hugging Face

0likes1.7kdownloads

config.json15 linesDownload Raw Back to root

1{2  "architectures": ["LDTModel"],3  "model_type": "ldt",4  "d_model": 320,5  "n_layers": 5,6  "n_heads": 5,7  "head_dim": 64,8  "ffn_dim": 896,9  "vocab_size": 12288,10  "max_position_embeddings": 512,11  "rope_theta": 10000.0,12  "tie_word_embeddings": true,13  "torch_dtype": "float32"14}15