datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
routing_analysis-finetuning-data
routing_analysis finetuning data
A mirror of routing_analysis/finetuning/data/: the JSONL training splits used
in the multilingual MoE language-expansion experiments, plus the generator
scripts, runners, manifests and logs that produced them.
Layout
Path
Contents
Size
splits/
Document-count tiers {lang}_{tier}.jsonl, {lang}_manifest.json, splits_summary.json
~36 GB
token_budget_splits/
Token-budget tiers {lang}_{1Btok,1p5Btok,2Btok}.jsonl and… See the full description on the dataset page: https://huggingface.co/datasets/lylybig8/routing_analysis-finetuning-data.NCSS_2023_Data_Analysis