shangeth/mls-mimi-codes

Multilingual LibriSpeech (MLS) — Mimi Codes

Pre-extracted Kyutai Mimi neural-codec tokens for Multilingual LibriSpeech — LibriVox audiobooks in 7 non-English languages.

English is intentionally excluded. For English Mimi codes, use:

• —shangeth/librispeech-mimi-codes — LibriSpeech (~280k rows, 7 splits)
• —shangeth/libritts-r-mimi-codes — LibriTTS-R (~360k rows, 7 splits, 24 kHz native)
• —shangeth/vctk-mimi-codes — VCTK (~44k rows, 110 speakers w/ accents)
• —shangeth/jenny-mimi-codes — Jenny TTS (~21k rows, single speaker)
• —shangeth/ljspeech-mimi-codes — LJSpeech (~13k rows, single speaker)

Configs (languages)

One HF dataset config per language:

Splits (per config)

Merged all config

A virtual all config aliases the per-language parquets via glob — no extra storage, just a YAML entry. Use it to train a single multilingual model:

ds = load_dataset("shangeth/mls-mimi-codes", "all", split="train")

Caveat: there is no language column, and speaker_id is per-language — so speaker IDs may collide across languages (e.g. German speaker 12345 is unrelated to French speaker 12345). If you need clean per-language speaker bookkeeping, load each language config separately.

Schema

Extraction details

• —Codec: `kyutai/mimi` @ 24 kHz, 12.5 fps
• —Resampling: MLS audio is 48 kHz opus → resampled to 24 kHz at extraction
• —Codebooks: all 8 extracted; slice codes[:k] for fewer
• —Source: `facebook/multilingual_librispeech`

Usage

from datasets import load_dataset
import torch

ds = load_dataset("shangeth/mls-mimi-codes", "german", split="dev")
ex = ds[0]
codes = torch.tensor(ex["codes"], dtype=torch.long)  # [8, n_frames]
print(ex["id"], "| speaker:", ex["speaker_id"], "|", ex["text"][:60])

# Decode back to 24 kHz audio
from transformers import MimiModel
mimi = MimiModel.from_pretrained("kyutai/mimi").cuda().eval()
with torch.no_grad():
    wav = mimi.decode(codes.unsqueeze(0).cuda()).audio_values[0].cpu()

Links

• —Dataset extraction code: github.com/shangeth/wren-datasets
• —Wren research project: github.com/shangeth/wren
• —TTS models trained on these codes: github.com/shangeth/wren-tts

Citation

@misc{wren2026,
  title  = {Wren: A Family of Small Open-Weight Models for Unified Speech-Text Modelling},
  author = {Shangeth Rajaa},
  year   = {2026},
  url    = {https://github.com/shangeth/wren}
}

@inproceedings{pratap2020mls,
  title     = {MLS: A Large-Scale Multilingual Dataset for Speech Research},
  author    = {Pratap, Vineel and Xu, Qiantong and Sriram, Anuroop and Synnaeve, Gabriel and Collobert, Ronan},
  booktitle = {Interspeech},
  year      = {2020}
}

License

CC-BY-4.0 (inherited from MLS / LibriVox).