MiniMaxAI/MiniMax-Music3

1---2library_name: diffusers3pipeline_tag: text-to-audio4tags:5  - music-generation6  - text-to-music7  - pytorch8  - sglang-omni9---10 11<div align="center">12  <img width="100%" src="figures/Music3.png" alt="MiniMax">13</div>14<p align="center">15  <a href="https://agent.minimax.io/" target="_blank"><img src="https://img.shields.io/badge/MiniMax%20Agent-FF6C37?logo=minimax&logoColor=white" alt="MiniMax Agent"></a>16  <a href="https://platform.minimax.io/docs/guides/text-generation" target="_blank"><img src="https://img.shields.io/badge/API-FF6C37?logo=minimax&logoColor=white" alt="API"></a>17  <a href="https://www.minimax.io" target="_blank"><img src="https://img.shields.io/badge/MiniMax%20Website-FF6C37?logo=minimax&logoColor=white" alt="MiniMax Website"></a>18  <br>19  <a href="https://modelscope.cn/organization/minimax" target="_blank" rel="noopener noreferrer"><img alt="ModelScope MiniMax AI" src="https://img.shields.io/badge/ModelScope-MiniMax%20AI-white?labelColor=%23EF3D5D"></a>20  <a href="https://platform.minimaxi.com/docs/faq/contact-us" target="_blank"><img src="https://img.shields.io/badge/WeChat-07C160?logo=wechat&logoColor=white" alt="WeChat"></a>21  <a href="https://discord.com/invite/DPC4AHFCBw" target="_blank"><img src="https://img.shields.io/badge/Discord-5865F2?logo=discord&logoColor=white" alt="Discord"></a>22  <a href="https://huggingface.co/MiniMaxAI" target="_blank"><img src="https://img.shields.io/badge/Hugging%20Face-FFD21E?logo=huggingface&logoColor=black" alt="Hugging Face"></a>23  <a href="https://github.com/MiniMax-AI/MiniMax-Music3" target="_blank"><img src="https://img.shields.io/badge/GitHub-181717?logo=github&logoColor=white" alt="GitHub"></a>24  <a href="https://huggingface.co/MiniMaxAI/MiniMax-Music3/blob/main/LICENSE" target="_blank"><img src="https://img.shields.io/badge/LICENSE-4CAF50?logo=creativecommons&logoColor=white" alt="LICENSE"></a>25</p>26 27# MiniMax Music 328 29**MiniMax Music 3** is a high-performance music generation model for creating complete songs up to **five minutes** long. Conditioned on lyrics and a detailed music description, it generates structurally coherent songs with expressive vocals, evolving arrangements, and stable long-form audio quality.30 31MiniMax Music 3 combines an **8B Global LLM** for long-range musical structure, a **0.6B Local LLM** for frame-level acoustic detail, and a continuous hidden-state synthesis system based on **Flow Matching** and **Flow-VAE**. The model produces 32 kHz, 16-bit stereo WAV audio.32## Demo33 34Explore music generation examples on the [MiniMax Music 3 Demo](https://minimax-ai.github.io/music3-demo/).35 36<p align="center">37  <img width="100%" src="figures/music3.0-Architecture-Diagram.png">38</p>39 40## Complete Songs with Long-Range Coherence41 42MiniMax Music 3 natively supports full-song generation up to five minutes. The model maintains musical themes, rhythm, vocal identity, and arrangement progression across long sequences, enabling complete structures such as intro, verse, pre-chorus, chorus, bridge, instrumental break, and outro.43 44## Fine-Grained Music Control45 46The model accepts two complementary inputs:47 48- **Lyrics** define the words to be sung and may include explicit section tags such as `[Intro]`, `[Verse]`, `[Pre-Chorus]`, `[Chorus]`, `[Post-Chorus]`, `[Bridge]`, `[Instrumental]`, `[Solo]`, and `[Outro]`.49- **Music description** defines the musical style, emotional progression, vocal performance, instrumentation, arrangement, and production profile.50 51For precise control, we recommend using a Structured Caption with three sections:52 53- **Global Metadata**: genre, subgenre, BPM, key, scale, emotional progression, listening scenario, and production profile.54- **Vocal Details**: vocal gender, timbre, performance style, harmony, backing vocals, and vocal effects.55- **Arrangement**: primary and secondary instruments, section-level instrument evolution, groove, bass, percussion, textures, and spatial effects.56 57This representation allows the model to follow not only a global style, but also the musical development of the song over time.58 59## Hybrid-LM60 61MiniMax Music 3 uses a hierarchical autoregressive architecture that separates global musical modeling from local acoustic modeling.62 63- The **Global LLM (8B)** predicts the first RVQ codebook frame by frame and models the song's long-range semantic and structural progression.64- The **Local LLM (0.6B)** predicts the remaining acoustic codebooks within each frame and restores fine-grained acoustic information.65 66The Global LLM is initialized from Qwen3-8B. During training, its embedding and output layers are first adapted to semantic music tokens. The Global and Local LLMs are then jointly trained to model all RVQ codebooks.67 68## Continuous Hidden-State Synthesis69 70Instead of decoding only from discrete RVQ tokens, the synthesis module fuses the final hidden states of the Global and Local LLMs. These continuous representations preserve richer acoustic information for vocal articulation, instrumental texture, and temporal continuity.71 72The synthesis path is:73 74```text75Global and Local LLM hidden states76                ↓77       Hidden-state fusion78                ↓79     Flow Matching (2.4B)80                ↓81        Flow-VAE latent82                ↓83    Flow-VAE Decoder (123M)84                ↓85       32 kHz stereo audio86```87 88The Flow-VAE architecture is adapted from MiniMax Speech and retrained for the dynamic range and spectral characteristics of music.89 90## Music Tokenizer91 92The training tokenizer uses eight layers of Residual Vector Quantization (RVQ):93 94- The first semantic codebook contains **16,384** entries and captures the core musical semantics and structure.95- The remaining seven acoustic codebooks contain **1,024** entries each and represent residual acoustic details.96 97Training first optimizes the semantic codebook, then jointly trains all eight codebooks. At inference time, waveform synthesis uses the fused LLM hidden states and does not require the discrete tokenizer decoder.98 99## How to Use100 101MiniMax Music 3 is supported by [SGLang-Omni](https://github.com/sgl-project/sglang-omni). Follow the official [installation guide](https://sgl-project.github.io/sglang-omni/get_started/installation.html) to prepare the runtime environment.102 103### Download the Model104 105```bash106hf download MiniMaxAI/MiniMax-Music3 --local-dir /path/to/minimax_ttm107```108 109We recommend the following inference frameworks to serve the model:110 111- [SGLang](https://docs.sglang.io/) \- see [cookbook](https://sgl-project.github.io/sglang-omni/cookbook/minimax_music3.html) 112 113- [diffusers](https://github.com/huggingface/diffusers) \- see [diffusers docs](https://github.com/huggingface/diffusers/blob/minimax-music3-integration/docs/source/en/api/pipelines/minimax_music3.md)114 115- [ComfyUI](https://github.com/Comfy-Org/ComfyUI) see [comfyUI tutorials](https://docs.comfy.org/tutorials/audio/minimax/minimax-music-3)116 117 118### Serve with SGLang-Omni119 120```bash121sgl-omni serve --model-path MiniMaxAI/MiniMax-Music3 --port 8000122```123 124### Generate Music125 126The service uses the shared speech API. Put the lyrics in `input` and the music description in `instructions`. Put lyric structure tags such as `[Verse]` and `[Chorus]` on their own lines.127 128```bash129curl http://127.0.0.1:8000/v1/audio/speech \130  -H 'Content-Type: application/json' \131  -d '{132    "model": "MiniMaxAI/MiniMax-Music3",133    "input": "[Verse]\nMorning light filtering through the pine\n[Chorus]\nSoftly the world begins to breathe",134    "instructions": "A warm acoustic pop song with intimate female vocals, fingerpicked guitar, soft piano, and a gradual emotional build into a wide final chorus.",135    "response_format": "wav",136    "seed": 7,137    "max_new_tokens": 750,138    "stream": false139  }' \140  --output minimax_music3.wav141```142 143`max_new_tokens` sets the maximum number of audio frames at 25 frames per second. Generation may finish before this limit when the model emits an end-of-audio token. The response is a 32 kHz, 16-bit stereo WAV file.144 145### Reproducible Example146 147The following end-to-end example contains the complete lyrics, music description, and generation parameters used to produce the reference audio.148 149| Use case | Request | Result |150|---|---|---|151| Text-to-music | [View script](https://huggingface.co/MiniMaxAI/MiniMax-Music3/blob/main/scripts/end_to_end/minimax_ttm_test.py) | [minimax_ttm.wav](https://huggingface.co/MiniMaxAI/MiniMax-Music3/blob/main/assets/minimax_ttm.wav) |152 153## 🧨 Diffusers154 155MiniMax Music 3 is available as a [diffusers](https://github.com/huggingface/diffusers) modular pipeline. Until [huggingface/diffusers#14456](https://github.com/huggingface/diffusers/pull/14456) is merged, install diffusers from the PR commit:156 157The snippet below fits 24GB+ VRAM GPUs158 159```bash160pip install git+https://github.com/huggingface/diffusers@dafe3733fcfdbf3c48915fe77be3aef65b5d6a2d transformers accelerate soundfile161```162 163```python164import soundfile as sf165import torch166from diffusers import ModularPipeline167 168pipe = ModularPipeline.from_pretrained("MiniMaxAI/MiniMax-Music3")169pipe.load_components(dtype=torch.bfloat16)170pipe.to("cuda")171 172lyrics = """[verse]173Morning light filtering through the pine174Every quiet street is yours and mine175[chorus]176Softly the world begins to breathe"""177 178prompt = (179    "Genre: acoustic pop. BPM: 96. Key: C major. Warm and intimate, building gently into the chorus. "180    "Vocals: soft female lead, close and breathy, light stacked harmonies in the chorus. "181    "Arrangement: fingerpicked guitar and soft piano; brushed drums and upright bass enter in the chorus."182)183 184audio = pipe(185    prompt=prompt,186    lyrics=lyrics,187    audio_duration=60.0,188    generator=torch.Generator("cuda").manual_seed(7),189    output="audios",190)[0]191 192sf.write("song.wav", audio.T.float().cpu().numpy(), pipe.sampling_rate)193```194 195### Low VRAM196 197The full precision fits under 24GB of VRAM. With automatic CPU offloading, generation takes in ~22 GB; additionally streaming the language model layer by layer makes it fit even 8 GB video cards:198 199```python200import torch201from diffusers import ComponentsManager, ModularPipeline202from diffusers.hooks import apply_group_offloading203 204manager = ComponentsManager()205manager.enable_auto_cpu_offload(device="cuda")206pipe = ModularPipeline.from_pretrained("MiniMaxAI/MiniMax-Music3", components_manager=manager)207pipe.load_components(dtype=torch.bfloat16)208 209# Only needed below ~22 GB of VRAM — slower, but fits in 8 GB.210apply_group_offloading(211    pipe.language_model, onload_device=torch.device("cuda"), offload_type="leaf_level", use_stream=True212)213 214 215```216 217## Prompt Enhancement218 219A concise natural-language description can be used directly. For richer prompts and more precise control, use the provided [`music-caption-rewriter`](https://github.com/MiniMax-AI/MiniMax-Music3/tree/main/skills/music-caption-rewriter) skill to expand it into a Structured Caption containing `Global Metadata`, `Vocal Details`, and `Arrangement`. The skill preserves musical instructions attached to lyric section tags in the arrangement description while keeping the lyric text in the lyrics input.220 221```bash222npx skills add MiniMax-AI/MiniMax-Music3 --skill music-caption-rewriter223```224 225## Limitations226 227- Inference requires CUDA.228- Only non-streaming generation is currently supported.229- The tokenized text prompt is limited to 5,000 tokens.230- Audio generation is limited to 9,000 acoustic frames.231- Section tags and music descriptions provide generative control rather than strict symbolic guarantees. The generated tempo, key, instrumentation, lyrics, and song structure may not always match every requested detail exactly.232 233## Contact Us234 235Contact us at [model@minimax.io](mailto:model@minimax.io).236