scrapegoat/Neural-Audio-Codec

XCodec Mini - Neural Audio Codec

Model Description

XCodec Mini is a state-of-the-art neural audio codec designed for high-quality music compression and reconstruction. It combines semantic and acoustic encoding approaches to achieve efficient compression while maintaining audio quality.

Key Features

• —Dual Encoding Architecture
• —Semantic encoder for high-level musical features
• —Acoustic encoder for detailed sound information
• —Multi-scale processing for efficient compression

• —Advanced Compression
• —Multiple codebooks for flexible quality/size tradeoff
• —Support for 44.1kHz high-fidelity audio
• —Separate processing paths for vocals and instrumentals

• —Technical Specifications
• —Input: Raw audio at 44.1kHz
• —Output: Compressed representations and reconstructed audio
• —Model Size: [Add total size]
• —Compression Ratio: [Add typical ratio]

Intended Uses

• —High-quality music compression
• —Audio archival and storage
• —Music streaming applications
• —Audio processing pipelines

Training Data

The model was trained on a diverse dataset of music, including:

• —Various genres and styles
• —Vocal and instrumental tracks
• —High-quality studio recordings

Performance and Limitations

Strengths

• —High-quality audio reconstruction
• —Efficient compression ratios
• —Separate handling of vocals and instrumentals
• —Support for high sample rates

Limitations

• —Computationally intensive for real-time applications
• —Requires significant GPU memory
• —Best suited for offline processing
• —May introduce artifacts in extreme compression settings

Technical Specifications

Model Architecture

1. 1.Semantic Encoder
2. 2.Based on HuBERT architecture
3. 3.Captures high-level musical features
4. 4.Outputs semantic tokens

1. 1.Acoustic Encoder
2. 2.Multi-scale convolutional architecture
3. 3.Processes detailed sound information
4. 4.Generates acoustic tokens

1. 1.Dual Decoders
2. 2.Separate decoders for vocals and instrumentals
3. 3.Multi-stage reconstruction process
4. 4.Quality-focused design

Input Requirements

• —Audio Format: WAV/MP3
• —Sample Rate: 44.1kHz
• —Channels: Mono/Stereo
• —Bit Depth: 16-bit

Output Format

• —Reconstructed Audio: 44.1kHz WAV
• —Intermediate Representations: Compressed tokens

Usage Guidelines

Hardware Requirements

• —GPU: NVIDIA GPU with 8GB+ VRAM
• —RAM: 16GB+ recommended
• —Storage: SSD recommended for faster processing

Software Requirements

• —Python 3.8+
• —PyTorch 2.0+
• —CUDA 11.0+
• —Additional dependencies listed in installation guide

Ethical Considerations

• —Copyright: Users should ensure they have proper rights to process copyrighted material
• —Attribution: Proper attribution should be given when using this model
• —Data Privacy: Consider data privacy implications when processing sensitive audio

Additional Information

Model Weights

The model requires several checkpoint files:

• —Semantic Encoder
• —Vocal Decoder
• —Instrumental Decoder
• —Final Checkpoint

Contact

For issues and questions, please use the GitHub repository's issue tracker.