WaveCut/LingBot-Video-MoE-30B-A3B-SDNQ-uint4-static

LingBot Video MoE 30B-A3B SDNQ UINT4 Static

This is a complete, loadable derivative of robbyant/lingbot-video-moe-30b-a3b with the diffusion transformer stored using static SDNQ UINT4 weights. It is tied to source model revision f2e538f64afe00cc4ae674db2aeb52e2945edfd5, LingBot Video code a2bb04b78edd848500dc27a26e035a95442ae186, and SDNQ d841c383ff7be38728d4df829e17af4f15d4fd66 (v0.2.1-17-gd841c38).

Text encoder, tokenizer/processor, scheduler, and VAE remain at their upstream precision. Convolutions and embeddings are not quantized. The recipe is uint4-static-transformer-only-with-3d-expert-adapter: weights_dtype=uint4, auto group size (group_size=0), no dynamic quantization, SVD, Hadamard transform, convolution quantization, or embedding quantization.

Coverage

Coverage is calculated from the original parameter inventory, not from model-file sizes.

Both the base transformer's and refiner's raw 3-D w1/w2/w3 expert tensors are packed. They are not excluded from the reported coverage. Exact module-level coverage and unquantized tensors are in `quantization_manifest.json` and `benchmark/coverage`.

Reproducible base benchmark

All five pairs use identical prompts, negative prompt, seeds 4201-4205, scheduler inputs, 832x480 dimensions, 73 frames, 24 fps, 40 steps, guidance 3.0, shift 3.0, batch_cfg=False, and null_cond_clone_zero=False. Resources were sampled every 250 ms from /proc, psutil, and nvidia-smi.

Observed base peak-VRAM reduction: 66.34%. Timing and memory are measurements on the environment recorded in `benchmark/environment`, not universal performance claims.

Frame-aligned aggregate quality across the five pairs: MAE 0.160701, RMSE 0.227453, PSNR 13.561 dB, SSIM 0.573403, LPIPS-Alex 0.484469.

[image]

The complete contact sheets and side-by-side MP4s are under `assets/comparison/base`. Quantized sample MP4s are under `samples/base`. Raw per-prompt CSV/JSONL, resource samples, commands, ffprobe records, output sizes, and SHA-256 values are under `benchmark`.

Refiner A/B

The refiner pair used the same 502b10f841d96aa101e69421b20083aeb60427c054b42fb0db29c5a6e70824cf initial latent generated from the same base MP4. Both outputs are 1920x1088, 73 frames at 24 fps, with 8 refiner steps and upstream-default null_cond_clone_zero=True.

Frame-aligned refiner quality: MAE 0.014650, RMSE 0.034053, PSNR 29.357 dB, SSIM 0.954816, LPIPS-Alex 0.063118.

Visual inspection found the same strong pink/red color clipping and cyan/magenta speckling in both the original-BF16 and SDNQ refiner outputs. Their close frame metrics therefore demonstrate pairwise similarity, not natural-color reconstruction quality; treat this as a shared refiner quality failure in this recorded sample.

[image]

Load the refiner by passing transformer_subfolder="refiner" to load_pipeline.

Installation and load

Use the exact pinned dependencies shipped with the repository:

git clone https://huggingface.co/WaveCut/LingBot-Video-MoE-30B-A3B-SDNQ-uint4-static
cd LingBot-Video-MoE-30B-A3B-SDNQ-uint4-static
python -m pip install -r runtime-requirements.txt

The tested MoE backend also needs the pinned SGLang userspace packages without replacing Torch:

python -m pip install --no-deps sglang==0.5.13.post1 apache-tvm-ffi==0.1.9 tilelang==0.1.8 triton==3.6.0

The repository includes the runtime adapter; no unmerged LingBot branch or local hidden file is needed:

import sys
from huggingface_hub import snapshot_download

root = snapshot_download("WaveCut/LingBot-Video-MoE-30B-A3B-SDNQ-uint4-static")
sys.path.insert(0, root)

from lingbot_sdnq_runtime import load_pipeline

pipe = load_pipeline(root, device="cuda")
# For the MoE refiner: load_pipeline(root, transformer_subfolder="refiner", device="cuda")

See `prompts.json` for the exact A/B inputs and `benchmark/summary.json` for portable metrics. Recorded consumer/offload smoke artifacts: benchmark/smokes/moe-refiner-sdnq-standard.json, benchmark/smokes/moe-refiner-sdnq-standard.mp4, benchmark/smokes/moe-sdnq-model.json, benchmark/smokes/moe-sdnq-model.mp4, benchmark/smokes/moe-sdnq-sequential.json, benchmark/smokes/moe-sdnq-sequential.mp4, benchmark/smokes/moe-sdnq-standard.json, benchmark/smokes/moe-sdnq-standard.mp4.

Runtime behavior and limitations

• —Generic SDNQ Linear layers use eager BF16 dequantization followed by F.linear in the tested Torch 2.8/CUDA 12.8 environment because the current SDNQ Triton quantized-matmul path is incompatible there.
• —Packed MoE experts are dequantized for each expert call and executed by the pinned SGLang Triton fused-MoE path. The adapter does not keep a persistent BF16 expert-weight cache.
• —Static UINT4 materially changes generated pixels. Inspect the published matrices and per-prompt metrics before choosing this derivative for quality-sensitive work.
• —The MoE SDNQ factory prompt is a severe framing/adherence regression: the valid 832x480 MP4 contains a smaller portrait-like factory view centered on a white canvas (MAE 0.338658). Treat that sample as a quality failure, not as a successful match to the original.
• —Peak residency and speed depend strongly on resolution, frame count, attention backend, offload mode, and GPU. The numbers above describe the exact recorded B200 run only.
• —The Apache-2.0 upstream license is retained. Users remain responsible for evaluating generated content for their application.

Evidence map

• —`quantization_manifest.json`: recipe, revisions, per-component and expert coverage.
• —`prompts.json`: exact structured prompts, negative prompt, seeds, and generation settings.
• —`benchmark/summary.json`: portable aggregate benchmark record.
• —`benchmark/base`: unmodified original and SDNQ raw metrics and resource samples.
• —`benchmark/comparison`: frame-aligned MAE/RMSE/PSNR/SSIM/LPIPS records.
• —`SHA256SUMS`: hashes for all published files, including model shards.