darkolorin/vibe_coding_router

Vibe Coding Router v5

A three-tier cascaded router for coding tasks that routes prompts between:

• —Local: Qwen3-Coder-Next (80B/3B active MoE, on-device via MLX)
• —Sonnet: Claude Sonnet 4.6 (medium-complexity cloud)
• —Opus: Claude Opus 4.6 (max-capability cloud)

What's New in v5

v4 suffered from inverted routing — simple queries went to cloud while complex ones stayed local. Root cause: length-quality anti-correlation in training data combined with PID loss reward-weight amplification. v5 fixes this with:

1. 1.7 new complexity features (45 handcrafted total): is_coding_task, junk_score, scope_breadth, imperative_verb_density, noun_phrase_density, interaction_complexity, requirement_clause_count
2. 2.Centered complexity premium: Adjusts training margins by premium * (complexity_score - center) so complex tasks push toward cloud and simple tasks push toward local
3. 3.Junk prompt clamping: 75 junk/greeting prompts neutralized (p_teacher=0.5, margin=0.0)
4. 4.Reward weight cap: PID loss reward_weight capped at 0.5 to prevent outlier margin dominance

Architecture

Two cascaded binary MLP routers trained with Privileged Information Distillation (PID):

• —Router A (local vs cloud): 77-dim → [32, 16] → 1, dropout=0.2, LayerNorm+ReLU
• —Router B (sonnet vs opus): 77-dim → [128, 64] → 1, dropout=0.0, LayerNorm+ReLU

Features: 45 handcrafted code features + 32 PCA-reduced sentence embeddings (all-MiniLM-L6-v2).

Training

• —Data: 1,644 coding prompts with real quality scores from all three models
• —Judge: GPT-5.4 scoring correctness, completeness, code quality, explanation
• —Loss: PID (reward-weighted CE + KL divergence), βkl=0.02, rewardcap=0.5
• —Label smoothing: ε=0.05, cost-aware margin for Router B (cost_premium=0.03)
• —Complexity premium: 2.0, centered at 0.3
• —HP sweep: 108 configurations, 3-way split (1150 train / 247 val / 247 test)
• —Threshold A: 0.60 (manually tuned for routing behavior — see note below)
• —Threshold B: 0.474 (calibrated on validation set)

Threshold Note

The utility-optimal Router A threshold (0.01) routes almost nothing to local because cloud quality is genuinely equal or better on nearly all prompts. The manual threshold of 0.60 trades ~1.4% utility for correct routing intuition: simple/fast tasks run locally with zero latency, while complex tasks go to cloud.

Real-World Routing (28 test queries, threshold_a=0.60)

v4 comparison: simple→local was 0/8 (now 5/8), complex→local was 6/6 (now 1/6).

Test Set Results (calibrated thresholds)

Files

• —router_a.safetensors — Router A weights (32×16 MLP, 13KB)
• —router_b.safetensors — Router B weights (128×64 MLP, 76KB)
• —config.json — Model config, thresholds, HP, training results
• —scaler.pkl — StandardScaler for feature normalization
• —embedding_extractor.pkl — PCA-reduced sentence-transformers extractor
• —sweep_results.json — Full 108-config HP sweep results

Usage

from router.three_tier_inference import ThreeTierRouter

router = ThreeTierRouter("models/three_tier_v5")
result = router.route("Write a Python function to sort a list")
# result.decision: "local", "sonnet", or "opus"
# result.p_cloud: probability of cloud routing
# result.p_opus: probability of opus (if routed to cloud)