gyung/Qwev-9B-RLCD

⚡ Qwev-9B-RLCD: Fast Non-Autoregressive System 1 Decision Model with Calibrated Uncertainty

<div align="center">

   

</div>

"Accept When Confident, Escalate When Unsure." Qwev-9B-RLCD is a fast non-autoregressive System 1 decision model aligned via Reinforcement Learning from Calibrated Decisions (RLCD) on top of the open-source parent model `jaredpalmer/kev-9b` (Qwen/Qwen3.5-9B-Base backbone with Pointer Head). Carnegie Mellon University's "JEV-as-a-Judge: Accept When Confident, Escalate When Unsure" (arXiv:2609.26550) paper is used as the 4-benchmark evaluation suite (RewardBench, HaluEval, JudgeBench, RM-Bench) and the &tau; &ge; 0.90 cascade validation methodology, allowing us to verify near-zero calibration error and single forward pass (168 ms) decision accuracy.

🌳 Model Lineage & Architecture

Qwen/Qwen3.5-9B-Base (9B Recurrent/DeltaNet Hybrid Backbone)
       │
       ▼
jaredpalmer/kev-9b (Pointer Head SFT Adaptation)
       │
       ▼  [Aligned via RLCD Reinforcement Learning on NVIDIA A100-80GB]
gyung/Qwev-9B-RLCD (Ours: Near-Zero Calibration Error & SOTA Accuracy)

• —Base Backbone: `Qwen/Qwen3.5-9B-Base`
• —Direct Parent Model: `jaredpalmer/kev-9b`
• —Adaptation Mechanism: Trainable LoRA Adapter + Pointer Softmax Readout Head (head.pt)
• —Evaluation Framework: CMU "JEV-as-a-Judge" Table 1 Benchmark Protocol (1,140 evaluation samples across 4 datasets)

🌟 Key Highlights

• —🚀 1-Pass Non-Autoregressive Inference: Zero token generation overhead. Decisions are made in 168.4 ms (approx. 11x faster than generative LLMs like GPT-6 Astra at 1,885 ms).
• —🏆 SOTA Decision Accuracy (CMU Table 1 Protocol):
• —RewardBench (400 samples): 99.2% (Outperforming CMU JEV 1.13: 92.2% & GPT-6 Astra: 93.5%)
• —HaluEval (240 samples): 98.8% (Outperforming CMU JEV 1.13: 87.5% & GPT-6 Astra: 86.7%)
• —RM-Bench-Hard (150 samples): 98.0% (Outperforming CMU JEV 1.13: 94.0%)
• —🎯 Calibrated Uncertainty (Near-Zero ECE):
• —On graduate-level 10-choice JudgeBench (random guess = 10%), Qwev-9B achieves 41.7% standalone accuracy with an average confidence of 42.3% (no overconfident hallucinations).
• —When filtering for confident answers (Confidence &ge; 0.90), accepted accuracy is 97.06% (33/34 correct), while unconfident queries escalate safely to GPT-6 for a 93.5% composite cascade accuracy.
• —🔌 100% Kev Compatible: Native drop-in LoRA adapter + pointer head architecture built on `jaredpalmer/kev-9b`.

📊 Comprehensive Benchmark Comparison (Full 1,140 Samples)

Evaluated under the exact protocol of Carnegie Mellon University's "JEV-as-a-Judge: Accept When Confident, Escalate When Unsure" (arXiv:2609.26550) on NVIDIA A100-SXM4-80GB:

\Note on Domain Specialization: convaiinnovations/laya (ModernBERT) and fastino/GLiNER2.5-Decide (DeBERTa-v3) achieve strong scores on binary pairs (HaluEval 95-99%), but drop to random chance (approx. 10%) on 10-choice STEM reasoning (JudgeBench), resulting in approx. 59-61% overall accuracy.*

⚡ Speed & Latency Comparison

🔬 Training Methodology & Full Loss Implementation

Mathematical Formulation

$$ L{\text{RLCD}} = L{\text{CE}} + 0.4 L{\text{Brier}} + 2.0 L{\text{Overconf}} + 0.3 L_{\text{Unknowable}} $$

• —1. Brier Calibration Loss ($L_{\text{Brier}}$): Minimizes squared distance between softmax probabilities and one-hot ground truth targets:

$$ L{\text{Brier}} = \frac{1}{K} \sum{k=1}^K (pk - yk)^2 $$

• —2. Asymmetric Overconfidence Penalty ($L_{\text{Overconf}}$): Exponentially penalizes high-confidence (&ge; 0.85) wrong predictions to eliminate confidently wrong errors:

$$ L{\text{Overconf}} = \max(0, p{\text{pred}} - \tau)^2 \cdot \exp(p_{\text{pred}}) \quad (\text{if } \text{pred} \ne \text{label}) $$

• —3. Unknowable Entropy Maximization ($L_{\text{Unknowable}}$): Enforces uniform probability distribution ($1/K$) when the context lacks sufficient evidence:

$$ L{\text{Unknowable}} = D{\text{KL}}\left(\text{Uniform}(1/K) \parallel p\right) $$

Complete PyTorch Loss Implementation:

import torch
import torch.nn as nn
import torch.nn.functional as F

class RLCDLoss(nn.Module):
    def __init__(self, brier_weight=0.4, overconf_weight=2.0, entropy_weight=0.3, conf_threshold=0.85):
        super().__init__()
        self.brier_weight = brier_weight
        self.overconf_weight = overconf_weight
        self.entropy_weight = entropy_weight
        self.conf_threshold = conf_threshold

    def forward(self, logits: torch.Tensor, label: int = None, soft_target: torch.Tensor = None, is_unknowable: bool = False):
        probs = F.softmax(logits, dim=-1)
        K = logits.size(-1)

        # 1. Unknowable Decision Regularization
        if is_unknowable:
            uniform_target = torch.full_like(probs, 1.0 / K)
            loss_unknowable = F.kl_div(F.log_softmax(logits, dim=-1), uniform_target, reduction="batchmean")
            return self.entropy_weight * loss_unknowable, {"unknowable": loss_unknowable.item()}

        # 2. Continuous Soft Target Distribution
        if soft_target is not None:
            log_probs = F.log_softmax(logits, dim=-1)
            loss_ce = -(soft_target * log_probs).sum()
            loss_brier = ((probs - soft_target) ** 2).sum()
            total_loss = loss_ce + self.brier_weight * loss_brier
            return total_loss, {"ce": loss_ce.item(), "brier": loss_brier.item()}

        # 3. Supervised Calibration Loss
        target = torch.tensor([label], device=logits.device)
        loss_ce = F.cross_entropy(logits.unsqueeze(0), target)

        one_hot = F.one_hot(target, num_classes=K).float()
        loss_brier = ((probs.unsqueeze(0) - one_hot) ** 2).sum(dim=-1).mean()

        # Asymmetric Overconfidence Penalty on False Hypotheses
        pred_idx = torch.argmax(probs)
        pred_conf = probs[pred_idx]
        loss_overconf = torch.tensor(0.0, device=logits.device)
        if pred_idx != label and pred_conf >= self.conf_threshold:
            loss_overconf = ((pred_conf - self.conf_threshold) ** 2) * torch.exp(pred_conf)

        total_loss = loss_ce + self.brier_weight * loss_brier + self.overconf_weight * loss_overconf
        return total_loss, {
            "ce": loss_ce.item(),
            "brier": loss_brier.item(),
            "overconf": loss_overconf.item()
        }

📂 Training Data Composition

The model was trained on a curated 5-in-1 Decision Alignment Mixture (4,800 records):

🔬 Ablation Study: Can 9B Decisions Scale on 10-Choice STEM? (JudgeBench Exploration)

A natural research question in non-autoregressive decision modeling is: Can a 9B model without chain-of-thought (CoT) solve complex 10-choice college STEM reasoning (MMLU-Pro / JudgeBench)?

We conducted an extensive series of ablation experiments exploring Test-Time Augmentation (TTA), Temperature Scaling, and Continual Knowledge Reinforcement (Option A):

💡 Key Takeaway: Why Selective Escalation Beats Brute-Force Capacity

1. 1.The 9B Non-autoregressive Ceiling:
2. 2.Without generating intermediate reasoning tokens (Chain-of-Thought), a 9B model's internal associative memory maxes out around 48% on college-level multi-step STEM proofs (compared to 59.1% on 27B and 78.6% on JEV 1.13 hosted ensemble). Continual SFT/RL yields modest gains (+3.15%p), but cannot bridge the fundamental capacity gap.
3. 3.The Power of Calibrated Refusal:
4. 4.The primary objective of RLCD is NOT to force a small 9B model into solving Olympiad mathematics, but to calibrate uncertainty: when unsure, the model honestly drops its confidence to approx. 42% rather than hallucinating.
5. 5.When confidence is &ge; 0.90, its accuracy is an astonishing 97.06%.
6. 6.By routing difficult queries to a flagship teacher LLM (GPT-6) and handling confident queries in 160ms, the Cascade Router achieves 93.5% overall accuracy while saving 71.4% of API expenditure.

💻 Standalone Inference & Cascade Usage

1. Direct Inference with Kev:

import torch
from kev.checkpoint import Checkpoint, LoadOptions

# Load Qwev-9B-RLCD adapter directly from Hugging Face
ck = Checkpoint("gyung/Qwev-9B-RLCD")
tok, model = ck.load(device="cuda", opts=LoadOptions(dtype=torch.bfloat16, merge=True))
model.eval()

# Input State and Options
record = {
    "state": "Context:\nParis is the capital of France.\n\nQuestion: What is the capital of France?\n\nCandidate Answer A: Paris.\nCandidate Answer B: London.",
    "questions": [{
        "instr": "Select the factually accurate answer.",
        "options": [
            "Answer A: Factually sound.",
            "Answer B: Factual error."
        ],
        "label": 0
    }]
}

enc = model.encode(tok, record)
probs = model.probs(enc)[0].cpu().numpy()
print(f"Option Probabilities: {probs}")
# -> [0.998, 0.002] (Confidence: 99.8% on Option A)

2. Cascade Escalation Router (CMU Protocol):

def route_decision(model, tok, record, tau=0.90):
    enc = model.encode(tok, record)
    probs = model.probs(enc)[0].cpu().numpy()
    pred_idx = probs.argmax()
    conf = probs.max()

    if conf >= tau:
        return {"decision": pred_idx, "confidence": float(conf), "escalated": False}
    else:
        # Escalate to Teacher Flagship (e.g., GPT-6)
        print(f"[!] Unconfident ({conf:.2f} < {tau}). Escalating to GPT-6...")
        return {"decision": call_flagship_llm(record), "confidence": 1.0, "escalated": True}

🇰🇷 한국어 안내 (Korean Overview)

Qwev-9B-RLCD는 오픈소스 의사결정 모델인 [`jaredpalmer/kev-9b`](https://huggingface.co/jaredpalmer/kev-9b)(Qwen/Qwen3.5-9B-Base 백본 + Pointer Head)을 부모 모델로 하여, RLCD(Reinforcement Learning from Calibrated Decisions, 확률 캘리브레이션 강화학습)을 적용해 과신 오답을 억제하고 불확실성 인지 능력을 극대화한 초저지연 비생성형 의사결정 모델입니다.

💡 CMU 논문과의 관계 명시: 카네기 멜론 대학교(CMU)의 "JEV-as-a-Judge: Accept When Confident, Escalate When Unsure" (arXiv:2609.26550) 논문의 Table 1 공식 4대 벤치마크(RewardBench, HaluEval, JudgeBench, RM-Bench) 전수 실측 평가 체계와 "확신도 90%(&tau; &ge; 0.90) 이상일 때 즉시 채택(Accept), 미만일 때 상위 모델로 이관(Escalate)"하는 2단계 캐스케이드(Cascade) 평가 아이디어를 실증 벤치마킹하는 데 활용하였습니다.

• —부모 기반 모델: `jaredpalmer/kev-9b` (Qwen/Qwen3.5-9B-Base 백본 + 128차원 Pointer Head)
• —초고속 1-Pass 추론: 토큰을 생성하지 않고 포인터 헤드로 단 0.16초(168.4ms)만에 정답을 결정 (GPT-6 Astra 대비 11배 고속).
• —SOTA 벤치마크: RewardBench 99.2%, HaluEval 98.8%, RM-Bench 98.0%로 CMU JEV 1.13 및 GPT-6 Astra를 능가.
• —정직한 확신도(Uncertainty Calibration): 10지선다 고난도 JudgeBench에서 무작정 찍지 않고 평균 확신도를 42.3%로 정직하게 낮추어, 확신도 90% 이상 채택 시 97.06%의 정확도를 보장합니다.
• —캐스케이드 비용 절감: 모르는 문제는 상위 플래그십 LLM으로 에스컬레이션하여 GPT-6급 성능(93.5%)을 유지하면서도 API 비용을 약 71.4% 절감합니다.

🔬 10지선다 고난도 STEM(JudgeBench) 한계 및 절제 연구(Ablation) 시사점

• —9B 비생성형의 본질적 한계: 생각 과정(CoT) 토큰을 생성하지 않고 0.16초 만에 10지선다 대학 수준 수학/물리를 푸는 것은 9B 파라미터 용량상 약 48%(TTA 적용 시)가 한계점입니다. 1,700건의 추가 STEM 강화학습을 진행해도 기본 1-Pass 정확도는 41.7%에서 44.9%(+3.2%p)로 소폭 상승하는 데 그칩니다 (3배 큰 Kev-27B도 59.1% 수준).
• —왜 캐스케이드(Cascade)가 최선인가?: 9B 모델을 억지로 쥐어짜서 풀게 만드는 것보다, "모르면 42%의 정직한 확신도로 자백하여 플래그십(GPT-6 등)으로 넘기고, 99% 이상 잘하는 인간 선호도·사실성·스타일 판정은 160ms로 처리하는 전략"이 CMU 논문이 증명한 가장 실용적이고 수학적으로 최적인 엔지니어링 해법입니다.

📜 Citation

@article{qwev2026rlcd,
  title={Qwev-9B-RLCD: Fast Non-Autoregressive Decision Alignment with Calibrated Uncertainty},
  author={Gyung},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/gyung/Qwev-9B-RLCD}}
}