Modelpublic

MSGEncrypted/ocf-typed-decisions-mbert-base

sourceHugging Faceapache-2.0updated 12d agoView on Hugging Face

0likes674downloads

Model Card

OCF typed-decisions CE smoke (ModernBERT-base)

Not a foundation classifier. Not RLCD. No Jev comparison.

Open Classification Foundation (OCF) specialist smoke: non-autoregressive System-1 decision head on `answerdotai/ModernBERT-base`, fine-tuned with cross-entropy on LocalLLaMA/typed-decisions train, temperature-calibrated on a held-out train slice (cal_frac=0.1).

Architecture

text

ModernBERT-base (bidirectional)
  + 2-layer decision Transformer head
  + OptionScorer at each option [MASK]
  → softmax(/T) over options

Knob	Value
Params	~164M
`max_len`	640
Option token budget	48 / option
Primitives	`choice` / `score` / `noul`

Training

Field	Value
Objective	CE on teacher-argmax labels (soft teachers in dataset)
Epochs / eff. batch	4 / 8
LR encoder / head	2.5e-5 / 1e-4
Seed (this upload)	1 (best of 0/1/2; mean acc 0.731, SD 0.028)
Temperatures (shipped)	choice 1.5 · score 1.25 · noul 1.5

Evaluation (`LocalLLaMA/typed-decisions` test, 2,000 decisions)

Arm	acc	NLL	ECE as-run	ECE after equal crossfit
Laya typed-decisions (reference)	0.767	0.707	0.214	0.021
This checkpoint (s1)	0.751	0.629	0.044	0.030
OCF stub (prior, ~8M)	≈0.61	≈0.90	≈0.04	—

Latency (warm, RTX 3050 Laptop): p50 59.6 ms / p95 87.8 ms per question (n=200). Not a cross-vendor claim.

Full board: see source-repo snapshot results-snapshot-2026-09-28-ocf-typed-decisions-mbert.md.

Files

—config.json — ClassifierConfig (includes temperatures)
—temperatures.json — fitted per-type T
—model.pt — full ClassifierBundle state_dict
—recipe.json — train hyperparams + acc

How to run

Requires the OCF stack from the source repository (packages.classify + models_ai.classify):

python

from classify import load

agent = load("path/to/this/repo")  # config.json + model.pt + temperatures.json
out = agent.predict(
    {"subject": "Refund", "body": "Billed twice"},
    {
        "department": {
            "type": "choice",
            "instructions": "Which department?",
            "criteria": {"billing": "invoices", "tech": "bugs", "other": "else"},
        }
    },
)

Not a drop-in for the external laya package.

Concurrent baseline

Laya — open System-1 concurrent (ModernBERT-large). Matched estimand only; not a dependency.

Non-claims

—Not a foundation / zero-shot System-1 model
—Not RLCD-trained
—Does not claim to beat TypeSafe Jev or Laya
—Soft teacher labels; teachers disagree on ~41% of test decisions