Modelpublic

Beko2210/statim-decide-multilingual-base

sourceHugging Faceotherupdated 7h agoView on Hugging Face

2likes174downloads

Model Card

Statim Decide Multilingual Base

Licence. This version was trained partly on data that is non-commercial, ShareAlike or under an unknown licence (the same training mixture as 0.7.0; findings in DATA_LICENSES.md). The weights are offered only under PolyForm Noncommercial 1.0.0. A version trained only on cleared data will follow.

A decision model for Statim, the native C++ engine for typed decisions: ask any text a choice, a score or a yes/no question and get calibrated answers from one forward pass, on CPU or GPU, without Python at runtime. Version 0.10.0, fine-tuned from `convaiinnovations/laya-multilingual` (mmBERT-base encoder).

[▶ Try it live in your browser](https://huggingface.co/spaces/Beko2210/statim): this model on a free CPU, no install and no key.

One support ticket, three typed answers, one forward pass: the 60-second film.

Quick start

# Statim release binary: https://github.com/BEKO2210/statim/releases
huggingface-cli download Beko2210/statim-decide-multilingual-base statim-decide-multilingual-base-q8_0.gguf --local-dir models
./statim serve -m multilingual=models/statim-decide-multilingual-base-q8_0.gguf --port 8080
curl -s localhost:8080/v1/systemone -d '{
  "state": {"subject": "Duplicate charge on invoice #4411",
            "body": "We were billed twice for March. Please refund the second charge."},
  "questions": {
    "department": {"type": "choice", "instructions": "Which department should handle this?",
      "criteria": {"billing": "invoices, refunds", "technical": "bugs, outages", "sales": "pricing, contracts"}},
    "urgency": {"type": "score", "instructions": "How urgent is this request?",
      "criteria": ["not urgent", "soon", "critical"]},
    "refund": {"type": "noul", "instructions": "Does the user explicitly request a refund?"}}}'

Open http://127.0.0.1:8080/ for the playground. API reference: docs/API.md.

Files

File	Size	Use
`statim-decide-multilingual-base-f32.gguf`	0.91 GB	reference precision, exact on GPU
`statim-decide-multilingual-base-q8_0.gguf`	0.36 GB	smaller; slower than f32 on AVX2 CPUs, faster on ARM dotprod and CUDA
`checkpoint/`	0.68 GB	Laya-format checkpoint for fine-tuning and the Python reference

Checksums in SHA256SUMS. The f32 file reproduces the reference implementation within 1e-4 on Statim's parity tests. q8_0 is 2.5x smaller with slightly different logits. Whether it is faster depends on the hardware: on x86-64 CPUs with AVX2 but without int8 dot-product instructions it is slower than f32; on ARM CPUs with dot-product instructions and with CUDA it is faster (measurements).

Evaluation

Measured by Statim's no-harm gate (`tools/finetune/gate.py`) on held-out test data the model selection never looked at. Against the previous release 0.7.0 (same held-out item pool), on 91 held-out suites: 6 significant gains, 85 within noise, 0 regressions (paired exact McNemar tests; gains and regressions are separately significant after Holm-Bonferroni over all suites).

Suite	Role	This model	0.7.0	Protocol
typed-decisions	trained	0.7750	0.7630	test split, first 2,000 decisions; its train split is replay data
Banking77	trained	0.9185	0.9140	test split, first 2,000 rows, all 77 intents in one question
MASSIVE intents	trained	0.8161	0.7995	mean over 12 languages, 150 seeded stratified test rows each
AG News	held out	0.9210	0.9295	zero-shot (never trained on), first 2,000 test rows
DAIR Emotion	held out	0.5305	0.5040	zero-shot, first 2,000 test rows
HWU64 intents	held out	0.8533	0.7867	English, 150 rows; rows overlapping MASSIVE removed
SIB-200 topics	held out	0.7967	0.7817	zero-shot, mean over 4 languages, 150 rows each
Sentiment	held out	0.6311	0.6050	zero-shot, mean over 12 languages, 150 rows each
HateCheck	held out	0.6461	0.6358	zero-shot, mean over 11 languages, 150 rows each
Belebele reading	held out	0.2800	0.2583	zero-shot, mean over 4 languages, 150 rows each

Decision categories

One held-out suite per decision category, built from splits of the training sources that the mixture never loads; any text that also occurs in the training mixture is dropped. 150 items per language, macro over languages.

Category	Languages	This model	0.7.0
complaint	en	0.820	0.767
emotion	de, en, es, fr, hi, pt, ru, zh	0.661	0.589
fact check	en	0.467	0.313
formality	ja, tr	0.953	0.773
intent	en, nl, tr	0.793	0.753
nli	en, ja, tr	0.769	0.747
pii	ar, de, en, es, fr, it, ja, nl, ru, sv, zh	0.909	0.856
reading	en	0.933	0.927
safety	en	0.880	0.727
sentiment	en, zh	0.877	0.800
similarity	pt	0.880	0.833
stance	en	0.980	0.893
topic	en	0.647	0.607
urgency	en	0.987	0.893

Published systems under the same protocol, for orientation: typed-decisions meraGPT 0.768, laya-typed-decisions 0.766, Jev 0.727; AG News zero-shot Laya 0.950, GPT-3 (CARP) 0.926, Jev 0.881; Banking77 supervised MPNet 0.941; MASSIVE XLM-R base 0.857 over 12 languages (full train set). Sources: docs/ROADMAP.md.

<details><summary>All 91 held-out suites</summary>

Suite	Accuracy	Rows
`amazon_massive_intent/ar`	0.7400	150
`amazon_massive_intent/de`	0.8267	150
`amazon_massive_intent/en`	0.8467	150
`amazon_massive_intent/es`	0.8333	150
`amazon_massive_intent/fr`	0.8067	150
`amazon_massive_intent/hi`	0.7800	150
`amazon_massive_intent/it`	0.8133	150
`amazon_massive_intent/ja`	0.8333	150
`amazon_massive_intent/pl`	0.8333	150
`amazon_massive_intent/ru`	0.8467	150
`amazon_massive_intent/tr`	0.8133	150
`amazon_massive_intent/zh-CN`	0.8200	150
`belebele/ar`	0.2600	150
`belebele/de`	0.3200	150
`belebele/en`	0.3133	150
`belebele/hi`	0.2267	150
`categories:complaint/en`	0.8200	150
`categories:emotion/de`	0.4867	150
`categories:emotion/en`	0.6400	150
`categories:emotion/es`	0.6200	150
`categories:emotion/fr`	0.8267	150
`categories:emotion/hi`	0.8600	150
`categories:emotion/pt`	0.5600	150
`categories:emotion/ru`	0.7400	150
`categories:emotion/zh`	0.5533	150
`categories:fact_check/en`	0.4667	150
`categories:formality/ja`	0.9067	150
`categories:formality/tr`	1.0000	150
`categories:intent/en`	0.8467	150
`categories:intent/nl`	0.5333	150
`categories:intent/tr`	1.0000	150
`categories:nli/en`	0.8000	150
`categories:nli/ja`	0.6600	150
`categories:nli/tr`	0.8467	150
`categories:pii/ar`	0.9333	150
`categories:pii/de`	0.8867	150
`categories:pii/en`	0.9333	150
`categories:pii/es`	0.8867	150
`categories:pii/fr`	0.9133	150
`categories:pii/it`	0.9000	150
`categories:pii/ja`	0.9267	150
`categories:pii/nl`	0.8533	150
`categories:pii/ru`	0.9667	150
`categories:pii/sv`	0.8733	150
`categories:pii/zh`	0.9267	150
`categories:reading/en`	0.9333	150
`categories:safety/en`	0.8800	150
`categories:sentiment/en`	0.8533	150
`categories:sentiment/zh`	0.9000	150
`categories:similarity/pt`	0.8800	150
`categories:stance/en`	0.9800	150
`categories:topic/en`	0.6467	150
`categories:urgency/en`	0.9867	150
`farstail/fa`	0.6533	150
`go_emotions/en`	0.4533	150
`hwu64/en`	0.8533	150
`indonli/id`	0.7000	150
`multi_hatecheck/ar`	0.6667	150
`multi_hatecheck/de`	0.6733	150
`multi_hatecheck/en`	0.6467	150
`multi_hatecheck/es`	0.6467	150
`multi_hatecheck/fr`	0.6600	150
`multi_hatecheck/hi`	0.5800	150
`multi_hatecheck/it`	0.6200	150
`multi_hatecheck/nl`	0.6467	150
`multi_hatecheck/pl`	0.6533	150
`multi_hatecheck/pt`	0.6733	150
`multi_hatecheck/zh`	0.6400	150
`multilingual_sentiments/ar`	0.6333	150
`multilingual_sentiments/de`	0.5600	150
`multilingual_sentiments/en`	0.7200	150
`multilingual_sentiments/es`	0.6000	150
`multilingual_sentiments/fr`	0.6267	150
`multilingual_sentiments/hi`	0.5400	150
`multilingual_sentiments/id`	0.7800	150
`multilingual_sentiments/it`	0.6467	150
`multilingual_sentiments/ja`	0.6800	150
`multilingual_sentiments/ms`	0.5333	150
`multilingual_sentiments/pt`	0.6400	150
`multilingual_sentiments/zh`	0.6133	150
`semrel/ar`	0.2733	150
`semrel/en`	0.2800	150
`semrel/hi`	0.2933	150
`sib200/ar`	0.7800	150
`sib200/de`	0.8267	150
`sib200/en`	0.8467	150
`sib200/hi`	0.7333	150
`test/ag_news`	0.9210	2000
`test/banking77`	0.9185	2000
`test/emotion`	0.5305	2000
`test/typed_decisions`	0.7750	2000

</details>

Reproduce these numbers: REPRODUCE.md.

Training

A uniform weight average (model soup, Wortsman et al., 2022) of 3 runs of train_multitask.py --clean from checkpoint laya-multilingual-big1 (0.4.0), which differ only in seed and batch order (best epochs 12/raw, 10/raw, 7/ema, each selected on validation data only); the temperatures were refitted on the validation items afterwards (--calibrate-only). Training data: the 0.7.0 mixture (Banking77, MASSIVE, typed-decisions replay, a tasksource mixture, Nemotron-Safety-Guard, IndicGuard, MINDS-14, SNIPS and further sources), including the sources the 2026-10-03 licence audit found non-commercial, ShareAlike or unlicensed; every source and finding is listed in DATA_LICENSES.md. Evaluation test rows were removed from the training data.

Intended use and limits

—Classification-style decisions over short texts and JSON: routing, triage, moderation, intent, yes/no checks, ordinal ratings. It does not generate text.
—Accuracy varies by task and language (see the table). Reading comprehension (Belebele) and semantic similarity are weak; do not use it for them without your own evaluation.
—Use the confidence: Statim's min_confidence option marks low-confidence answers with escalate: true so a person can review them. Do not automate high-stakes decisions about people without human review.

Licence

The weights may be used only under PolyForm Noncommercial 1.0.0 (text in LICENSE-MODEL.md); the Small Business, Free Trial and commercial licences do not apply to this version. The Statim engine is Apache-2.0.

Built on Laya (Apache-2.0) and mmBERT-base (MIT). Training data attribution: Banking77 (Casanueva et al., 2020, PolyAI), MASSIVE (FitzGerald et al., 2022, Amazon), and the CC-BY sources in DATA_LICENSES.md. Statim is independent and not affiliated with the Laya authors.