Modelpublic

latent-lab/smaller-than-truth-bitnet-2b

sourceHugging Facemitupdated 7mo agoView on Hugging Face

0likes

Model Card

lmprobe: Linear Probe on bitnet-b1.58-2B-4T

Truth probe for 'X is smaller than Y' statements. Near-perfect accuracy (99.0%) — structural/relational knowledge survives ternary quantization.

Classes

—0: false_statement
—1: true_statement

Usage

python

from lmprobe import LinearProbe

probe = LinearProbe.from_hub("latent-lab/smaller-than-truth-bitnet-2b", trust_classifier=True)
predictions = probe.predict(["your text here"])

Probe Details

—Base model: microsoft/bitnet-b1.58-2B-4T
—Model revision: 04c3b9ad9361b824064a1f25ea60a8be9599b127
—Layers: all (0–29, 30 layers)
—Pooling: last_token
—Classifier: logistic_regression
—Task: classification
—Random state: 42

Evaluation

Metric	Value
accuracy	0.9899
auroc	0.9995
f1	0.9899
precision	0.9899
recall	0.9899

Training Data

—Positive examples: 792
—Negative examples: 792
—Positive hash: sha256:55f43cad0c06e9599839603d3c513543de4ca71eed2b22cccb16542930f6e2b8
—Negative hash: sha256:d9f02b07f2025712a6dd175310f7af58ee761c73b40792e10cf06335953628c0

—Evaluation samples: 396
—Evaluation hash: sha256:546af552622d90b1e54d88d3111f68aa201178f0458267da51ecbe393443e821

Reproducibility

—lmprobe version: 0.5.8
—Python: 3.12.3
—PyTorch: 2.10.0+cu128
—scikit-learn: 1.8.0
—transformers: 5.3.0