Mohaaxa/quantbench-leaderboard-data

QuantBench leaderboard data

Raw benchmark data behind the QuantBench leaderboard: calibration-quality GPTQ/AWQ quantization results across model sizes, calibration corpora, and GPU tiers. 331 rows (239 ok / 92 failed — failed runs are published too; a documented failure is a finding, not noise).

Models

• —Qwen/Qwen2.5-1.5B-Instruct (1.5B)
• —HuggingFaceTB/SmolLM2-1.7B-Instruct (1.7B)
• —deepgrove/Bonsai (0.5B)
• —Qwen/Qwen2.5-3B-Instruct (3B) — licence pending, rows only, no weights here
• —HuggingFaceTB/SmolLM3-3B (3B)

Files

• —rows.csv — the full table (see the leaderboard site for column definitions and the methodology page for the eval protocol).
• —rows.parquet — the same table with an explicit Arrow schema (numeric columns nullable int64/double, blanks as null; everything else string). This is what the dataset viewer reads (configs: above); rows.csv is unchanged for humans.
• —<row_id>.json — one file per row, same fields as its rows.csv line.
• —niche_rows.csv — task-level niche columns over the same artifacts (HumanEval[0:50] pass@1, FiQA-2018 sentiment accuracy [flare-finqa is gated for this token — instrument recorded per row], SEC-filing context drift, 10-prompt ROUGE-L consistency), each vs a same-stack fp16 baseline; failures/skips recorded as rows.
• —logs/ — the driver log, last heartbeat, and spend ledger from the run that produced this data (transparency, not polish — these are raw operational logs).
• —pools/BUILD.md — exact recipe to reproduce the calibration/eval text pools. No third-party corpus text is redistributed here.

Honesty notes carried into every row

• —GPTQ rows in this sweep ran gptqmodel's torch-fallback kernel path (load_backend column), not the optimized kernel — perplexity is unaffected, throughput numbers understate what a working kernel would show.
• —fp16 baselines are per-stack (autoawq vs gptqmodel use different torch/transformers pins), so ppl_delta_vs_fp16 is only comparable within one stack and one GPU tier, never across them.
• —webgpu_runnable is [UNVERIFIED] on every row — no browser measurement was run.
• —Machine-generated benchmark output. Independent verification welcome; treat any single row as a data point, not a certified result.