Datasetpublic

asingh15/amazon-c11-distillation-filtered

Amazon C11 quality-filtered distillation This is a high-signal SFT view of asingh15/amazon-c11-distillation. It contains six balanced rubric-writer/criterion-judge configurations. The original source remains unchanged. Filter A trajectory is retained only when its selected rubric has gold-score spread greater than 0.10 on both the selection panel and the paired held-out panel, non-constant proxy scores on both, positive Spearman correlation on both, positive… See the full description on the dataset page: https://huggingface.co/datasets/asingh15/amazon-c11-distillation-filtered.

sourceHugging Faceupdated 23d agoView on Hugging Face

0likes420downloads

Dataset Card

Amazon C11 quality-filtered distillation

This is a high-signal SFT view of `asingh15/amazon-c11-distillation`. It contains six balanced rubric-writer/criterion-judge configurations. The original source remains unchanged.

Filter

A trajectory is retained only when its selected rubric has gold-score spread greater than 0.10 on both the selection panel and the paired held-out panel, non-constant proxy scores on both, positive Spearman correlation on both, positive random-to-oracle normalized tie-robust harmonic Best@1..32 on both, and no writer truncation or recovery. The source stopping rubric is never reselected using held-out results.

All writer-prefix rows are retained. An equal number of judge rows is selected deterministically per trajectory, covering every final criterion first and then maximizing ICL-width and activation-label coverage. The native train, validation, and test splits are preserved.

Config	Train rows	Validation	Test	Train panels kept	Source-published retention
`latent-state-spearman`	66,012	582	694	4,688	47.4%
`latent-state-best32-norm-tr`	70,872	664	818	4,299	43.5%
`latent-state-harmonic32-norm-tr`	74,032	664	824	4,565	46.2%
`non-diverse-spearman`	69,180	812	672	5,241	57.2%
`non-diverse-best32-norm-tr`	78,876	842	844	4,869	53.1%
`non-diverse-harmonic32-norm-tr`	79,964	942	918	5,239	57.2%

Gold-score spread audit

Gold spread is max(gold) - min(gold) over each frozen 40-answer panel. Raw covers all source reviewers, Source reflects the original corpus's spread gate, and Filtered reflects this release. The filter raises mean spread and therefore enriches for higher-signal panels; it is not a difficulty-representative benchmark sample. The p10–p90 columns show that the retained data still covers a range of panel score spreads.

Config	Side	Raw mean	Source mean	Filtered mean	Filtered p10–p90
`latent-state-spearman`	own	0.735	0.742	0.776	0.600–0.900
`latent-state-spearman`	cross	0.610	0.615	0.701	0.400–0.900
`latent-state-best32-norm-tr`	own	0.735	0.742	0.778	0.600–0.900
`latent-state-best32-norm-tr`	cross	0.610	0.615	0.703	0.450–0.900
`latent-state-harmonic32-norm-tr`	own	0.735	0.742	0.777	0.600–0.900
`latent-state-harmonic32-norm-tr`	cross	0.610	0.615	0.703	0.416–0.900
`non-diverse-spearman`	own	0.610	0.657	0.699	0.400–0.900
`non-diverse-spearman`	cross	0.735	0.751	0.774	0.600–0.900
`non-diverse-best32-norm-tr`	own	0.610	0.657	0.700	0.450–0.900
`non-diverse-best32-norm-tr`	cross	0.735	0.751	0.776	0.600–0.900
`non-diverse-harmonic32-norm-tr`	own	0.610	0.657	0.700	0.450–0.900
`non-diverse-harmonic32-norm-tr`	cross	0.735	0.751	0.776	0.600–0.900

Quality sidecar

Aggregate per-trajectory diagnostics are under quality/<split>/*.parquet. They include inclusion status and exclusion reasons, own/cross gold spread, proxy dispersion, Spearman, normalized harmonic Best@1..32, and role-row counts. Raw score or activation vectors and private targets are not published. The signed aggregate report is quality_audit.json.

Provenance

—Source dataset: asingh15/amazon-c11-distillation at 3f7302f2eb78cfa8a90110bd370237fd11e1638c
—Source objective catalog SHA-256: faff332d2bf465d37c7d6e9c0bb068522d80e3ae1e5f777688ec6c607f99f1d2
—Collection manifest SHA-256: 866c7bc6801f5fa8813a498c51eaa176ce6f5ff4042197a160f2e6162aa42d45
—Filter policy SHA-256: 3bafbdac25d0c745d75a44fa420ed647916ebf9f0ef64f26870c8690fe11d60c
—Filtered catalog SHA-256: 1b756f6f8b6124b7682db6a65c4054265b034b3e530eb03558fa675fad0e6f98
—Loss mask: qwen3_final_response