Datasetpublic

asingh15/amazon-c2-varied-rubrics

Amazon C2 varied-rubric distillation This release exposes six balanced C2 SFT configurations: latent-state and non-diverse candidate panels at K=1, K=2, and K=4 rubrics per retained reviewer. Each rubric-writer target is paired with one full-rubric listwise judge target over the same variant's frozen 40-candidate panel. The K arms within a variant share one reviewer cohort and are exact nested prefixes. Config Train rows Validation Test Train reviewers… See the full description on the dataset page: https://huggingface.co/datasets/asingh15/amazon-c2-varied-rubrics.

sourceHugging Faceupdated 17d agoView on Hugging Face

0likes293downloads

Dataset Card

Amazon C2 varied-rubric distillation

This release exposes six balanced C2 SFT configurations: latent-state and non-diverse candidate panels at K=1, K=2, and K=4 rubrics per retained reviewer. Each rubric-writer target is paired with one full-rubric listwise judge target over the same variant's frozen 40-candidate panel. The K arms within a variant share one reviewer cohort and are exact nested prefixes.

Config	Train rows	Validation	Test	Train reviewers
`latent-state-varied-k1`	9,202	98	98	4,601
`latent-state-varied-k2`	18,404	196	196	4,601
`latent-state-varied-k4`	36,808	392	392	4,601
`non-diverse-varied-k1`	9,202	98	98	4,601
`non-diverse-varied-k2`	18,404	196	196	4,601
`non-diverse-varied-k4`	36,808	392	392	4,601

Selection and provenance

Every chosen rubric is an accepted snapshot from one sequential teacher trajectory, not an independent teacher sample. It must pass the pinned quality policy on both candidate panels. Selection ranks eligible snapshots on the arm's own panel, then greedily maximizes an equal blend of rank disagreement and exact criterion-set Jaccard distance. Additional rubrics must clear the pinned combined-distance floor.

The 102 aggregate selection audits are transported as deterministic gzip under selections/<split>/part-*.json.gz; their uncompressed hashes remain signed by the block manifests. They contain rubric quality and distance diagnostics but no private review targets or raw oracle vectors. The companion `selection-audit viewer` reads those sidecars at a pinned dataset commit. selection_transport.json maps every signed source sidecar to its compressed transport path, encoding, size, and hash so the unchanged block manifests remain machine-resolvable.

—Rubric source: shared-sequential-trajectory-snapshot
—Source collection manifest SHA-256: 866c7bc6801f5fa8813a498c51eaa176ce6f5ff4042197a160f2e6162aa42d45
—Source config SHA-256: f0c8a1ee29cc3b2ad93d3ad14b55149da73d6920e496e26037ee984749b83c52
—Quality policy SHA-256: 3bafbdac25d0c745d75a44fa420ed647916ebf9f0ef64f26870c8690fe11d60c
—Selection policy SHA-256: 60d1678f791fec71809667fff5d895ac0a66a7d60e8c8daa27968be62dc481ff
—Dataset catalog SHA-256: 8f6080e56114ac5ea1bfffdb497d11f598e03a552b98b588e6e713487c0d89a7
—Row schema: amazon-c2-varied-rubrics-sft-row-v1
—Loss mask: qwen3_final_response