Datasetpublic

LawrenceYin/annotation-pack-a

Annotation pack A — does a response genuinely follow an instruction? 50 rows. Each row: a user prompt, one instruction from it, and a model response that an automatic checker marks as satisfying that instruction. Annotators judge whether it is satisfied genuinely. For annotators / 标注人: Read RUBRIC_HUMAN.md (English + 中文). Annotator A downloads annotator_A.csv; annotator B downloads annotator_B.csv (same items). Fill label (GENUINE / LOOPHOLE / GARBLED), helpfulness (1–5)… See the full description on the dataset page: https://huggingface.co/datasets/LawrenceYin/annotation-pack-a.

sourceHugging Facecc-by-4.0updated 2d agoView on Hugging Face

0likes41downloads

Dataset Card

Annotation pack A — does a response genuinely follow an instruction?

50 rows. Each row: a user prompt, one instruction from it, and a model response that an automatic checker marks as satisfying that instruction. Annotators judge whether it is satisfied genuinely.

For annotators / 标注人:

1.Read RUBRIC_HUMAN.md (English + 中文).
2.Annotator A downloads annotator_A.csv; annotator B downloads annotator_B.csv (same items).
3.Fill label (GENUINE / LOOPHOLE / GARBLED), helpfulness (1–5), optional notes. Work alone.
4.Send the filled CSV back directly to the person who asked you (do not upload it here).

Expected time: 30–45 minutes.