closed-book
flan_t5_large-sciq_Multiple_Choice_Closed_Book_flan_t5_large-sciq_Direct_Question_Closed_Book_closedbook_qa_distillabs_ggufflan2021-closed-book-qa-held-out-llama-3-8b-1-epochs-ITflan2021-closed-book-qa-held-out-mistral-7b-1-epochs-ITflan_t5_xl-sciq_Multiple_Choice_Closed_Book_flan_t5_xl-sciq_Direct_Question_Closed_Book_closed-book-19-06-2023
harvey-closed-book-qwen35-9b-notes-conditioned-30m-kl-0p05
harvey-closed-book-qwen35-9b-notes-conditioned-30m-kl-0p05
Completed closed-book C&H knowledge evaluation of violetxi/qwen35-9b-harvey-v4-notes-conditioned-30m-kl-0p05,
revision 1bced92ee267198876b452d59e13cf17c41d5df9, job 122412. All 7,933 probes completed
without API errors. Each probe category is a separate split in this single repo.
Split
Probes
Accuracy
forward_qa
2,515
16.90%
mc_logprob
1,780
50.45%
negative_abstain
1,799
41.30%
reversed_qa
1,839
7.94%… See the full description on the dataset page: https://huggingface.co/datasets/violetxi/harvey-closed-book-qwen35-9b-notes-conditioned-30m-kl-0p05.harvey-closed-book-qwen35-9b-notes-conditioned-30m-kl-0p01
harvey-closed-book-qwen35-9b-notes-conditioned-30m-kl-0p01
Completed closed-book C&H knowledge evaluation of violetxi/qwen35-9b-harvey-v4-notes-conditioned-30m-kl-0p01,
revision 8c81d012091313d40b3619f15dec11bf47d19c73, job 122411. All 7,933 probes completed
without API errors. Each probe category is a separate split in this single repo.
Split
Probes
Accuracy
forward_qa
2,515
18.21%
mc_logprob
1,780
50.11%
negative_abstain
1,799
36.85%
reversed_qa
1,839
9.52%… See the full description on the dataset page: https://huggingface.co/datasets/violetxi/harvey-closed-book-qwen35-9b-notes-conditioned-30m-kl-0p01.harvey-closed-book-qwen35-9b-notes-conditioned-10m
harvey-closed-book-qwen35-9b-notes-conditioned-10m
Completed closed-book C&H knowledge evaluation of violetxi/qwen35-9b-harvey-v4-notes-conditioned-10m,
revision 4bf295a036c010040d76960e167bfbf0efd05927, job 122446. All 7,933 probes completed
without API errors. Each probe category is a separate split in this single repo.
Split
Probes
Accuracy
forward_qa
2,515
13.04%
mc_logprob
1,780
41.97%
negative_abstain
1,799
51.70%
reversed_qa
1,839
6.14%
forward_qa:… See the full description on the dataset page: https://huggingface.co/datasets/violetxi/harvey-closed-book-qwen35-9b-notes-conditioned-10m.harvey-closed-book-qwen35-9b-notes-conditioned-30m-kl-0p1
harvey-closed-book-qwen35-9b-notes-conditioned-30m-kl-0p1
Completed closed-book C&H knowledge evaluation of violetxi/qwen35-9b-harvey-v4-notes-conditioned-30m-kl-0p1,
revision d657eca6271c10bdd83a5275f58cf0b3d3dc823d, job 122413. All 7,933 probes completed
without API errors. Each probe category is a separate split in this single repo.
Split
Probes
Accuracy
forward_qa
2,515
15.35%
mc_logprob
1,780
50.22%
negative_abstain
1,799
30.41%
reversed_qa
1,839
6.14%… See the full description on the dataset page: https://huggingface.co/datasets/violetxi/harvey-closed-book-qwen35-9b-notes-conditioned-30m-kl-0p1.harvey-closed-book-qwen35-9b-notes-conditioned-5m
harvey-closed-book-qwen35-9b-notes-conditioned-5m
Completed closed-book C&H knowledge evaluation of violetxi/qwen35-9b-harvey-v4-notes-conditioned-5m,
revision 906f2b45e2962698c489cfd1f53ab19a4ebe1458, job 122445. All 7,933 probes completed
without API errors. Each probe category is a separate split in this single repo.
Split
Probes
Accuracy
forward_qa
2,515
7.24%
mc_logprob
1,780
36.85%
negative_abstain
1,799
37.08%
reversed_qa
1,839
2.83%
forward_qa:… See the full description on the dataset page: https://huggingface.co/datasets/violetxi/harvey-closed-book-qwen35-9b-notes-conditioned-5m.harvey-closed-book-qwen35-9b-notes-conditioned-100m-kl-0p1
harvey-closed-book-qwen35-9b-notes-conditioned-100m-kl-0p1
Completed closed-book C&H knowledge evaluation of violetxi/qwen35-9b-harvey-v4-notes-conditioned-100m-kl-0p1,
revision 04390b5989d8d9adfe5597339f24a27b9b411c37, job 122416. All 7,933 probes completed
without API errors. Each probe category is a separate split in this single repo.
Split
Probes
Accuracy
forward_qa
2,515
17.06%
mc_logprob
1,780
59.38%
negative_abstain
1,799
27.24%
reversed_qa
1,839
6.42%… See the full description on the dataset page: https://huggingface.co/datasets/violetxi/harvey-closed-book-qwen35-9b-notes-conditioned-100m-kl-0p1.