Datasetpublic

mlfoundations-dev/a1_code_stackexchange_codereview_1744693103_eval_1331

mlfoundations-dev/a1_code_stackexchange_codereview_1744693103_eval_1331 Precomputed model outputs for evaluation. Evaluation Results Summary Metric AIME24 AMC23 MATH500 GPQADiamond JEEBench MMLUPro LiveCodeBench CodeElo Accuracy 18.0 52.5 74.8 42.8 40.3 28.6 25.6 6.8 AIME24 Average Accuracy: 18.00% ± 1.35% Number of Runs: 10 Run Accuracy Questions Solved Total Questions 1 26.67% 8 30 2 20.00% 6 30 3… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations-dev/a1_code_stackexchange_codereview_1744693103_eval_1331.

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes7downloads

Dataset Card

mlfoundations-dev/a1codestackexchangecodereview1744693103eval1331

Precomputed model outputs for evaluation.

Evaluation Results

Summary

Metric	AIME24	AMC23	MATH500	GPQADiamond	JEEBench	MMLUPro	LiveCodeBench	CodeElo
Accuracy	18.0	52.5	74.8	42.8	40.3	28.6	25.6	6.8

AIME24

—Average Accuracy: 18.00% ± 1.35%
—Number of Runs: 10

Run	Accuracy	Questions Solved	Total Questions
1	26.67%	8	30
2	20.00%	6	30
3	13.33%	4	30
4	16.67%	5	30
5	16.67%	5	30
6	20.00%	6	30
7	16.67%	5	30
8	13.33%	4	30
9	13.33%	4	30
10	23.33%	7	30

AMC23

—Average Accuracy: 52.50% ± 1.77%
—Number of Runs: 10

Run	Accuracy	Questions Solved	Total Questions
1	52.50%	21	40
2	52.50%	21	40
3	47.50%	19	40
4	62.50%	25	40
5	52.50%	21	40
6	60.00%	24	40
7	52.50%	21	40
8	47.50%	19	40
9	55.00%	22	40
10	42.50%	17	40

MATH500

—Accuracy: 74.80% | Accuracy | Questions Solved | Total Questions | |----------|-----------------|----------------| | 74.80% | 374 | 500 |

GPQADiamond

—Average Accuracy: 42.76% ± 0.36%
—Number of Runs: 3

Run	Accuracy	Questions Solved	Total Questions
1	41.92%	83	198
2	42.93%	85	198
3	43.43%	86	198

JEEBench

—Average Accuracy: 40.29% ± 0.92%
—Number of Runs: 3

Run	Accuracy	Questions Solved	Total Questions
1	38.16%	196.5	515
2	41.99%	216.25	515
3	40.73%	209.75	515

MMLUPro

—Average Accuracy: 28.60% ± 0.00%
—Number of Runs: 1

Run	Accuracy	Questions Solved	Total Questions
1	28.60%	143	500

LiveCodeBench

—Average Accuracy: 25.57% ± 0.57%
—Number of Runs: 3

Run	Accuracy	Questions Solved	Total Questions
1	24.66%	126	511
2	26.61%	136	511
3	25.44%	130	511

CodeElo

—Average Accuracy: 6.82% ± 0.90%
—Number of Runs: 3

Run	Accuracy	Questions Solved	Total Questions
1	8.18%	32	391
2	7.16%	28	391
3	5.12%	20	391