mlfoundations-dev/a1_code_magicoder_1744693377_eval_1331
mlfoundations-dev/a1_code_magicoder_1744693377_eval_1331 Precomputed model outputs for evaluation. Evaluation Results Summary Metric AIME24 AMC23 MATH500 GPQADiamond JEEBench MMLUPro LiveCodeBench CodeElo Accuracy 16.3 56.2 72.0 34.0 39.1 29.8 34.5 9.2 AIME24 Average Accuracy: 16.33% ± 1.45% Number of Runs: 10 Run Accuracy Questions Solved Total Questions 1 20.00% 6 30 2 13.33% 4 30 3 16.67% 5 30 4… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations-dev/a1_code_magicoder_1744693377_eval_1331.
09
mlfoundations-dev/a1codemagicoder1744693377eval_1331
Precomputed model outputs for evaluation.
Evaluation Results
Summary
AIME24
- Average Accuracy: 16.33% ± 1.45%
- Number of Runs: 10
AMC23
- Average Accuracy: 56.25% ± 1.67%
- Number of Runs: 10
MATH500
- Accuracy: 72.00% | Accuracy | Questions Solved | Total Questions | |----------|-----------------|----------------| | 72.00% | 360 | 500 |
GPQADiamond
- Average Accuracy: 34.01% ± 1.22%
- Number of Runs: 3
JEEBench
- Average Accuracy: 39.09% ± 0.12%
- Number of Runs: 3
MMLUPro
- Average Accuracy: 29.80% ± 0.00%
- Number of Runs: 1
LiveCodeBench
- Average Accuracy: 34.51% ± 0.35%
- Number of Runs: 3
CodeElo
- Average Accuracy: 9.21% ± 0.53%
- Number of Runs: 3