GPUMODE/kernelbot-data

1---2configs:3- config_name: amd_submissions4  data_files: "submissions.parquet"5- config_name: amd_successful_submissions6  data_files: "successful_submissions.parquet"7- config_name: amd_1_1m_competition8  data_files: "amd_1_1m_competition_submissions.parquet"9- config_name: helion_b200_nebius10  data_files: "helion_b200_nebius_submissions.parquet"11- config_name: trimul_submissions12  data_files: "trimul_submissions.parquet"13- config_name: nvidia_nvfp4_submissions14  data_files: "nvidia_nvfp4_submissions.parquet"15- config_name: pmpp_v2_submissions16  data_files: "pmpp_v2_submissions.parquet"17- config_name: linalg_submissions18  data_files: "linalg_submissions.parquet"19- config_name: leaderboards20  data_files: "leaderboards.parquet"21tags:22- code23license: other24---25 26# KernelBot Competition Data27 28This dataset contains GPU kernel submissions from the KernelBot competition platform. Submissions are optimized GPU kernels written for specific hardware targets.29 30## Data Files31 32### AMD MI300 Submissions33| File | Description |34|------|-------------|35| `submissions.parquet` | All AMD competition submissions |36| `successful_submissions.parquet` | AMD submissions that passed correctness tests |37| `deduplicated_submissions.parquet` | AMD submissions deduplicated by (user, code) |38| `deduplicated_successful_submissions.parquet` | Deduplicated passing AMD submissions |39 40**AMD Problems:** fp8-gemm, moe (mixture of experts), mla-decode, all2all, gemm+reducescatter, allgather+gemm, mxfp4-mm, moe-mxfp4, mixed-mla41 42### AMD 1.1M Competition43| File | Size | Description |44|------|------|-------------|45| `amd_1_1m_competition_submissions.parquet` | ~699 MB | Deduplicated submissions with code for `amd-mxfp4-mm` (763), `amd-moe-mxfp4` (764), and `amd-mixed-mla` (765) |46 47### Trimul48| File | Size | Description |49|------|------|-------------|50| `trimul_submissions.parquet` | ~120 MB | Deduplicated submissions with code for `trimul` (leaderboard 496) |51 52`trimul` is a separate mixed-GPU problem and is not grouped with the AMD competition exports.53 54### Helion B200_Nebius55| File | Size | Description |56|------|------|-------------|57| `helion_b200_nebius_submissions.parquet` | ~4 MB | Deduplicated submissions with code for `causal_conv1d` (766), `fp8_quant` (767), `gated_deltanet_chunk_fwd_h` (768), `gated_deltanet_chunk_fwd_o` (769), and `gated_deltanet_recompute_w_u` (770) |58 59**Measurement note:** these problems were run on `B200_Nebius`, and the measurements for this problem set are brittle. Treat leaderboard scores from this export with extra caution.60 61### NVIDIA Blackwell NVFP4 Submissions62| File | Size | Description |63|------|------|-------------|64| `nvidia_nvfp4_submissions.parquet` | ~1.4 GB | NVFP4 submissions deduplicated by (user, code), with full code content |65 66 67**NVFP4 Problems:** gemv (leaderboard 595), gemm (597), dual_gemm (598), modal_dual_gemm (697), group_gemm (730)68 69**Note on Dual GEMM:** There are two variants of the dual_gemm problem. Midway through the competition, on-prem hardware measurements became unreliable, so a second leaderboard was created on Modal infrastructure. The Modal measurements (leaderboard 697, `modal_nvfp4_dual_gemm`) are more trustworthy.70 71**Note:** Scores are execution time in seconds. **Lower is better.**72 73### PMPP v2 Submissions74| File | Size | Description |75|------|------|-------------|76| `pmpp_v2_submissions.parquet` | ~28 MB | All PMPP v2 submissions with full code content |77 78**PMPP v2 Problems:** conv2d_v2 (537), grayscale_v2 (538), histogram_v2 (539), matmul_v2 (540), prefixsum_v2 (541), sort_v2 (542), vectoradd_v2 (543), vectorsum_v2 (544)79 80### Linear Algebra Submissions81| File | Size | Description |82|------|------|-------------|83| `linalg_submissions.parquet` | ~4.3 GB | Deduplicated submissions with code for `qr_v2` (leaderboard 774), `eigh` (leaderboard 775), and `cholesky` (leaderboard 776) |84 85The earlier `qr` leaderboard (773) is not included.86 87## Helper Scripts88 89- `analyze_submissions.py` - Python functions for analyzing submissions90- `skills.md` - Documentation for data processing workflows91 92### Quick Start93 94```python95from analyze_submissions import load_submissions, top_contestants, author_progression96 97# Load NVIDIA NVFP4 data98df = load_submissions()99 100# Get top 20 for a problem101leaders = top_contestants(df, problem_name='nvfp4_gemm', n=20)102 103# See a user's progression over time104progression = author_progression(df, user_name='username', problem_name='nvfp4_gemm')105```106 107## Learn More108 109- Competition platform: [gpumode.com](https://gpumode.com)110- Reference kernels and problem specs: [github.com/gpu-mode/reference-kernels](https://github.com/gpu-mode/reference-kernels)111 112## License113 114This dataset is licensed under the [June 9 Researcher Reciprocity License](LICENSE).115 116You are free to use, share, analyze, transform, and redistribute the material for research, education, benchmarking, publication, commercial analysis, and other lawful purposes, provided you give appropriate credit.117 118This license adapts the Open RAIL-D structure and adds one specific use restriction: training, fine-tuning, distillation, synthetic-data generation for training, embedding for training, or otherwise using this dataset to improve an AI model or AI service requires Researcher Reciprocity.119 120> If you train on it, you let us generate.121 122Covered AI model and service providers may not use this dataset while imposing terms that prevent GPU Mode, dataset contributors, or authorized researchers from generating outputs, evaluating models, benchmarking, publishing research, or exploring their own research ideas on materially equal terms to ordinary users.123 124**Attribution:** Please cite GPU Mode and link to this dataset. For academic papers, use the citation below.125 126## Citation127 128If you use this dataset in your work, please cite:129 130```bibtex131@inproceedings{132  kernelbot2025,133  title={KernelBot: A Competition Platform for Writing Heterogeneous {GPU} Code},134  author={Alex L Zhang and Matej Sirovatka and Erik Schultheis and Benjamin Horowitz and Mark Saroufim},135  booktitle={Championing Open-source DEvelopment in ML Workshop @ ICML25},136  year={2025},137  url={https://openreview.net/forum?id=bq9U4dmuyJ}138}139```140