kcxain/KernelZero-CUDA-SFT-7B

KernelZero-CUDA-SFT-7B

KernelZero-CUDA-SFT-7B is the supervised CUDA kernel generation checkpoint used to initialize the CUDA Coder in KernelZero. It is based on Qwen2.5-Coder-7B-Instruct and trained on 42,454 CUDA code-generation examples.

Paper

KernelZero: Continual Co-Evolution of LLMs for Kernel Generation

KernelBench results

The paper reports the following results with 10 sampled candidates per problem.

Intended use

This checkpoint generates CUDA extensions from PyTorch modules. Use the KernelZero CUDA prompt and validation environment for reproducible evaluation.

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "kcxain/KernelZero-CUDA-SFT-7B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

Checkpoint provenance

This release contains the merged Hugging Face inference weights from SFT checkpoint 3606. Optimizer states, scheduler state, RNG state, and distributed training artifacts are excluded.