datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
openswe-tasks-patched-v5static-analysis-evalA dataset of 76 Python programs taken from real Python open source projects (top 100 on GitHub),
where each program is a file that has exactly 1 vulnerability as detected by a particular static analyzer (Semgrep), used in the paper Patched MOA: optimizing inference for diverse software development tasks.
OpenAI used the synth-vuln-fixes and fine-tuned
a new version of gpt-4o is now the SOTA on this benchmark. More details and code is available from their repo.
More details on the benchmark… See the full description on the dataset page: https://huggingface.co/datasets/patched-codes/static-analysis-eval.vulnerability-cwe-patch
Description
This dataset, CIRCL/vulnerability-cwe-patch, provides structured, real-world vulnerabilities enriched with CWE identifiers and corresponding patches from platforms like GitHub and GitLab. It is designed to support the development of tools for vulnerability classification, triage, and automated remediation. Each entry includes metadata such as CVE/GHSA ID, a description, CWE categorization, and links to verified patch commits with associated diff content and commit… See the full description on the dataset page: https://huggingface.co/datasets/CIRCL/vulnerability-cwe-patch.patchlet-embed-preprocessedgenerate-readme-eval
Generate README Eval
The generate-readme-eval is a dataset (train split) and benchmark (test split) to evaluate the effectiveness of LLMs
when summarizing entire GitHub repos in form of a README.md file. The datset is curated from top 400 real Python repositories
from GitHub with at least 1000 stars and 100 forks. The script used to generate the dataset can be found here.
For the dataset we restrict ourselves to GH repositories that are less than 100k tokens in size to allow us to… See the full description on the dataset page: https://huggingface.co/datasets/patched-codes/generate-readme-eval.cc12m_openai_clip-vit-base-patch32_image_image_retrieval_pairs_2022-09-13patchaudit-artifact
PatchAudit Artifact
PatchAudit audits security patches. You give it a CVE's initial fix — commit C1 — and a later commit Ci,
and it tells you whether Ci is a future commit: a later commit that had to keep fixing the same problem
because C1 was incomplete (it left the vulnerability reachable) or incorrect (its own change
introduced a new defect). When such a future commit exists, C1 was a bad patch. When even the latest fix
still leaves the hole open, the bug is a lingering… See the full description on the dataset page: https://huggingface.co/datasets/zhcharyzhang/patchaudit-artifact.liveswebench-patchesvessel-detection-labeled-patches
Vessel Detection Labeled Patches
Validated/confirmed satellite image patches exported from the military-boat-detection review workflow.
Contents
images/: patch images.
metadata.csv: one row per patch, compatible with Hugging Face image-folder metadata.
metadata.jsonl: rich patch metadata with nested objects.
annotations.csv: one row per vessel annotation.
annotations.jsonl: JSONL version of the object annotations.
labels/: YOLO-format labels. Hard negatives have empty… See the full description on the dataset page: https://huggingface.co/datasets/DefendIntelligence/vessel-detection-labeled-patches.PatchEval
👋 Overview
PatchEval-Verified is a benchmark for evaluating LLMs and coding agents on automated repair of real-world vulnerabilities. It contains 230 CVE cases with Docker-based evaluation environments, covering vulnerabilities reported between 2015 and 2025 across Go, JavaScript, and Python.
PatchEval-Verified updates the evaluation environments from the original PatchEval release. In the original benchmark, some PoC tests were adapted from project regression tests and were… See the full description on the dataset page: https://huggingface.co/datasets/ByteDance/PatchEval.github-patchesLLaVa-CC3M-PostTraining-clip-vit-base-patch16SynthMat_Patches_DScybersec-chatml-vuln-patch-v1
Cybersecurity ChatML SFT Dataset (Detection + Patch + Multitask)
This dataset contains ChatML records for 2 security tasks:
Vulnerability detection (is_vulnerable, cwe, severity JSON output)
Secure patch generation (assistant returns patched code only)
Files
chatml_detection_train.jsonl
chatml_detection_val.jsonl
chatml_patch_train.jsonl
chatml_patch_val.jsonl
chatml_multitask_train.jsonl
chatml_multitask_val.jsonl
chatml_build_manifest.json… See the full description on the dataset page: https://huggingface.co/datasets/Kushalkhemka/cybersec-chatml-vuln-patch-v1.rl__24GPU_base__swe_rebench_patched_oracle__r2egym-nl2bash-stacka3-rl-DCAgent_r2egym-patched-full-oracleswe_rebench_patchedcc12m_openai_clip-vit-base-patch32_image_image_retrieval_pairs_2022-09-15LLaVa-Instruct-150K-clip-vit-base-patch32PatchBench
PatchBench
PatchBench is a benchmark for evaluating AI agents on realistic vulnerability patching tasks: 213 tasks drawn from 32 popular GitHub C/C++ projects. It selects vulnerabilities whose ground-truth fixes lie outside the crash stack, and uses vulnerability transplant plus code mutation to mitigate surface-level fixes and patch memorization.
This repository holds the task metadata, one row per task to identify the project, the exact repository state, the crash, and the… See the full description on the dataset page: https://huggingface.co/datasets/ai-sec-lab/PatchBench.Patchnoisseur
Patchnoisseur
A connoisseur's cellar of CVEs: every NVD CVE joined to its fixing-commit
diff (when one could be found), its NVD description, and its
associated CWE(s) (id, name, short description) — served as a single
Parquet dataset.
351 884 CVEs · 25 015 with a real git diff attached · CVE-1999 → CVE-2026
· ~744 MB on disk (zstd-compressed Parquet, sharded ~300 MB each).
What's in it
One row per CVE in the NVD feed. CVEs without a retrievable patch are… See the full description on the dataset page: https://huggingface.co/datasets/michoo42/Patchnoisseur.R2E-TestgenAgent-Patchesrl__24GPU_shaped__swe_rebench_patched_oracle__r2egym-nl2bash-stackKimi-2.5-swe_rebench_patched-maxeps-32kgithub-patches-genesysimport re
import json
from datasets import load_dataset
PROMPT_TEMPLATE = """\
We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
{issue}
--- END ISSUE ---
Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
{file_context}
--- END FILES ---
Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format… See the full description on the dataset page: https://huggingface.co/datasets/rasdani/github-patches-genesys.swe_gym_annotate_with_patchpatch_region_128mtg-scryfall-cropped-art-embeddings-siglip-so400m-patch14-384cc12m_openai-clip-vit-patch32mscoco_train_2014_openai_clip-vit-base-patch32_image_image_retrieval_pairs_2022-09-15