datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
benchmark-bcplusRoadmapBench
RoadmapBench
A benchmark for evaluating AI coding agents on multi-target, long-horizon software development tasks derived from open-source project version upgrades.
Overview
RoadmapBench contains 115 tasks spanning 17 open-source repositories across 5 programming languages (Python, TypeScript, Go, Rust, C++). Each task requires an agent to implement multiple interdependent features that correspond to a real version upgrade of the target project.
Task Structure… See the full description on the dataset page: https://huggingface.co/datasets/benchmark-anon-2026/RoadmapBench.explicit-edit-benchmark
Explicit Edit Benchmark
226 deterministic exact-edit tasks, run by different agents, harnesses, models and configurations. Every observation records what the harness did and whether the resulting files matched byte for byte.
Source code and benchmark runner: GitHub — Explicit Edit Benchmark
Open the interactive Explorer to compare agents, harnesses, models, versions, reasoning modes, correctness, recovery, time, cost and tokens.
Leaderboard by model route
Score v2… See the full description on the dataset page: https://huggingface.co/datasets/alexshpunt/explicit-edit-benchmark.regx-benchmark
RegX
Cross-Domain Multi-View Point Cloud Registration Benchmark
RegX evaluates multi-view point cloud registration across scales spanning nine orders
of magnitude — nanometre-scale microscopy to kilometre-scale airborne maps — and sensors
never designed to be compared: clinical colonoscopes, RGB-D cameras, spinning and
solid-state LiDAR, terrestrial and airborne laser scanners.
Most registration benchmarks fix one sensor and one scale. RegX asks a narrower question
instead: does… See the full description on the dataset page: https://huggingface.co/datasets/YuePanEdward/regx-benchmark.cvdp-benchmark-datasetImportant please see "Files and versions" above for full list of files in the CVDP dataset.
Please see LICENSE and NOTICE for licensing information. See CHANGELOG for changes.
This is the Comprehensive Verilog Design Problems (CVDP) benchmark dataset to use with the CVDP infrastructure on GitHub.
funes-handoff-recall-benchmark
handover-vs-recall
A long investigation bloats an agent session until each new turn costs more to carry the context than to
do the work. Switching to a fresh session avoids that — but the findings have to travel somehow, and the
ways of moving them differ in cost. This benchmark measures those ways, as cost per successful task,
on tasks that genuinely require the prior investigation:
arm
channel
A branch-only
switch, carry nothing — the fresh session re-derives the… See the full description on the dataset page: https://huggingface.co/datasets/dacorvo/funes-handoff-recall-benchmark.ai-humanizer-benchmark
AI Humanizer Benchmark: AI humanizers tested against 7 AI detectors (October 2026)
AI Humanizer Benchmark measures how well AI humanizers rewrite AI-generated text so that AI detectors classify it as human-written, and how much meaning and readability the rewrite loses. In each monthly cycle, 11 AI humanizers rewrite the same 33 source texts across 7 writing categories, using each tool's default settings. Each output gets three kinds of score: 7 AI detectors (GPTZero… See the full description on the dataset page: https://huggingface.co/datasets/ai-humanizer-benchmark/ai-humanizer-benchmark.amazon-benchmark
Amazon query–bundle benchmark
Canonical, category-organized query and reference-positive data. Experiment traces should reference this repository by commit SHA, category, split, and candidate_id, rather than republishing the dataset.
Musical Instruments
Split
Examples
agent_dev
2,028
agent_hidden
1,960
Each record contains a query and 3–7 reference product IDs. These are observed reference positives, not exhaustive labels for all valid… See the full description on the dataset page: https://huggingface.co/datasets/iaouali/amazon-benchmark.benchmark-research
Benchmark Research
面向金融 Deep Research Agent 的评测数据集（仅数据）。
代码、builder、prompt 模板与评测脚本在独立代码仓库维护；本 Hub 仓库只发布标准化 ready seeds，支持按子集按需加载。
快速开始
from datasets import load_dataset
# 按题型子集加载（推荐）
a1 = load_dataset("sselaine27/benchmark-research", "a1")
c = load_dataset("sselaine27/benchmark-research", "c")
d = load_dataset("sselaine27/benchmark-research", "d")
print(a1["train"][0]["task_id"])
print(a1["train"][0]["prompt"][:200])
# 按时间层筛选
t2 = a1["train"].filter(lambda x:… See the full description on the dataset page: https://huggingface.co/datasets/sselaine27/benchmark-research.FinQAFinQA is a large-scale dataset with 2.8k financial reports for 8k Q&A pairs to study numerical reasoning with structured and unstructured evidence.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/FinQA", "queries")
documents = datasets.load_dataset("embedding-benchmark/FinQA", "corpus")
pair_labels = datasets.load_dataset("embedding-benchmark/FinQA", "default")
FinanceBenchThe FinanceBench dataset is derived from the PatronusAI/financebench-test dataset, containing only the PASS examples processed into a clean format for question-answering tasks in the financial domain. FinanceBench-rtl has been repurposed for retrieval.
Usage
import datasets
# Download the dataset
queries = datasets.load_dataset("embedding-benchmark/FinanceBench", "queries")
documents = datasets.load_dataset("embedding-benchmark/FinanceBench", "corpus")
pair_labels =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/FinanceBench.ogd4all-benchmark
OGD4All Benchmark
This is a 199-question benchmark that was used to evaluate the overall performance of OGD4All and different configurations (LLM, orchestration, ...).
OGD4All is an LLM-based prototype system enabling an easy-to-use, transparent interaction with Geospatial Open Government Data through natural language.
Each question requires GIS, SQL and/or topological operations on zero, one, or multiple datasets in GPKG or CSV formats to be answered.
Tasks
The… See the full description on the dataset page: https://huggingface.co/datasets/michael7ma/ogd4all-benchmark.hack-ignition-benchmark
hack-ignition benchmark — data, v0.1.6
Training trajectories of reinforcement-learning runs on exploitable graders, for studying and predicting when RL
comes to produce exploits. Each family is a set of GRPO runs over configurations of (start model, prompt,
training set, grader / reward structure, recipe), with one or more seeds per configuration. Every family stores
what its training logs contain — per-step exploit, task and reward rates, the item × step exploit record… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/hack-ignition-benchmark.Open-LLM-Benchmark
Open-LLM-Benchmark
Dataset Description
The Open-LLM-Leaderboard tracks the performance of various large language models (LLMs) on open-style questions to reflect their true capability. The dataset includes pre-generated model answers and evaluations using an LLM-based evaluator.
License: CC-BY 4.0
Dataset Structure
An example of model response files looks as follows:
{
"question": "What is the main function of photosynthetic cells within a plant?"… See the full description on the dataset page: https://huggingface.co/datasets/Open-Style/Open-LLM-Benchmark.HC3FinanceThe HC3 dataset comprises tens of thousands of comparison responses from both human experts and ChatGPT across various domains, including open-domain, financial, medical, legal, and psychological areas. The data collection process involved sourcing publicly available question-answering datasets and wiki texts, ensuring that the human answers were either expert-provided or high-quality user responses, thereby minimizing mislabeling and enhancing the dataset's reliability.
Usage
import datasets… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/HC3Finance.big-finance-benchmark
BigFinanceBench Public Release
arXiv | Website | GitHub | Blog post
Finance answers are only useful when another analyst can audit how they were produced. BigFinanceBench evaluates that full workflow: agents must produce a numerical answer, and their traces are graded against point-weighted rubrics for source choice, period, accounting definition, assumptions, adjustments, and calculation.
This release contains a 50-question stratified subset of the 928-item BigFinanceBench… See the full description on the dataset page: https://huggingface.co/datasets/RogoAI/big-finance-benchmark.GCDG-Benchmark
GCDG Benchmark
Dataset accompanying Minimal-Intervention Target Retrieval in Cluttered Scenes with Grasp-Conditioned Dependency Graphs.
Code and evaluation · Project page · Paper
The viewer above is a target index, not the full annotation content. The eleven archives contain the scene configurations, grasp proposals and dependency annotations. Download all shards for a split and extract them into the same directory.
GCDG dense-scene benchmark
The benchmark… See the full description on the dataset page: https://huggingface.co/datasets/WanhaoX25/GCDG-Benchmark.SWE-QA-Benchmark
SWE-QA Benchmark
A comprehensive benchmark dataset for Software Engineering Question Answering, containing 720 questions across 15 popular Python repositories.
Dataset Summary
Total Questions: 720
Repositories: 15
Format: JSONL (JSON Lines)
Fields: question, answer
Repository Coverage
Each repository contains 48 questions:
astropy
conan
django
flask
matplotlib
pylint
pytest
reflex
requests
scikit-learn
sphinx
sqlfluff
streamlink
sympy
xarray… See the full description on the dataset page: https://huggingface.co/datasets/swe-qa/SWE-QA-Benchmark.data-product-benchmark
DPDisc Dataset
Paper | Code
Dataset Description
This dataset provides a benchmark for automatic data product creation. The task is framed as follows: given a natural language data product request and a corpus of text and tables, the objective is to identify the relevant tables and text documents that should be included in the resulting data product which would useful to the given data product request. The benchmark brings together three variants: HybridQA, TAT-QA, and… See the full description on the dataset page: https://huggingface.co/datasets/ibm-research/data-product-benchmark.Act2Cap_benchmarkCollected data from GUI-Action-Narrator
last-translation-benchmark
Last Translation Benchmark
Abstract: For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases.
Standard benchmarks for machine translation evaluation are often either trivial (having few authentic mistakes) or unrealistic (overly synthetically contrived).
Furthermore, automatic translation metrics become less reliable and reward-hacked as models get stronger, and their outputs are… See the full description on the dataset page: https://huggingface.co/datasets/zouhar/last-translation-benchmark.ChatDoctor_HealthCareMagicThe ChatDoctor-HealthCareMagic-100k dataset comprises 112,000 real-world medical question-and-answer pairs, providing a substantial and diverse collection of authentic medical dialogues. There is a slight risk to this dataset since there are grammatical inconsistencies in many of the questions and answers, but this can potentially help separate strong healthcare retrieval models from weak ones.
Usage
import datasets
# Download the dataset
queries =… See the full description on the dataset page: https://huggingface.co/datasets/embedding-benchmark/ChatDoctor_HealthCareMagic.k-verify-benchmark-v1
Part of the SZL Holdings governed estate — claims are designed to carry checkable receipts. Verification proves integrity & origin, never accuracy or performance.
K-Verify Benchmark v1
Author: Yachay / SZL Holdings · Version: 1.0.0 · Items: 100
K-Verify measures whether an AI's claimed factual answer is verifiable via a
receipt chain — not just whether it is correct. It is the first benchmark we
know of that scores provenance and honest refusal as… See the full description on the dataset page: https://huggingface.co/datasets/SZLHOLDINGS/k-verify-benchmark-v1.MSU-Benchmark
MSU-Bench: Towards Speaker-Centric Understanding in Conversational Multi-Speaker Scenarios
Interspeech 2026 · ASLP@NPU (Northwestern Polytechnical University), in collaboration with Li Auto.
Zhaokai Sun*, Shuai Wang*, Zhennan Lin*, Chengyou Wang, Dehui Gao, Yuang Cao, Chunjiang He, Pan Zhou, Lei Xie**
Audio, Speech and Language Processing Group (ASLP@NPU), School of Software, Northwestern Polytechnical University, China
School of Intelligent Science and Technology, Nanjing… See the full description on the dataset page: https://huggingface.co/datasets/ASLP-lab/MSU-Benchmark.mcp-agent-trajectory-benchmark
MCP Agent Trajectory Benchmark
A benchmark dataset of 49 MCP (Model Context Protocol) agent trajectories (38 single-pass + 11 multi-conv) with complete tool-use traces in the ATIF v1.2 (Agent Trajectory Interchange Format) format. Each agent operates in a distinct business domain with custom tools, realistic user conversations, and full execution traces.
Designed for training and evaluating tool-use / function-calling capabilities of LLMs.
Overview
Item
Details… See the full description on the dataset page: https://huggingface.co/datasets/obaydata/mcp-agent-trajectory-benchmark.map-spatial-benchmark
Map-based Spatial Reasoning Benchmark
A multi-view map-based spatial reasoning benchmark. Each row is one multiple-choice
question instance over a registered map image; models must answer with a single option
letter. Four tasks (T1–T4), four base-map views, and controlled evidence conditions
(direct / query / oracle) and world perturbations (transform / world layers) allow
fine-grained analysis of spatial reasoning robustness.
Task overview
Task
Question… See the full description on the dataset page: https://huggingface.co/datasets/mapspatial/map-spatial-benchmark.scene-mem-benchmark
scene-mem-benchmark
A benchmark for scene memory in embodied agents: an agent watches a mobile manipulator work
in a house for several minutes, then is asked to retrieve an object it has to remember — one
that was moved, dropped, or merely seen along the way — or (resume) to go back and finish the
job it was interrupted in, remembering how far it had got — or (routine) to put a new object away
where this household keeps that kind of thing, a rule it was never told and can only… See the full description on the dataset page: https://huggingface.co/datasets/Keh0t0/scene-mem-benchmark.omr_benchmark
Muse OMR Benchmark
What this is
A small, clean benchmark dataset for OMR (Optical Music Recognition — recognizing music notation from images/PDFs).
It contains 1077 pairs:
a symbolic music score (the “ground truth”, see dataset fields below)
a corresponding PDF rendering with data augmentation applied
All underlying works are Public Domain.
Why it exists
OMR is often evaluated on private or inconsistent datasets. This dataset aims to provide the community… See the full description on the dataset page: https://huggingface.co/datasets/musegroup/omr_benchmark.invoice-extraction-benchmark
Invoice Extraction Benchmark v1
A synthetic test set for invoice data extraction (invoice OCR, intelligent document
processing, accounts-payable capture): 181 documents with answer keys and a scorer.
Run any invoice reader over the documents, write its output as one JSON file, and score it
field by field.
Source, scorer and generator: https://github.com/DrewKraken/invoice-extraction-benchmark
(this dataset is a mirror of corpus/v1/ there; the GitHub repository is canonical).
Who… See the full description on the dataset page: https://huggingface.co/datasets/drew-ipp/invoice-extraction-benchmark.FiVE-Fine-Grained-Video-Editing-Benchmark
FiVE-Bench
FiVE-Bench: A Fine-Grained Video Editing Benchmark for Evaluating Diffusion and Rectified Flow Models
Minghan Li1*, Chenxi Xie2*, Yichen Wu13, Lei Zhang2, Mengyu Wang1†
1Harvard University 2The Hong Kong Polytechnic University 3City University of Hong Kong
*Equal contribution †Corresponding Author
💜 Leaderboard (coming soon) |
💻 GitHub |
🤗 Hugging Face
📝 Project Page |
📰 Paper |
🎥 Video Demo
FiVE is a benchmark comprising 100 videos for… See the full description on the dataset page: https://huggingface.co/datasets/LIMinghan/FiVE-Fine-Grained-Video-Editing-Benchmark.