datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
tasksThis dataset is for storing assets for https://huggingface.co/tasks and https://github.com/huggingface/huggingface.js/tree/main/packages/tasks
HF_ML_Tasksmith
HF ML Tasksmith
Fifty PR-derived Harbor tasks from Accelerate, Diffusers, PEFT, Transformers and TRL, including CPU and GPU tasks.
Contains 50 Harbor tasks generated with the owned
tasksmith recipe in Repo2RLEnv.
Browse the complete task bundles in Harbor Visualiser or
open the task folders. Each folder is a runnable Harbor task:
tasks/<task_id>/
├── task.toml # Harbor configuration and provenance
├── instruction.md # Task shown to the coding agent… See the full description on the dataset page: https://huggingface.co/datasets/FineEnvs/HF_ML_Tasksmith.mmluMMLU (hendrycks_test on huggingface) without auxiliary train. It is much lighter (7MB vs 162MB) and faster than the original implementation, in which auxiliary train is loaded (+ duplicated!) by default for all the configs in the original version, making it quite heavy.
We use this version in tasksource.
Reference to original dataset:
Measuring Massive Multitask Language Understanding - https://github.com/hendrycks/test
@article{hendryckstest2021,
title={Measuring Massive Multitask Language… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/mmlu.OpenMLE-Tasks
OpenMLE Tasks
📄 Paper
•
🌐 Project
•
💻 Code
•
🤗 Models
•
📚 SFT Traces
OpenMLE Tasks provides machine-learning Task environments. The public SFT trajectories are released separately in OpenMLE-SFT-Traces. These resources accompany the paper Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering and the OpenRSI code release.
Release form
What is included… See the full description on the dataset page: https://huggingface.co/datasets/FrontisAI/OpenMLE-Tasks.bigbenchBIG-Bench but it doesn't require the hellish dependencies (tensorflow, pypi-bigbench, protobuf) of the official version.
dataset = load_dataset("tasksource/bigbench",'movie_recommendation')
Code to reproduce:
https://colab.research.google.com/drive/1MKdLdF7oqrSQCeavAcsEnPdI85kD0LzU?usp=sharing
Datasets are capped to 50k examples to keep things light.
I also removed the default split when train was available also to save space, as default=train+val.
@article{srivastava2022beyond… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/bigbench.nucleotide_transformer_downstream_tasks
Dataset Card for Dataset Name
The nucleotide_transformer_downstream_tasks dataset features the 18 downstream tasks presented in the Nucleotide Transformer paper. They consist of both binary and multi-class classification tasks that aim at providing a consistent genomics benchmark.
⚠️We note that we have revised and improved our benchmark during the peer-review process. The datasets featured in this repository are used up to this release. We highly encourage to move to the new… See the full description on the dataset page: https://huggingface.co/datasets/InstaDeepAI/nucleotide_transformer_downstream_tasks.reclorhttps://whyu.me/reclor/
@inproceedings{yu2020reclor,
author = {Yu, Weihao and Jiang, Zihang and Dong, Yanfei and Feng, Jiashi},
title = {ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning},
booktitle = {International Conference on Learning Representations (ICLR)},
month = {April},
year = {2020}
}
proofwriter
Dataset Card for "proofwriter"
More Information needed
frontier-ml-tasks
Frontier MLE tasks
Data for the tasks in induction-labs/frontier-ml.
Each task has one folder:
<slug>/public/ given to the coding agent verbatim, at /task/public
<slug>/private/ held-out data for the trusted verifier, at /tests/private
<slug>/artifacts/ optional starting files for the agent, at /artifacts
<slug>/manifest.json optional provenance summary
Tasks pin this repository by commit in tasks/<slug>/task.yaml. Starting model weights are
downloaded from their… See the full description on the dataset page: https://huggingface.co/datasets/inductionlabs/frontier-ml-tasks.robocasa_22_tasksNemotron-Terminal-Synthetic-Tasks
Terminal-Corpus: Task Structure Specification
This repository contains the skill-based synthetic tasks within the Terminal-Corpus. These tasks are designed to evaluate and train autonomous agents in realistic Linux terminal environments.
🏗️ Task Anatomy
Each task is contained within a dedicated directory and follows a strict four-component architecture:
1. Instruction (instruction.md)
Purpose: Provides the natural language description of the objective.… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-Terminal-Synthetic-Tasks.sandboxes-tasksbabi_nli
bAbi_nli
bAbI tasks recasted as natural language inference.
https://github.com/facebookarchive/bAbI-tasks
tasksource recasting code:
https://colab.research.google.com/drive/1J_RqDSw9iPxJSBvCJu-VRbjXnrEjKVvr?usp=sharing
@article{weston2015towards,
title={Towards ai-complete question answering: A set of prerequisite toy tasks},
author={Weston, Jason and Bordes, Antoine and Chopra, Sumit and Rush, Alexander M and Van Merri{\"e}nboer, Bart and Joulin, Armand and Mikolov, Tomas}… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/babi_nli.tasksource-jev-typed-decisions
tasksource-jev-typed-decisions
2.5 million typed decisions (choices, ratings and probabilities) from 670 sources.
Why use it
Real supervision. Labels, ratings, and annotator votes come from
established datasets, not a teacher model. Every row names its source.
Breadth. Over 300 dataset families: NLI and reasoning, QA and
commonsense, sentiment, intent and topic, toxicity and safety, preference
pairs, fact checking, entity tagging, and dozens of languages. GLUE… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/tasksource-jev-typed-decisions.strategy-qanucleotide_transformer_downstream_tasks_revised
Dataset Card for Dataset Name
The nucleotide_transformer_downstream_tasks dataset features the 18 downstream tasks presented in the Nucleotide Transformer paper. They consist of both binary and multi-class classification tasks that aim at providing a consistent genomics benchmark.
We note that this is an updated version of this benchmark after the paper has been through peer-review. We highly encourage to move to this version in detriment of the older version.Keypoints about the… See the full description on the dataset page: https://huggingface.co/datasets/InstaDeepAI/nucleotide_transformer_downstream_tasks_revised.osworld_v2_tasks
OSWorld V2 Task Classes
This gated dataset contains the official root-level task_*.py Python task classes for OSWorld V2.
The public GitHub repository keeps the task loader, helper utilities, and documentation. The task implementations are gated to reduce benchmark leakage and to help prevent evaluated agents from finding task answers, setup logic, or evaluator details online while executing a task.
Download from the public repository root with:
uvx --from huggingface_hub hf… See the full description on the dataset page: https://huggingface.co/datasets/xlangai/osworld_v2_tasks.webgym_tasks
WebGym Tasks Dataset
Dataset Description
This dataset contains web navigation tasks for training and evaluating autonomous web agents. Each task consists of a natural language instruction that describes an action to be performed on a specific website, along with evaluation criteria and metadata.
Dataset Summary
Total Training Tasks: 292,092
Total Test Tasks: 1,167
Domains: Multiple domains including Lifestyle & Leisure, Sports & Fitness, and more
Source… See the full description on the dataset page: https://huggingface.co/datasets/microsoft/webgym_tasks.dev_set_bespoke_tasksCountdown-Tasks-3to4tasksdataesci
Dataset Card for "esci"
ESCI product search dataset
https://github.com/amazon-science/esci-data/
Preprocessings:
-joined the two relevant files
-product_text aggregate all product text
-mapped esci_label to full name
@article{reddy2022shopping,
title={Shopping Queries Dataset: A Large-Scale {ESCI} Benchmark for Improving Product Search},
author={Chandan K. Reddy and Lluís Màrquez and Fran Valero and Nikhil Rao and Hugo Zaragoza and Sambaran Bandyopadhyay and Arnab Biswas and Anlu… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/esci.kilt_tasks
Dataset Card for KILT
Dataset Summary
KILT has been built from 11 datasets representing 5 types of tasks:
Fact-checking
Entity linking
Slot filling
Open domain QA
Dialog generation
All these datasets have been grounded in a single pre-processed Wikipedia dump, allowing for fairer and more consistent evaluation as well as enabling new task setups such as multitask and transfer learning with minimal effort. KILT also provides tools to analyze and understand the… See the full description on the dataset page: https://huggingface.co/datasets/facebook/kilt_tasks.lsat-lr
Dataset Card for "lsat-lr"
More Information needed
Multi-SWE-smith-taskslsat-rc
Dataset Card for "lsat-rc"
More Information needed
rlbench-18-tasks
RLBench 18 Tasks Dataset
Overview
This repository provides the RLBench dataset for 18 tasks, originally hosted by PerAct in Google Drive. Since downloading large files from Google Drive via terminal can be problematic due to various limits, we have mirrored the dataset on Hugging Face for easier access. To know more about the details of this dataset, please refer to PerAct.
Dataset Structure
The dataset is organized into three splits:
data/
├── train/ #… See the full description on the dataset page: https://huggingface.co/datasets/hqfang/rlbench-18-tasks.foliohttps://github.com/Yale-LILY/FOLIO
@article{han2022folio,
title={FOLIO: Natural Language Reasoning with First-Order Logic},
author = {Han, Simeng and Schoelkopf, Hailey and Zhao, Yilun and Qi, Zhenting and Riddell, Martin and Benson, Luke and Sun, Lucy and Zubova, Ekaterina and Qiao, Yujie and Burtell, Matthew and Peng, David and Fan, Jonathan and Liu, Yixin and Wong, Brian and Sailor, Malcolm and Ni, Ansong and Nan, Linyong and Kasai, Jungo and Yu, Tao and Zhang, Rui and Joty, Shafiq and… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/folio.lawma-tasks
Lawma legal classification tasks
This repository contains the legal classification tasks from Lawma.
These tasks were derived from the Supreme Court and Songer Court of Appeals databases.
See the project's GitHub repository for more details.
Please cite as:
@misc{dominguezolmedo2024lawmapowerspecializationlegal,
title={Lawma: The Power of Specialization for Legal Tasks},
author={Ricardo Dominguez-Olmedo and Vedant Nanda and Rediet Abebe and Stefan Bechtold and Christoph… See the full description on the dataset page: https://huggingface.co/datasets/ricdomolm/lawma-tasks.tbench-tasks_migrated