datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Multitask-National-Speech-Corpus-v1Multitask-National-Speech-Corpus (MNSC v1) is derived from IMDA's NSC Corpus.
MNSC is a multitask speech understanding dataset derived and further annotated from IMDA NSC Corpus. It focuses on the knowledge of Singapore's local accent, localised terms, and code-switching.
ASR: Automatic Speech Recognition
SQA: Speech Question Answering
SDS: Spoken Dialogue Summarization
PQA: Paralinguistic Question Answering
from datasets import load_dataset
data =… See the full description on the dataset page: https://huggingface.co/datasets/MERaLiON/Multitask-National-Speech-Corpus-v1.multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR
PreFLMR M2KR Dataset Card
Dataset details
Dataset type:
M2KR is a benchmark dataset for multimodal knowledge retrieval. It contains a collection of tasks and datasets for training and evaluating multimodal knowledge retrieval models.
We pre-process the datasets into a uniform format and write several task-specific prompting instructions for each dataset. The details of the instruction can be found in the paper. The M2KR benchmark contains three types of tasks:… See the full description on the dataset page: https://huggingface.co/datasets/BByrneLab/multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR.Multitask-National-Speech-Corpus-v1-extendmulti-task-tcc-robosuite
XIRL
Overview
Setup
Datasets
Code Navigation
Experiments: Reproducing Paper Results
Extending XIRL
Acknowledgments
Overview
Code release for our CoRL 2021 conference paper:
XIRL: Cross-embodiment Inverse Reinforcement Learning
Kevin Zakka1,3, Andy Zeng1, Pete Florence1, Jonathan Tompson1, Jeannette Bohg2, and Debidatta Dwibedi1
Conference on Robot Learning (CoRL) 2021
1Robotics at Google,
2Stanford… See the full description on the dataset page: https://huggingface.co/datasets/Renton-Ren/multi-task-tcc-robosuite.so101-ai20k-multitaskThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/hungho77/so101-ai20k-multitask.multitask_german_examples_32kp2-etf-multitask-gp-resultsso101-multitask-calib
SO-101 multitask — calibration pool (LeRobot v2.1)
The observation pool that post-training quantization of a GR00T N1.7 SO-101
policy is calibrated on, in the LeRobot v2.1 layout. Same recordings as
hungho77/so101-multitask,
which is stored in the v3.0 layout; GR00T's data loader reads v2.1 only, so this
is the copy a quantization or evaluation run actually opens.
143 episodes · 67,496 frames · 3 tasks · 30 fps · single SO-101 arm ·
two 480×640 cameras (top, wrist) · 6-D state… See the full description on the dataset page: https://huggingface.co/datasets/hungho77/so101-multitask-calib.multitask_5blocks_v3_867ep_trimmed_mergedmultitask_5blocks_v3_865ep_trimmed_mergedThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/eslab1234/multitask_5blocks_v3_865ep_trimmed_merged.multitask_vqa_benchmarkThis dataset is a part of .
🍈 MMT-47: Multimodal Multi-Task Benchmark
47 Tasks · 7 Categories · 3 Modalities (Image, Video, Text)
Cite our ICML-2026 paper for this dataset
@article{kowsher2026lime,
title={LiME: Lightweight Mixture of Experts for Efficient Multimodal Multi-task Learning},
author={Kowsher, Md and Mansoor, Haris and Prottasha, Nusrat Jahan and Garibay, Ozlem and Zhu, Victor and Ji, Zhengping and Chen, Chen},
journal={arXiv preprint… See the full description on the dataset page: https://huggingface.co/datasets/Kowsher/multitask_vqa_benchmark.busybox_multitask
Single-arm BusyBox multitask dataset
LeRobot v3 dataset containing 66 episodes and
12141 frames at 20 FPS across 27 single-arm
BusyBox tasks. The green-button source contributes only its first three
episodes; all other listed sources contribute every episode.
Visualization
Open episode 0 in the LeRobot Dataset Visualizer.
The original unique language instruction from each source is preserved.
RoboCandyWrapper busybox_state and robot_telemetry plugin data is… See the full description on the dataset page: https://huggingface.co/datasets/AutoRLBench/busybox_multitask.multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR_CN
PreFLMR M2KR Dataset Card
Dataset details
Dataset type:
M2KR is a benchmark dataset for multimodal knowledge retrieval. It contains a collection of tasks and datasets for training and evaluating multimodal knowledge retrieval models.
We pre-process the datasets into a uniform format and write several task-specific prompting instructions for each dataset. The details of the instruction can be found in the paper. The M2KR benchmark contains three types of tasks:… See the full description on the dataset page: https://huggingface.co/datasets/BByrneLab/multi_task_multi_modal_knowledge_retrieval_benchmark_M2KR_CN.VQA-MultiTaskur5e_multitask_000This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": null,
"total_episodes": 270,
"total_frames": 106900,
"total_tasks": 11,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:270"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/Perseus101/ur5e_multitask_000.multitask_5blocks_v2_530epThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/eslab1234/multitask_5blocks_v2_530ep.multitask_5blocks_v3_704ep_hil_r1_mergedThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/eslab1234/multitask_5blocks_v3_704ep_hil_r1_merged.so101_multitask_oldsim_pick_place_stack_onehotThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"observation.state": {
"dtype": "float32",
"shape": [
6
],
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos"… See the full description on the dataset page: https://huggingface.co/datasets/Chaenn/so101_multitask_oldsim_pick_place_stack_onehot.multi_task_multi_modal_knowledge_retrieval_benchmark_M2KRso101_multitask_newsim_place_stack_onehotThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"observation.state": {
"dtype": "float32",
"shape": [
6
],
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos"… See the full description on the dataset page: https://huggingface.co/datasets/Chaenn/so101_multitask_newsim_place_stack_onehot.busybox_multitask
Single-arm BusyBox dataset
LeRobot v3 dataset containing 132 episodes and
25734 frames at 20 FPS across
27 BusyBox task(s), merged from 4 source
dataset(s). This revision adds 3 cell-3 source dataset(s) on top of armnet/busybox_multitask. Green-button cell-3 contributes only its first three episodes. Cell-3 language instructions are normalized to the original multitask strings.
Visualization
Open episode 0 in the LeRobot Dataset Visualizer.
The original unique… See the full description on the dataset page: https://huggingface.co/datasets/armnet/busybox_multitask.so101-multitaskThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 143,
"total_frames": 67496,
"total_tasks": 3,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:143"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/hungho77/so101-multitask.extract_multitask_1_DPAThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos"
],
"shape": [
4
]
},
"observation.state": {… See the full description on the dataset page: https://huggingface.co/datasets/liam-martiiin/extract_multitask_1_DPA.smolvla_multitask_hil_285k_v1_220ep_trimmedThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/eslab1234/smolvla_multitask_hil_285k_v1_220ep_trimmed.touchdown_multitask_1_DPAThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos"
],
"shape": [
4
]
},
"observation.state": {… See the full description on the dataset page: https://huggingface.co/datasets/liam-martiiin/touchdown_multitask_1_DPA.multi-task-dataset
Multi-Task Dataset
Description
A large-scale multi-task dataset designed for training and evaluating AI models across reasoning, mathematics, code, research, verification, data analysis, and general problem solving.
Content
100,000,001 examples
20+ task families
English + French
Train / Validation / Test splits
Structured reasoning and verification signals
Multiple difficulty levels
OOD and generalization-oriented examples
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/Lelonthecodeur/multi-task-dataset.multitask_5blocks_v3_684ep_trimmed_mergedThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/eslab1234/multitask_5blocks_v3_684ep_trimmed_merged.multitask-tabular-datasetsThis is a port of the Multi-Label Classification Dataset Repository (link).
We convert the datasets from there to simple csvs, resulting in 32 csvs (many of their mulan files fail to parse into python for us)
The targets in each csv are labeled with the suffix __target
Dataset
Domain
m
d
q
Card
Dens
Div
avgIR
rDep
m×q×d
0
3s-bbc1000
Text
352
1000
6
1.125
0.188
0.234
1.718
0.733
2.11e+06
1
3s-guardian1000
Text
302
1000
6
1.126
0.188
0.219
1.773
0.667
1.81e+06
2
3s-inter3000… See the full description on the dataset page: https://huggingface.co/datasets/imodels/multitask-tabular-datasets.ur10e-multitask
UR10e Multitask Teleop
335 teleoperated manipulation episodes on a Universal Robots UR10e with a Robotiq 2F gripper, covering 7 tabletop tasks. Packaged in LeRobot v2.1 format with precomputed normalization statistics, so it can be dropped into a VLA finetuning run the same way LIBERO is.
Episodes
335
Frames
60,416
Tasks
7
Control rate
15 fps
Robot
UR10e, 6-DoF, Robotiq 2F gripper
Cameras
2 exterior (Azure Kinect) + 1 wrist (RealSense)
Image size
180 x… See the full description on the dataset page: https://huggingface.co/datasets/bag100/ur10e-multitask.so101_multitask_newsim_pick_place_stack_rawThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"observation.state": {
"dtype": "float32",
"shape": [
6
],
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos"… See the full description on the dataset page: https://huggingface.co/datasets/Chaenn/so101_multitask_newsim_pick_place_stack_raw.