datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
kitchen-workspace-understanding-safe-manipulation
Kitchen Workspace Understanding & Safe Manipulation
Generated by datapack-import.ts
This dataset mirrors public data-pack render outputs from Physicl.
Each row represents one render view. The image column contains a stable URL to the primary render image uploaded under /data; image_path stores the relative repository path and data_commit_sha pins the Hugging Face dataset commit used by those URLs. Files are uploaded as downloaded unless optional PNG recompression is enabled by… See the full description on the dataset page: https://huggingface.co/datasets/physicl/kitchen-workspace-understanding-safe-manipulation.everyday-manipulation-3d-raw
Everyday Manipulation 3D (raw RGB-D)
1,513 clips · 10.28 hours · 279 GiB · 4 participants · 10 manipulation tasks · 42 recording sittings
Chest-mounted iPhone Pro capture of everyday two-handed manipulation by
CaryX AI. Clips were recorded with
Record3D, an iOS app that captures the
iPhone's LiDAR RGB-D stream. Each clip is the app's .r3d recording with the
audio track removed; the sensor streams are unmodified: synchronised RGB,
metric LiDAR depth, per-frame ARKit 6-DoF camera… See the full description on the dataset page: https://huggingface.co/datasets/CaryxAI/everyday-manipulation-3d-raw.Arena-G1-Loco-Manipulation-Task
Dataset Description:
The Arena-G1-Loco-Manipulation-Task dataset is multimodal collections of trajectories generated in Isaac Lab. It supports humanoid (G1) loco-manipulation task in IsaacLab-Arena environment. Each entry provides the full context (state, vision, language, action) needed to train and evaluate generalist robot policies for box pick and place task.
Dataset Name
# Trajectories
G1 Loco-Manipulation Task
50
This dataset is ideal for behavior cloning… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Arena-G1-Loco-Manipulation-Task.berkeley_fanuc_manipulation_lerobotThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "fanuc_mate",
"total_episodes": 415,
"total_frames": 62613,
"total_tasks": 32,
"total_videos": 830,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 10,
"splits": {
"train": "0:415"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/IPEC-COMMUNITY/berkeley_fanuc_manipulation_lerobot.ego-tactile-manipulation
Ego-Tactile Manipulation
Egocentric video + dense two-hand tactile + touch-grounded action labels - by OpenGraph Labs.
Four episodes visualized in our dashboard - egocentric video with live tactile & sensor signals.
Synchronized ego + touch human-manipulation data is rare. This is a clean 1.28-hour sample from OpenGraph Labs' Physical-AI data pipeline: a head camera plus our OGLO tactile gloves on both hands, with action labels derived from the physical contact signal.… See the full description on the dataset page: https://huggingface.co/datasets/OpenGraphLabs-Research/ego-tactile-manipulation.PhysicalAI-Robotics-Manipulation-Kitchen
PhysicalAI Robotics Manipulation in the Kitchen
Dataset Description:
PhysicalAI-Robotics-Manipulation-Kitchen is a dataset of automatic generated motions of robots performing operations such as opening and closing cabinets, drawers, dishwashers and fridges. The dataset was generated in IsaacSim leveraging reasoning algorithms and optimization-based motion planning to find solutions to the tasks automatically [1, 3]. The dataset includes a bimanual manipulator built with… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Kitchen.berkeley_fanuc_manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "unknown",
"total_episodes": 415,
"total_frames": 62613,
"total_tasks": 32,
"total_videos": 830,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 10,
"splits": {
"train": "0:415"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/lerobot/berkeley_fanuc_manipulation.utokyo_pr2_tabletop_manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "unknown",
"total_episodes": 240,
"total_frames": 32708,
"total_tasks": 3,
"total_videos": 240,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 5,
"splits": {
"train": "0:240"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/lerobot/utokyo_pr2_tabletop_manipulation.PhysicalAI-Robotics-Manipulation-ObjectsPhysicalAI-Robotics-Manipulation-Objects is a dataset of automatic generated motions of robots performing operations such as picking and placing objects in a kitchen environment. The dataset was generated in IsaacSim leveraging reasoning algorithms and optimization-based motion planning to find solutions to the tasks automatically [1, 3]. The dataset includes a bimanual manipulator built with Kinova Gen3 arms. The environments are kitchen scenes where the furniture and appliances were… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Objects.factory-manipulation-videos
Factory manipulation videos
Procedural Robotics is open sourcing a small set of our factory data so teams can assess its quality. The videos show workers performing factory tasks.
Contents
Seven continuous takes, 109 minutes in total.
Task
Station
Worker
Duration
File
cardboard manipulation
01
041
23.6 min
cardboard_manipulation_station01_worker041.mp4
cardboard manipulation
04
026
16.5 min
cardboard_manipulation_station04_worker026.mp4
defect… See the full description on the dataset page: https://huggingface.co/datasets/procedural-robotics/factory-manipulation-videos.conq_hose_manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "unknown",
"total_episodes": 139,
"total_frames": 8277,
"total_tasks": 3,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:139"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4"… See the full description on the dataset page: https://huggingface.co/datasets/lerobot/conq_hose_manipulation.Arena-GR1-Manipulation-PlaceItemCloseDoor-Task
Dataset Description:
The Arena-GR1-Manipulation-PlaceItemCloseDoor-Task dataset is a multimodal collection of trajectories generated in Isaac Lab. It supports humanoid (GR1) manipulation tasks in the IsaacLab-Arena environment. Each entry provides the full context (state, vision, language, and action) needed to train and evaluate generalist robot policies for a sequential task (e.g. putting object into a fridge and closing the door).
Dataset Name
# Trajectories
GR1… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Arena-GR1-Manipulation-PlaceItemCloseDoor-Task.Arena-GR1-Manipulation-Task
Dataset Description:
The Arena-GR1-Manipulation-Task dataset is multimodal collections of trajectories generated in Isaac Lab. It supports humanoid (GR1) manipulation task in IsaacLab-Arena environment. Each entry provides the full context (state, vision, language, action) needed to train and evaluate generalist robot policies for opening microwave task.
Dataset Name
# Trajectories
GR1 Manipulation Task
50
This dataset is ideal for behavior cloning, policy learning… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Arena-GR1-Manipulation-Task.Arena-GR1-Manipulation-Task-v3This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "GR1",
"total_episodes": 50,
"total_frames": 4928,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 50,
"splits": {
"train": "0:50"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Arena-GR1-Manipulation-Task-v3.zerograv-manipulation-trajectories-v0
⚠️ Superseded — do not use for distillation
These trajectories were collected from policies trained before the sim-real
action-space alignment. Measured on this data: mean |arm action| is 0.72, with
46-53% of actions at the clip boundary of [-1, 1]. For comparison, the real
robot's own pi0.5-DROID policy outputs a mean of 0.066 and human teleop 0.101
— so these actions sit 7-11x outside the distribution the base model has seen.
Distilling them would push pi0.5-DROID's action… See the full description on the dataset page: https://huggingface.co/datasets/stray-light/zerograv-manipulation-trajectories-v0.egocentric-manipulation-sample
SmartDeer — Egocentric Human Demonstration Data (Public Sample)
SmartDeer collects egocentric human demonstration data for embodied AI: real operators, real tasks, real environments. No teleoperation rigs, no lab mock-ups, no actors.
This repository is a small public sample, published so you can inspect our capture quality, annotation schema and file conventions. Production data is delivered under commercial terms.
This sample
Task twist_cap — bimanual bottle-cap… See the full description on the dataset page: https://huggingface.co/datasets/SmartDeer/egocentric-manipulation-sample.franka-fr3-manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "franka",
"total_episodes": 101,
"total_frames": 21006,
"total_tasks": 4,
"chunks_size": 1000,
"fps": 8,
"splits": {
"train": "0:101"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4"… See the full description on the dataset page: https://huggingface.co/datasets/binsabit/franka-fr3-manipulation.so101_gba_direct_manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so101_leader",
"total_episodes": 200,
"total_frames": 47567,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:200"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/giacomoran/so101_gba_direct_manipulation.crab-manipulation-dataset-labeled
Crab Manipulation Dataset — Reward Labeled
Same demonstrations as crab-manipulation-dataset, augmented with per-episode reward parameters for Safety-Aware Reward-Weighted Flow Matching (SA-RWFM) training.
What's Different
Each session's meta/ folder contains an additional reward_params.json with:
Tactile analysis: contact fractions, force statistics, damage flags
Gripper analysis: open/close thresholds, velocity analysis
Reward parameters: success/drop/damage rewards… See the full description on the dataset page: https://huggingface.co/datasets/armteam/crab-manipulation-dataset-labeled.berkeley_fanuc_manipulation_lerobotThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "fanuc_mate",
"total_episodes": 415,
"total_frames": 62613,
"total_tasks": 32,
"total_videos": 830,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 10,
"splits": {
"train": "0:415"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/introvoyz041/berkeley_fanuc_manipulation_lerobot.zerograv-manipulation-trajectories-state-v0
⚠️ Superseded — do not use for distillation
These trajectories were collected from policies trained before the sim-real
action-space alignment. Measured on this data: mean |arm action| is 0.72, with
46-53% of actions at the clip boundary of [-1, 1]. For comparison, the real
robot's own pi0.5-DROID policy outputs a mean of 0.066 and human teleop 0.101
— so these actions sit 7-11x outside the distribution the base model has seen.
Distilling them would push pi0.5-DROID's action… See the full description on the dataset page: https://huggingface.co/datasets/stray-light/zerograv-manipulation-trajectories-state-v0.crab-manipulation-dataset
Crab Manipulation Dataset
Teleoperated demonstration dataset for contact-rich manipulation on the Crab bimanual mobile manipulator (dual SO-101 arms). Collected at 15 Hz with 3 RGB cameras (640×480) and dual 10×10 tactile force sensor matrices.
Dataset Structure
Clean demonstrations (27 sessions, used for training)
Folder
Sessions
Task
Hardness
egg_carton_to_tray/
11
Open carton, place egg on tray
Soft
marmalade_jar/
7
Pick and place jar
Medium… See the full description on the dataset page: https://huggingface.co/datasets/armteam/crab-manipulation-dataset.manipulation-tasksThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/anant9/manipulation-tasks.unitree-g1-manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "Unitree_G1_Dex3",
"total_episodes": 2,
"total_frames": 603,
"total_tasks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:2"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4"… See the full description on the dataset page: https://huggingface.co/datasets/Breno-de-Angelo/unitree-g1-manipulation.OXE_berkeley_fanuc_manipulation_embeddingsLanguage Table (LeRobot) — Embedding-Only Release
(DINOv3 + SigLIP2 image features; EmbeddingGemma task-text features)
This repository packages a re-encoded variant of IPEC-COMMUNITY/berkeley_fanuc_manipulation_lerobot where raw videos are replaced by fixed-length image embeddings, and task strings are augmented with text embeddings. All indices, splits, and semantics remain consistent with the source dataset while storage and I/O are substantially lighter. To make the dataset practical to… See the full description on the dataset page: https://huggingface.co/datasets/saaduddinM/OXE_berkeley_fanuc_manipulation_embeddings.door-opening-manipulation-training-pack-next-pack-fbe147bb-03e76805
Outdoor Door Handle Push/Pull Training Set
Synthetic outdoor dataset staged in an alleyway to train a robot to detect door handles and determine whether a door must be pushed or pulled. 20 renders at 1024x1024 with albedo and metric depth passes, per-frame annotations, and authored lighting.
This dataset mirrors public data-pack render outputs from Physicl.
Each row represents one render view. The image column contains a stable URL to the primary render image uploaded under… See the full description on the dataset page: https://huggingface.co/datasets/physicl-community/door-opening-manipulation-training-pack-next-pack-fbe147bb-03e76805.oversonic-r1916-manipulation-v4This dataset was created using LeRobot v3.0 format via Forge.
Robot: oversonic (21-DOF)
Cameras: observation.images.nav, observation.images.head, observation.images.wrist_left
Episodes: 2
Frames: 378
FPS: 27
Arena-G1-Loco-Manipulation-TaskThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "G1",
"total_episodes": 5,
"total_frames": 4354,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 50,
"splits": {
"train": "0:5"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/nvkartik/Arena-G1-Loco-Manipulation-Task.kitchen-object-grasping-manipulation-policy-training-next-pack-5534d90d-f7cb0d04
Cluttered Home Kitchens for Cooking-Assistant Robot
Training dataset of richly cluttered, realistic home kitchens to train a YOLOv8-based cooking-assistant robot that navigates the kitchen, detects and segments objects, distinguishes food from non-food, identifies what needs cleaning, avoids obstacles and manipulates objects. Renders are 640x640 with metric depth, world-space normals (OpenGL linear), albedo and material-index passes, per-frame annotations, and midday lighting… See the full description on the dataset page: https://huggingface.co/datasets/physicl-community/kitchen-object-grasping-manipulation-policy-training-next-pack-5534d90d-f7cb0d04.factory-manipulation-lerobot-v2