datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
PhysicalAI-Robotics-GR00T-Teleop-Sim
Simulation GR1 Tabletop Task 1K Dataset
Dataset Description:
The PhysicalAI-Robotics-GR00T-Teleop-GR1 dataset consists of 1000 teleoperation trajectories in simulation using the GR1 robot with upper body control. The simulation setup mimics tabletop manipulation tasks and uses RGB observations with a virtual camera. The robot is equipped with simulated Fourier hands.
This dataset is ready for non-commercial use.
Dataset Owner(s):
NVIDIA GEAR… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-GR00T-Teleop-Sim.PhysicalAI-Robotics-GR00T-Teleop-GR1
Introduction
TL;DR: DreamDojo is a generalist robot world model pretrained on 44k hours of human egocentric data, showing unprecedented generalization to diverse objects and environments.
Project page: https://dreamdojo-world.github.io/
Paper: https://arxiv.org/abs/2602.06949
Code: https://github.com/NVIDIA/DreamDojo
How to Use
Check out https://github.com/NVIDIA/DreamDojo
Citation
@article{gao2026dreamdojo,
title={DreamDojo: A Generalist Robot… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-GR00T-Teleop-GR1.PhysicalAI-Robotics-GR00T-Teleop-G1
Unitree G1 Fruits Pick and Place 1K Dataset
Dataset Description:
The PhysicalAI-Robotics-GR00T-Teleop-G1 dataset consists of1000 teleoperation trajectories of real robot data using Unitree G1, with upper body control. The robot chooses the correct fruit to pick and place on the plate according to the language prompt. A total of 4 fruits are used: Apple, Pear, Starfruit, Grape. The robot is equipped with the default realsense camera, and a pair of Unitree G1 Tri-fingers… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-GR00T-Teleop-G1.PhysicalAI-Robotics-Manipulation-Kitchen
PhysicalAI Robotics Manipulation in the Kitchen
Dataset Description:
PhysicalAI-Robotics-Manipulation-Kitchen is a dataset of automatic generated motions of robots performing operations such as opening and closing cabinets, drawers, dishwashers and fridges. The dataset was generated in IsaacSim leveraging reasoning algorithms and optimization-based motion planning to find solutions to the tasks automatically [1, 3]. The dataset includes a bimanual manipulator built with… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Kitchen.PhysicalAI-Robotics-Manipulation-ObjectsPhysicalAI-Robotics-Manipulation-Objects is a dataset of automatic generated motions of robots performing operations such as picking and placing objects in a kitchen environment. The dataset was generated in IsaacSim leveraging reasoning algorithms and optimization-based motion planning to find solutions to the tasks automatically [1, 3]. The dataset includes a bimanual manipulator built with Kinova Gen3 arms. The environments are kitchen scenes where the furniture and appliances were… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Objects.PhysicalAI-GR00T-Tuned-Tasks
Dataset Description:
This dataset is multimodal collections of trajectories generated in Isaac Lab. It supports humanoid (GR1) tabletop manipulation tasks for industrial settings. Each dataset entry provides the full context (state, vision, language, action) needed to train and evaluate generalist robot policies for tasks like pouring nuts or sorting pipes by color.
Dataset Name
# Trajectories
Exhaust-Pipe-Sorting-task
1000
Nut-Pouring-task
1000
This dataset is… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-GR00T-Tuned-Tasks.physical_ai_tuneupso101_1200ep_dataset_20260803_104129This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/physicalairi/so101_1200ep_dataset_20260803_104129.physicalaiThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/slvrfivo/physicalai.physical-ai
DecisionFacts Physical AI Dataset — SO-101 Robotic Arm Teleoperation
Data Summary
This dataset is a curated collection of real-world teleoperation data captured on the SO-101 robotic arm (so_follower), built to support training and evaluation of modern robot-learning models — from imitation-learning policies to large-scale Vision-Language-Action (VLA) and world models.
Each episode is a human-teleoperated demonstration of a manipulation task, recorded… See the full description on the dataset page: https://huggingface.co/datasets/DecisionFacts/physical-ai.Physical_AI_SO101_Cup_Nesting_Task
DecisionFacts Physical AI Dataset — SO-101 Robotic Arm Teleoperation
Data Summary
This dataset is a curated collection of real-world teleoperation data captured on the SO-101 robotic arm (so_follower), built to support training and evaluation of modern robot-learning models — from imitation-learning policies to large-scale Vision-Language-Action (VLA) and world models.
Each episode is a human-teleoperated demonstration of a manipulation task, recorded… See the full description on the dataset page: https://huggingface.co/datasets/salehin21/Physical_AI_SO101_Cup_Nesting_Task.ur3_stack_cube_camera_sim_v2This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"observation.state": {
"dtype": "float32",
"shape": [
12
],
"names": [
"shoulder_pan_joint.pos",
"shoulder_lift_joint.pos",
"elbow_joint.pos",
"wrist_1_joint.pos",
"wrist_2_joint.pos"… See the full description on the dataset page: https://huggingface.co/datasets/physicalairi/ur3_stack_cube_camera_sim_v2.ur10e-gear-pick-place-demos
UR10e Gear Pick-and-Place Demonstrations
Private archive of the exact LeRobot v3.0 train and validation directories used by the revised Rho delta-action IL run. The task is to pick up the large gear and place it in the bin using a simulated UR10e with a Robotiq 2F-140 gripper.
📊 Dataset identity
Split
Episodes
Frames
Original data bytes
train/
90
154,593
277,712,776
validation/
10
17,170
30,883,658
Total
100
171,763
308,596,434
The corpus was… See the full description on the dataset page: https://huggingface.co/datasets/physical-ai-toolchain/ur10e-gear-pick-place-demos.PhysicalAI-Robotics-Manipulation-ObjectsPhysicalAI-Robotics-Manipulation-Objects is a dataset of automatic generated motions of robots performing operations such as picking and placing objects in a kitchen environment. The dataset was generated in IsaacSim leveraging reasoning algorithms and optimization-based motion planning to find solutions to the tasks automatically [1, 3]. The dataset includes a bimanual manipulator built with Kinova Gen3 arms. The environments are kitchen scenes where the furniture and appliances were… See the full description on the dataset page: https://huggingface.co/datasets/muhammadshihab/PhysicalAI-Robotics-Manipulation-Objects.so101_pick_orange_sim_v2This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so101_follower",
"total_episodes": 45,
"total_frames": 46278,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:45"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/physicalairi/so101_pick_orange_sim_v2.PhysicalAI-ADE-DEPhysicalAI-ADE-US
PhysicalAI-US-ADE
Dataset Summary
PhysicalAI-US-ADE contains per-sample evaluation outputs for autonomous driving waypoint prediction on the US subset of the PhysicalAI NVIDIA dataset.
This dataset stores inference-time predictions and evaluation statistics for models evaluated on the dataset, organized by model name at the top level. Each model directory contains sample-level records for that model’s predictions against ground truth.
The current release includes… See the full description on the dataset page: https://huggingface.co/datasets/mjf-su/PhysicalAI-ADE-US.physical-ai-bucharest-1This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 1,
"total_frames": 594,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:1"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/vladfatu/physical-ai-bucharest-1.physical_AiThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 2,
"total_frames": 1004,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:2"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/young331/physical_Ai.PhysicalAI-SimReady-Kitchens-v1
PhysicalAI SimReady Kitchens: 800 Scenes
800 simulation-ready OpenUSD kitchen environments for robotics, embodied AI, and physical AI evaluation.
This release is a public sample of Imagine.io's programmable world-generation infrastructure. It is designed to help research and commercial teams evaluate controlled, metadata-rich indoor environments for perception, scene understanding, simulation, and physical AI workflows.
License notice. Public access is CC BY-NC 4.0 for… See the full description on the dataset page: https://huggingface.co/datasets/imagineio/PhysicalAI-SimReady-Kitchens-v1.physical-ai-hack-test-setedition_1235_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade
edition_1235_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade
A Readymade by TheFactoryX
Original Dataset
nvidia/PhysicalAI-Robotics-GR00T-X-Embodiment-Sim
Process
This dataset is a "readymade" - inspired by Marcel Duchamp's concept of taking everyday objects and recontextualizing them as art.
What we did:
Selected the original dataset from Hugging Face
Shuffled each column independently
Destroyed all row-wise relationships
Preserved structure… See the full description on the dataset page: https://huggingface.co/datasets/TheFactoryX/edition_1235_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade.edition_2130_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade
edition_2130_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade
A Readymade by TheFactoryX
Original Dataset
nvidia/PhysicalAI-Robotics-GR00T-X-Embodiment-Sim
Process
This dataset is a "readymade" - inspired by Marcel Duchamp's concept of taking everyday objects and recontextualizing them as art.
What we did:
Selected the original dataset from Hugging Face
Shuffled each column independently
Destroyed all row-wise relationships
Preserved structure… See the full description on the dataset page: https://huggingface.co/datasets/TheFactoryX/edition_2130_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade.