datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
PhysicalAI-Robotics-PhysicalAssets-VoMP-Eval
VoMP: Predicting Volumetric Mechanical Properties
Dataset Description:
The Pre-Processed 3D Dataset is a dataset that is composed of 4 individual 3D asset datasets which are processed to render them from multiple views, voxelize the assets, and propagate VLM annotations for material properties.
We release pre-processed data derived from the 3D assets, specifically: voxels, rendered images, and LLM-annotated material descriptions.
This dataset is for research and… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-PhysicalAssets-VoMP-Eval.robotics-end-datarobotics-perception-action-alignment-v0.1What this dataset tests
Whether robot actions match current perception
Whether the system acts on stale, wrong-frame, or hallucinated state
Why this exists
Robots fail when perception and action decouple
stale frames
latency
occlusion
misclassification
hallucinated targets
This set makes those failures measurable
Data format
Each row contains
sensor_snapshot
world_state_change
commanded_action
executed_action
outcome
The task is to label alignment and… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/robotics-perception-action-alignment-v0.1.robotics-human-intent-alignment-v0.1What this dataset tests
The robot correctly interprets human signals
The robot respects safety constraints
The robot asks clarifying questions when needed
Why this exists
Robots fail around humans when they
ignore stop signals
act too literally
overreach without confirmation
miss gestures
treat ambiguity as certainty
Data format
human_signal
context
robot_interpretation
robot_action
outcome
Task
Emit one intent label
Give one short reason
Intent… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/robotics-human-intent-alignment-v0.1.robotics-constraint-aware-motion-v0.1What this dataset tests
Motion plans respect limits
Safety rules hold near humans
Paths avoid obstacles
Why this exists
Robots fail when planners ignore limits.This set catches that.
Data format
robot_state
planned_trajectory
constraints
environment
execution_result
Task
Emit one label
Give a short reason
Scoring
Correct label
Reference the broken limit
Reference space or motion
Constraint pressures
collision_risk
joint_limit
torque_limit… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/robotics-constraint-aware-motion-v0.1.robotics-action-outcome-coherence-v0.1What this dataset tests
Whether claimed success matches physical outcome
Whether constraints are respected
Whether failure is acknowledged
Why this exists
Robots often narrate successPhysics disagrees
This set forces contact with reality
Data format
Each row contains
initial_state
commanded_action
physical_constraints
observed_outcome
claimed_outcome
Feed the model
initial_state
commanded_action
physical_constraints
observed_outcome
claimed_outcome
Score… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/robotics-action-outcome-coherence-v0.1.robotics-sensor-fusion-coherence-failure-v0.1What this repo is for
You use it to detect when multiple sensors stop agreeing about reality.
It captures real-world autonomy failures:
camera sees clear path but lidar disagrees
IMU drift destabilises pose
odometry diverges from world model
fusion confidence collapses before failure
Applies to:
warehouse robots
mobile robots
drones
industrial arms
field robotics
Prompt format
Return exactly one token
coherent or incoherent
nl-robotics-semantic-parsing-info_structure-2k-novelty-context-TESTrobotics-failure-recovery-integrity-v0.1What this dataset tests
Failure is detected
Failure is acknowledged
Recovery is appropriate and safe
Why this exists
Robots often fail twice
first in action
second in recovery
This set makes that visible
Data format
failure_signal
detection_response
recovery_plan
executed_recovery
outcome
Task
Emit one recovery label
Give one short reason
Recovery pressures
slip_event
torque_spike
mechanical_break
blocked_path
repeat_error
human_interaction… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/robotics-failure-recovery-integrity-v0.1.robotics-world-model-consistency-v0.1What this dataset tests
The internal world model matches perception and reality
No duplication, teleporting, or stale entities
Identity tracking stays stable
Why this exists
Embodied systems fail when state tracking breaks
duplicated objects
teleport assumptions
stale memory
wrong identity
impossible combined states
This set labels those breaks
Data format
perception_updates
memory_state_before
world_model_after
observed_world_after
Task
Emit one… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/robotics-world-model-consistency-v0.1.robotics-multi-agent-coordination-coherence-risk-v0.1What this repo is for
You use it to detect when robot fleets stop coordinating properly.
It captures real deployment failure signals:
shared map divergence
task allocation conflicts
comms latency desync
deadlocks in corridors
swarm formation collapse
Applies to:
warehouse robot fleets
hospital delivery robots
drone swarms
factory material handling
Prompt format
Return exactly one token
coherent or incoherent
robotics-blind-spots-phi3
Dataset: Robotics Blind Spots in Microsoft Phi-3.5-mini-instruct
Model Tested
This dataset evaluates the base model microsoft/Phi-3.5-mini-instruct (3.8B parameters), released in 2024.
Model Loading Code
The model was loaded in Google Colab with a T4 GPU using 4-bit quantization:
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import torch
model_name = "microsoft/Phi-3.5-mini-instruct"
quantization_config =… See the full description on the dataset page: https://huggingface.co/datasets/MS100/robotics-blind-spots-phi3.robotics-temporal-action-sequencing-v0.1What this dataset tests
Whether actions occur in the correct order
Whether prerequisite steps are respected
Whether unsafe ordering is detected
Why this exists
Robots often fail by doing the right actionsin the wrong order
This set detects temporal incoherence
Data format
planned_sequence
executed_sequence
observed_result
Task
Emit one sequencing label
Give a short explanation
Sequencing pressures
ordering_error
premature_force
missing_tilt
loop_error… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/robotics-temporal-action-sequencing-v0.1.robotics_1000_dataset.csvrobotics-world-model-coherence-drift-v0.1What this repo is for
You use it to detect when a robot’s internal world model no longer matches reality.
It captures failures like:
map outdated after environment change
localisation drift
object position mismatch
robot navigating into blocked space
Applies to:
warehouse robots
delivery robots
drones
inspection robots
industrial manipulators
Prompt format
Return exactly one token
coherent or incoherent
robotics-command-actuation-coherence-loss-v0.1What this repo is for
You use it to detect when a robot sends the right command but the hardware does not execute it faithfully.
It captures deployment failures such as:
controller instability
actuator saturation
excessive latency
torque or thrust limits exceeded
feedback diverging from command
Applies to:
industrial robot arms
AMRs and mobile bases
drones
grippers and end effectors
Prompt format
Return exactly one token
coherent or incoherent
clinical-adverse-events-dataset
Clinical Adverse Events Dataset
Overview
This dataset contains 400 curated adverse event reports based on the FDA FAERS (FDA Adverse Event Reporting System) structure. It is designed for training AI systems in pharmaceutical safety analysis and regulatory compliance.
Dataset Description
The dataset provides comprehensive adverse event information including patient demographics, drug exposures, event narratives, outcomes, and medical histories. Each record has… See the full description on the dataset page: https://huggingface.co/datasets/Bumblebee-Robotics/clinical-adverse-events-dataset.robotics-plan-execution-coherence-drift-v0.1What this repo is for
You use it to detect when a robot has a valid plan but drifts off it in the real world.
It captures common deployment failures:
localisation drift
control tracking error
obstacle surprises that require replan
feasible in sim but fails in reality
Typical uses:
warehouse AMRs
hospital delivery robots
manipulator trajectory execution
field inspection robots
Prompt format
Output must be exactly one token
coherent or incoherent
robotics-task-intent-outcome-coherence-risk-v0.1What this repo is for
Detect when robots complete actions but fail the real task.
This is what companies care about:
robot did something
but not what mattered.
Use it to track:
wrong bin placement
incomplete inspections
wrong delivery
skipped assembly steps
This becomes the top-level KPI dataset for robotics deployments.
nl-robotics-translation-simple_english-30k-no-contextnl-robotics-semantic-parsing-info_structure-2k-no-context-TESTnl-robotics-semantic-parsing-info_structure-30k-contextnl-robotics-translation-simple_english-2k-no-context-TESTnl-robotics-semantic-parsing-info_structure-2k-novelty-no-context-TESThumanoid-robotics-dataset-01nl-robotics-translation-simple_english-30k-contextnl-robotics-translation-simple_english-2k-novelty-context-TESTrobotics-goal-persistence-under-disturbance-v0.1What this dataset tests
Goal remains stable when the world shifts
Replanning happens instead of goal drift
Safety interrupts persistence when needed
Why this exists
Robots often fail under disturbance
they abandon goals
they substitute easier goals
they keep pushing unsafely
This set labels those outcomes
Data format
primary_goal
disturbance_event
chosen_action_after
rationale_after
Task
Emit one persistence label
Give one short reason
Disturbance pressures… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/robotics-goal-persistence-under-disturbance-v0.1.robotics-human-proximity-safety-coherence-risk-v0.1What this repo is for
You use it to detect safety envelope drift in human-robot environments.
It targets failures that precede incidents:
human detected but robot fails to slow
stop not triggered when required
unsafe overrides used to keep throughput
near misses that signal a rising accident risk
Applies to:
warehouse AMRs
factory cobots
hospital delivery robots
mixed human-robot zones
Prompt format
Return exactly one token
coherent or incoherent
nl-robotics-semantic-parsing-info_structure-30k-no-context