Datasetpublic

vLAR/PhysInOne

PhysInOne: Visual Physics Learning and Reasoning in One Suite vLAR Group | The Hong Kong Polytechnic University | Syai Singapore | Meta CVPR 2026 🧭 Navigation 📌 Summary 🚀 Release Timetable 📦 Repositories & Downloads 📊 Data Splits 🧱 3D Assets 🛠️ Data Processing 🏆 Leaderboard Evaluation Data 🎞️ Rendered Data    1. Download Scripts    2. Install Dependencies… See the full description on the dataset page: https://huggingface.co/datasets/vLAR/PhysInOne.

sourceHugging Facecc-by-nc-sa-4.0updated 5d agoView on Hugging Face

22likes35kdownloads

Dataset Card

.physinone-repo-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(260px, 1fr)); gap: 8px; margin: 12px 0 8px; } .physinone-repo-card { display: flex; flex-direction: column; min-width: 0; box-sizing: border-box; padding: 9px 10px; overflow: hidden; border: 1px solid rgba(148, 163, 184, 0.28); border-radius: 9px; background: linear-gradient(145deg, rgba(148, 163, 184, 0.07), rgba(148, 163, 184, 0.025)); } .physinone-repo-card-main { grid-column: 1 / -1; background: linear-gradient(135deg, rgba(253, 224, 71, 0.14), rgba(249, 115, 22, 0.055)); border-color: rgba(234, 179, 8, 0.30); } .physinone-repo-card-header, .physinone-repo-card-bottom { display: flex; flex-wrap: wrap; align-items: center; min-width: 0; gap: 5px 7px; } .physinone-repo-card-header { justify-content: space-between; flex-wrap: nowrap; } .physinone-repo-card-header > strong { overflow: hidden; font-size: 13px; line-height: 1.2; text-overflow: ellipsis; white-space: nowrap; } .physinone-repo-card-tags { display: inline-flex; flex: 0 0 auto; align-items: center; margin-left: auto; gap: 3px; } .physinone-repo-split, .physinone-repo-status { padding: 1px 4px; font-size: 7.5px; font-weight: 700; line-height: 1.5; text-transform: uppercase; border-radius: 999px; white-space: nowrap; } .physinone-repo-split { color: #1d4ed8; background: rgba(59, 130, 246, 0.10); border: 1px solid rgba(59, 130, 246, 0.18); } .physinone-repo-status { color: #15803d; background: rgba(34, 197, 94, 0.11); border: 1px solid rgba(34, 197, 94, 0.18); } .physinone-repo-card-main .physinone-repo-status { color: #a16207; background: rgba(234, 179, 8, 0.12); border-color: rgba(234, 179, 8, 0.22); } .physinone-repo-card-bottom { margin-top: 6px; } .physinone-repo-size { flex: 0 0 auto; color: #d32f2f; font-size: 12px; line-height: 18px; white-space: nowrap; } .physinone-repo-downloads { display: flex; flex-wrap: wrap; align-items: center; min-width: 0; margin-left: auto; gap: 4px; } .physinone-repo-downloads a { display: inline-flex; max-width: 100%; } .physinone-repo-downloads img { display: block; max-width: 100%; height: 18px; } .physinone-repo-link { display: inline-flex; flex: 0 0 auto; align-items: center; margin-left: 0; padding: 1px 4px; font-size: 8.5px; font-weight: 700; line-height: 1.35; text-decoration: none; white-space: nowrap; background: rgba(59, 130, 246, 0.08); border: 1px solid rgba(59, 130, 246, 0.15); border-radius: 6px; }

@media (max-width: 900px) { .physinone-card { display: block !important; } .physinone-card aside { width: auto !important; min-width: 0 !important; max-width: none !important; position: static !important; margin-bottom: 20px; box-sizing: border-box; } .physinone-card main { width: 100% !important; min-width: 0 !important; box-sizing: border-box; } .physinone-summary-grid { grid-template-columns: repeat(2, minmax(0, 1fr)) !important; } }

@media (max-width: 560px) { .physinone-summary-grid { grid-template-columns: 1fr !important; } }

</style>

<h1 align="center" style="font-size: 42px;">PhysInOne: Visual Physics Learning and Reasoning in One Suite</h1>

<p align="center"> vLAR Group | The Hong Kong Polytechnic University | Syai Singapore | Meta </p>

<aside class="physinone-sidebar" style="flex: 0 1 250px; width: 250px; max-width: 100%; box-sizing: border-box; position: sticky; top: 16px; padding: 16px; border: 1px solid rgba(148, 163, 184, 0.28); border-radius: 12px; background: rgba(148, 163, 184, 0.12); font-size: 13.5px; line-height: 1.55;"> <strong>🧭 Navigation</strong><br><br> <a href="#summary">📌 Summary</a><br> <a href="#release-timetable">🚀 Release Timetable</a><br> <a href="#dataset-repositories">📦 Repositories & Downloads</a><br> <a href="#data-splits">📊 Data Splits</a><br> <a href="#three-d-assets">🧱 3D Assets</a><br> <a href="#data-processing">🛠️ Data Processing</a><br> <a href="#leaderboard-evaluation-data">🏆 Leaderboard Evaluation Data</a><br> <a href="#rendered-data">🎞️ Rendered Data</a><br>    <a href="#download-scripts">1. Download Scripts</a><br>    <a href="#install-dependencies">2. Install Dependencies</a><br>    <a href="#filter-cases">3. Filter Cases</a><br>       <a href="#filter-by-split">3.1 Split</a><br>       <a href="#filter-by-split-and-activity">3.2 Split + Activity</a><br>       <a href="#filter-by-phenomenon">3.3 Phenomenon</a><br>       <a href="#random-sampling">3.4 Random Sampling</a><br>    <a href="#download-selected-cases">4. Download Cases</a><br> <a href="#dataset-structure">📦 Dataset Structure</a><br> <a href="#annotation-details">📝 Annotation Details</a><br>    • <a href="#depth">Depth</a><br>    • <a href="#segmentation">Segmentation</a><br>    • <a href="#captions">Captions</a><br>    • <a href="#trajectories">Trajectories</a><br>       ◦ <a href="#trajectory-json-file-structure">JSON file structure</a><br>       ◦ <a href="#trajectory-actor-level-pose-storage">Actor-level pose storage</a><br>       ◦ <a href="#trajectory-pose-entry-format">Pose entry format</a><br>    • <a href="#cameras">Cameras</a><br>       ◦ <a href="#camera-json-structure">Camera JSON structure</a><br>       ◦ <a href="#dynamic-camera-behavior">Dynamic camera behavior</a><br>    • <a href="#point-clouds">Point Clouds</a><br> <a href="#data-fields">🧩 Data Fields</a><br>    • <a href="#physical-phenomena">Physical Phenomena</a><br> <a href="#license">📜 License</a><br> <a href="#citation">📚 Citation</a><br> <a href="#contact">📮 Contact</a><br> </aside>

<h2 id="summary" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">📌 Summary</h2> <div style="margin-left: 24px; text-align: justify;">

<strong>PhysInOne</strong> is a large-scale synthetic dataset for visual physics learning and reasoning, containing <strong>153,810 dynamic 3D scenes</strong> and <strong>2 million annotated videos</strong> across <strong>71 physical phenomena</strong> in mechanics, optics, fluid dynamics, and magnetism. Each scene features complex multi-object and multi-physics interactions with rich annotations, including RGB videos, depth maps, object masks, 3D trajectories, camera poses, object meshes, material properties, and textual descriptions. The dataset supports research on physics-aware video generation, future frame prediction, physical property estimation, motion transfer, physical reasoning, and world models.

<div style="padding: 12px; border: 1px solid rgba(148, 163, 184, 0.28); border-radius: 10px; background: rgba(148, 163, 184, 0.12);"> <strong>153,810</strong><br> <span style="font-size: 13px;">Dynamic 3D scenes</span> </div>

<div style="padding: 12px; border: 1px solid rgba(148, 163, 184, 0.28); border-radius: 10px; background: rgba(148, 163, 184, 0.12);"> <strong>2M</strong><br> <span style="font-size: 13px;">Annotated videos</span> </div>

<div style="padding: 12px; border: 1px solid rgba(148, 163, 184, 0.28); border-radius: 10px; background: rgba(148, 163, 184, 0.12);"> <strong>71</strong><br> <span style="font-size: 13px;">Physical phenomena</span> </div>

<div style="padding: 12px; border: 1px solid rgba(148, 163, 184, 0.28); border-radius: 10px; background: rgba(148, 163, 184, 0.12);"> <strong>4</strong><br> <span style="font-size: 13px;">Physics domains</span> </div>

</div>

<h2 id="release-timetable" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">🚀 Release Timetable</h2> <div style="margin-left: 24px;">

Component	Progress	Status	Notes
Rendered Data - Train	`██████████` 100%(122988/122988)	Released	Last updated: Aug 21
Rendered Data - Test	`██████████` 100%	Released	All Leaderboard user inputs released; GT excluded
Rendered Data - Val	`░░░░░░░░░░` 1%(103/15411)	In progress
3D Assets	`██████████` 100%	Released	14,847 Val scenes, 120,660 Train scenes, and Train/Val resource libraries
Leaderboard	`██████████` 100%	Released	Public evaluation inputs for all four tasks; GT excluded
Baseline code	`████░░░░░░` 50%	In progress	Last updated: Sept 18
Data processing	`██████████` 100%	Released	Camera placement, UE rendering, EXR conversion, and point-cloud generation

<h3 id="dataset-repositories" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2;">📦 Dataset Repositories & Downloads</h3>

Due to the large scale of PhysInOne, the rendered data and annotations are split across 16 Hugging Face repositories. Each entry shows the shard size, release status, live all-time downloads, live downloads in the last 30 days, and its repository link.

Combined snapshot (Oct 1, 2026): P01–P16 have 1,233,150 all-time downloads and 408,806 downloads in the last 30 days. Including the main repository, the per-repository sums are 1,280,414 and 442,702.

<div class="physinone-repo-card physinone-repo-card-main"> <div class="physinone-repo-card-header"><strong>Main repository</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-status">Main</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/vLAR/PhysInOne" aria-label="Open main repository">Open ↗</a></div> <div class="physinone-repo-card-bottom"><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/vLAR/PhysInOne"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FvLAR%2FPhysInOne%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOne total downloads"></a> <a href="https://huggingface.co/datasets/vLAR/PhysInOne"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FvLAR%2FPhysInOne%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOne 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP01</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 01/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP01/PhysInOneP01" aria-label="Open PhysInOneP01">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">4.52 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP01/PhysInOneP01"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP01%2FPhysInOneP01%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP01 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP01/PhysInOneP01"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP01%2FPhysInOneP01%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP01 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP02</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 02/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP02/PhysInOneP02" aria-label="Open PhysInOneP02">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">7.09 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP02/PhysInOneP02"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP02%2FPhysInOneP02%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP02 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP02/PhysInOneP02"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP02%2FPhysInOneP02%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP02 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP03</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 03/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP03/PhysInOneP03" aria-label="Open PhysInOneP03">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">7.59 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP03/PhysInOneP03"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP03%2FPhysInOneP03%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP03 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP03/PhysInOneP03"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP03%2FPhysInOneP03%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP03 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP04</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 04/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP04/PhysInOneP04" aria-label="Open PhysInOneP04">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">7.58 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP04/PhysInOneP04"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP04%2FPhysInOneP04%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP04 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP04/PhysInOneP04"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP04%2FPhysInOneP04%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP04 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP05</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 05/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP05/PhysInOneP05" aria-label="Open PhysInOneP05">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">7.16 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP05/PhysInOneP05"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP05%2FPhysInOneP05%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP05 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP05/PhysInOneP05"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP05%2FPhysInOneP05%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP05 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP06</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 06/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP06/PhysInOneP06" aria-label="Open PhysInOneP06">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">7.19 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP06/PhysInOneP06"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP06%2FPhysInOneP06%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP06 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP06/PhysInOneP06"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP06%2FPhysInOneP06%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP06 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP07</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 07/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP07/PhysInOneP07" aria-label="Open PhysInOneP07">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">7.21 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP07/PhysInOneP07"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP07%2FPhysInOneP07%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP07 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP07/PhysInOneP07"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP07%2FPhysInOneP07%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP07 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP08</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 08/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP08/PhysInOneP08" aria-label="Open PhysInOneP08">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">7.21 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP08/PhysInOneP08"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP08%2FPhysInOneP08%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP08 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP08/PhysInOneP08"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP08%2FPhysInOneP08%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP08 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP09</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 09/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP09/PhysInOneP09" aria-label="Open PhysInOneP09">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">7.20 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP09/PhysInOneP09"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP09%2FPhysInOneP09%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP09 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP09/PhysInOneP09"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP09%2FPhysInOneP09%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP09 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP10</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 10/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP10/PhysInOneP10" aria-label="Open PhysInOneP10">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">7.25 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP10/PhysInOneP10"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP10%2FPhysInOneP10%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP10 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP10/PhysInOneP10"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP10%2FPhysInOneP10%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP10 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP11</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 11/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP11/PhysInOneP11" aria-label="Open PhysInOneP11">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">7.48 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP11/PhysInOneP11"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP11%2FPhysInOneP11%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP11 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP11/PhysInOneP11"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP11%2FPhysInOneP11%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP11 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP12</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 12/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP12/PhysInOneP12" aria-label="Open PhysInOneP12">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">6.68 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP12/PhysInOneP12"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP12%2FPhysInOneP12%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP12 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP12/PhysInOneP12"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP12%2FPhysInOneP12%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP12 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP13</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 13/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP13/PhysInOneP13" aria-label="Open PhysInOneP13">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">6.66 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP13/PhysInOneP13"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP13%2FPhysInOneP13%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP13 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP13/PhysInOneP13"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP13%2FPhysInOneP13%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP13 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP14</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 14/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP14/PhysInOneP14" aria-label="Open PhysInOneP14">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">6.72 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP14/PhysInOneP14"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP14%2FPhysInOneP14%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP14 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP14/PhysInOneP14"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP14%2FPhysInOneP14%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP14 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP15</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 15/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP15/PhysInOneP15" aria-label="Open PhysInOneP15">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">7.93 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP15/PhysInOneP15"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP15%2FPhysInOneP15%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP15 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP15/PhysInOneP15"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP15%2FPhysInOneP15%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP15 30d downloads"></a></div></div> </div>

<div class="physinone-repo-card"> <div class="physinone-repo-card-header"><strong>PhysInOneP16</strong><span class="physinone-repo-card-tags"><span class="physinone-repo-split">Train · 16/16</span><span class="physinone-repo-status">Complete</span></span><a class="physinone-repo-link" href="https://huggingface.co/datasets/PhysInOneP16/PhysInOneP16" aria-label="Open PhysInOneP16">Open ↗</a></div> <div class="physinone-repo-card-bottom"><strong class="physinone-repo-size">1.57 TB</strong><div class="physinone-repo-downloads"><a href="https://huggingface.co/datasets/PhysInOneP16/PhysInOneP16"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP16%2FPhysInOneP16%3Fexpand%3DdownloadsAllTime&query=%24.downloadsAllTime&label=total&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP16 total downloads"></a> <a href="https://huggingface.co/datasets/PhysInOneP16/PhysInOneP16"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fdatasets%2FPhysInOneP16%2FPhysInOneP16%3Fexpand%3Ddownloads&query=%24.downloads&label=30d&logo=huggingface&color=FFD21E&cacheSeconds=3600" alt="PhysInOneP16 30d downloads"></a></div></div> </div>

</div>

Download badges query the official Hugging Face API and update automatically. Counts are repository-level download events, not deduplicated users; accessing multiple shards can produce one event in each shard.

For large-scale rendered-data downloading and filtering, please refer to the Rendered Data section below.

</div>

<h2 id="data-splits" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">📊 Data Splits</h2> <div style="margin-left: 24px; text-align: justify;">

PhysInOne is divided into <strong>Train, Val, and Test</strong> splits, containing 122,988, 15,411, and 15,411 scenes, respectively. The three splits are generated with <strong>completely distinct 3D meshes and backgrounds</strong>, ensuring that no scene or visual content is shared across splits. This separation provides a robust benchmark for evaluating generalization and physical reasoning.

</div>

<h2 id="three-d-assets" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">🧱 3D Assets</h2> <div style="margin-left: 24px; text-align: justify;">

The planned public 3D asset release is now <strong>100% complete</strong>. It contains shared Unreal Engine 5.5.4 project files, Scene and Trajectory resources for <strong>14,847 validation scenes and 120,660 training scenes</strong>, six validation resource archives, and seven training resource archives. The maintained download contains <strong>4,585 files</strong>, including <strong>285 ZIP archives</strong>, totaling approximately <strong>139.67 GiB</strong>. The 273 validation origins reserved for the Motion Transfer test set are intentionally excluded to prevent evaluation leakage.

The Scene and Trajectory archives remain separate. Their ZIP members begin with <code>Train/</code> or <code>Val/</code>; extracting each archive into its current parent produces <code>Content/PhysInOne/Scenes/{Train,Val}/</code> and <code>Content/PhysInOne/Trajectories/{Train,Val}/</code> without collisions.

The Train Scene and Trajectory series each span batches <code>0000–0120</code>; the Val series each span batches <code>0000–0014</code>. The final Train batch contains 660 scenes, and the final Val batch contains 847 scenes.

Install <strong>Unreal Engine 5.5.4</strong>. Windows is recommended for the simplest setup; Linux is also supported with additional platform configuration. The dependency-free GitHub downloader retrieves and assembles the complete release, safely extracts all 285 ZIP files into their correct parent directories, preserves the Unreal Engine package hierarchy, supports interrupted-run recovery, and records detailed logs.

bash

git clone https://github.com/vLAR-group/PhysInOne.git
cd PhysInOne
python scripts/download_data.py --task 3d-assets --output-dir ./PhysInOne_data

After download and extraction, open:

text

PhysInOne_data/assets/PhysicBenchmark/PhysInOne.uproject

Do not rename or flatten directories such as Content/PhysInOne/, Content/__ExternalActors__/, or Content/__ExternalObjects__/; Unreal Engine asset references depend on the exact project-relative structure. See the full setup guide for manual extraction paths, disk-space guidance, resume behavior, and troubleshooting.

</div>

<h2 id="data-processing" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">🛠️ Data Processing</h2> <div style="margin-left: 24px; text-align: justify;">

The public data-processing toolkit is released as ordinary source files rather than a ZIP archive. It covers static-camera placement, moving-camera and Level Sequence generation, unattended Unreal Engine rendering, EXR conversion, and multi-view point-cloud generation.

The source directory contains the camera tools, Windows headless-render launcher, post-processing modules, dependency list, examples, and illustrated documentation. Install the post-processing dependencies with:

bash

python -m pip install -r requirements-render.txt

The ten supplied Unreal configuration assets must remain under PhysicBenchmark/Content/PhysInOne/Scripts/, which Unreal Engine exposes as /Game/PhysInOne/Scripts/. The rendering configuration refers to that virtual path directly. The tools require no Hugging Face account or access token.

</div>

<h2 id="leaderboard-evaluation-data" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">🏆 Leaderboard Evaluation Data</h2> <div style="margin-left: 24px; text-align: justify;">

All user-facing evaluation inputs required for the four PhysInOne Leaderboard tasks have been released. Ground-truth targets remain private and are not included in the public release. Exact task-specific file lists and a resumable Python downloader allow users to download one task, multiple tasks, or individual scenes without scanning unrelated dataset shards.

Task	Released files	Public data
Video Generation	75,865	Initial RGB observations, camera metadata, and captions
Future Prediction	103 ZIPs	Training-view RGB prefixes and train/val/test transform metadata
Physical Properties Estimation	72 scene ZIPs + 2 shared files	Training/novel-view inputs, camera metadata, and shared supplementary resources
Motion Transfer	214 ZIPs	Source videos, target reference frames, and captions

Clone the GitHub repository once. The public downloader uses only the Python standard library and reads its maintained manifests locally:

bash

git clone https://github.com/vLAR-group/PhysInOne.git
cd PhysInOne

Interactive downloads show one live progress bar with file count, transferred and estimated total size, current speed, and ETA. Redirected output uses periodic plain-text snapshots, while full per-file details remain in _download_logs/.

Each row is a complete command that can be copied directly. The remote Leaderboard folder names remain unchanged, while the downloader creates space-free local task directories.

Task	Copy command	Local directory
Video Generation	`python scripts/download_data.py --task video-generation --output-dir ./PhysInOne_data`	`PhysInOne_data/leaderboard/video_generation/`
Future Prediction	`python scripts/download_data.py --task future-prediction --output-dir ./PhysInOne_data`	`PhysInOne_data/leaderboard/future_prediction/`
Physical Properties Estimation	`python scripts/download_data.py --task physical-properties-estimation --output-dir ./PhysInOne_data`	`PhysInOne_data/leaderboard/physical_properties_estimation/`
Motion Transfer	`python scripts/download_data.py --task motion-transfer --output-dir ./PhysInOne_data`	`PhysInOne_data/leaderboard/motion_transfer/`
All four Leaderboard tasks	`python scripts/download_data.py --task video-generation future-prediction physical-properties-estimation motion-transfer --output-dir ./PhysInOne_data`	`PhysInOne_data/leaderboard/`

Parameter	Purpose
`--task NAME [NAME ...]`	Select one or more tasks. Use the values in the table above, `3d-assets`, or `all`.
`--output-dir PATH`	Set the space-free local output root. Default: `./PhysInOne_data`.
`--scene TEXT`	Match a scene name or six-character scene ID; repeatable.
`--workers N`	Set concurrent downloads. Default: `4`.
`--extract` / `--no-extract`	Enable or disable safe ZIP extraction.
`--delete-zip-after-extract`	Delete each ZIP only after verified extraction.
`--revision REV`	Select the dataset revision. Default: `main`.
`--dry-run` / `--list-only`	Preview counts/output paths or exact public URLs.

Downloads are resumable, and every run records successes and failures for review. No Hugging Face account, access token, or Python package installation is required.

See the complete GitHub download guide, download script, and bundled manifests. The Hugging Face utility guide documents the same dependency-free interface.

</div>

<h2 id="rendered-data" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">🎞️ Rendered Data</h2>

<h3 id="download-scripts" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2; margin-left: 24px;">1. Download Scripts</h3>

Please download the `Utils` folder from the main dataset repository. Its public scripts mirror the GitHub scripts/ tree:

text

Utils/
|-- metadata/
|   |-- repo_assignment.txt
|   +-- repo_map.json
|-- 3d_assets_lists/
|-- leaderboard_lists/
|-- 3D_ASSETS_DOWNLOAD.md
|-- LEADERBOARD_DOWNLOAD.md
+-- scripts/
    |-- download_data.py
    |-- download_selected.py
    |-- filter_cases.py
    +-- download_lists/
        |-- video_generation.txt
        |-- future_prediction.txt
        |-- physical_properties_estimation.txt
        |-- motion_transfer.txt
        |-- 3d_assets.txt
        |-- repo_assignment.txt
        +-- repo_map.json

Run the following commands from inside the downloaded Utils/ folder:

bash

cd "Utils"

PhysInOne is distributed across multiple Hugging Face dataset repositories because of its large scale. We provide utilities for selecting and downloading full-dataset cases and Leaderboard evaluation data:

—download_data.py: downloads all four Leaderboard tasks and assembles the currently released Unreal Engine project subset.
—filter_cases.py: exports selected full-dataset cases to JSON.
—download_selected.py: downloads selected scene ZIPs from the corresponding public shards.
—download_lists/: contains the exact manifests used by all three scripts.

</div>

<h3 id="install-dependencies" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2; margin-left: 24px;">2. Requirements</h3>

Python 3.9 or newer is recommended. The download utilities use only the Python standard library and anonymous HTTPS requests; no package installation or authentication is required.

</div>

<h3 id="filter-cases" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2; margin-left: 24px;">3. Filter Cases</h3>

Export a JSON file containing case information matching your filter criteria, for use in subsequent download scripts. The filter_cases.py script supports selection by:

—Split: train, val, test
—Activity complexity: single, double, triple
—Physical phenomenon abbreviation: MovingHitsFixed, FrictionStop, LiquidTension, ..., GranularFall
—Number of cases: globally sample num cases after filtering

The following examples show common filtering workflows.

</div>

<h4 id="filter-by-split" style="font-size: 1.12em; margin-top: 0.55em; margin-bottom: 0.14em; line-height: 1.18; margin-left: 48px;">3.1 Filter by split</h4>

bash

python scripts/filter_cases.py \
  --split train \
  --output selected_cases.json

</div>

<h4 id="filter-by-split-and-activity" style="font-size: 1.12em; margin-top: 0.55em; margin-bottom: 0.14em; line-height: 1.18; margin-left: 48px;">3.2 Filter by split and activity complexity</h4>

bash

python scripts/filter_cases.py \
  --split train \
  --activity_type double \
  --output selected_cases.json

</div>

<h4 id="filter-by-phenomenon" style="font-size: 1.12em; margin-top: 0.55em; margin-bottom: 0.14em; line-height: 1.18; margin-left: 48px;">3.3 Filter by physical phenomenon abbreviation</h4>

By default, phenomenon matching uses contains mode. For example, the following command selects all double-physics cases that contain FrictionStop(For all physical phenomena and their abbreviations, please refer to the Data Fields section below.):

bash

python scripts/filter_cases.py \
  --split train \
  --activity_type double \
  --phenomena FrictionStop \
  --match_mode contains \
  --output selected_cases.json

For exact matching, use --match_mode exact. Order does not matter. For example, this command selects cases whose phenomenon set is exactly {AccelConcaveSpin, AccelSurfaceSpin}:

bash

python scripts/filter_cases.py \
  --split train \
  --activity_type double \
  --phenomena AccelConcaveSpin AccelSurfaceSpin \
  --match_mode exact \
  --output selected_cases.json

</div>

<h4 id="random-sampling" style="font-size: 1.12em; margin-top: 0.55em; margin-bottom: 0.14em; line-height: 1.18; margin-left: 48px;">3.4 Randomly sample a fixed number of cases</h4>

--num is the global number of cases sampled after filtering (This is because the number of cases meeting the filtering criteria may be much larger than the number you need.). The default random seed is 42.

bash

python scripts/filter_cases.py \
  --split train \
  --activity_type double \
  --phenomena FrictionStop \
  --num 3000 \
  --seed 42 \
  --output selected_cases.json

</div>

<h3 id="download-selected-cases" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2; margin-left: 24px;">4. Download Selected Cases</h3>

Each selected case is downloaded as a zip file from the corresponding Hugging Face shard repository.

bash

python scripts/download_selected.py \
  --selection selected_cases.json \
  --output_dir ./PhysInOne

By default, downloaded zip files are saved into a flat output folder. To preserve shard/split/activity structure:

bash

python scripts/download_selected.py \
  --selection selected_cases.json \
  --output_dir ./PhysInOne \
  --keep_shard_structure

The preserved structure is:

text

PhysInOne/
  physinone_part1/
    Train/
      DoublePhysics/
        AccelConcaveSpin_AccelSurfaceSpin__bg070__K5ER39_trajectory.zip

</div>

<h2 id="dataset-structure" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">📦 Dataset Structure</h2> <div style="margin-left: 24px;">

text

AccelConcaveSpin__bg010__m3Zrtz_trajectory/
├── AccelConcaveSpin__bg010__m3Zrtz_trajectory.json
├── blender_CineCamera_0.json
├── blender_CineCamera_1.json
├── blender_CineCamera_2.json
├── blender_CineCamera_3.json
├── blender_CineCamera_5.json
├── blender_CineCamera_6.json
├── blender_CineCamera_7.json
├── blender_CineCamera_8.json
├── blender_CineCamera_9.json
├── blender_CineCamera_10.json
├── blender_CineCamera_11.json
├── blender_CineCamera_12.json
├── blender_CineCamera_Moving.json
├── cameras.ply
├── caption.txt
├── points3d.ply
├── recorder_stats.json
├── static_camera_list.txt
├── CineCamera_0/
│   ├── depth/  (0000.npz ~ 0089.npz, 90 files)
│   ├── rgb/    (0000.jpg ~ 0089.jpg, 90 files)
│   └── seg/    (0000.npz ~ 0089.npz, 90 files)
├── CineCamera_1/
│   ...
├── CineCamera_2/
│   ...
├── CineCamera_3/
│   ...
├── CineCamera_5/
│   ...
├── CineCamera_6/
│   ...
├── CineCamera_7/
│   ...
├── CineCamera_8/
│   ...
├── CineCamera_9/
│   ...
├── CineCamera_10/
│   ...
├── CineCamera_11/
│   ...
├── CineCamera_12/
│   ...
└── CineCamera_Moving/
    ├── depth/  (0000.npz ~ 0089.npz, 90 files)
    ├── rgb/    (0000.jpg ~ 0089.jpg, 90 files)
    └── seg/    (0000.npz ~ 0089.npz, 90 files)

</div>

<h2 id="annotation-details" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">📝 Annotation Details</h2> <div style="margin-left: 24px; text-align: justify;">

PhysInOne provides synchronized visual and physical annotations for each dynamic 3D scene.

<h3 id="depth" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2; margin-left: 0;"><span style="display: inline-block; width: 1.08em;">•</span>Depth</h3> <div style="margin-left: 16px;">

Depth maps are provided in .npz format and are expressed in meters.

</div>

<h3 id="segmentation" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2; margin-left: 0;"><span style="display: inline-block; width: 1.08em;">•</span>Segmentation</h3> <div style="margin-left: 16px;">

Segmentation masks encode background, static foreground objects, and dynamic foreground objects.

Expected encoding:

Pixel Value	Meaning
`0`	Background
`1-127`	Static foreground objects
`128-255`	Dynamic foreground objects

</div>

<h3 id="captions" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2; margin-left: 0;"><span style="display: inline-block; width: 1.08em;">•</span>Captions</h3> <div style="margin-left: 16px;">

Each scene includes a caption.txt file containing an English paragraph that describes the visual elements and the physical activity.

</div>

<h3 id="trajectories" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2; margin-left: 0;"><span style="display: inline-block; width: 1.08em;">•</span>Trajectories</h3> <div style="margin-left: 16px;">

Each sequence includes a JSON file with the same name as the sequence. This file records the per-frame poses of selected dynamic objects, including their rotations and translations in the world coordinate system.

<details> <summary id="trajectory-json-file-structure"><strong>JSON file structure</strong></summary>

The JSON file contains two main parts:

—sequence_info: basic information about the sequence, such as the total number of frames and the frame rate (fps).
—actors: pose and metadata for dynamic objects, including object category, asset index path, actor type, and transformation data.

</details>

<details> <summary id="trajectory-actor-level-pose-storage"><strong>Actor-level pose storage</strong></summary>

For each actor, the storage format depends on its actor_type:

—If actor_type is solid, the object is treated as a single rigid body. Its per-frame pose is stored in the actor-level transform_data field, while the components field is empty.
—If actor_type is interactable, the object is represented by multiple components. The actor-level transform_data field is empty, and the per-component poses are stored under components[component_name].transform_data.

</details>

<details> <summary id="trajectory-pose-entry-format"><strong>Pose entry format in <code>transform_data</code></strong></summary>

Each entry in transform_data describes the pose of the object or component at a specific frame:

—frame: frame index in the sequence
—time_seconds: timestamp of the frame in seconds
—transform: object or component transformation, including:
—location: 3D position
—rotation: rotation quaternion
—scale: 3D scale

</details>

</div>

<h3 id="cameras" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2; margin-left: 0;"><span style="display: inline-block; width: 1.08em;">•</span>Cameras</h3> <div style="margin-left: 16px;">

Each scene includes multiple camera viewpoints to support 3D perception, reconstruction, and physics-based visual reasoning tasks.

—Taichi-based MPM simulations (elastic solids, plasticine, granular substances, some Newtonian and Non-Newtonian fluids): 15 static cameras + 1 dynamic camera
—Other scenes: 12 static cameras + 1 dynamic camera

Note: Camera IDs are not fixed across cases. The camera index mapping for each case is stored in static_camera_list.txt under that case directory.

<details> <summary id="camera-json-structure"><strong>Camera JSON structure</strong></summary>

Per camera:

—frames: list of per-frame 4×4 camera-to-world (C2W) transformation matrices in Blender coordinate system
—camera_angle_x: horizontal field of view (FOV) in radians, can be used to compute intrinsic focal length
—img_w, img_h: image width and height in pixels
—trajectory_name: name of the camera trajectory
—total_frames, fps: number of frames and frames per second

</details>

<details> <summary id="dynamic-camera-behavior"><strong>Dynamic camera behavior</strong></summary>

—The dynamic camera moves along a path randomly sampled on a hemisphere surrounding the center of the main objects, providing diverse viewpoints.
—JSON frame indices correspond directly to video frame indices.

</details>

</div>

<h3 id="point-clouds" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2; margin-left: 0;"><span style="display: inline-block; width: 1.08em;">•</span>Point Clouds</h3> <div style="margin-left: 16px;">

Each scene should include points.ply.

The initial point cloud was obtained by back-projecting the pixels from each camera view in the first frame based on depth, followed by randomly sampling 100,000 points from the resulting data.

</div>

<h2 id="data-fields" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">🧩 Data Fields</h2> <div style="margin-left: 24px;">

<h3 id="physical-phenomena" style="font-size: 1.35em; margin-top: 0.75em; margin-bottom: 0.22em; line-height: 1.2; margin-left: 0;"><span style="display: inline-block; width: 1.08em;">•</span>Physical Phenomenon and Abbreviations</h3> <div style="margin-left: 16px; text-align: justify;">

ID	Physical Phenomenon	Abbreviation	Related Physical Laws
`1`	Object collide with static, stationary objects	MovingHitsFixed	Laws of Momentum
`2`	Moving objects collide with non-static stationary objects	MovingHitsStationary	Laws of Momentum
`3`	Two moving objects collide	MovingHitsMoving	Laws of Momentum
`...`	...	...	...

<details> <summary>Click to expand full abbreviation table</summary>

ID	Physical Phenomenon	Abbreviation	Related Physical Laws
`4`	Objects in equilibrium of wind and gravity	WindGravityBalance	Equilibrium, Aerodynamics, Gravity
`5`	Wind applied to a stationary object	WindPushStationary	Aerodynamics
`6`	Wind applied to objects moving in same direction	WindPushSameDir	Aerodynamics
`7`	Wind applied to objects moving in the opposite direction	WindPushOppDir	Aerodynamics
`8`	Wind applied to moving objects changes its velocity (applied at an angle)	WindDeflectMotion	Aerodynamics
`9`	Object thrown up with angle	ObliqueProjectile	Gravity
`10`	Objects falling straight down	VerticalFall	Gravity
`11`	Objects rolling down a straight slope	RollDownSlope	Gravity, Friction
`12`	Objects rolling up a slope	RollUpSlope	Gravity, Friction
`13`	Magnetic Attraction	MagnetAttract	Magnetism
`14`	Magnetic Repulsion	MagnetRepel	Magnetism
`15`	Objects near uniformly rotating pillar	UniformPanelSpin	Laws of Rotation
`16`	Objects near acceleratingly rotating pillar	AccelPanelSpin	Laws of Rotation
`17`	Objects inside uniformly rotating bowl	UniformConcaveSpin	Equilibrium, Laws of Rotation, Gravity, Friction
`18`	Objects inside acceleratingly rotating bowl	AccelConcaveSpin	Equilibrium, Laws of Rotation, Gravity, Friction
`19`	Objects on uniformly rotating plane	UniformSurfaceSpin	Laws of Rotation, Friction
`20`	Objects on acceleratingly rotating plane	AccelSurfaceSpin	Laws of Rotation, Friction
`21`	Objects on coarse surface with friction	FrictionStop	Friction
`22`	Spring is compressed	SpringCompress	Laws of Elasticity
`23`	Spring is stretched	SpringStretch	Laws of Elasticity
`24`	Breakable object shatters	ImpactFracture	Laws of Plasticity
`25`	Mirror shatters	MirrorFragmentReflect	Laws of Plasticity, Law of Reflection
`26`	Elastic rope connection	ElasticCouple	Laws of Elasticity
`27`	Object bounces off spring board	SpringboardRebound	Laws of Elasticity, Laws of Momentum
`28`	Objects interact with a balanced seesaw	SeesawCenterPivot	Laws of Torque
`29`	Objects interact with an imbalanced seesaw	SeesawOffsetPivot	Laws of Torque
`30`	Free balloon floats to ceiling	BalloonFloat	Laws of Buoyancy
`31`	Tethered balloon pulls string taut	BalloonTether	Laws of Buoyancy, Rope Restraint
`32`	Multiple balloons lifting	BalloonLift	Laws of Buoyancy, Rope Restraint, Gravity
`33`	Laser hits flat mirror and reflects	FixedPlanarRedirect	Law of Reflection
`34`	Laser reflects off multiple mirrors	FixedArrayRedirect	Law of Reflection
`35`	Laser hits and reflects off concave mirror	FixedConcaveRedirect	Law of Reflection
`36`	Laser hits and reflects off convex mirror	FixedConvexRedirect	Law of Reflection
`37`	Mirror sweeps beam	DynMirrorRedirect	Law of Reflection
`38`	Laser blocked by object	LaserBlock	Light Obstruction
`39`	Mirror reflection	MirrorReflect	Law of Reflection
`40`	Cart moving forward with rolling wheels	CartMove	Complex Mechanical Structure Constraints
`41`	Objects on rotating turntable flies off	RotTurnableInertia	Laws of Rotation, Friction, Laws of Inertia
`42`	Rotating block pushes another objects	RotBoardInertia	Laws of Inertia, Laws of Rotation, Laws of Momentum
`43`	One object carrying another	LinCarryInertia	Laws of Inertia, Friction
`44`	Catapult launches objects	CatapultLaunch	Special Mechanical Structure, Gravity
`45`	Chain suspends objects	ChainSuspend	Complex Mechanical Structure Constraints, Gravity
`46`	Objects Swing	SimplePendulum	Laws of Pendulum Motion, Gravity
`47`	Double Pendulum Moves	DoublePendulum	Laws of Multiple Pendulum Motion, Gravity
`48`	Crank push objects	CrankPush	Special Mechanical Structure
`49`	Wall composed of square blocks collapses	BlockWallCollapse	Gravity, Structural Stability
`50`	Wooden board supported by sticks collapses	StickSupportFail	Gravity, Structural Stability
`51`	Objects float on the fluid surface	FloatOnLiquid	Laws of Buoyancy, Fluid Dynamics
`52`	Objects drop into the fluid	DropInLiquid	Laws of Buoyancy, Fluid Dynamics
`53`	Objects' movement causes fluid motion	MovingObjDriveLiquid	Fluid Dynamics
`54`	Flowing fluid carries objects along	LiquidCarryMovingObj	Laws of Buoyancy, Fluid Dynamics
`55`	Fluid flows against stationary objects	LiquidHitFixedObj	Fluid Dynamics
`56`	Fluid transfers from one container to another	LiquidTransfer	Fluid Dynamics, Conservation of Mass, Surface Tension
`57`	Fluid passes through several connected containers	LiquidMultiTransfers	Fluid Dynamics, Conservation of Mass, Surface Tension
`58`	Fluid flows through grid-like structures	LiquidThroughGrid	Fluid Dynamics, Conservation of Mass
`59`	Fluid moves across mountainous or uneven landscapes	LiquidAcrossUneven	Fluid Dynamics
`60`	Increasing fluid volume elevates the surface level	LiquidRise	Fluid Dynamics
`61`	Fluid flows along the contours of an object's surface	LiquidAlongContours	Fluid Dynamics
`62`	Jet-like fluid projection upward or outward	JetLiquid	Fluid Dynamics
`63`	Fluid exhibits surface tension	LiquidTension	Fluid Dynamics, Laws of Surface Tension
`64`	Fluid refracts light when crossing media	LiquidRefraction	Fluid Dynamics, Optics, Law of Refraction (Snell's Law)
`65`	Sticky fluid drips and accumulates on objects	StickyToObjects	Laws of Cohesion, Viscous Flow
`66`	Sticky fluid falls from an object's surface	StickyFromObjects	Laws of Cohesion, Laws of Viscous Flow (Navier-Stokes)
`67`	An elastic object falls and bounces on another surface	ElasticFall	Laws of Elasticity
`68`	A plasticine object falls and deforms on a surface	PlasticineFall	Laws of Plasticity
`69`	A Newtonian fluid falls and spreads across a surface	NewtonianFluidFall	Laws of Viscous Flow (Navier-Stokes)
`70`	A Non-Newtonian fluid falls and flows with variable resistance	NonNewtonianFluidFall	Laws of Viscoplastics Flow
`71`	A granular substance falls and disperses across a surface	GranularFall	Laws of Friction

</details>

Example: MovingHitsStationaryWindDeflectMotionBalloonFloat_bg140_TZlWl1

—The first three components correspond to specific physical phenomenon abbreviations (see table above).
—bg140 indicates that the scene uses background number 140.
—The last six-character string (TZlWl1) is a unique hash code generated for this scene to guarantee uniqueness.

This naming convention allows users to quickly parse scene metadata and link it to the corresponding physical phenomena, background, and unique identifier.

</div>

<h2 id="license" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">📜 License</h2>

All 3D assets and materials included in PhysInOne have been sourced from publicly available platforms and verified to carry licenses compatible with non-commercial use. These include:

—SketchFab: assets under various licenses, verified that AI-related usage is allowed.
—Fab: assets under CC BY or Unreal Engine Standard License, explicitly permitting AI-related usage.
—BlenderKit: distributed under Royalty-Free (RF) license.
—ShareTextures: textures under CC0 license.

In total, assets comply with licenses including CC BY-NC, CC BY-SA, CC BY-NC-SA, CC0, CC BY, and RF, ensuring all files can be legally used for building a non-commercial dataset. Users must adhere to the original licenses for any redistribution or derivative work.

</div>

<h2 id="citation" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">📚 Citation</h2>

If you use PhysInOne in your research, please cite:

bibtex

@misc{zhou2026physinonevisualphysicslearning,
      title={PhysInOne: Visual Physics Learning and Reasoning in One Suite}, 
      author={Siyuan Zhou and Hejun Wang and Hu Cheng and Jinxi Li and Dongsheng Wang and Junwei Jiang and Yixiao Jin and Jiayue Huang and Shiwei Mao and Shangjia Liu and Yafei Yang and Hongkang Song and Shenxing Wei and Zihui Zhang and Peng Huang and Shijie Liu and Zhengli Hao and Hao Li and Yitian Li and Wenqi Zhou and Zhihan Zhao and Zongqi He and Hongtao Wen and Shouwang Huang and Peng Yun and Bowen Cheng and Pok Kazaf Fu and Wai Kit Lai and Jiahao Chen and Kaiyuan Wang and Zhixuan Sun and Ziqi Li and Haochen Hu and Di Zhang and Chun Ho Yuen and Bing Wang and Zhihua Wang and Chuhang Zou and Bo Yang},
      year={2026},
      eprint={2604.09415},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2604.09415}, 
}

</div>

<h2 id="contact" style="margin-left: 0; font-size: 1.9em; line-height: 1.22; margin-top: 1.2em; margin-bottom: 0.5em;">📮 Contact</h2>

For questions about the dataset, please contact:

—{siyuan.zhou, hejun.wang, hu123.cheng, jinxi.li}@connect.polyu.hk, bo.yang@polyu.edu.hk

</div>

</main> </div>