buaaplay/SVCBench

SVCBench: Streaming Video Counting Benchmark

This dataset contains the clipped video segments for SVCBench, a Streaming Video Counting Benchmark for Spatial-Temporal State Maintenance. It repositions counting as a minimal, controlled probe for diagnosing how video understanding models maintain world state along the video timeline.

• —Project Page: https://buaa-colalab.github.io/SVCBench/
• —Code: https://github.com/buaa-colalab/SVCBench

Dataset Description

This repository provides 4,574 clipped video segments used by SVCBench, designed for evaluating spatial-temporal state maintenance capabilities in video understanding models.

Dataset Summary

• —Total Videos: 4,574 clips
• —Total Size: ~80 GB
• —Video Format: MP4 (H.264)
• —Categories: 8 subcategories across object counting and event counting tasks

Categories

Object Counting (2,297 clips):

• —O1-Snap: Current-state snapshot (252 clips)
• —O1-Delta: Current-state delta (98 clips)
• —O2-Unique: Global unique counting (1,869 clips)
• —O2-Gain: Windowed gain counting (78 clips)

Event Counting (2,277 clips):

• —E1-Action: Instantaneous action (1,281 clips)
• —E1-Transit: State transition (205 clips)
• —E2-Periodic: Periodic action (280 clips)
• —E2-Episode: Episodic segment (511 clips)

File Naming Convention

Multi-query clips

Format: {category}_{question_id}_{query_index}.mp4

Example: e1action_0000_00.mp4, e1action_0000_01.mp4

Single-query clips

Format: {category}_{question_id}.mp4

Example: o1delta_0007.mp4, o2gain_0000.mp4

Video Properties

• —Encoding: H.264 (using -c copy for lossless clipping)
• —Frame Rates: Preserved from source (3fps, 24fps, 25fps, 30fps, 60fps)
• —Duration Accuracy: ±0.1s from annotation timestamps
• —Quality: Original quality maintained (no re-encoding)

Source Datasets

Videos are clipped from multiple source datasets:

• —YouTube walking tours and sports videos
• —RoomTour3D (indoor navigation)
• —Ego4D (first-person view)
• —ScanNet, ScanNet++, ARKitScenes (3D indoor scenes)
• —TOMATO, CODa, OmniWorld (temporal reasoning)
• —Simulated physics videos

Usage

Loading with Python

from huggingface_hub import hf_hub_download
import cv2

# Download a specific video
video_path = hf_hub_download(
    repo_id="buaaplay/SVCBench",
    filename="e1action_0000_00.mp4",
    repo_type="dataset"
)

# Load with OpenCV
cap = cv2.VideoCapture(video_path)

Batch Download

# Install huggingface-cli
pip install huggingface_hub

# Download entire dataset
huggingface-cli download buaaplay/SVCBench --repo-type dataset --local-dir ./svcbench_videos

Annotations

For complete annotations including questions, query points, and ground truth answers, please refer to the SVCBench code repository (https://github.com/buaa-colalab/SVCBench):

• —Object counting annotations: object_count_data/*.json
• —Event counting annotations: event_counting_data/*.json

Each annotation file contains:

• —id: Question identifier
• —source_dataset: Original video source
• —video_path: Original video filename
• —question: Counting question
• —query_time or query_points: Timestamp(s) for queries
• —count: Ground truth answer(s)

Quality Validation

All videos have been validated for:

• —✓ Duration accuracy (100% within ±0.1s)
• —✓ Frame rate preservation (original fps maintained)
• —✓ No frame drops or speed changes
• —✓ Lossless clipping (no re-encoding artifacts)

Citation

If you use this dataset, please cite the SVCBench paper:

@inproceedings{liu2026svcbench,
  title     = {SVCBench: A Streaming Video Counting Benchmark for
               Spatial-Temporal State Maintenance},
  author    = {Liu, Pengyiang and Shi, Zhongyue and Hao, Hongye and Fu, Qi and
               Bi, Xueting and Zhang, Siwei and Hu, Xiaoyang and Wang, Zitian and
               Huang, Linjiang and Liu, Si},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}

Licensing

• —Annotations (parquet/JSON) and evaluation code: CC-BY-4.0
• —Self-generated physics-simulation videos: CC-BY-4.0
• —Source videos: retain their respective licenses — Ego4D (Ego4D License), ScanNet / ScanNet++ (respective Terms of Use, non-commercial research), ARKitScenes (Apple ARKitScenes license), RoomTour3D (CC-BY-NC), CODa (Apache-2.0), OmniWorld (CC-BY-NC), TOMATO (academic / non-commercial only; self-created clips CC BY-NC-SA, YouTube-sourced clips under original Creative Commons licenses), and other YouTube content (platform / creator terms). The E2-Periodic clips are loop-concatenation derivatives of TOMATO and are therefore released for non-commercial research under CC BY-NC-SA, with attribution to TOMATO and the original video creators.

By downloading this dataset, you agree to adhere to CC-BY-4.0 and to the licenses of all source datasets. If you are a rights holder with any concern about specific content, please open an issue and we will address it promptly.

Dataset Statistics

Contact

For questions or issues, please open an issue in the SVCBench repository.