tuhink/cambench_binary_eval

1---2configs:3- config_name: default4  data_files:5  - split: train6    path: data.jsonl7task_categories:8- visual-question-answering9- video-classification10language:11- en12size_categories:13- n<1K14---15 16 17# CameraBench Binary Evaluation Dataset18 19A balanced VQA dataset for evaluating camera motion understanding in videos.20 21## 📊 Dataset Statistics22 23- **Total Questions**: 38424- **Unique Videos**: 11925- **Unique Questions**: 3126- **Yes Answers**: 192 (50.0%)27- **No Answers**: 192 (50.0%)28- **Balance Ratio**: 1.0029- **Total Size**: 126.16 MB (0.12 GB)30- **Average Video Size**: 1.06 MB31 32## 🎯 Task Categories33 34This dataset covers various camera motion tasks including:35 36- **Static**: 42 questions37- **Move In**: 29 questions38- **Pan Left**: 24 questions39- **Tilt Up**: 24 questions40- **Move Out**: 21 questions41- **Move Right**: 19 questions42- **Roll Counterclockwise**: 18 questions43- **Pan Right**: 17 questions44- **Zoom Out**: 16 questions45- **Move Left**: 16 questions46- **Has Pan Left**: 15 questions47- **Roll Clockwise**: 15 questions48- **Zoom In**: 14 questions49- **Tilt Down**: 14 questions50- **Is The Fixed Camera Shaking Or Not**: 13 questions51- **Has Forward Motion**: 13 questions52- **Has Pan Right**: 12 questions53- **Is Scene Static Or Not**: 11 questions54- **Move Up**: 11 questions55- **Move Down**: 11 questions56- **Is The Camera Stable Or Shaky**: 9 questions57- **Has Truck Left**: 8 questions58- **Has Backward Motion**: 7 questions59- **Has Truck Right**: 6 questions60- **Has Forward Vs Backward Ground**: 4 questions61- **Has Zoom Out Not Move Vs Has Move Not Zoom Out**: 2 questions62- **Is Camera Movement Slow Or Fast**: 2 questions63 64## 📝 Dataset Format65 66The dataset consists of:67- `videos/`: Directory containing all MP4 video files68- `metadata.jsonl`: JSONL file with question annotations69 70Each record in `metadata.jsonl` contains:71- `video_name`: Original video filename72- `video_path`: Relative path to video file (e.g., `videos/video.mp4`)73- `question`: Binary question about camera motion74- `label`: Answer ("Yes" or "No")75- `task`: Task category76- `label_name`: Detailed label identifier77 78## 🚀 Usage79 80### Loading the Dataset81 82```python83import json84import os85 86# Load metadata87metadata = []88with open("metadata.jsonl", "r") as f:89    for line in f:90        metadata.append(json.loads(line))91 92# Access a sample93sample = metadata[0]94print(f"Question: {sample['question']}")95print(f"Answer: {sample['label']}")96print(f"Task: {sample['task']}")97print(f"Video path: {sample['video_path']}")98```99 100### Downloading the Dataset101 102Download the entire dataset using huggingface-cli or git:103 104```bash105# Using huggingface-cli106huggingface-cli download tuhink/cambench_binary_eval --repo-type dataset --local-dir ./cambench_data107 108# Or using git109git clone https://huggingface.co/datasets/tuhink/cambench_binary_eval110```111 112This will download all videos and metadata to your local machine.113 114### Loading Videos115 116```python117import json118import cv2119 120# Load metadata121with open("metadata.jsonl", "r") as f:122    metadata = [json.loads(line) for line in f]123 124# Load a video125sample = metadata[0]126video_path = sample['video_path']  # e.g., "videos/video_name.mp4"127 128# Use OpenCV to read the video129cap = cv2.VideoCapture(video_path)130while cap.isOpened():131    ret, frame = cap.read()132    if not ret:133        break134    # Process frame135    pass136cap.release()137```138 139### Batch Processing140 141For evaluation tasks:142 143```python144import json145 146# Load all questions147with open("metadata.jsonl", "r") as f:148    dataset = [json.loads(line) for line in f]149 150correct = 0151total = 0152 153for sample in dataset:154    video_path = sample['video_path']155    question = sample['question']156    ground_truth = sample['label']157    158    # Your model inference here159    # prediction = your_model(video_path, question)160    161    # if prediction == ground_truth:162    #     correct += 1163    # total += 1164 165# accuracy = correct / total if total > 0 else 0166# print(f"Accuracy: {accuracy:.2%}")167```168 169### Using with HuggingFace Datasets Library170 171```python172from datasets import load_dataset173 174# Load the dataset175dataset = load_dataset("tuhink/cambench_binary_eval")176 177# Access samples178for sample in dataset['train']:179    print(f"Question: {sample['question']}")180    print(f"Answer: {sample['label']}")181    print(f"Video: {sample['video_path']}")182```183 184## 📊 Evaluation185 186This dataset is designed for binary classification tasks. Evaluate your model using:187- Accuracy188- Precision/Recall189- F1 Score190- Per-task performance191 192## 📄 License193 194Please refer to the original CameraBench dataset for licensing information.195 196## 🙏 Citation197 198If you use this dataset, please cite the original CameraBench paper.199 200## 📧 Contact201 202For questions or issues, please open an issue on the repository.203 204---205 206**Note**: All videos are provided in original MP4 format. The dataset maintains temporal dynamics for accurate camera motion evaluation.207