Datasetpublic

CircleRadon/EOC-Bench

EOC-Bench : Can MLLMs Identify, Recall, and Forecast Objects in an Egocentric World? 🔍 Overview we introduce EOC-Bench, an innovative benchmark designed to systematically evaluate object-centric embodied cognition in dynamic egocentric scenarios. Specially, EOC-Bench features 3,277 meticulously annotated QA pairs categorized into three temporal categories: Past, Present, and Future, covering 11 fine-grained evaluation dimensions and 3 visual object… See the full description on the dataset page: https://huggingface.co/datasets/CircleRadon/EOC-Bench.

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

6likes802downloads

Dataset Card

EOC-Bench : Can MLLMs Identify, Recall, and Forecast Objects in an Egocentric World?

![arXiv preprint](https://arxiv.org/abs/2506.05287) ![GitHub](https://github.com/alibaba-damo-academy/EOCBench/) ![Project Page](https://circleradon.github.io/EOCBench/) ![Learderboard](https://circleradon.github.io/EOCBench/#leaderboard)

🔍 Overview

we introduce EOC-Bench, an innovative benchmark designed to systematically evaluate object-centric embodied cognition in dynamic egocentric scenarios. Specially, EOC-Bench features 3,277 meticulously annotated QA pairs categorized into three temporal categories: Past, Present, and Future, covering 11 fine-grained evaluation dimensions and 3 visual object referencing types. To ensure thorough assessment, we develop a mixed-format human-in-the-loop annotation framework with four types of questions and design a novel multi-scale temporal accuracy metric for open-ended temporal evaluation.

📚 Tasks Definition

EOC-Bench structures questions into three temporally grounded categories: Past, Present, and Future, with a total of 11 categories.

📈 Evaluation

Please see our GitHub.