suijinru/MultiViewBench

MultiView-Bench

Evaluation data for MultiView-Bench: A Diagnostic Benchmark for World-Centric Multi-View Integration in VLMs (arXiv:2607.08970).

This repository currently contains the synthetic image-and-text portion of the benchmark. The real-world subset is not included; see the scope note below.

Contents

• —2,400 evaluation samples across 24 benchmark variants
• —7,900 PNG views (2.94 GiB logical image data)
• —One, three, or six images per sample
• —English question text and deterministic coordinate answer
• —Single-agent and multi-agent question schemas

The 3D Real World / exp3D_real_world_50 subset is intentionally excluded. It uses 3DCoMPaT++ source assets and is not part of this release. This package also contains no GLTF, BIN, model asset, source code, credentials, or model outputs.

Task previews

Each sample asks for a relative 3D position and must be solved by integrating the evidence across its views.

Colored-object three-view task

Question: What is the relative position of the olive cone to the indigo cube? Answer: (-X, -Y, -Z)

3D isometric task

Question: What is the relative position of the cylinder to the cone? Answer: (0X, 0Y, -Z) The full sample contains six views; three are shown here.

Full 3D reasoning (DoF=3)

Task variant: 3D DoF=3 (unconstrained placement along X, Y, and Z) Question: What is the relative position of the cube to the cylinder? Answer: (-X, +Y, +Z) The full sample contains six views; three are shown here.

Single-view 3D task

Task variant: 3D Single View Question: What is the relative position of the cube to the cylinder? Answer: (-X, +Y, +Z) Unlike the multi-view variants, this sample exposes only one isometric view.

[image]

Grid-assisted three-view task

Task variant: 2D Three Views Grids Question: What is the relative position of the cylinder to the sphere? Answer: (0X, -Y, -Z) The projected coordinate grids provide additional alignment cues in each view.

Rotated-coordinate task (90°)

Task variant: 2D Three Views Rotation (90°) Question: What is the relative position of the cube to the cone? Answer: (+X, 0Y, -Z) The coordinate frame is rotated to test whether the model follows the visible axes rather than relying on a conventional orientation.

Multi-agent three-view task

Task variant: 2D Three Views multiagent (Rotation 0) Question: What is the relative position of the sphere to the cone? Answer: (-X, -Y, +Z) Each view has its own axis-pair prompt (front_question, side_question, or top_question) so three view-specific responses can be combined into the full XYZ answer.

Load

After upload:

from datasets import load_dataset

dataset = load_dataset("suijinru/MultiViewBench", split="test")
sample = dataset[0]
print(sample["benchmark"], sample["question"], sample["answer"])
print(sample["images"])

For a local pre-upload check:

dataset = load_dataset("imagefolder", data_dir="test", split="train")

ImageFolder derives the images column from the images_file_names list in test/metadata.jsonl. The Hub recognizes columns ending in _file_names as image lists.

Fields

• —benchmark: benchmark variant identifier
• —task_variant: human-readable task variant label used to group corresponding results in the paper; it is not a row number. For the 3D variants, DoF describes the object-placement constraint: DoF=1 restricts objects to the same axis, DoF=2 restricts them to the same plane, and DoF=3 allows unconstrained placement in 3D space.
• —task_id: task identifier within the variant
• —mode: single or multi
• —question: complete prompt for single-agent samples
• —front_question, side_question, top_question: per-view prompts for multi-agent samples
• —answer: expected coordinate answer
• —images_file_names: ordered relative paths to all views
• —num_views: number of views

All rows are evaluation data; do not use them for training when reporting MultiView-Bench scores.

License

The released synthetic images, questions, answers, and metadata are available under the MIT License. Copyright (c) 2026 MultiView-Bench authors.