Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs

• Multi-view understanding supports consistent environmental perception and reliable navigation, manipulation, and interaction, but remains underexplored in MLLMs.Source: §1 · p. 2
• The benchmark asks whether MLLMs can simultaneously understand multiple viewpoints and identifies the key challenges limiting that capability.Source: §1 · p. 2
• All-Angles Bench contains over 2,100 human-annotated question-answer pairs across 90 scenes, six geometric reasoning tasks, paired questions, and evaluations of 27 MLLMs against humans.Source: §1 · p. 2
• MLLMs struggle to identify the same object across views, while chain-of-thought prompting does not consistently improve multi-view reasoning.Source: §1 · p. 2
• Models often fail to estimate coarse camera poses, impeding relative-direction and object-manipulation performance.Source: §1 · p. 3

2. All-Angles Bench

• The benchmark integrates multiple viewpoints to test geometric understanding, object relationships, and interaction outcomes in complex real-world environments.Source: §2 · p. 3, §2.1 · p. 3

2.1. Overview of All-Angles Bench

• All-Angles Bench targets six dimensions of multi-view reasoning: counting, attributes, relative distance, relative direction, object manipulation, and camera pose estimation.Source: §2.1 · p. 3
• The benchmark contains 2,132 three-option question-answer pairs from 90 scenes sourced from Ego-Exo4D and EgoHumans, spanning varied activities and environments.Source: §2.1 · p. 3

2.2. Benchmark Collection Process

• Scenes include at least three viewpoints, with manually designed tasks covering object enumeration, attributes, spatial relationships, cross-view changes, and camera perspectives.Source: §2.2 · p. 4
• An MLLM generates initial questions, then eight human annotators remove invalid items, revise wording and options, and mark the single correct answer.Source: §2.2 · p. 4
• Paired questions rephrase queries or alter viewpoints while preserving visual correspondences; human checks yield paired counterparts for 85.3% of questions, excluding counting.Source: §2.2 · p. 4

3.1. Evaluation Setup

• The evaluation covers closed- and open-source MLLMs using standard protocols, with a 250-question human-evaluated subset for comparison.The benchmark results consolidate performance for 27 MLLMs across the reported subtasks.Source: §3.1 · p. 4, §3.2 · p. 4, Table 1 · p. 5

3.2. Results

• Human annotators reach 88.9% on camera-pose ordering, while leading MLLMs trail by over 50 percentage points and some open-source models perform below random guessing.These errors commonly involve unreconciled viewpoint transitions and misinterpreted geometric relationships.Source: §3.2 · p. 5
• Ovis2-34B and Qwen2.5-VL-72B outperform Gemini-2.0 and Claude-3.7-Sonnet on object manipulation and relative direction.The authors associate this pattern with video-focused training and fine-grained visual grounding, while noting that closed-source training strategies are unclear.Source: §3.2 · pp. 5–6

3.3. MLLM’s Robustness on Paired Questions

• Paired questions are classified as both correct, both wrong, or inconsistent when a model answers one semantically equivalent version correctly and the other incorrectly.This consistency measure tests whether isolated correctness reflects stable multi-view understanding.Source: §3.3 · p. 6
• GPT-4o reaches around 70% inconsistency on relative-distance pairs, while all six models exceed 40% inconsistency on relative-direction pairs.Gemini-2.0-Flash and Claude-3.7-Sonnet are comparatively balanced across question types, whereas Ovis2-34B and GPT-4o vary substantially.Source: §3.3 · p. 6

4.1. Failure of Multi-View Correspondence

• Models often count complete-view scenes correctly but fail when individuals are distributed across partial views, sometimes selecting the largest per-view count instead of reconciling identities.This failure reflects incorrect cross-view correspondence rather than simple per-view enumeration.Source: §4.1 · p. 6, Figure 6 · p. 7
• Identification CoT substantially improves GPT-4o under partial visibility, whereas prompting benefits diminish or can degrade performance for models already strong at multi-view counting.The strategy explicitly describes visible entities, cross-references them across views, and produces a unique-entity tally.Source: §4.1 · pp. 6–7, Figure 7 · p. 7

4.2. Failure with Coarse Camera Estimation

• The findings indicate that reasoning-prompt refinement alone is insufficient, motivating domain-specific training or architectures that emphasize viewpoint consistency.The conclusion applies across both partial-visibility correspondence failures and coarse camera-estimation failures.Source: §4.1 · p. 7, Figure 7 · p. 7, §4.2 · p. 8
• Orientation-sensitive errors arise when models misalign camera coordinates, overlook geometric background cues, and fail to maintain object correspondences across viewpoints.These errors affect camera-pose estimation as well as relative direction, trajectory tracking, and object manipulation.Source: §4.2 · p. 7, Figure 8 · p. 8
• Models can provide detailed reasoning yet fail completely when asked to order camera poses clockwise.The visualization analysis asks GPT-4o and Gemini-2.0-Flash to infer object and camera positions and orientations from multi-view images.Source: §4.2 · p. 7, Figure 9 · p. 8

Related Work

• The paper frames multi-view understanding as necessary for consistently integrating scene layout and object relationships in real-world interaction.This capability supports geometric understanding and safer anticipation of interaction outcomes.Source: §5 · p. 8
• The work provides a benchmark for assessing MLLMs’ multi-view perception and an analysis of why current models struggle with it.The authors position these contributions as relevant to fundamental capabilities for 3D and 4D tasks.Source: §5 · p. 8
• Unlike VideoMME, VSI-Bench, and MV-Bench’s primary emphases on temporal, egocentric spatial, or multimodal temporal reasoning, this work evaluates alignment across multiple viewpoints.It also provides a detailed breakdown of multi-view deficiencies.Source: §5 · p. 8

• All-Angles Bench evaluates MLLM multi-view understanding across 27 models and over 2,100 annotated question-answer pairs spanning six tasks.Source: §6 · p. 8
• The strongest limitations occur in cross-view identification and camera pose estimation.Source: §6 · p. 9
• The findings motivate domain-specific training to improve multi-view reasoning toward human-level performance.Source: §6 · p. 9

• Supplementary Section 7 provides details on the construction and annotation pipeline of All-Angles Bench.Source: Supplementary Material · p. 12
• Supplementary Section 8 reports evaluation setup, chain-of-thought implementation details, result visualizations, and complete tiny-benchmark results.Source: Supplementary Material · p. 12
• Supplementary Section 9 contains additional visualization results and prompts.Source: Supplementary Material · p. 12

7.1. Dataset Collection

• The dataset retains 4–5 spatially dispersed views from 90 scenes and standardizes all images to 796 × 448 pixels.Source: §7.1 · p. 12

7.2. Question Creation

• Questions record source and task metadata, while camera-pose items use a dedicated template instead of the general MLLM generation process.Source: §7.2 · p. 12

7.3. Human Annotation and Quality Check

• Eight annotators followed guidelines, cross-checked annotations, resolved disagreements collaboratively, and underwent random-sampling quality reviews using a GUI platform.Source: §7.3 · p. 12
• Dataset collection and processing required over 300 person-hours.Source: §7.3 · p. 12

8.1. Evaluation Setup

• The evaluation uses VLMEvalKit with greedy decoding and standardized answer formatting, followed by Qwen2.5-32B extraction of single-letter predictions.Source: §8.1 · p. 12
• Human evaluation uses two independent Ph.D. students on 250 questions, with shuffled and view-controlled inputs to reduce contextual and sequential biases.Source: §8.1 · p. 13

8.2. Implementation Details of CoT Methods

• The benchmark compares Zero-Shot CoT, Self-Consistency, and Identification CoT, with the last prompting models to list visible target entities for counting.Source: §8.2 · pp. 13–14

8.3. Evaluation Results Visualization

• Figure 23 encodes precision across six task categories using cooler colors for lower accuracy and warmer colors for higher accuracy.Source: §8.3 · p. 14

8.4. More Evaluation Results

• Human performance significantly surpasses that of all 27 MLLMs on the 250-question benchmark.Source: §8.4 · p. 14

9. Visualization Results

• The visualization section provides relevant scene views during inference while omitting views unnecessary for individual questions.This setup supports consistent scene visualization without requiring every available view for every question.Source: §9 · p. 14

9.1. Benchmark Examples

• Figures 16 and 17 show diverse multi-view image inputs with their corresponding question-and-answer pairs.The examples illustrate the benchmark’s range and complexity.Source: §9.1 · pp. 14–15

9.2. Pair Data Examples

• Figures 18 and 19 compare primary and generated paired Q&A side by side, showing their structural alignment and transformation.The comparison demonstrates how paired questions are derived from primary questions while preserving dataset consistency.Source: §9.2 · p. 15

9.3. Reasoning Examples

• Reasoning cases show that GPT-4o and Gemini-2.0-Flash can select incorrect answers, produce logically inconsistent explanations, or reason coherently when tasks align with learned knowledge.One GPT-4o answer is correct despite inconsistent reasoning, whereas both models reason correctly in a favorable case.Source: §9.3 · p. 15

9.4. Human Evaluation Failure Cases

• Human evaluator errors both underscore the challenge of the benchmark and provide evidence for reviewing annotation quality and spatial reasoning difficulty.Figure 22 examines three questions answered incorrectly by human evaluators.Source: §9.4 · p. 15

9.5. Visualization Prompt

• The supplementary prompts define benchmark-generation and evaluation procedures for counting across multi-views, identifying objects on a 10x10 grid, visualizing camera pose alignment, and extracting multiple-choice answers.They also include task-specific prompts for designing benchmark questions and generating five tasks with the MLLM.Source: §9.5 · p. 15, Figure 24 · p. 26, Figure 25 · p. 26