PaperlayerView exact revision on arXiv

Source-linked AI summary

Seek-and-View Reasoning for Multi-View Spatial Understanding

Qixiang Chen, Cheng Zhang, Fucai Ke, Chi-Wing Fu, Jianfei Cai, Jingwen Ye

arXiv:2610.11810v1cs.CV

TL;DR

Multi-view spatial reasoning relies on sparse, fixed views, making cross-view alignment and spatial inference challenging. Vantage pairs a VLM with a 3D foundation model to seek question-relevant views and augment reasoning with synthesized evidence. It consistently improves performance across five benchmarks and six VLMs without fine-tuning.

  • Problem

    Multi-view spatial questions require associating evidence across sparse, fixed observations, while VLM spatial reasoning remains limited and prone to hallucination.

    Source: §1 · p. 1

  • Method

    Vantage uses a VLM to analyze questions and plan a relevant viewpoint, then a 3D foundation model to synthesize evidence for final reasoning.

    Source: §1 · p. 2

  • Results

    A 6.7% mean accuracy gain: Vantage consistently boosts six VLMs across five spatial reasoning benchmarks without fine-tuning.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Seeking question-relevant visual evidence can alleviate reliance on semantic textual reasoning for global scene understanding.

    Source: §1 · pp. 1–2

    Most residual errors arise in viewpoint-grounded reasoning, where incorrect question analysis and view planning remain bottlenecks.

    Source: §4.4 · p. 9

Abstract

from arXiv · show

Existing approaches to multi-view spatial reasoning operate largely on sparse input views. Vision-language models (VLMs) are thus restricted to understand a scene and infer spatial relations within these fixed views, leading to fragile cross-view alignment and geometry-to-language bottleneck. To address these issues, we formulate a novel Seek-and-View reasoning approach to find implicit cross-view spatial evidence by locating a question-relevant view to support the spatial reasoning. To realize this approach, we propose Vantage, a training-free model-agnostic reasoning framework that pairs a VLM with a 3D foundation model: a viewpoint-grounded reasoning stage for question analysis and view planning, followed by a geometry-grounded evidence augmentation stage to effectively synthesize and incorporate visual evidence into the final reasoning. Comprehensive experiments on six VLMs demonstrate consistent improvements on five benchmarks without fine-tuning. Overall, by revealing spatial evidence through view-grounded reasoning, Vantage can largely reduce reliance on language-based cross-view alignment and improve multi-view spatial understanding. Our code is available at https://github.com/q1xiangchen/Vantage.

1 INTRODUCTION

Vantage addresses fragile cross-view alignment and the geometry-to-language bottleneck by seeking question-relevant views and pairing a VLM with a 3D foundation model. Across five benchmarks and six VLMs, it improves performance without fine-tuning.

1 INTRODUCTION

  • Existing approaches rely on VLMs to align views and reason in language, risking fragile geometric consistency and loss of fine-grained spatial information.The paper frames these as fragile cross-view alignment and a geometry-to-language bottleneck.Source: §1 · p. 1
  • Seek-and-View seeks a question-relevant viewpoint to consolidate cross-view evidence into an observation for spatial reasoning, rather than relying only on fixed views.Figure 1 contrasts reasoning over fixed views with seeking a view that makes spatial evidence directly observable.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • Vantage instantiates Seek-and-View with viewpoint-grounded reasoning followed by geometry-grounded evidence augmentation using a VLM and a 3D foundation model.The VLM analyzes the question and plans a view; the 3D foundation model synthesizes evidence for the final reasoning.Source: §1 · p. 2
  • Across five spatial reasoning benchmarks and six VLMs, Vantage consistently boosts performance without fine-tuning, with a mean accuracy gain of 6.7%.The evaluations cover multi-view spatial VQA in static real-world scenes where answers depend on geometric relations.Source: §1 · p. 2

2 RELATED WORK

Related work improves multi-view spatial reasoning through geometry-to-language methods, geometric priors, visual operations, world models, and 3D foundation models. Seek-and-View differs by acquiring a question-specific view while keeping the core VLM training-free.

2 RELATED WORK

  • Geometry-to-language methods use scene structures for downstream reasoning, while geometric-prior methods modify models, but both typically leave observations fixed.The former can require per-scene annotations and supervised fine-tuning; the latter includes geometric encoders, 3D positional encoding, and reconstructive objectives.Source: §2 · p. 3
  • Thinking-with-images methods externalize intermediate reasoning through visual operations, but address mathematical geometry and 2D spatial tasks.Examples include sketching, marking, zooming, and depth estimation.Source: §2 · p. 3
  • World-model exploration can generate unseen observations but incurs substantial inference cost and requires dedicated fine-tuning for cross-view consistency.Stitched panoramas and top-down representations instead offer generic global views rather than views tailored to a specific question.Source: §2 · p. 3
  • 3D foundation models reconstruct consistent scene representations and estimate camera poses, enabling viewpoint planning and evidence-seeking view synthesis.Unlike conventional novel-view synthesis, the sought view need not be photorealistic if it preserves the scene and reveals evidence needed to answer the question.Source: §2 · p. 3

3 METHOD

Vantage plans question-relevant views from sparse observations, then uses a 3D reconstruction to synthesize those views and augment final reasoning. Its stages connect viewpoint planning with geometry-grounded evidence and preserve the reasoning context that explains what the new view should reveal.

3.1 OVERVIEW OF SEEK-AND-VIEW REASONING PARADIGM

  • A VLM selects a reference view and plans a relative camera action from the question, while retaining analysis and guidance for interpreting the planned view.The action is grounded in an observed view rather than a directly predicted metric camera pose.Source: §3.1 · pp. 3–4, Figure 2 · p. 4

3.2 VIEWPOINT-GROUNDED REASONING

  • The VLM resolves queried directions in a semantic frame grounded in the reference view, treating its forward direction as north unless the question specifies another convention.It determines which relation remains unverified and translates that observation need into one of 17 rotation, translation, or hybrid actions at three coarse magnitudes.Source: §3.2 · p. 5
  • The VLM also generates guidance before synthesis, recording expected spatial configuration, visual anchors, and cues to inspect without assuming the expected evidence will appear.This guidance is passed to the final VQA stage to connect the planned observation with the original evidence.Source: §3.2 · p. 5

3.3 GEOMETRY-GROUNDED EVIDENCE AUGMENTATION

  • A 3D foundation model reconstructs scene geometry and camera poses from unposed views; the planned action is grounded relative to the reference pose and reprojection produces a synthesized view.Gravity alignment supplies a canonical frame for converting semantic actions into geometric camera transformations.Source: §3.3 · pp. 5–6
  • Final VQA combines the synthesized view with the original views, question-analysis context, and reasoning guidance, so incomplete synthesis supplements rather than replaces observed evidence.The three augmentation components are evaluated separately in an ablation study.Source: §3.3 · p. 6

4 EXPERIMENTS

Experiments evaluate six VLMs on multi-view spatial VQA, with additional single-view and dynamic-scene subsets. Across the reported benchmarks, Vantage improves model accuracy, while ablations and failure analysis examine the roles and limits of its components.

4.2 MAIN RESULTS

  • Vantage consistently improves average accuracy for all six VLMs across five benchmarks without additional training.Table 1 reports the main results across the five benchmarks.Source: §4.2 · p. 6, Table 1 · p. 7
  • Vantage reaches comparable or superior performance on several multi-view subsets against methods using extensive task training or substantially larger VLMs.The comparisons use a training-free, diagnostic pipeline.Source: §4.2 · p. 7
  • Gains depend on question-analysis and view-planning quality, and stronger results often pair Qwen3.6-27B analysis with planning by a smaller model.Replacing both steps with outputs from Qwen3.6-27B does not always yield the best performance.Source: §4.2 · p. 7

4.3 ABLATION STUDIES

  • Removing question analysis lowers Qwen3-VL-4B accuracy from 38.9% to 30.0% on MindCube-tiny and from 33.7% to 26.8% on MMSI-Bench.Vantage outperforms QA-skip and QA-only in most settings.Source: §4.3 · p. 8, Table 3 · p. 8
  • Across four models, 6-DoF prediction underperforms structured action selection, while random actions and interpolation provide limited gains overall.Interpolation surpasses the method only for Qwen3-VL-8B.Source: §4.3 · p. 8, Table 4 · p. 8
  • The synthesized view improves average accuracy on three benchmarks for all four models; combining it with textual context generally adds gains.The full context consistently outperforms using only question analysis and reasoning guidance.Source: §4.3 · p. 8, Table 5 · p. 8

4.4 DISCUSSION

  • Incorrect analysis and view planning account for most residual errors, making question interpretation and executable camera-action grounding the main view-seeking bottleneck.The audit covers 600 incorrect predictions across four models and three benchmarks; Figure 3 summarizes failure prevalence and representative cases.Source: §4.4 · p. 9, Figure 3 · p. 9
  • Synthesis failures are rare and occur mainly under challenging visual conditions, while final VLMs can underuse useful views or reason inconsistently.The cited conditions include reflections, transparency, severe occlusion, close-ups, darkness, blur, and low resolution.Source: §4.4 · p. 9

5 CONCLUSION

Seek-and-View uses question-relevant viewpoints to make spatial evidence across views more observable, and Vantage instantiates it by pairing a VLM with a 3D foundation model. Across five benchmarks and six VLMs, the framework consistently improves spatial reasoning, though rollout quality may vary with the underlying models.

5 CONCLUSION

  • Across five benchmarks and six VLMs, training-free, model-agnostic Vantage consistently improves spatial reasoning.The results suggest that seeking question-relevant visual evidence supports cross-view alignment and subsequent reasoning.Source: §5 · p. 9
  • Vantage instantiates Seek-and-View by pairing a VLM and a 3D foundation model in viewpoint-grounded reasoning and geometry-grounded evidence augmentation stages.The paradigm seeks question-relevant viewpoints to make spatial evidence distributed across views more directly observable.Source: §5 · p. 9
  • As a training-free framework, Vantage may produce rollouts of varying quality depending on the capabilities of its underlying VLM and 3D foundation model.The authors suggest using high-quality rollouts for future end-to-end optimization toward more reliable spatial reasoning.Source: §5 · p. 9

APPENDIX

The appendix collects additional implementation details, benchmark descriptions, and qualitative results with failure analysis.

Appendix

  • Appendix A provides additional Vantage design and implementation details, including its action set, view synthesis pipeline, gravity alignment, reconstruction backend, and prompt templates.Source: APPENDIX · p. 14
  • Appendix B describes the evaluation benchmarks and subset selection.Source: APPENDIX · p. 14
  • Appendix C presents additional qualitative results and the detailed failure analysis protocol.Source: APPENDIX · p. 14

A MORE DETAILS OF VANTAGE

This section outlines additional Vantage details, covering the action space, view synthesis, gravity alignment, and prompts.

A MORE DETAILS OF VANTAGE

A.1 ACTION SET FOR VIEW PLANNING

Vantage’s view-planning actions map to geometric transformations, with translation scaled to the spread of observed viewpoints because reconstructed point clouds lack metric scale.

A.1 ACTION SET FOR VIEW PLANNING

  • Rotation changes viewing direction; translation moves the camera by a scene-unit distance, while orbit rotates around the reconstructed point-cloud centroid.Bird’s-eye and worm’s-eye views add pitch rotations to create elevated or lowered viewpoints.Source: §A.1 · p. 14
  • Because the reconstructed point cloud is not metrically scaled, scene unit s is the mean distance from the reference camera center to the other input camera centers.This scene-adaptive scale reflects the spatial spread of the observed viewpoints, so absolute translation distances do not transfer directly across scenes.Source: §A.1 · p. 14
  • The action set maps semantic actions to geometric transformations at small, medium, and large magnitudes, except turn around, which uses a fixed 180° rotation.Table 6 lists the action set and magnitude levels; translation magnitudes use scene unit s.Source: Table 6 · p. 14

A.2 VIEW SYNTHESIS PIPELINE

The view-synthesis pipeline uses gravity-aligned reconstruction to execute relative camera actions and selects camera coverage for the planned transformation. Its prompts analyze the question, plan an action, and use the synthesized view as additional context for answering, with a fallback for format errors.

A.2 VIEW SYNTHESIS PIPELINE

  • FoV is set to preserve scene coverage: rotation and translation use 120°, while hybrid transformations adapt FoV to the reconstructed scene.The synthesized output follows the reference view’s aspect ratio; reprojection runs at twice the target width and height before downsampling.Source: §A.2 · pp. 14–15
  • Gravity alignment gives the reconstruction a ground-consistent frame for relative camera actions; without it, the realized viewpoint can deviate from the intended ground-relative motion.The action interface defines translations such as forward, backward, and lateral motion relative to the ground plane.Source: Figure 4 · p. 15, §A.2 · p. 15
  • Reducing diagonal FoV from 120° to 80° lowers accuracy by 0.7% for Qwen3.6-27B and 2.5% for Gemma-4-31B on MindCube.The upstream viewpoint-grounded reasoning is held fixed; the passage relates the effect to spatial context captured in the synthesized view.Source: Table 7 · p. 15, §A.2 · p. 15
  • G3T and VGGT-Ω with GeoCalib yield comparable downstream performance, while the latter requires approximately 1.4× the inference time.G3T directly produces gravity-aligned reconstruction and avoids the additional calibration step.Source: §A.2 · p. 15, Table 8 · p. 16

A.3 PROMPT TEMPLATES

  • Prompt 1 has the VLM identify the reference view, anchor entity, frame of reference, and observation need without answering or selecting an action.This produces the structured viewpoint analysis from the input views and question.Source: §A.3 · p. 16
  • Prompt 2 selects a relative camera action and generates guidance for the later answer, representing the action by a predefined type and magnitude level.The action is conditioned on the input views, question, and viewpoint analysis rather than specified as direct numerical camera parameters.Source: §A.3 · p. 16
  • Prompt 3 appends the synthesized view to the original views for final answering, while Prompt 4 falls back to the original views and available textual context after a format error.The available textual context comprises question analysis and, if available, reasoning guidance.Source: §A.3 · p. 16

B BENCHMARK DETAILS

This section adds benchmark details, including evaluated subsets and the criteria used to select them.

B BENCHMARK DETAILS

  • The section describes benchmark details beyond Table 1, including evaluated subsets and subset-selection criteria.The supplied passage does not specify the benchmarks or criteria themselves.Source: B BENCHMARK DETAILS · p. 16

B.1 BENCHMARK DESCRIPTIONS

The benchmark descriptions identify which spatial reasoning settings are evaluated and where each dataset extends beyond the core static multi-view setting.

B.1 BENCHMARK DESCRIPTIONS

  • MindCube-tiny evaluates spatial mental modeling from limited multi-view observations using the Rotation, Among, and Around subsets.Its questions cover object positions, orientations, and hypothetical viewpoint changes.Source: §B.1 · p. 16
  • MMSI-Bench evaluation uses static multi-view spatial reasoning subsets and excludes Motion and Multi-Step Reasoning, which introduce temporal changes or additional compositional reasoning factors.The benchmark covers relations among cameras, objects, and regions, as well as attribute, motion, and multi-step reasoning.Source: §B.1 · p. 16
  • The evaluation uses BLINK’s Multi-view Reasoning subset, which requires reasoning across observations from different viewpoints.BLINK also includes relative depth, visual correspondence, and spatial-relations tasks.Source: §B.1 · p. 16
  • Figure 5 categorizes 150 sampled incorrect predictions per model across three multi-view benchmarks into five failure types and shows each category’s share.The figure reports proportions of annotated failures, not overall prediction accuracy.Source: Figure 5 · p. 17
  • OmniSpatial evaluation uses only Perspective Taking, covering egocentric, allocentric, and hypothetical viewpoint reasoning from a single observed scene.Its other categories are not designed around multi-view observations of real-world scenes.Source: §B.1 · p. 17
  • SPINBench adds Dynamic Rotation and Dynamic Translation subsets, a limited extension testing queried entities that change pose or position while the surrounding scene largely remains static.Its broader diagnostic scope includes translation, rotation, relative pose, and viewpoint change.Source: §B.1 · p. 17

C ADDITIONAL QUALITATIVE ANALYSIS

This section supplements Vantage’s qualitative cases with a failure-analysis protocol and prompts for planning evidence-revealing views and answering questions from the input images, prior analysis, and view planning.

Qualitative Examples

Failure Analysis Protocol

  • The failure analysis annotates 600 errors across four models and three benchmarks, then weights failure-type proportions by empirical error rates before macro-averaging across benchmarks.The 50 incorrect predictions sampled per model per benchmark are categorized as incorrect analysis, incorrect view plan, unreliable synthesis, reasoning failure, or format error; Figure 5 shows per-model distributions and Figure 3 reports aggregated estimates.Source: C ADDITIONAL QUALITATIVE ANALYSIS · p. 17

Failure Case Visualizations

Prompt 1: Stage 1 - Question Analysis

  • Stage 1 analysis constructs a viewpoint-centric question representation, resolves spatial relations into the reference-view frame, and identifies missing or uncertain evidence without answering the question.The prompt makes the reference viewpoint the executable coordinate system and requires intermediate coordinate systems to be projected back into it.Source: C ADDITIONAL QUALITATIVE ANALYSIS · pp. 22–23

Prompt 2: Stage 1 - View Planning

  • Stage 1 view planning verifies the observation gap, selects a pose and move magnitude to reveal the missing evidence, and supplies spatial clues for connecting the new view to relevant input views.The narrative describes viewing geometry, observable anchors in each relevant input view, and cross-view correspondences for the downstream reasoner to combine, without stating the conclusion.Source: C ADDITIONAL QUALITATIVE ANALYSIS · pp. 24–25

Prompt 3: Stage 2 - Final VQA

  • Stage 2 answers from the original views plus one synthesized view, which may contain rendering artifacts and must be treated as supporting evidence rather than ground truth.The prompt instructs the reasoner to cross-check the synthesized view against the original views.Source: C ADDITIONAL QUALITATIVE ANALYSIS · p. 26

C ADDITIONAL QUALITATIVE ANALYSIS

  • The fallback asks the model to combine all images with its earlier question analysis and reasoning guidance to answer the original question.The prompt labels this as “Stage 2 - Fallback for Final VQA.”Source: C ADDITIONAL QUALITATIVE ANALYSIS · p. 26
Loading 2610.11810v1…