Source-linked AI summary
Reason, Then Re-reason: Cross-view Revisiting Improves Spatial Reasoning
Chaofan Ma, Zhenjie Mao, Yuhuan Yang, Fanqin Zeng, Yue Shi, Yingjie Zhou, Xiaofeng Cao, Jiangchao Yao
TL;DR
Egocentric videos limit spatial reasoning because camera trajectories leave 3D layouts and object relations underdetermined. ReRe forms a hypothesis, then verifies or revises it using synthesized complementary views, producing broad gains across spatial benchmarks and architectures.
Problem
Egocentric videos provide trajectory-limited, often ambiguous evidence, leaving 3D layouts and object relations underdetermined for single-turn spatial reasoning.
Method
ReRe separates inference into hypothesis formation from the original video and verification or revision using synthesized complementary cross-view evidence.
Results
ReRe delivers broad gains across VSI-Bench and STI-Bench architectures, including a 5.8% increase for Qwen3-VL-4B that rivals GPT-4o.
Takeaways & Limitations
Treating geometry as observable evidence enables models to validate reasoning against complementary 3D structure without architectural modifications.
Takeaways & Limitations
ReRe relies on inherently ill-posed monocular 3D reconstruction and may fall back to the original video when synthesized views are incomplete.
Abstract
from arXiv · showhide
Spatial reasoning from egocentric videos is inherently challenging because the observable evidence is constrained by the camera trajectory. Existing methods rely on single-turn inference, forcing models to resolve geometric ambiguity through semantic priors rather than verifiable evidence. We argue that spatial reasoning should be revisitable: conclusions formed under limited evidence should remain open to revision when complementary viewpoints become available. Building on this insight, we propose Reason, then Re-reason (ReRe), a training-free, inference-time framework with two phases: in the Reason Phase, an MLLM forms a spatial hypothesis from the original video; in the Re-reason Phase, it verifies or revises the hypothesis by observing a synthesized novel-view video. To enable effective cross-view revisiting, we design a Geometry-to-Video pipeline that renders strategically complementary novel views from predicted 3D geometry. These views feature an elevated, oblique perspective with scene-spanning coverage, while preserving the MLLM's native video interface without architectural modifications. Extensive evaluations on VSI-Bench and STI-Bench demonstrate that ReRe substantially boosts open-source MLLMs to rival proprietary state-of-the-art performance. Project page: https://zhenjiemao.github.io/ReRe/
1. Introduction
Egocentric-video spatial reasoning is difficult because camera trajectories provide limited, ambiguous evidence, making single-turn inference structurally fragile. ReRe addresses this by revisiting an initial hypothesis with strategically synthesized cross-view video evidence through a training-free, two-phase framework.
- Motivation: Egocentric spatial reasoning requires inferring geometric constraints, relationships, and three-dimensional layouts from evidence distributed across frames and camera motion.
- Motivation: Viewpoint-limited and potentially inconsistent egocentric evidence makes spatial reasoning challenging, while single-turn inference forces models to finalize answers despite unresolved 3D ambiguities.
- ReRe framework: ReRe treats spatial reasoning as revisitable: the Reason Phase forms an initial hypothesis from the original video, and the Re-reason Phase verifies or revises it using synthesized cross-view evidence.
- Geometry-to-Video pipeline: The Geometry-to-Video pipeline plans strategically complementary viewpoints and renders predicted 3D geometry into standard video frames consumable by frozen MLLMs.
- Evaluation: Evaluations on VSI-Bench and STI-Bench report broad gains across diverse architectures, while ablations identify revisiting and egocentric–allocentric cross-view synergy as essential.
2. Related Work
Prior work has progressed from static-image spatial understanding toward video-based reasoning and increasingly exploits monocular 3D geometry. However, existing geometry-based approaches treat geometry as latent context rather than verifiable visual evidence, motivating ReRe’s explicit cross-view hypothesis verification.
- Visual Spatial Understanding: Static-image methods ground objects and reason about spatial layout within a single frame, but fixed and partially occluded viewpoints limit spatial reasoning.Recent MLLMs perform well on static-image spatial understanding, while the field has begun shifting toward video-based understanding.
- Leveraging 3D Geometry for Spatial Reasoning: Recent monocular geometry prediction, particularly VGGT, enables high-fidelity 3D-structure recovery from 2D inputs at scale for spatial reasoning.A growing body of work builds on VGGT’s geometric representations to enhance spatial reasoning.
- Leveraging 3D Geometry for Spatial Reasoning: Existing geometry-based strategies extract latent spatial features and align them with models, but cannot verify spatial hypotheses against novel views, leaving them vulnerable to occlusion-induced hallucinations.ReRe instead uses generative geometry as observable visual evidence so MLLMs can explicitly verify reasoning training-free.
3. Methodology
ReRe reframes egocentric spatial reasoning as revisitable inference: an MLLM first forms a provisional hypothesis, then verifies or revises it using complementary synthesized views. Its Geometry-to-Video pipeline provides targeted geometric counter-evidence through a native video interface without architectural modifications.
- Revisitable Reasoning: ReRe replaces single-turn inference with two-phase reasoning that forms an intermediate hypothesis before cross-view verification.The final prediction conditions on the hypothesis, synthesized novel-view video, and query: A = P_M(A | H, Vexo, Q).
- Reason Phase: The Reason Phase produces a provisional hypothesis H = (T, Ã), comprising a thinking trace and task-formatted answer grounded in the original egocentric video.Explicit reasoning makes observations, inferences, and assumptions visible for later verification.
- Re-reason Phase: The Re-reason Phase compares synthesized allocentric evidence against the prior thinking trace, then confirms or revises spatial claims before producing the final answer.Its protocol proceeds through Compare, Reflect, and Confirm objectives.
- Geometry-to-Video Pipeline: The Geometry-to-Video pipeline plans complementary trajectories and renders temporally coherent novel-view frames from recovered 3D geometry.Figure 3 specifies VGGT point-cloud prediction, an Oblique Sweep path, and point-based rasterization.
- Design Principles: The synthesized view targets hidden spatial information by reducing occlusion and increasing coverage while remaining compatible with the MLLM’s native video interface without architectural modifications.The revisiting protocol uses the view as targeted counter-evidence that turns the second phase into a focused critique of the initial hypothesis.
4. Experiments
ReRe consistently improves spatial reasoning across open-source MLLMs and benchmarks, with gains driven by cross-view verification rather than simply adding visual information or reasoning steps. Ablations and efficiency analysis identify complementary oblique views as effective, while 3D reconstruction dominates inference cost.
- VSI-Bench: ReRe boosts diverse open-source architectures on VSI-Bench, including a 5.8% gain for Qwen3-VL-4B and parity with proprietary GPT-4o.Improvements primarily arise in configurational reasoning and measurement estimation.
- STI-Bench: ReRe generalizes to STI-Bench, improving Qwen3-VL-2B by 8.0 on average and by 18.5 in Spatial Relation reasoning.InternVL2.5-8B reaches 34.8 on average, surpassing proprietary GPT-4o at 31.0.
- Effectiveness of Revisiting Protocol: Sequential revisiting outperforms Baseline, Concat, and Interleaved, showing that revisiting—not merely combining views—drives performance.Concat disrupts temporal coherence, whereas Interleaved provides only marginal gains.
- Deconstructing the Re-Reasoning Components: Re-reasoning on Vego alone degrades performance, while Vexo alone also underperforms the full pipeline, establishing the need for both new evidence and original visual detail.The findings attribute gains to cross-view verification rather than increased reasoning depth on unchanged observations.
- Impact of Allocentric Trajectories: Oblique Sweep performs best because its elevated angle exposes hidden layouts while preserving canonical viewpoints; Mid-level Traverse leaves occlusions unresolved and Bird’s-eye Orbit causes viewpoint mismatch.All trajectory variants use the same Geometry-to-Video pipeline and have identical rendering cost.
- Analysis and Efficiency: 71.6% of changed samples are positive flips versus 28.4% negative flips, a 2.52:1 ratio, while the full pipeline runs in ∼11 s per sample.VGGT accounts for ∼9 s, rendering takes < 1 s, and the second MLLM call uses the same 8-frame budget.
5. Limitations and Discussion
ReRe tolerates imperfect monocular 3D geometry through multi-frame reconstruction, confidence filtering, and original-video fallback, while adding roughly 10 seconds per sample. Its modular design can benefit from improved 3D priors and reduced reconstruction costs.
- Robustness to Imperfect Geometry: ReRe mitigates imperfect monocular geometry by fusing the full video into a unified point cloud and filtering low-confidence points before rendering.Uncertain regions are left blank rather than fabricated.
- Robustness to Imperfect Geometry: Vego remains the semantic anchor, while Vexo provides cross-view verification and allows fallback to the original video when novel views are incomplete.The passage reports positive gains across diverse backbones and benchmarks.
- Computational Considerations and Future Acceleration: ReRe’s modular design can inherit efficiency gains from advancing 3D priors, while future priors with more accurate geometry and confidence estimates may further reduce limitations.The framework is described as agnostic to the underlying 3D backbone.
- Computational Considerations and Future Acceleration: ∼10 s of additional latency per sample is incurred over single-turn inference on an A100, with most cost coming from 3D reconstruction.Reducing VGGT input to 20 frames brings total latency to ∼4.
6. Conclusion
ReRe addresses viewpoint limitations in egocentric spatial reasoning by making inference revisitable through hypothesis formation and verification, using synthesized complementary allocentric views grounded in 3D structure.
- Conclusion: ReRe reformulates egocentric spatial inference as a revisitable process of hypothesis formation and verification.It is presented as a training-free framework.
- Conclusion: Its Geometry-to-Video pipeline synthesizes complementary allocentric views to provide additional spatial evidence.These views are used to address inherent viewpoint limitations.
- Conclusion: ReRe enables models to validate reasoning against 3D structural evidence without architectural modifications.The framework performs this validation through synthesized views rather than changes to the model architecture.
Impact Statement
The paper advances video-based spatial reasoning for MLLMs through ReRe, aiming to improve reliable and accurate spatial perception in egocentric scenarios. This could benefit embodied AI agents, robotics, and assistive technologies requiring robust environmental understanding.
- Impact Statement: ReRe targets more reliable and accurate spatial perception for MLLMs in egocentric video scenarios.The work is positioned as advancing machine learning in video-based spatial reasoning.
- Impact Statement: The framework could support embodied AI agents, robotics, and assistive technologies that require robust environmental understanding.
A. Qualitative Results
Figure 5 qualitatively shows that ReRe corrects erroneous initial spatial judgments by using synthesized geometric evidence to resolve ambiguities from incomplete egocentric observations.
- Qualitative Results: ReRe’s Re-reason Phase uses newly synthesized geometric evidence to correct erroneous initial judgments in four representative cases.The cases illustrate resolution of spatial ambiguities caused by incomplete egocentric observations.
- Qualitative Results: For object counting, novel views reveal a second monitor on the same desk that was not initially observed.This example appears in Figure 5(a).
- Qualitative Results: For object counting, synthesized views reveal a second bed outside the original visible region.This example appears in Figure 5(b).
B. Prompt Template · Prompt Example for ReRe
The ReRe Reason Phase prompts the model to form a hypothesis by carefully observing the video, inferring a plausible answer despite incomplete visual information, and concluding with a final answer. It also encourages internal self-reflection and verification before presenting reasoning and the answer in specified tags.
- Reason Phase: Hypothesis Formation: The Reason Phase frames the task as hypothesis formation from the input video.
- Reason Phase: Hypothesis Formation: The prompt instructs the model to observe key visual elements, infer a plausible answer despite incomplete information, and conclude with a final answer.
- Reason Phase: Hypothesis Formation: The model is encouraged to use natural-language self-reflection and verification expressions during its reasoning process.
- Reason Phase: Hypothesis Formation: The prompt requires detailed reasoning between <think> and </think> tags, followed by the final answer between <answer> and </answer> tags.
Re-Reason Phase: Cross-View Verification
The Re-Reason Phase gives the model its prior reasoning as a baseline, then presents a VGGT-reconstructed new-view video to support cross-view comparison and verification. The model reflects on whether its original conclusion holds, while preserving the first-round answer for temporal-order questions and outputting one final answer in the required format.
- Cross-View Verification: The model receives its previous reasoning and answer as a baseline before analyzing the reconstructed new-view video.The new view depicts the same scene and addresses errors caused by limited viewpoints.
- Cross-View Verification: It compares the old and new views, reflects on the prior conclusion, and confirms a final answer.For temporal-order questions, the model should primarily maintain its first-round answer.
- Output Constraints: The response must place step-by-step reasoning and one specific final answer inside <think> and <answer> tags.No text may appear outside the tags, and vague terms such as “around” or “approximately” are disallowed.
- Output Constraints: Task-specific templates constrain the answer content, including a single option letter or exactly one numeric value when required.Numeric answers may contain only digits and an optional decimal point inside the <answer> tags.