PaperlayerView exact revision on arXiv

Source-linked AI summary

Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Views

Jiho Choi, Seonho Lee, Seojeong Park, Hyunjung Shim

arXiv:2606.23557v1cs.CV

TL;DR

MV3D-VQA challenges MLLMs to integrate partial views into coherent 3D reasoning, yet sparse answer-level supervision leaves cross-view consistency and view selection brittle. DR-MV3D uses global and egocentric maps, question-conditioned trajectories, and dense VFM-grounded rewards optimized with GRPO. It consistently improves across MindCube, VSI-Bench, and BLINK (MV), while remaining limited by its static-scene assumption.

  • Problem

    Existing MLLMs struggle to construct coherent 3D representations and maintain consistent spatial reasoning from partially observable multi-view inputs.

  • Method

    DR-MV3D combines allocentric map construction, question-conditioned view trajectories, egocentric grounding, and dense global and local rewards optimized through trajectory-level GRPO.

  • Results

    DR-MV3D improves across MindCube, VSI-Bench, and BLINK (MV), reaching 66.5% accuracy on MindCube-Tiny, 37.1 Avg on VSI-Bench, and 56.4 on BLINK (MV).

  • Takeaways & Limitations

    Dense structural supervision provides a more effective learning signal for multi-step spatial reasoning than standard answer-level optimization.

  • Takeaways & Limitations

    The framework focuses on static scenes and can misinterpret independently moving foreground objects as camera motion.

Abstract

from arXiv · show

Multi-view 3D Visual Question Answering (MV3D-VQA) requires integrating partial observations into a coherent 3D scene representation and selecting informative viewpoints for multi-step spatial reasoning. However, current multimodal LLMs are typically trained with sparse, answer-level supervision, which often yields inconsistent cross-view reasoning and brittle view selection. We present DR-MV3D (Dense Reward for MV3D-VQA), a map-grounded learning framework that provides dense, verifiable rewards to supervise the reasoning process. Our approach decomposes MV3D-VQA into (i) allocentric global map construction, (ii) question-conditioned view-trajectory planning, and (iii) egocentric grounding for answer prediction. To make intermediate steps learnable without manual annotations, we introduce two rewards: a global consistency reward that aligns the predicted map with geometry-consistent pseudo targets from frozen 3D vision foundation models (e.g., VGGT + SAM3), and a local trajectory reward that supervises ordered viewpoint selection. We optimize the full pipeline with trajectory-level policy optimization (GRPO). Experiments on MindCube, VSI-Bench, and BLINK (MV) show that DR-MV3D consistently improves over strong multi-image baselines, supporting the effectiveness of process-level dense supervision for multi-view 3D reasoning.

1 Introduction

MV3D-VQA requires coherent 3D scene representations from partial views, but existing MLLMs often struggle with consistent spatial reasoning. DR-MV3D addresses this with map-grounded representations and dense supervision for map construction and viewpoint selection.

  • Motivation: MV3D-VQA integrates fragmented observations across viewpoints to reason about 3D spatial relationships.This capability supports embodied and physical intelligence under partial observability.
  • Challenges: Existing MLLMs often produce inconsistent cross-view predictions, unstable reasoning under occlusion, and near-random performance on compositional spatial queries.These failures point to difficulties maintaining a coherent scene-level representation, rather than merely lacking visual inputs.
  • Prior Direction: Prior work finds that constructing an intermediate cognitive map before reasoning is more effective than simply adding views.The map acts as a persistent spatial workspace organizing fragmented observations into scene-level structure.
  • DR-MV3D: DR-MV3D combines a global allocentric cognitive map, viewpoint-aligned egocentric grounding, question-conditioned trajectory planning, and trajectory-level policy optimization.A 3D vision foundation model provides a structural prior for geometrically consistent map construction.
  • Dense Supervision: VGGT+SAM3-derived cognitive maps provide geometry-consistent pseudo-structural targets without requiring manual annotations.The signals softly align language-model maps with physically plausible 3D layouts.
  • Results: 66.5% accuracy on MindCube-Tiny represents a 28.7%p improvement over Qwen2.5-3B, alongside 37.1 Avg on VSI-Bench and 56.4 on BLINK (MV).The reported results span multiple multi-view benchmarks.

2 Related Work

Related work improves multi-view 3D reasoning through explicit 3D representations, reasoning supervision, and verifiable-reward reinforcement learning. These directions motivate dense, intermediate supervision for multimodal reasoning.

  • 3D Understanding and Spatial Reasoning: Recent benchmarks show MLLMs remain brittle in 3D understanding under partial observability and multi-view evidence.Recurring issues include viewpoint consistency, relative spatial relations, and spatial memory.
  • 3D Understanding and Spatial Reasoning: Prior methods inject 3D-aware representations or improve reasoning behavior through explicit traces, perspective-aware imagery, and multi-reward reinforcement learning.These approaches target complementary aspects of geometric representation and reasoning behavior.
  • Verifiable Rewards and GRPO: Verifiable-reward methods replace human preference supervision with objective signals from exact-match checks, rule-based constraints, or metric-driven evaluators.GRPO estimates baselines from group-wise samples without an explicit value or critic model.
  • Verifiable Rewards and GRPO: Recent vision-language extensions apply step-wise, vision-guided, or metric-driven reinforcement learning to intermediate reasoning and grounded perception outputs.Examples include R1-VL, Vision-R1, and Visual-RFT.
  • Verifiable Rewards and GRPO: Related work also explores curriculum RL, two-stage rule-based RL, iterative self-improvement, cross-modal formalization, and reasoning-aware training for grounded outputs.These efforts broaden verifiable-reward learning and multimodal grounding beyond the core settings above.

3 Proposed Method

DR-MV3D formulates MV3D-VQA as trajectory-level reasoning: it builds a global allocentric map, selects question-conditioned views, grounds them egocentrically, and predicts an answer under dense multi-level rewards.

  • Problem Formulation: MV3D-VQA is formulated as trajectory-level policy optimization over multi-view observations and a spatial question, with the trajectory ending in answer prediction.The trajectory includes map construction, sequential view selection, egocentric map generation, and final answering.
  • Overall Framework: The pipeline constructs a global allocentric cognitive map, plans an ordered question-conditioned view trajectory, converts selected evidence into egocentric maps, and predicts the final answer.Allocentric mapping provides a stable world-centered reference, while egocentric grounding aligns evidence with selected viewpoints.
  • Global Map Construction: The global reward structurally aligns the predicted allocentric map with a geometry-consistent pseudo target generated by frozen VFMs such as VGGT and SAM3.This supervision anchors the learned map to a physically plausible 3D structure without manual annotations.
  • View-Trajectory Reasoning: The local reward supervises ordered viewpoint selection by measuring step-wise matches between the predicted trajectory and a reference trajectory derived from benchmark metadata.The policy samples each viewpoint autoregressively conditioned on prior selections, the global map, and the question.
  • Reward Design and Policy Optimization: The dense trajectory reward combines global-map, local-trajectory, answer-correctness, and format terms, with GRPO optimizing the MLLM policy using group-relative advantages.Answer reward uses exact final-answer matching, while format reward checks validity of the structured output.

4 Experiments

Experiments evaluate DR-MV3D on MindCube, VSI-Bench, and BLINK (MV), with ablations showing benefits from combining intermediate supervision and dense rewards. The method improves multi-view spatial reasoning and sequential route planning over strong baselines.

  • Evaluation setup: Experiments use MindCube, VSI-Bench, and BLINK (MV), following accuracy or per-category average-score protocols.MindCube evaluates view-dependent 3D spatial reasoning; VSI-Bench evaluates route planning and compositional spatial relations; BLINK (MV) evaluates distributed multi-view reasoning.
  • VSI-Bench: 37.1 Avg on VSI-Bench is the best score among compared methods, with 32.4 Route Plan, 46.6 Rel. Dir., and 37.8 Rel. Dist.SFT reaches 34.2 Avg from the 30.4 Qwen2.5-VL-3B-Instruct baseline, while GRPO reaches 37.1.
  • BLINK (MV): 56.4 on BLINK (MV) improves over the Qwen2.5-VL-3B baseline at 42.1 and exceeds RoBoBrain at 55.6.The gain is obtained with the same 3B backbone and also improves over Think3D at 53.4.
  • Ablations: 62.4 overall is achieved when allocentric-map, egocentric-map, and trajectory supervision are combined during SFT.Trajectory-only supervision reaches 58.2, while supervising both maps reaches 53.6, indicating complementary but insufficient map supervision alone.
  • Ablations: 66.5 overall is achieved with both global and local rewards, compared with 63.8 using only answer-and-format reward and 64.9 with global reward added.The ablation starts from an SFT model that already produces valid answers and formats.

5 Conclusion

The conclusion presents DR-MV3D as a dense-reward framework for MV3D-VQA under partial observability. It combines global scene mapping, question-conditioned viewpoint trajectories, and egocentric reasoning, with consistent gains across three benchmarks.

  • 5 Conclusion: DR-MV3D formulates MV3D-VQA as trajectory-level policy optimization under partial observability.The pipeline constructs an allocentric cognitive map, selects informative viewpoint trajectories, and aligns observations into egocentric maps for reasoning and answer prediction.
  • 5 Conclusion: Dense rewards supervise cross-view consistency and viewpoint selection using pseudo-structural signals from vision foundation models.These rewards make intermediate reasoning processes learnable without relying only on answer-level outcomes.
  • 5 Conclusion: Experiments on MindCube, VSI-Bench, and BLINK show consistent gains over strong supervised baselines.The conclusion highlights process-level supervision as valuable for 3D reasoning.

S1 Overview

The supplementary material expands the proposed map-grounded framework with implementation, data, optimization, evaluation, analysis, and qualitative-example details.

  • S1 Overview: The supplement describes the structured reasoning pipeline, supervised training-data construction, and SFT and GRPO implementation and optimization details.It also covers dataset-specific evaluation protocols.
  • S1 Overview: Additional analyses and qualitative examples complement the main-paper results.

S2 Limitations & Future Work

The framework is evaluated on static scenes; extending it to dynamic environments remains challenging because temporal correspondence, motion, and scene changes must be handled.

  • Dynamic environments such as video and 4D scenes remain outside the static-scene focus and pose challenges involving temporal correspondence, motion, and scene changes.

S3 Method Details

The method represents multi-view scenes with a lightweight allocentric cognitive map and computes a structural similarity reward that prioritizes relative geometry and viewpoint orientation over exact coordinates.

  • The allocentric cognitive map is a discretized 10 × 10 bird’s-eye-view abstraction in a world-centered reference frame.It integrates multi-view observations into a unified coarse spatial representation.
  • Each map serializes salient objects and camera viewpoints as structured entries containing names, grid positions, and available facing attributes.The representation is designed to remain lightweight and interpretable for MLLM generation.
  • The map serves as a global intermediate representation for subsequent local trajectory planning and egocentric grounding.
  • The global reward compares a predicted allocentric map with a pseudo target using a structural similarity function.The objective evaluates preserved spatial structure rather than exact absolute placement on the grid.
  • Similarity combines directional and facing components as α_sim s_dir + (1-α_sim) s_face, with both scores bounded between 0 and 1.α_sim balances directional similarity against viewpoint-facing similarity.
  • Directional similarity compares coarse pairwise object relations, while facing similarity checks discrete orientations for valid viewpoints.The resulting reward is invariant to global translation and tolerant of small coordinate perturbations when relative layout is preserved.

S4 Implementation Details

Implementation combines programmatically generated structured supervision with SFT and GRPO, using VFM-derived cognitive-map targets and dense rewards over the full reasoning trajectory.

  • The framework uses Qwen2.5-VL as its multimodal backbone and extends the official MindCube codebases.The default maximum sequence length is 8,192 tokens.
  • Each training instance contains multi-view images, a question, and a unified autoregressive target with CogMap, EgoMap, reasoning, and answer blocks.These blocks jointly train intermediate spatial representations and final answer prediction.
  • SFT targets are generated deterministically from benchmark metadata through predefined templates and rule-based transformations rather than LLM-generated rationales.
  • The pipeline constructs a world-centered allocentric map, derives a question-conditioned egocentric map, and generates a grounded reasoning chain before serializing the complete target.
  • The SFT corpus contains 10,000 training examples with 2 to 6 input images, alongside a held-out split of 1,050 examples.
  • Pseudo cognitive-map targets use frozen 3D vision foundation models, including VGGT and SAM3, followed by a coarse scene-to-map transformation.The pseudo map is intended as stable, verifiable proxy supervision rather than perfect annotation.
  • After SFT, GRPO optimizes a trajectory-level reward combining global-map, local-trajectory, exact-answer, and format components.

S5 Dataset and Evaluation Details

Evaluation covers MindCube, VSI-Bench, and BLINK (MV), spanning viewpoint-based spatial reasoning, video-based spatial intelligence, and multi-view visual perception.

  • The study reports accuracy on MindCube and BLINK (MV), and category-wise scores plus an overall average on VSI-Bench.
  • MindCube tests partial-observation spatial reasoning across Rotation, Among, and Around camera-movement settings.Its questions include viewpoint-dependent relations, perspective taking, and hypothetical motion.
  • VSI-Bench uses real indoor-scene videos with object-level 3D annotations across tasks including counting, distance, direction, routing, size, room size, and appearance order.The protocol reports category-wise results and an overall average.
  • For VSI-Bench, sampled video frames are handled by prompting the model to construct an egocentric map before applying coordinate-based reasoning.This adapts the pipeline to differences in question format and visual input distribution.
  • BLINK (MV) evaluates whether models infer camera or scene changes from multiple images under partial and distributed visual evidence using exact-match accuracy.

S6 Additional Experimental Results

Additional experiments examine qualitative reasoning, training dynamics, egocentric grounding, and failure cases across the evaluated benchmarks. The results show complementary supervision, stable GRPO optimization, benefits from egocentric representations, and reduced reliability in dynamic scenes.

  • Additional Qualitative Results: Qualitative examples show grounded reasoning across MindCube-Tiny and BLINK (MV), including allocentric scene understanding, egocentric grounding, and camera-motion resolution.The BLINK (MV) example contrasts incorrect frozen-model reasoning with correct reference-view selection and ego-consistent reasoning.
  • Training Plot and Reward Plot Analysis: GRPO rewards increase and then stabilize under full, without Rlocal, and without both Rglobal and Rlocal settings.The comparison uses EMA-smoothed mean rewards over 200 training steps because raw step-wise rewards are noisy.
  • Effects of Egocentric Grounding: Egocentric grounding raises the MindCube-Tiny overall score from 41.43 with CogMap input to 45.52 with EgoMap input and 52.19 with both maps.Allocentric-only variants remain nearly identical at 41.33 and 41.43, whereas adding egocentric representations improves viewpoint-dependent reasoning.
  • Failure Cases: The cognitive-map approach is less reliable when independently moving foreground objects create ambiguity between object motion and camera motion.In the cited white-car example, the model attributes object motion to forward camera motion despite the camera moving backward.
Loading 2606.23557v1…