PaperlayerView exact revision on arXiv

Source-linked AI summary

MV-STRIDE: Enabling MLLMs to Master Multi-View Spatial Reasoning via Hierarchical Capability Modeling

Jin Xu, Xiaojian Huang, Zhuodong Luo, Zhihong Zhang, Xin Liu, Jiansheng Wei, Xinzhi Wang, Jie Zhao, Xuejin Chen

arXiv:2609.07258v1cs.CVcs.AI

TL;DR

MLLMs still struggle with robust multi-view spatial reasoning, while existing datasets often lack structured capability dependencies. MV-STRIDE builds a hierarchical dataset and training framework; its multi-stage training reports improvements across benchmarks, including a leading open-source result on MMSI-Bench.

  • Problem

    Existing spatial datasets often lack structured dependencies linking perception, scene understanding, and high-level multi-view reasoning, while MLLMs struggle to maintain cross-view correspondences and geometric consistency.

    Source: §1 · p. 2

  • Method

    MV-STRIDE organizes spatial tasks into three dependency-linked levels and generates cross-view-constrained QA groups with multi-stage training and CoT supervision.

    Source: §1 · pp. 2–3, §3 · p. 5, Level III: Multi-View High-Level… · p. 6

  • Results

    MV-STRIDE-SFT(Full) achieves 38.9% average accuracy on MMSI-Bench, 9.7% above the Base Model, and the paper reports state-of-the-art performance across multiple spatial reasoning benchmarks.

    Source: Quantitative Results on Spatial… · p. 10, §5 · p. 15

  • Takeaways & Limitations

    The evaluations support a hierarchical learning pathway for multi-view spatial reasoning, with the conclusion describing RL-refined models as developing structured, interpretable reasoning chains.

    Source: §5 · p. 15

Abstract

from arXiv · show

Despite the rapid progress of Multimodal Large Language Models (MLLMs) in 2D vision-language tasks, robust multi-view spatial reasoning remains a fundamental bottleneck due to the lack of structured 3D cognitive pathways in existing datasets. To address this, we introduce MV-STRIDE, a Multi-View hierarchical SpaTial Reasoning dataset with Interdependent and DEcomposed capabilitiEs. Moving beyond flat data structures, MV-STRIDE explicitly models the dependency relationships between foundational perception, scene understanding, and complex contextual reasoning, providing a coherent learning pathway aligned with human spatial cognition. We develop a systematic QA generation pipeline leveraging diverse 3D scene sources that enforces cross-view dependency constraints to prevent single-view solvability, generating multi-level spatial reasoning tasks supported by cognitively grounded chain-of-thought supervision for complex inference. Extensive evaluations demonstrate that our multi-stage training framework based on our hierarchical dataset achieves state-of-the-art performance across multiple spatial reasoning benchmarks, notably the multi-view oriented MMSI-Bench. Our approach enables MLLMs to maintain robust, 3D-consistent spatial reasoning across diverse viewpoints. The code and dataset are available at https://co1dspring.github.io/MV-STRIDE/.

1 Introduction

MV-STRIDE addresses multi-view spatial reasoning by organizing capabilities into a dependency-aware hierarchy and using that structure for training. The paper reports consistent benchmark improvements from multi-stage training.

1 Introduction

  • Existing spatial datasets often lack dependency-structured pathways from foundational perception to high-level multi-view reasoning [5] [25] [26] [39].Some multi-view datasets emphasize low-level perception, while tasks are often designed independently.Source: §1 · p. 2
  • MV-STRIDE organizes single-view perception, cross-view scene understanding, and complex contextual reasoning through explicit capability dependencies.The hierarchy is intended to provide a coherent learning pathway aligned with human cognition.Source: §1 · pp. 2–3
  • Its generation pipeline constructs Level III tasks under cross-view constraints, then decomposes them into prerequisite sub-questions and CoT annotations.The resulting structure supports training from supervised fine-tuning to reinforcement learning.Source: §1 · p. 3
  • Multi-stage training with MV-STRIDE yields consistent improvements across multiple spatial reasoning benchmarks.The paper also reports state-of-the-art performance across multiple benchmarks, particularly in multi-view settings.Source: §1 · p. 3

2 Related Work

Prior spatial datasets provide large-scale or accurate supervision, but the paper identifies a shortage of structured pathways for complex, multi-view spatial reasoning.

2.1 Multimodal Spatial Reasoning Dataset.

  • Recent datasets use pseudo-labeling or precise 3D ground truth from real-world reconstructions and simulated environments, but lack structured pathways for complex geometric consistency [7,9,6,51,52,34,46,11,48,13].These efforts have scaled single-view reasoning and provide supervision across spatial tasks.Source: §2.1 · pp. 3–4

2.2 Multimodal Multi-view Spatial Reasoning Dataset.

  • Multi-view datasets cover low-level perception or isolated capabilities, whereas MV-STRIDE models dependencies from foundational perception to high-level contextual reasoning [11] [34] [46] [51] [44] [50] [43] [24].The paper frames this hierarchy as a route to 3D-consistent spatial inference.Source: §2.2 · p. 4

3 Method

The method builds a cognition-driven, three-level spatial reasoning dataset and uses its dependencies to structure data generation and progressive model training.

3 Method

  • The method organizes spatial tasks around a “Perception, Modeling, Reasoning” hierarchy with explicit capability dependencies.The hierarchy is presented as a progression from spatial perception through scene-level understanding to contextual reasoning.Source: §3 · p. 5

3.1 Hierarchical Definition of Spatial Reasoning Capabilities

3.2 Data Construction Pipeline

  • The pipeline combines procedural scenes and real-world reconstructions, selecting camera pairs and visible, identifiable objects before QA generation.Camera pairs are chosen to have reasonable spatial separation and shared objects as spatial anchors.Source: Scene Assets and Preprocessing · p. 7, Viewpoint and Object Filtering · p. 7
  • Level III questions are generated first from filtered views and objects, then paired with Level I and II sub-questions using structured metadata and templates.Level I and II tasks use short answers, while Level III uses multiple choice; sampled verification reports 88.0% QA-pair accuracy and 97.5% CoT-annotation accuracy.Source: Hierarchical QA Generation · p. 8, Construction of Long-Chain Reasoning… · pp. 8–9
  • For Level III tasks, prerequisite answers are sequenced and synthesized into natural-language CoT chains that follow perception, cross-view integration, and contextual inference.The authors describe these annotations as transparent and interpretable supervision for long-chain reasoning.Source: Construction of Long-Chain Reasoning… · p. 8

3.3 Multi-Stage Training Strategy.

  • Training progresses from spatial SFT to CoT cold-start training and then GRPO reinforcement learning on samples selected for moderate difficulty.The stages use 60% of scene data for SFT, 20% of ScanNet++ scenes for CoT cold-start, and remaining data for RL sample selection.Source: §3.3 · p. 9

4 Experiments

The experiments evaluate MV-STRIDE across spatial reasoning benchmarks and compare training stages, data sources, and reasoning outputs. The results report strong benchmark performance and examine how hierarchical data and multi-stage training relate to accuracy and structured reasoning.

4.1 Experimental Setup

  • Experiments use Qwen3-VL-8B-Instruct with large-scale SFT, CoT cold-start training, and GRPO reinforcement learning across four benchmarks and varied MLLM baselines.The benchmarks cover foundational single-view perception, multi-view localization and reasoning, and 3D-aware inference.Source: Implementation Details · pp. 9–10, Benchmarks · p. 10, Baselines · p. 10

4.2 Main Results

  • 38.9% average accuracy on MMSI-Bench gives MV-STRIDE-SFT(Full) open-source MLLM SOTA and a 9.7% improvement over the Base Model.The result is reported alongside comparisons with larger open-source models and leading proprietary MLLMs.Source: Quantitative Results on Spatial… · p. 10, Table 1 · p. 11
  • MV-STRIDE-SFT(Stage 1) achieves the overall best ViewSpatial-Bench performance and top open-source 3DSRBench performance, while MV-STRIDE-SFT(Full) leads on CV-Bench.The paper also reports consistent improvements across benchmarks for models with different parameter scales and architectures.Source: Quantitative Results on Spatial… · p. 11
  • Fig. 3 examples show MV-STRIDE-RL reasoning through perception, cross-view 3D modeling, and scene reasoning, including camera estimates of a 0.85m shift and 30.5° rotation.The paper describes the chains as more structured and interpretable, while noting that RL does not always surpass other variants in quantitative benchmarks.Source: Qualitative Analysis of 3D-Consistent… · pp. 11–12, Figure 3 · p. 12

4.3 Ablation Studies

  • Adding MV-STRIDE produces a substantial performance surge, whereas open-source auxiliary data alone yields marginal gains across most benchmarks and almost negligible gains on MMSI-Bench.The ablation compares Stage 1 training data combinations; the paper characterizes MV-STRIDE as the primary driver of gains, especially for multi-view scenarios.Source: Figure 4 · p. 12, Effectiveness of MV-STRIDE · p. 13
  • Removing Stage 1 leads to RL convergence failure on high-order tasks, omitting Stage 2 degrades CV-Bench and 3DSRBench, and omitting Stage 3 leaves cold-start training imitating SFT templates.The paper says RL supports autonomous verification of spatial logic and perspective consistency.Source: Figure 4 · p. 12, Effect of Progressive Multi-Stage… · p. 13
  • CoT-style supervision may lower some direct-answer scores; RL benefits multi-view benchmarks, while pooled SFT can score higher on MMSI-Bench through stronger pattern matching.The authors describe the score shift from short answers to CoT outputs as a “reasoning tax” and say staged RL derives reasoning rather than passively imitating it.Source: Effect of Progressive Multi-Stage… · pp. 13–14
  • With equal quantities of Level III data, hybrid synthetic and real-world sources yield optimal performance, combining precise synthetic ground truth with real-world complexity.The paper notes synthetic data’s diversity is limited by procedural algorithms and its fidelity gap relative to real-world scenes.Source: Impact of Synthetic vs. Real-world… · p. 14, Table 2 · p. 14

5 Conclusion

MV-STRIDE organizes spatial reasoning data around dependencies from perception to complex 3D reasoning. The paper concludes that reinforcement learning, structured reasoning chains, and combined synthetic and real-world data are central to its approach.

5 Conclusion

  • MV-STRIDE combines hierarchical capability dependencies, reinforcement learning, interpretable reasoning chains, and synthetic–real data synergy to support multi-view spatial cognition in MLLMs.The conclusion contrasts RL-refined structured reasoning with rote memorization and highlights geometric accuracy alongside robustness.Source: §5 · p. 15

Table of Contents (Supplementary Material)

The supplementary material contents list appendices on implementation, experiments, dataset construction, dataset examples, and hierarchical reasoning data.

Table of Contents (Supplementary Material)

  • The supplementary material includes appendices on training details, extended results, dataset specifications and examples, and hierarchical CoT and RL data construction.The listed appendices are A through E.Source: Table of Contents (Supplementary… · p. 19

A Implementation and Training Details

This section documents the three-stage training and inference setup, then reports reasoning-chain comparisons and generalization across scenario types.

A.1 Hyperparameters

  • Stage 3 uses GRPO to refine long-chain spatial reasoning, with group size 8, temperature 1.0, and KL penalty coefficient 0.001.The reward combines terminal-answer accuracy, output-format consistency, and penalties for repetitive or overly long reasoning.Source: §A.1 · pp. 19–20

A.2 Inference Setting

  • Cold-start and RL models receive Chain-of-Thought prompts, while the Stage 1 SFT model and most baselines receive direct-answer prompts.The reasoning prompt requests thinking in <think> tags and the final choice in <answer> tags.Source: §A.2 · p. 20, Inference Prompt for Reasoning… · p. 20

B.1 Detailed Results on Other Benchmarks

  • Beyond the MMSI-Bench per-category results in Tab. 1, detailed per-category performance metrics for the other three benchmarks are provided in Tabs. B.2 to B.4.Source: §B.1 · p. 20

B.2 Ablation of Multiple Stages

  • Table B.5 provides detailed performance by training stage and ablations of the multi-stage strategy to complement Figure 4b.Source: §B.2 · p. 20

B.3 Reasoning Chain Comparisons

  • MV-STRIDE-RL produces more flexible, coherent step-by-step reasoning that consistently leads to accurate answers than MV-STRIDE-ColdStart.ColdStart tends to imitate training-data patterns and can make intermediate perception or logic errors.Source: §B.3 · p. 20

B.4 Generalization across scenario types

  • MV-STRIDE-SFT(Full) improves spatial reasoning across indoor, dynamic, egocentric, and outdoor MMSI-Bench scenarios.The comparison is between the base model and MV-STRIDE-SFT(Full).Source: §B.4 · p. 21

B.5 Validation across Architectures and Scales of Base Model

The dataset improves results across evaluated model architectures and scales; comparisons with recent methods and a hierarchical-level ablation further examine its performance and data design.

B.5 Validation across Architectures and Scales of Base Model

  • MV-STRIDE consistently improves performance across all evaluated models and benchmarks, including different parameter scales and architectures.The results are presented in Table B.7.Source: §B.5 · p. 21, Table B.3 · p. 22

B.6 Comparison with Recent Spatial Reasoning Methods

  • MV-STRIDE performs much better than SpatialLadder on multi-view spatial reasoning, while 3DThinker scores higher on some benchmarks using model-level modifications.The paper characterizes the data-centric MV-STRIDE and architecture-modifying 3DThinker approaches as complementary.Source: §B.6 · p. 22

B.7 Ablation on Hierarchical Data Levels

  • Removing lower-level data decreases performance across most benchmarks in a Stage 1 ablation using 50K QA pairs per level.The ablation supports the contribution of each level and the benefit of training with data from all levels.Source: §B.7 · p. 22

C.1 Synthetic Data Generation via Infinigen

The dataset combines procedurally generated indoor scenes with real-world ScanNet++ scenes, extracts aligned spatial annotations, and verifies sampled QA and reasoning annotations.

C.1 Synthetic Data Generation via Infinigen

  • Infinigen scene generation produced 441 indoor environments and 4,410 high-resolution images, with 10 viewpoints per scene.Scenes are constrained to one room, and camera poses are filtered from generated candidates.Source: §C.1 · pp. 22–23

C.2 Real-world Data Processing of ScanNet++

  • ScanNet++ contributes 124 real-world indoor scenes, using only iPhone frames with verified camera pose annotations.3D attributes are extracted from annotated meshes, while 2D labels are rendered using camera poses and coordinate conventions are unified with Infinigen.Source: §C.2 · pp. 24–25

C.3 Coordinate System and Bounding Boxes

  • The pipeline unifies camera and bounding-box conventions across data sources and represents 2D object locations as [x1, y1, x2, y2].The coordinates are normalized and rescaled to [0, 1000].Source: §C.3 · pp. 25–26

C.4 Quality Verification

  • Randomly sampled QA and CoT annotations achieved reported overall accuracies of 88% and 97.5%, respectively.Two human annotators checked 200 QA samples; Gemini-3-Flash checked 200 CoT samples.Source: §C.4 · p. 26

D.1 Hierarchical Dataset Definitions and Examples

MV-STRIDE defines three levels of spatial tasks, composes cross-level dependencies into question groups, and uses those groups to construct Level III CoT annotations.

D.1 Hierarchical Dataset Definitions and Examples

  • The hierarchy progresses from single-view spatial perception (Level I), through 3D-consistent scene understanding (Level II), to multi-view high-level contextual reasoning (Level III).The levels are presented with task-category definitions and examples.Source: §D.1 · p. 27

D.2 Spatial Cognition Process Question Group.

  • Compositional question structures rooted in Level III tasks explicitly define dependencies between capability items across levels.Figure D.2 visualizes the question compositions.Source: §D.2 · p. 40, Figure D.2 · p. 41

D.3 CoT annotation of Level III

  • The question groups are used to construct Level III CoT annotations, except for Virtual Perspective, which is excluded because ScanNet++ does not reliably establish reference-object forward orientation.Source: §D.3 · p. 41

E.1 CoT Generation Method

The CoT generation method builds Level III answers through a staged progression from single-view perception to cross-view modeling and high-level contextual reasoning. For RL fine-tuning, it filters candidate questions by model accuracy to retain moderately difficult samples.

E.1 CoT Generation Method

  • CoT annotations use 20% of ScanNet++ scenes, with question groups culminating in a Level III question and converted into multi-turn dialogues for Gemini-3-Flash.The dialogues include the corresponding images and request complete reasoning chains.Source: §E.1 · p. 41
  • Level III CoT follows a three-stage pipeline: single-view perception, cross-view 3D modeling, then high-level contextual inference.The stages move from identifying spatial information in images to integrating perspectives and reasoning over global scene information.Source: §E.1 · p. 48
  • The prompt requires reasoning to connect Level I perception with Level II modeling and Level III inference, while prioritizing image evidence when QA content conflicts with it.It also requires a detailed reasoning process in <think> tags before the final answer in <answer> tags.Source: §E.1 · p. 48

E.2 RL Data Filtering and Reward Design

  • RL candidates retain 20% of ScanNet++ scenes and 40% of Infinigen scenes, then exclude samples scoring 0% or 100% accuracy across 10 inference runs.The remaining samples form a moderately difficult dataset for GRPO fine-tuning.Source: §E.2 · pp. 48–49

Related papers

Loading 2609.07258v1…