SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models

Hongxing Li; Dingming Li; Zixuan Wang; Yuchen Yan; Hang Wu; Wenqi Zhang; Yongliang Shen; Weiming Lu; Jun Xiao; Yueting Zhuang

Source-linked AI summary

SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models

Hongxing Li, Dingming Li, Zixuan Wang, Yuchen Yan, Hang Wu, Wenqi Zhang, Yongliang Shen, Weiming Lu, Jun Xiao, Yueting Zhuang

arXiv:2510.08531v1Oct 9, 2025cs.CVcs.AIcs.CL

TL;DR

VLMs struggle with spatial reasoning, partly because existing approaches learn it without hierarchical perceptual foundations. SpatialLadder introduces a multimodal dataset and three-stage progressive training framework, producing strong in-domain results and out-of-domain generalization. The paper also identifies limits in model scale, dataset coverage, domain diversity, and fixed training order.

Problem
Current VLMs struggle with spatial reasoning, while existing datasets and methods lack systematic multimodal coverage and hierarchical foundations linking perception to reasoning.
Method
SpatialLadder combines the 26,610-sample SpatialLadder-26k dataset with progressive training through localization, seven-dimensional multimodal understanding, and GRPO reinforcement learning.
Results
SpatialLadder achieves state-of-the-art spatial reasoning performance, including 62.3% overall accuracy, 23.4% improvement over the base model, and 7.2% average out-of-domain improvement.
Takeaways & Limitations
The results support progressively building spatial intelligence from perception to understanding and reasoning across multimodal spatial tasks.
Takeaways & Limitations
Experiments use only 3B-parameter models and a 26,610-sample dataset dominated by ScanNet indoor scenes, while the fixed training sequence may not suit every task.

Abstract

from arXiv · show

Spatial reasoning remains a fundamental challenge for Vision-Language Models (VLMs), with current approaches struggling to achieve robust performance despite recent advances. We identify that this limitation stems from a critical gap: existing methods attempt to learn spatial reasoning directly without establishing the hierarchical foundations of perception and understanding. To address this challenge, we present a comprehensive methodology for building spatial intelligence progressively. We introduce SpatialLadder-26k, a multimodal dataset containing 26,610 samples spanning object localization, single image, multi-view, and video spatial reasoning tasks, constructed through a standardized pipeline that ensures systematic coverage across modalities. Building on this dataset, we design a three-stage progressive training framework that (1) establishes spatial perception through object localization, (2) develops spatial understanding through multi-dimensional spatial tasks, and (3) strengthens complex reasoning via reinforcement learning with verifiable rewards. This approach yields SpatialLadder, a 3B-parameter model that achieves state-of-the-art performance on spatial reasoning benchmarks, with 23.4% average improvement over the base model, surpassing GPT-4o by 20.8% and Gemini-2.0-Flash by 10.1%. Notably, SpatialLadder maintains strong generalization with 7.2% improvement on out-of-domain benchmarks, demonstrating that progressive training from perception to reasoning is essential for robust spatial intelligence.

1 INTRODUCTION

SpatialLadder addresses the perception–reasoning gap in VLM spatial reasoning with a multimodal dataset and progressive training from perception to complex reasoning. The resulting model achieves strong benchmark performance and out-of-domain generalization.

Current VLMs struggle with spatial queries, limiting deployment in robotics navigation, autonomous driving, and virtual reality.
5.0% accuracy improves with bounding-box hints, followed by another 4.5% gain from directional cues in controlled spatial-orientation experiments.
SpatialLadder-26k contains 26,610 samples covering object localization and spatial reasoning across single-image, multi-view, and video modalities.
The three-stage framework builds spatial capability through object localization, multidimensional understanding, and reinforcement learning with verifiable rewards.
62.3% overall accuracy surpasses the base model by 23.4%, GPT-4o by 20.8%, and Gemini-2.0-Flash by 10.1%.
7.2% average improvement on out-of-domain benchmarks indicates strong generalization beyond the evaluated training distribution.

2 RELATED WORKS

Related work shows that VLM spatial reasoning remains difficult despite progress in visual reasoning and reinforcement-learning-based methods.

Spatial reasoning remains a serious bottleneck for VLMs despite advances in basic visual tasks and spatial benchmarks.
Recent work transfers reinforcement learning techniques from LLMs to VLMs, improving visual mathematical reasoning and extending into video reasoning.

3 METHODS

The method combines SpatialLadder-26k with a three-stage curriculum that progresses from spatial perception to understanding and explicit reasoning. Data span multiple modalities and dimensions, while GRPO uses structured rewards for reasoning and answer correctness.

Dataset: SpatialLadder-26k provides 26,610 multimodal samples spanning spatial tasks from basic perception to complex reasoning.
Dataset: The dataset contains object localization, single-image, multi-view, and video categories with 5,929, 5,929, 5,752, and 9,000 samples respectively.
Dataset: A standardized pipeline uses ScanNet 3D reconstructions, SR-91k videos, and 3D-to-2D transformations to generate unified multimodal data.
Three-stage progressive training framework: Stage 1 grounds spatial perception through object localization and JSON outputs containing object identities and 2D bounding boxes.
Three-stage progressive training framework: Stage 2 trains size, distance, orientation, counting, room-size, appearance-order, and related reasoning across single-image, multi-view, and video modalities.
Three-stage progressive training framework: Stage 3 uses chain-of-thought reinforcement learning with GRPO and rewards for format, task-specific accuracy, and reasoning quality.

4 EXPERIMENTS

Experiments evaluate SpatialLadder across in-domain and out-of-domain benchmarks, ablations, training dynamics, uncertainty, attention, and hierarchical reasoning. Results show strong performance, generalization, component dependence, and progressively focused internal behavior.

Evaluation Setup: SpatialLadder uses Qwen2.5-VL-3B with supervised fine-tuning in Stages 1–2 and GRPO reinforcement learning in Stage 3.The three-stage schedule is configured separately by stage.
Main Results: 62.3% overall accuracy establishes state-of-the-art in-domain performance, including 70.2% on SPBench-SI and 70.9% on SPBench-MV.The model reaches 45.7% on VSI-Bench and surpasses the base model by 23.4%.
Main Results: 50.8% overall accuracy on out-of-domain benchmarks surpasses GPT-4o’s 48.1% and improves 7.2% over the base model.ViewSpatial-Bench person-perspective tasks show a 16.5% improvement.
Ablation Analysis: Removing Stage 2 causes a 9.4% accuracy drop, while excluding single-image and multi-view data causes a 16.4% loss.Stages 1 and 3 produce smaller drops of 1.8% and 2.1%, respectively.
Training Dynamics: The complete model shows smoother convergence and lower reward variance than variants missing Stage 1 or Stage 2, with Stage 2 removal causing the largest VSI-Bench decline.The full framework achieves the highest VSI-Bench accuracy among the compared variants.
Uncertainty Analysis: Semantic entropy rises from 1.24 to 1.47 during Stages 1–2, then declines to 0.66 during Stage 3 reinforcement learning.The reported dynamics mark a shift from broader exploration toward focused reasoning convergence.
Visual Attention Analysis: SpatialLadder achieves 73.5% accuracy and 37.7% visual attention IoU, compared with 32.1% and 33.8% for the base model.Its attention entropy is also lower, at 0.176 versus 0.193.
Hierarchical Reasoning: Qualitative analyses show accurate spatial element identification supporting structured reasoning chains in relative-distance and path-planning tasks.The model is reported to produce correct conclusions with clear reasoning.

5 CONCLUSION

The paper addresses the perception–reasoning gap in VLM spatial tasks with a multimodal dataset and progressive training framework. SpatialLadder achieves strong in-domain and out-of-domain benchmark performance, while ablations support the contribution of its components.

Conclusion: SpatialLadder-26k covers object localization and single-view, multi-view, and video-based spatial reasoning.The framework builds spatial intelligence progressively from perception to understanding and reasoning.
Conclusion: SpatialLadder achieves state-of-the-art results across multiple spatial reasoning benchmarks with strong in-domain and out-of-domain performance.Ablation studies confirm the effectiveness of each component.

ETHICS STATEMENT

The work reports no human subjects, personal data, or sensitive information and uses publicly available visual spatial reasoning benchmarks.

Ethics Statement: The experiments use publicly available benchmark datasets and involve no human-subject data collection.The authors report adherence to ethical research practices without identified privacy, security, or fairness concerns.

REPRODUCIBILITY STATEMENT

The authors describe extensive reproducibility documentation covering dataset construction, training, evaluation, and analysis implementation details.

Reproducibility Statement: Reproducibility materials include dataset construction, training configurations, hyperparameters, dataset descriptions, evaluation results, and analysis implementation details.The referenced documentation spans Appendices B and C.

A PRELIMINARY ANALYSIS

Controlled orientation experiments show that progressively adding perceptual grounding improves spatial reasoning, motivating SpatialLadder’s staged training from localization and supervised foundations to reinforcement learning. The implementation combines filtered multimodal task generation, task-specific prompts and rewards, and a cold-start phase for reliably formatted reasoning outputs.

Diagnostic Experiments: 36.5% baseline accuracy rose to 41.5% with location hints and 46.0% with directional hints on 200 spatial orientation tasks.The diagnostic conditions used raw images, colored bounding boxes, and directional arrows within bounding boxes.
Diagnostic Experiments: SpatialLadder showed 82.0% accuracy without hints, 82.5% with location hints, and 83.5% with full hints.The 1.5% range across conditions indicates comparatively limited dependence on added perceptual hints after training.
Dataset Construction: The dataset generates counting, distance, size, relative-distance, and relative-direction tasks from unified ScanNet metadata and 3D scene information.Construction includes cross-view instance aggregation, Euclidean distance, maximum 3D bounding-box dimension, and constrained comparisons to reduce ambiguity.
Dataset Construction: Quality assurance limits questions per scene and object type while filtering noisy objects such as walls, floors, and ceilings.These filters are intended to preserve scene and object diversity and focus the tasks on human-scale spatial reasoning.
Training Implementation: Stage 1 uses supervised localization training, while Stage 2 uses supervised multi-dimensional spatial reasoning before Stage 3 reinforcement learning refinement.Stages 1–2 optimize cross-entropy loss, and Stage 3 uses GRPO with task-specific verifiable rewards.

C.3 DETAILS OF IN-DOMAIN BENCHMARKS RESULTS

SpatialLadder achieves strong in-domain spatial reasoning performance across VSI-Bench, SPBench-SI, and SPBench-MV, while attention analyses associate its gains with more focused task-relevant visual attention and explicit reasoning dynamics.

In-domain benchmark results: 45.7% accuracy on VSI-Bench places SpatialLadder ahead of all compared models except Spatial-MLLM, including models with 2–3 times larger parameter sizes.It improves performance by 16.3% on average across VSI-Bench sub-tasks.
In-domain benchmark results: 70.2% and 71.2% accuracy on SPBench-SI and SPBench-MV correspond to relative improvements of 29.9% and 34.6% over Qwen2.5-VL-3B.SpatialLadder consistently outperforms all compared baselines on these benchmarks.
Attention analysis: Visual Attention IoU measures attention concentration within target-object bounding boxes using min-max normalized attention weights.The related entropy metric instead measures concentration from the probability distribution induced by normalized attention weights.
Attention analysis: On object size estimation examples, SpatialLadder allocates more focused attention to task-relevant objects than Qwen2.5-VL-3B.The analysis interprets this pattern as improved perceptual ability supporting more reliable spatial reasoning.
Reasoning dynamics: Figure 9 analyzes training with and without chain-of-thought using accuracy rewards, reward standard deviation, and VSI-Bench performance.The supplied passage states that later accuracy reward curves are comparable, while training stability and convergence differ.

D.2 COMPARISON WITH OTHER SPATIAL DATASET

Comparisons indicate that SpatialLadder-26k achieves stronger performance than larger alternative spatial datasets despite using fewer samples, and that progressive training order materially affects VSI-Bench accuracy.

Dataset comparison: 43.9% VSI-Bench accuracy with 26,610 samples exceeds SpaceR-151k at 35.1% and Spatial-MLLM-120k at 40.0%.The comparison uses supervised fine-tuning on Qwen2.5-VL-3B as the common base model.
Dataset comparison: Integrating object localization with single-image, multi-view, and video reasoning provides broader coverage than datasets focused on individual modalities or limited combinations.The passage attributes SpatialLadder-26k’s performance gain to this comprehensive dataset design.
Dataset scaling: Overall performance rises from 36.2% to 60.2% as dataset scaling increases from 0% to 100%.The upward trajectory shows no saturation at full scale in the reported analysis.
Dataset scaling: VSI-Bench accuracy increases from 29.4% to 43.9% across the same 0%-to-100% dataset-scaling range.The reported trend suggests additional dataset expansion remains potentially beneficial within the tested range.
Training order: 43.9% VSI-Bench accuracy for progressive perception-to-spatial training surpasses spatial-only training at 42.7% and mixed training at 40.7%.The ablation identifies training order as an important design factor.

E LIMITATIONS AND FUTURE WORK

The authors identify limitations in model scale, dataset size and diversity, domain coverage, and the fixed sequential training structure, while proposing broader scaling and data expansion as future directions.

Limitations: Experiments are limited to 3B-parameter models, leaving scalability to larger models unexplored.The authors propose evaluating 7B, 13B, and larger models.
Limitations: The 26,610-sample dataset may be insufficient to capture the full complexity of spatial reasoning scenarios.The authors identify dataset scaling as a future research direction.
Limitations: Reliance primarily on ScanNet scenes introduces indoor-environment bias and limits generalization to diverse real-world scenarios.Proposed expansions include outdoor landscapes, urban environments, and domain-specific imagery.
Limitations: The fixed sequential three-stage framework may not be optimal for every spatial reasoning task because it lacks task-specific flexibility.The authors suggest adapting the approach to varied requirements in future work.