PaperlayerView exact revision on arXiv

Source-linked AI summary

SpaceVista: All-Scale Visual Spatial Reasoning from mm to km

Peiwen Sun, Shiqiang Lang, Dongming Wu, Yi Ding, Kaituo Feng, Huadai Liu, Zhen Ye, Rui Liu, Yun-Hui Liu, Jianan Wang, Xiangyu Yue

arXiv:2510.09606v2cs.CV

TL;DR

Spatial reasoning research has made progress on indoor scenes but remains limited across diverse scales and settings, with existing data dependent on indoor scans and costly annotations. SpaceVista combines an automated multiscale dataset with scale-aware modeling and a manually assembled benchmark. It reports competitive results across spatial benchmarks and substantially stronger performance on its all-scale benchmark than proprietary and open-source models.

  • Problem

    Spatial reasoning methods remain limited to indoor settings, specific objects, and constrained scales, while building large wild-scene datasets and physically aligned evaluations is difficult.

    Source: §1 · pp. 1–2

  • Method

    SpaceVista pairs an automated all-scale spatial QA dataset with dense spatial inputs and LoRA-like scale experts coordinated by a scale router.

    Source: §1 · p. 2, §4 · pp. 5–6

  • Results

    SpaceVista performs competitively across spatial benchmarks, approaches state of the art on VSI-Bench, and exceeds proprietary and open-source models by over 3% on average on SpaceVista-Bench.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The authors present SpaceVista-7B as a promising robust baseline for all-scale spatial reasoning.

    Source: §6 · p. 9

Abstract

from arXiv · show

With the current surge in spatial reasoning explorations, researchers have made significant progress in understanding indoor scenes, but still struggle with diverse applications such as robotics and autonomous driving. This paper aims to advance all-scale spatial reasoning across diverse scenarios by tackling two key challenges: 1) the heavy reliance on indoor 3D scans and labor-intensive manual annotations for dataset curation; 2) the absence of effective all-scale scene modeling, which often leads to overfitting to individual scenes. In this paper, we introduce a holistic solution that integrates a structured spatial reasoning knowledge system, scale-aware modeling, and a progressive training paradigm, as the first attempt to broaden the all-scale spatial intelligence of MLLMs to the best of our knowledge. Using a task-specific, specialist-driven automated pipeline, we curate over 38K video scenes across 5 spatial scales to create SpaceVista-1M, a dataset comprising approximately 1M spatial QA pairs spanning 19 diverse task types. While specialist models can inject useful domain knowledge, they are not reliable for evaluation. We then build an all-scale benchmark with precise annotations by manually recording, retrieving, and assembling video-based data. However, naive training with SpaceVista-1M often yields suboptimal results due to the potential knowledge conflict. Accordingly, we introduce SpaceVista-7B, a spatial reasoning model that accepts dense inputs beyond semantics and uses scale as an anchor for scale-aware experts and progressive rewards. Finally, extensive evaluations across 5 benchmarks, including our SpaceVista-Bench, demonstrate competitive performance, showcasing strong generalization across all scales and scenarios. Our dataset, model, and benchmark will be released on https://peiwensun2000.github.io/mm2km .

1. Introduction

Spatial reasoning research has largely centered on indoor scenes, while comprehensive all-scale modeling and data remain lacking. SpaceVista addresses this gap with an all-scale dataset, scale-aware modeling, and a manually curated evaluation benchmark.

1. Introduction

  • Existing spatial reasoning work is primarily limited to indoor settings, specific objects, and constrained scales, with no dedicated framework for holistic all-scale scene understanding.The paper frames its contribution across data, model, and evaluation.Source: §1 · p. 1
  • The work presents a six-order-of-magnitude scope, from mm to km, integrating cues from micro-objects to macro-scenes.The paper connects this breadth to rich semantics and diverse tasks.Source: §1 · p. 1, Figure 1 · p. 2
  • SpaceVista-1M provides 1 million QA pairs across 5 scales and 19 tasks, using an automated pipeline over wild video scenes.It supplies rationale answers for SFT and regression or multiple-choice answers for RL.Source: §1 · p. 2
  • Existing spatial datasets rely heavily on indoor scans and manual 3D annotation, making large-scale wild data and physically aligned evaluation difficult to obtain.The authors address evaluation separately with SpaceVista-Bench, assembled through manual recording or authoritative sources and human annotation.Source: §1 · pp. 1–2
  • SpaceVista-7B adds dense spatial information beyond semantic visual tokens and uses scale experts with a customized training strategy to mitigate cross-scale conflicts.The paper describes naive fine-tuning as often suboptimal and uses scale as an anchor for progressive rewards.Source: §1 · pp. 2–3

2. Related Works

Prior work develops spatial reasoning across video, geometry-aware models, and scale-related perception tasks, but systematic all-scale video data and modeling remain unexplored. The paper situates its approach within a field whose spatial MLLMs are still early-stage.

2. Related Works

  • Earlier visual reasoning work introduced GRPO-based rewards and broader data and training structures, while spatial MLLMs remain at an early stage (Feng et al., 2025; Liao et al., 2025; Li et al., 2025c; Chen et al., 2025; Liu et al., 2025c).Video spatial claims are grouped as 2D plane-based or 3D space-based reasoning; this paper focuses on the latter.Source: §2 · p. 3
  • VSI-Bench introduced video spatial tasks such as counting, direction, and planning, and highlighted substantial performance gaps between MLLMs and humans (Yang et al., 2025a).Other datasets expanded spatial supervision through QA pairs across indoor and other limited settings (Deng et al., 2025b; Zhang et al., 2025c; Ouyang et al., 2025).Source: §2 · p. 3
  • Spatial models span image, multi-image, multi-view, video, and simulation inputs, with video posing data-acquisition and modeling challenges (Ma et al., 2025; Liu et al., 2025b; Xu et al., 2025; Li et al., 2025b; Wu et al., 2025c; Li et al., 2025a).Geometry-aware dual encoders have also been used to capture geometry cues and infer occluded structures from monocular inputs (Wu et al., 2025c; Zheng et al., 2025; Fan et al., 2025).Source: §2 · p. 3
  • Systematic data and designed models for all-scale video scenes remain unexplored despite work on long, omni, ego-centric, and aerial video reasoning (Zhang et al., 2025a; Dongfang et al., 2025; Wu et al., 2025e; Sun et al., 2026).The passage presents these video settings as separately explored directions.Source: §2 · p. 3
  • Here, “all-scale” refers to real-world distances, semantics, and object states across scales, extending beyond low-resolution patch information loss (Nikouei et al., 2025).Related work has explored perception and reconstruction from meters to kilometers, depth from close range to infinity, and object annotations from screws to airplanes (Deng et al., 2025a; Wen et al., 2025; Liu et al., 2025a).Source: §2 · p. 3

3. Dataset

SpaceVista-1M is an open-source real-world spatial video QA dataset built to broaden spatial reasoning beyond indoor-focused resources. Its automated pipeline assembles multiscale scenes, structured annotations, and task-specific QA, while a manually reviewed benchmark supports evaluation.

3. Dataset

  • The dataset pipeline combines semantic, 2D, and 3D annotations, integrating specialized models to prepare, transform, and generate all-scale QA data.The figure describes scenes and sources, object sizes from mm to 100 m, object-to-camera distances typically from 10 to 600 m, and scene diversity.Source: §3 · p. 4, Figure 3 · p. 4
  • Data preparation uses video datasets with camera parameters, estimates depth and normal maps, extracts semantics and boxes, and associates object identities and masks across frames.The source videos are mostly static scenes without moving objects.Source: §3 · p. 4
  • The 19 task workflows use camera parameters to derive object positions and dimensions, including tabletop manipulation and drone-view area estimation.The counting example detects objects, propagates masks, tracks identities, filters ambiguous scenes, and derives temporally consistent counts.Source: §3 · p. 4
  • Fixed questions such as counting and object size use templates, while flexible planning questions use GPT-generated natural-language answers and randomized options for RL.The authors manually curate more than 3,000 templates; filtered CoT rationales generated with Qwen2.5-VL-72B-Instruct support SFT.Source: §3 · pp. 4–5
  • SpaceVista-Bench contains about 1,600 QA pairs across approximately 300 video scenes, with answers checked against the physical world through manual review and source verification.Training-set quality control manually verifies a small portion and reports approximately 83% average global accuracy; drone-scenario exploration remains preliminary.Source: §3 · p. 5
  • SpaceVista-1M contains 1 million QA pairs spanning 19 tasks, 5 scale types, and more than 50 subscene categories.The paper presents it as an open-source, real-world, all-scale spatial video QA dataset.Source: §3 · p. 5

4. Method

SpaceVista addresses cross-scale knowledge conflict with dense visual features, scale-specific experts, and rewards that guide reasoning through semantic, scale, and answer anchors. Its training proceeds from cold-start supervised fine-tuning to scale-router training and then reinforcement learning.

Overview

  • The method targets cross-scale knowledge conflict, where similar visual patterns receive different interpretations at different scales.The paper reports that undifferentiated cross-scale knowledge mixtures hinder rather than facilitate reasoning.Source: §4 · p. 5, Figure 4 · p. 6

Beyond Semantics

  • DINOv3 dense patch features are fused with video features to provide spatial information beyond semantics.The dense features are mapped to the video-feature channel dimensions and temporally pooled before fusion.Source: §4 · p. 5

Scale Experts Design

  • Scale experts preserve scale-level knowledge independence, with a router dynamically allocating experts across layers.Each expert fine-tunes 0.5% of overall parameters; the router uses an MLP and softmax to produce learned scaling factors.Source: §4 · p. 6

Process Reward Design

  • Three reinforcement-learning anchors guide reasoning through task semantics, overall scene scale, and the answer.The semantic and scale anchors are omitted when unnecessary for a task, such as camera rotation.Source: §4 · p. 6

Training Strategy

  • Training progresses from cold-start optimization of projection, feature-fusion, and scale-expert modules to router training and then reinforcement learning.Router training encourages each scale-specific expert to specialize on appropriate inputs.Source: §4 · p. 7

5. Experiment

Experiments evaluate SpaceVista across five benchmarks and report strong SpaceVista-Bench performance, alongside ablations of model components and input modalities. The appendices provide additional visualizations, experiments, and analyses.

Benchmarks

  • Evaluation covers five benchmarks: three video-based benchmarks and two multi-image benchmarks, treated as related tests of cross-frame understanding.SpaceVista-1M is used for supervised fine-tuning and reinforcement learning.Source: §5 · pp. 7–8

Comparison on Subsets of SpaceVista-Bench

  • SpaceVista-7B scores comparatively highly across scenarios and overall, despite only minor improvements on indoor scenes.Small-scale subsets challenge commercial and general models; the authors suggest pretraining-corpus biases as a likely explanation.Source: §5 · p. 8

Comparison on Spatial Reasoning Datasets

  • SpaceVista-7B exceeds open-source models by around 6% in comprehensive all-scale spatial reasoning and exceeds proprietary and open-source models by more than 3% on average on SpaceVista-Bench.The reported results also describe competitive performance across spatial reasoning benchmarks and comparable, near-state-of-the-art results on VSI-Bench.Source: Table 5 · p. 8, §5 · p. 8

Ablation on Each Component

  • Scale experts yield substantial gains, but adding experts can pressure the router and create load imbalance; progressive rewards outperform unconstrained reasoning, and DINO v3 yields greater gains than VGGT.The authors describe optional anchors as useful halfway points in all-scale reasoning.Source: §5 · p. 8

More Experiments

  • The appendix adds visualizations, patch-level encoder ablations, all-scale observations, and out-of-distribution analysis on customized data including Guinness Records.The model uses Qwen2.5-VL-7B for main experiments and Qwen2.5-VL-3B for ablations.Source: §5 · pp. 7–9

6. Conclusion

The paper introduces an all-scale visual spatial reasoning task, SpaceVista-1M, SpaceVista-Bench, and SpaceVista-7B. Experiments show promising performance as a baseline for all-scale reasoning.

Conclusion

  • The paper introduces a multimodal task requiring machines to answer questions and provide rationales from visual spatial context.It develops SpaceVista-1M for cold start and reinforcement learning, handcrafts a physically grounded multi-scale video benchmark, and reports promising SpaceVista-7B performance as an all-scale baseline.Source: §6 · p. 9

Impact Statement

The authors present SpaceVista as a foundational step toward applications of all-scale spatial reasoning, from manufacturing and surgery to remote sensing and cartography.

Impact Statement

  • The authors frame SpaceVista as a foundational step for industrial automation, embedded systems, and autonomous driving.They also anticipate applications in spatial captioning, guided visual generation, and interactive world models.Source: Impact Statement · p. 9
  • The envisioned spatial scales range from precision manufacturing at µm and surgery at mm to remote sensing at km and cartography at 10km.Source: Impact Statement · p. 9
  • The authors describe the intended setting as complex, unconstrained environments in the wild.Source: Impact Statement · p. 9

Appendices Contents

The appendix contents list data resources, model ablations, and leaderboard-setting details.

Appendices Contents

A.1. Task Distribution

SpaceVista-1M combines general and scale-specific spatial tasks, and the paper compares multimodal models across five spatial reasoning benchmarks. SpaceVista-7B is reported to improve across those benchmarks.

A.1. Task Distribution

  • SpaceVista-1M includes general and scale-specific tasks, with bubble sizes indicating each task’s relative data volume.Figure A5 visualizes the data composition for each scene task.Source: §A.1 · p. 16
  • Task abbreviations include “Est.” for Estimation, “Dist.” for Distance, “Loc.” for Location, and “Com.” for Comparison.The supplied text also describes Figure A5 as a statistical chart of QA types.Source: §A.1 · p. 16, Figure A5 · p. 16

A.2. Performance Radar

  • SpaceVista-7B achieves significant improvement across five spatial reasoning benchmarks.The comparison evaluates eight multimodal models using a radar chart.Source: §A.2 · p. 16

B. Data Construction

SpaceVista-1M spans five scene categories and combines public datasets with collected data. Its sources range from tiny-object videos and tabletop scenes to indoor, outdoor, and drone views.

B. Data Construction

  • SpaceVista-1M contains 1 million QA pairs across 19 spatial reasoning task types and 38 thousand videos from diverse scenes.Its sources span tiny, tabletop, indoor, outdoor, and drone-view scenarios.Source: B. Data Construction · p. 16, §B.1 · p. 17

B.1. Data Comparison

  • The authors filter redundant or inconsistent samples and remove training scenes appearing in all benchmarks to separate training from test data.Table B6 summarizes the datasets used for SpaceVista-1M and SpaceVista-Bench, including sources reserved for evaluation.Source: §B.1 · p. 17, Table B6 · p. 17

B.2.1. TINY TABLETOP SCENE; B.2.2. TABLETOP SCENE

  • Tiny-object videos come from a filtered uCO3D subset, while tabletop data uses multi-object WildRGB-D videos and the smaller SMOT collection.The tabletop category also includes varied planar surfaces such as sand, beds, wardrobes, and floors.Source: §B.2.1 · p. 18, §B.2.2 · p. 18

B.2.3. INDOOR SCENE; B.2.4. WILD INDOOR SCENE

  • Indoor sources include ScanNet and ScanNet++, while DL3DV adds wild indoor videos with greater geometric and appearance variability.ScanNet datasets provide aligned RGB, depth, and 3D reconstructions; DL3DV uses video capture and camera-parameter estimation.Source: §B.2.3 · p. 18, §B.2.4 · p. 18

B.2.5. OUTDOOR SCENE

  • DL3DV outdoor footage covers landmarks, street corners, courtyards, and parks, with camera parameters calibrated using COLMAP.The dataset includes 10,510 videos at 4K resolution and about 51.2 million frames across 65 location types.Source: §B.2.5 · p. 19

B.2.6. DRONE SCENE

  • DL3DV drone footage adds aerial views of varied settings, expanding scene diversity and viewpoint range despite its smaller scale.The subset contains more than 100 videos, including plazas, pathways, rooftop platforms, and landmark facades.Source: §B.2.6 · p. 19

B.2.7. OUR OWN COLLECTED DATA

The paper supplements existing video sources with manually measured or authoritative-reference data, then constructs spatial QA through task-specific processing. It also describes dataset quality controls, scene diversity, and an encoder comparison.

B.2.7. OUR OWN COLLECTED DATA

  • For tiny and tabletop evaluation, the authors measure and annotate collected data; for indoor and outdoor scenarios, they retrieve and verify public information.Their self-recorded data covers nearly 50 object categories in over 200 videos and 1,000 QA pairs, while landmark dimensions are retrieved from authoritative sources.Source: §B.2.7 · pp. 19–20

B.3.1. DATA PREPARATION

  • Known camera parameters support processing; depth models provide metric maps, Video-Depth-Anything addresses frame inconsistency, and grounding models with SAM2 provide tracked masks and object IDs.The authors note that extreme scales such as drone views and tiny objects require adjusted depth normalization.Source: §B.3.1 · pp. 20, 22

B.3.2. TYPE: DISTANCE

  • Distance tasks convert depth keypoints into 3D point clouds and use PCA to estimate dimensions, with camera calibration and multiple frames supporting metric measurements.The method assumes piecewise rigid scenes, focuses on objects larger than 20×20 pixels, and depends on depth and segmentation quality.Source: §B.3.2 · p. 22

B.3.3. TYPE: COUNTING

  • Counting workflows combine detection, tracking, and confirmation, with outdoor filtering for scenes containing 2 to 10 objects and minimum 32-pixel boxes.The tabletop workflow uses grounding models and SAM2 to distinguish instances.Source: §B.3.3 · pp. 22–23

B.3.4. TYPE: PLANNING; B.3.5. TYPE: RELATION

  • Planning uses object geometry, camera parameters, and RRT to create collision-free paths, while relation analysis combines semantic candidates with 3D plausibility checks.The planned paths yield projected trajectory answers and natural-language movement instructions.Source: §B.3.4 · p. 23, §B.3.5 · pp. 23–24

B.3.6. DATA POST-PROCESSING; B.3.7. BENCHMARK CONSTRUCTION

  • Rationale distillation and confidence filtering produce training QA, while the benchmark combines human-annotated questions with about 1,600 fully human-annotated evaluation QA pairs.The benchmark includes measurement-related and non-measurement questions.Source: §B.3.7 · p. 24, §B.3.6 · p. 24

B.4. Data Statistics; B.4.1. TARGET CATEGORY DISTRIBUTION; B.4.2. DATA QUALITY CONTROL

  • The dataset’s scene categories differ in object distributions, while its training-set preference rates measure human acceptability rather than strict physical accuracy.The reported data include over 100 million raw frames and 1 million QA pairs; route-planning, navigation, and obstacle-avoidance preference standards are notably stringent.Source: §B.4.1 · p. 24, §B.4 · p. 24, §B.4.2 · p. 26

C.1. Parameter Setting; C.2. Patch Level Encoder Ablation

  • DINOv3 offers a more favorable efficiency–accuracy trade-off than VGGT in the patch-encoder comparison, which the authors attribute partly to self-supervised pretraining.Most ablations use Qwen2.5-VL-3B for resource reasons; the default number of experts is four.Source: §C.2 · p. 27, §C.1 · p. 27

C.3. LoRA Like Expert Ablation

The appendix examines scale-aware experts, size memorization, scenario difficulty, and geometric representations. Its reported findings include gains from expert routing and 2.5D inputs, alongside persistent size-bias concerns.

C.3. LoRA Like Expert Ablation

  • Model-wise LoRA-like experts outperform full-parameter fine-tuning and vanilla LoRA, with further gains from higher-capacity layer-wise experts.The ablation compares three training strategies on the same 3B pretrained base model.Source: §C.3 · pp. 27–28, Table C13 · p. 28

C.4. Memorization effect observation (Out-of-Distribution Problem)

  • All-scale training benefits overall reasoning, but models still memorize typical object sizes; GWR size questions challenge both Qwen2.5-VL and SpaceVista without a clear performance gap.The GWR set uses precise measurements and human-annotated questions about unusual objects; results compare size-related subsets for fairness.Source: §C.4 · pp. 28–29, Table C14 · p. 28

C.5. Challenging Scenario Analysis

  • Indoor data appears easiest in the controlled scene comparison; the authors hypothesize that human-scale estimation bias from meter-expressible objects in pretraining may explain this.Models were trained on diverse scenes with similar quantities of QA pairs and video samples.Source: §C.5 · p. 30, Table C16 · p. 30

C.6. 3D Geometric Feature Observation

  • 2.5D inputs are usually more robust than 3D-only features, using rendered views to exploit pretrained image tokenizers for more reliable semantics.The paper describes 3D as pure VGGT geometry and 2.5D as adding 12 rendered viewing angles; native VGGT features are less suited to semantics and overall scenes, especially at low resolution and in uncommon scenarios.Source: §C.6 · p. 30, Table C17 · p. 30

C.7. Leaderboard Settings Detail

  • The leaderboard evaluates closed- and open-source models across Tiny Tabletop, Tabletop, Indoor, and Outdoor scenarios, with release dates, sources, and generation settings documented separately.The paper points to Tables C18–C20 for model and configuration details.Source: §C.7 · p. 31, Table C18 · p. 31, Table C19 · p. 31

D.1. Error Accumulation

The appendix notes possible depth-related error accumulation and incomplete scale coverage, then describes dataset usage, scene characteristics, and task templates. It also outlines validation steps and possible future expansion.

D.1. Error Accumulation

  • Metric-depth estimation and canonical-view transformation may accumulate errors, especially because current metric-depth models perform poorly at full scale.The authors cite manual verification of a training subset, full human annotation of the test set, and a real-world measured test subset as quality checks.Source: §D.1 · p. 32

D.2. All Scale Possibilities

  • Current coverage does not span the full spectrum, including nanometer-scale manufacturing and 10-kilometer-or-greater remote sensing; these applications involve imaging and 3D modeling beyond conventional sensing.The authors aim to expand modeling across broader dimensions.Source: §D.2 · p. 32

D.3. Dataset Usage Discussion

  • About 100K SPAR-7M free-form samples—roughly 1% of that dataset—were filtered with scan datasets for cold-start CoT, but the authors do not count their model as trained on SPAR-7M.They report adding filtering and labeling steps and integrating the processed sources with wild video data into SpaceVista-1M.Source: §D.3 · p. 32

E.1. Scene Preview

  • Indoor scenes are simple, clean room-scale environments, while wild indoor scenes have denser, irregular layouts, diverse objects, reflections, occlusions, and uneven lighting.Outdoor data includes ground and aerial views; drone views reveal structures that are difficult to see from the ground.Source: §E.1 · pp. 32–33
  • Tabletop videos use dynamic mobile multi-view capture for varied objects and backgrounds, while tiny tabletop turntable videos capture objects through 360-degree views.Self-collected tabletop and tiny tabletop data emphasize detailed multi-view coverage for fine-grained analysis.Source: §E.1 · p. 33

E.2. Template Preview

  • Templates demonstrate point-input object counting, bounding-box distance questions, and original-input spatial relations; other scenes and tasks use similar templates.The appendix gives example phrasings for point-input counting and original-input spatial-relation questions.Source: §E.2 · pp. 33–34, Table E21 · p. 34
Loading 2510.09606v2…