PaperlayerView exact revision on arXiv

Source-linked AI summary

SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments

Dinging Li, Yingxiu Zhao, Xinrui Cheng, Kangheng Lin, Hongbo Peng, Hongxing Li, Zixuan Wang, Yuhong Dai, Haodong Li, Jia Wang, Yukang Shi, Liang Zhao, Jianjian Sun, Zheng Ge, Xiangyu Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen

arXiv:2604.14144v1cs.CVcs.CL

TL;DR

Continuous 3D spatial-reasoning improvement is limited by costly annotation and consensus-based pseudo-labels. SpatialEvo uses deterministic geometric verification and role-based policy co-evolution, achieving the highest average score across nine benchmarks at both 3B and 7B scales.

  • Problem

    Continuous 3D spatial-reasoning improvement remains limited by costly geometric annotation, while self-evolving methods rely on model-generated consensus signals.

  • Method

    SpatialEvo uses a Deterministic Geometric Environment to compute exact ground truth from 3D geometry and co-evolves one policy across Questioner and Solver roles with adaptive scheduling.

  • Results

    SpatialEvo achieves the highest average score at both 3B and 7B scales across nine benchmarks, with consistent spatial-reasoning gains and no degradation in general visual understanding.

  • Takeaways & Limitations

    Deterministic physical feedback replaces model consensus with zero-noise supervision for continual 3D spatial-reasoning evolution.

  • Takeaways & Limitations

    SpatialEvo requires high-fidelity indoor 3D assets, calibrated camera poses, and complete scene coverage, limiting current applicability mainly to static indoor environments.

Abstract

from arXiv · show

Spatial reasoning over three-dimensional scenes is a core capability for embodied intelligence, yet continuous model improvement remains bottlenecked by the cost of geometric annotation. The self-evolving paradigm offers a promising path, but its reliance on model consensus to construct pseudo-labels causes training to reinforce rather than correct the model's own geometric errors. We identify a property unique to 3D spatial reasoning that circumvents this limitation: ground truth is a deterministic consequence of the underlying geometry, computable exactly from point clouds and camera poses without any model involvement. Building on this insight, we present SpatialEvo, a self-evolving framework for 3D spatial reasoning, centered on the Deterministic Geometric Environment (DGE). The DGE formalizes 16 spatial reasoning task categories under explicit geometric validation rules and converts unannotated 3D scenes into zero-noise interactive oracles, replacing model consensus with objective physical feedback. A single shared-parameter policy co-evolves across questioner and solver roles under DGE constraints: the questioner generates physically valid spatial questions grounded in scene observations, while the solver derives precise answers against DGE-verified ground truth. A task-adaptive scheduler endogenously concentrates training on the model's weakest categories, producing a dynamic curriculum without manual design. Experiments across nine benchmarks demonstrate that SpatialEvo achieves the highest average score at both 3B and 7B scales, with consistent gains on spatial reasoning benchmarks and no degradation on general visual understanding.

1. Introduction

3D spatial reasoning needs scalable improvement beyond costly static annotation and consensus-based pseudo-labels. SpatialEvo addresses this with exact geometric verification, role-based policy co-evolution, and adaptive curriculum learning, achieving leading results across nine benchmarks.

  • Motivation: Static annotated datasets impose a fixed training distribution that cannot adapt to current weaknesses or generate harder examples as models improve.They also require annotation investment to scale.
  • Motivation: Self-evolving methods provide dynamic curricula but commonly construct training signals through majority voting or self-consistency over model predictions.This makes model consensus the source of ground-truth supervision.
  • Key insight: 3D spatial answers are deterministic consequences of point clouds, camera poses, and well-formed geometric questions, so exact answers can be computed programmatically without model judgment.Examples include nearest-point computations for distance and rotation-matrix arithmetic for camera orientation.
  • Method: SpatialEvo’s Deterministic Geometric Environment covers 16 task categories and converts 3D scene assets into zero-noise online reward judges.Its geometric rules validate questions and compute exact ground truth from point clouds and camera pose sequences.
  • Method: A single VLM co-evolves as Questioner and Solver under DGE constraints, while task scheduling adapts sampling to historical accuracy for curriculum learning.The Questioner generates physically valid questions and the Solver derives answers through geometric reasoning.
  • Results: SpatialEvo achieves the highest average score at both 3B and 7B scales, with consistent spatial-reasoning gains and no degradation in general visual understanding.Ablations identify replacing DGE ground truth with majority-vote pseudo-labels as the largest performance drop.

2. Related Works

Prior work improves spatial reasoning through data construction, reasoning-oriented training, and multimodal self-play, but existing self-evolving approaches remain limited by static corpora and model-consensus rewards. SpatialEvo instead uses deterministic geometric computation for 3D self-evolution.

  • Spatial Reasoning in Vision-Language Models: Vision-language spatial reasoning methods have pursued multi-view fusion, depth injection, data construction, and reasoning-oriented training.Examples include SpatialVLM, SpatialBot, and SpatialLadder.
  • Self-Evolution of Vision-Language Models: Self-evolution has expanded from language models to vision-language models, and spatial reasoning is suited to it because visual inputs contain physical information for deterministic ground-truth computation.This property can reduce reliance on model consensus.
  • Self-Evolution of Vision-Language Models: Existing multimodal self-play methods advance self-evolution but remain constrained by static image corpora and commonly use model consensus as a reward proxy.The related methods cited include VisPlay, EvolMM, V-Zero, Vision-Zero, and MM-Zero.
  • Self-Evolution of Vision-Language Models: SpatialEvo replaces model-derived judgment with deterministic geometric computation for 3D spatial reasoning.Its framework combines physical verification with self-evolution rather than consensus-based rewards.

3. Methodology

SpatialEvo combines a deterministic geometric environment with shared-parameter questioner–solver co-evolution to train 3D spatial reasoning from multi-view scenes. Its verification rules make 16 task categories programmatically decidable and provide objective feedback throughout the self-evolution loop.

  • Problem formulation: SpatialEvo represents spatial reasoning over multi-view RGB images from 3D scenes containing point clouds, semantic annotations, and camera poses.The policy observes RGB frames while the underlying scene assets support geometric verification and answer computation.
  • Task space: The task space covers 16 spatial reasoning categories spanning scene-level, dual-image, metric, topological, and camera-pose reasoning.Examples include counting, distance, direction, visibility, camera position, elevation, motion, and attribute measurement.
  • Deterministic Geometric Environment: The DGE maps generated natural-language questions to 3D scene assets, validates them with task-specific rules, and computes exact ground truth as a noise-free physical judge.Its rules filter ambiguous or degenerate cases and enforce conditions such as sufficient point-cloud density and meaningful viewpoint disparity.
  • Verification pipeline: An automated three-stage pipeline parses question entities, verifies legality and physical consistency, then synthesizes answers using geometric operators such as coordinate transformation and point-cloud analysis.The pipeline can also retain intermediate geometric states for qualitative analysis and error attribution.
  • Training signal: Programmatic physical computation removes reward noise so policy updates remain anchored to objective geometric laws.This feedback replaces black-box judgment with deterministic supervision during self-evolution.
  • Policy co-evolution: A single role-conditioned policy alternates between questioner and solver roles, enabling shared parameters to transfer geometric learning between question generation and answer derivation.The co-evolution procedure includes self-play, task scheduling, reward design, and GRPO training.

4. Experiments

Experiments evaluate SpatialEvo across nine benchmarks, model scales, baselines, and ablations. The results show strong spatial-reasoning gains, preserved general visual capability, and substantial dependence on deterministic geometric feedback.

  • Main Results: 51.1 (3B) and 54.7 (7B) are SpatialEvo’s average scores, the highest under both backbone settings.The model outperforms all baselines by a consistent margin across both scales.
  • Main Results: 39.2 (3B) and 46.1 (7B) are SpatialEvo’s VSI-Bench scores, with consistent gains also reported on EmbSpatial and ViewSpatial.These benchmarks correspond to multi-view, embodied, and perspective-dependent spatial reasoning.
  • Main Results: 55.2 (3B) and 62.5 (7B) on MMStar show general visual capability retention after spatial specialization.SpatialEvo also achieves 66.5 (3B) and 66.7 (7B) on RealWorldQA.
  • Ablation Studies: 49.6 is the ablated average after replacing DGE ground truth with majority-voting pseudo-GT, the largest reported degradation.VSI-Bench falls to 18.8, while removing the Solver yields an average of 51.5 and removing the Questioner yields 53.1.

5. Analysis and Discussion

SpatialEvo is evaluated against static learning paradigms and across iterative scheduler settings. Its online training improves benchmark performance by adapting sampling to the solver’s evolving weaknesses.

  • Online Evolution vs. Static Learning: 46.3 and 43.9 are SpatialEvo’s highest averages against the SpatialLadder RL baseline and static dataset SFT counterparts, respectively.The comparison uses a narrower task scope than the reference datasets.
  • Online Evolution vs. Static Learning: SpatialEvo continuously aligns its training distribution with the solver’s current cognitive frontier through Questioner–DGE interaction.This enables adaptive hard-sample mining that static datasets cannot replicate.
  • Curriculum Emergence Analysis: 46.1 is the full SpatialEvo average at Iter 4 with the Adaptive Scheduler, compared with 43.4 without it.Both groups start from the same base model and differ only in scheduler use.
  • Curriculum Emergence Analysis: The Adaptive Scheduler increasingly samples persistent weak spots, producing late-stage gains on Abs. Dist. (32.8), Rel. Dist. (45.1), and Appr. Order (40.1).Uniform task sampling instead stagnates and declines after early iterations.
  • Curriculum Emergence Analysis: Rel. Dir. reaches 21.8% and Rel. Dist. 18.7% sampling, while Room Area falls to 12.5% and Obj. Size to 13.4% from a uniform 16.7%.The scheduler’s allocation diverges from uniform sampling as training progresses.
  • Curriculum Emergence Analysis: Questioner validity reward stabilizes near 1.0, while Solver accuracy rises and the Invalid Ratio declines during training.These trajectories indicate increasingly valid question generation and progressive geometric-reasoning internalization.

6. Conclusions

SpatialEvo introduces physically grounded self-evolution for 3D spatial reasoning by replacing model-voted pseudo-labels with deterministic geometric feedback. Its experiments report gains across multiple spatial reasoning benchmarks and motivate physically verifiable self-exploration for embodied intelligence.

  • Conclusions: SpatialEvo uses physical information in point clouds and camera poses to compute exact ground truth programmatically.This converts unannotated 3D scenes into zero-noise reward judges.
  • Conclusions: A single policy co-evolves as Questioner and Solver under DGE constraints, while a task scheduler enables adaptive curriculum self-emergence.The roles generate valid questions and solve them against deterministic geometric feedback.
  • Conclusions: SpatialEvo reports significant gains across multiple spatial reasoning benchmarks.The conclusion states the result at the benchmark level without specifying a single metric.
  • Conclusions: Physically verifiable environments provide a basis for continual spatial-reasoning improvement driven by interaction with the objective world rather than costly human annotation.The paper presents this as a reference direction for broader embodied intelligence research.

A. Limitations

SpatialEvo’s applicability is constrained by its dependence on high-fidelity, explicit 3D scene representations and reliable language-based entity parsing. These constraints limit deployment across environments and can introduce geometric or verification noise.

  • Dependency on high-fidelity 3D assets: SpatialEvo requires complete, high-quality indoor point clouds, calibrated camera poses, and comprehensive scene coverage.This currently restricts the framework mainly to static indoor environments such as ScanNet.
  • Dependency on high-fidelity 3D assets: Outdoor and dynamic settings challenge geometric consistency because of sparse point clouds, scale variation, and moving objects.These conditions can undermine reliable ground-truth computation.
  • Sensitivity to entity parsing quality: Ambiguous references or underspecified targets can cause entity-parsing errors that propagate through verification and computation.Deterministic geometric reasoning alone cannot fully mitigate this noise source.
  • Sensitivity to point cloud quality: Point-cloud artifacts, sparsity, and occlusions can reduce the precision of bounding-box fitting and depth estimation.Relative-error tolerance bands partially absorb reconstruction noise, but remain bounded by underlying data quality.
  • Future directions: The limitations largely stem from reliance on explicit 3D representations.Future work may explore implicit representations and on-demand geometry construction to improve scalability and generalization.

B. Implementation Details of the DGE

The DGE validates spatial questions through explicit task-specific rules, structured extraction, grounded candidate pools, and deterministic geometric execution. It supports both valid-question ground truth and interpretable rejection feedback.

  • Validation framework: DGE acceptance requires compatible input modality, successful entity extraction, grounded candidate labels, valid task structure, and an executable geometric solver.These checks jointly reject malformed or physically unsolvable questions before downstream computation.
  • Grounding and visibility: Visibility pools use per-frame instance counts with v_min ≥0.1, while stricter tool-specific thresholds override the default.Grounded pools distinguish unique scene labels, repeated labels, and visible per-frame objects.
  • Grounding and visibility: Structural constraints prevent same-object role assignments, conflicting distinct-field roles, duplicate candidates, empty lists, and query targets appearing among candidates.These constraints ensure that generated questions remain semantically and geometrically well-posed.
  • Validation framework: The DGE covers 16 spatial reasoning categories with task-specific validity rules and deterministic solvers for consistent, verifiable evaluation.Rules constrain grounding and well-posedness, while geometric operators compute the corresponding answers.
  • Verification pipeline: The verification pipeline normalizes tasks, injects scene metadata, extracts and canonicalizes entities, rejects invalid structures, and executes the prescribed geometric rubric.For accepted questions, the pipeline records intermediate geometric states such as coordinates, pose matrices, and depth projections.
  • Feedback: Invalid questions receive diagnostic rejection evidence, allowing the system to provide noise-free geometric ground truth for valid questions and interpretable feedback for invalid ones.This unified interface turns rejection outcomes into usable supervision rather than simply discarding invalid samples.

C.1.3. Task Sampling from Smoothed Accuracy

Task sampling adapts to smoothed historical accuracy, emphasizing weaker categories while preserving coverage of mastered tasks. Questioner and solver rewards further shape training toward valid, informative questions and grounded answers.

  • Adaptive scheduling: w_k = max(δ, 1 − ā_k) with δ = 0.05 assigns higher sampling weight to tasks with lower historical accuracy.The floor ensures well-mastered tasks are not entirely excluded from sampling.
  • Questioner rewards: Questioner rewards combine format, geometric validity, and observation quality, with the default valid-question form weighted as 0.1 f_fmt + 0.9 f_valid f_obs.A lightweight text judge evaluates whether observations move from global scene structure to the local task target.
  • Questioner rewards: DGE validity down-weights formally valid but weakly discriminative cases, including singleton counts and same outcomes in depth, elevation, or visibility tasks.This suppresses repeated sampling of geometrically trivial edge cases.
  • Observation quality: Observation judging scores groundedness, global caption quality, local focus, transition, support, and sample specificity on a four-level scale.The intended observation has a global-to-local flow and naturally leads into the generated question.
  • Solver rewards: Solver rewards valid answers through format and accuracy, while invalid-question explanations are scored against DGE-provided diagnostic reasons.Hard format failure on a valid question incurs an overall penalty of −1.

C.3.2. Accuracy Scoring for All 16 Tasks

Accuracy scoring compares predictions directly with DGE-computed ground truth rather than model consensus, using task-specific metrics across the 16 spatial reasoning tasks. Invalid questions also generate explanation-based supervision tied to authoritative diagnostics.

  • Deterministic scoring: DGE scoring evaluates samples against deterministic geometric ground truth instead of majority-vote consensus.This applies the same objective source of supervision used to validate generated questions.
  • Metric-specific accuracy: Continuous-valued tasks use relative-error accuracy over 11 equally spaced tolerance thresholds from coarse to strict.Object size, absolute distance, room size estimation, and camera-object distance use this metric.
  • Metric-specific accuracy: Object counting receives piecewise absolute-error rewards, while relative distance uses exact label matching and directional tasks use normalized direction-set comparison.The remaining categorical tasks use exact matching after task-specific label normalization.
  • Invalid-question supervision: Rejected questions become explanation-training examples rather than being discarded, teaching the model to identify the applicable geometric or task constraint.Explanation quality is graded against structured invalidity reasons and validation evidence.
  • Invalid-question supervision: The explanation judge receives the final invalid reason, error code, failure stage, extraction status, and validation issues as authoritative diagnostic signals.It prioritizes simulator conclusions over fluent but unsupported explanations.

D.1.1. Backbone Model and Training Configuration

SpatialEvo evaluates 3B and 7B Qwen2.5-VL backbones with online GRPO as the default training paradigm, using a two-round Questioner–Solver architecture and pre-filtered multi-source visual contexts.

  • Backbones: Main experiments use Qwen2.5-VL-7B-Instruct and Qwen2.5-VL-3B-Instruct, while other experiments default to the 7B backbone.The setup therefore tests SpatialEvo at both reported model scales.
  • Training paradigm: Online GRPO reinforcement learning is the default and is applied without an SFT warm-start; SFT appears only as a comparison baseline.Hyperparameters for the SFT and RL stages are reported separately.
  • Training configuration: The RL implementation runs on 1 node × 8 H800 GPUs with two rounds: Questioner generation followed by Solver answer generation.Four rollouts per round yield up to 16 candidate question-answer chains per context before deduplication.
  • Training configuration: Auxiliary entity extraction and judging calls share a GPT-OSS-120B text backend, keeping these operations text-only and reducing system heterogeneity.The shared backend handles DGE extraction, observation judging, and invalid-question explanation judging.
  • Data configuration: Online RL samples pre-filtered scene-level, image-pair, and single-image contexts from ScanNet, ScanNet++, and ARKitScenes while generating questions and answers online.Filtering imposes visibility and shared-object requirements and limits contexts sampled per modality.

D.3. Ablation and Analysis Experiment Details

The ablation isolates physically grounded supervision by replacing DGE-derived ground truth with model-voted pseudo-labels, while supplementary experiments specify reproduced baselines and a controlled SpatialEvo configuration.

  • Ablation Setup: Round 2 supervision is the sole variable in the majority-vote pseudo-label ablation.All other ablation experiments use the main method’s training configuration.
  • Ablation Setup: The w/o Physical Grounding variant replaces DGE ground truth with task-specific aggregates of the model’s sampled responses.Aggregation uses majority vote, integer-rounded plurality, or a median from the largest tolerance-clustered group, depending on task type.
  • Ablation Setup: The Round 2 reward keeps the main method’s format–content decomposition but evaluates content against vote-based pseudo-labels.This design isolates the contribution of physically grounded supervision.
  • Baseline Comparisons: SpatialLadder is reproduced from the Qwen2.5-VL-3B backbone on its official 26K corpus under unified SFT settings, while SpaceR and Spatial-SSRL use released Qwen2.5-VL-3B checkpoints.The GRPO comparison trains on SpatialLadder with rollout multiplicity 8 to match the original paper.
  • Controlled Configuration: The controlled SpatialEvo configuration uses ScanNet’s 992 training scenes and six task categories, with training continuing to approximately 20K non-redundant examples.RL hyperparameters follow Table 9 except for dynamically determined training epochs.

E. Supplementary Qualitative Cases

Qualitative cases track Questioner and Solver evolution on single-view relative direction, showing increasingly grounded observations and increasingly structured geometric reasoning during training.

  • Qualitative Setup: The qualitative analysis uses single-view relative direction as a representative task for Questioner and Solver evolution.Figures 4 and 5 visualize these changes across training steps.
  • Questioner Evolution: By Step 1116, the Questioner produces more structured observations involving object sizes, foreground-background relationships, and relative positions.At Step 20, its brief descriptions provide only coarse positional information and weak visual grounding.
  • Solver Evolution: At Step 136, the Solver gives an incorrect answer with an unstructured reasoning process that lacks a coherent geometric chain.This contrasts with later reasoning that explicitly identifies positions and infers spatial relationships.
  • Solver Evolution: At Steps 1116 and 1860, the Solver uses multi-step or enumerated reasoning to derive precise answers through explicit spatial sub-steps.Both later examples receive Reward 1.0.
Loading 2604.14144v1…