Source-linked AI summary
ConsiSpace: Learning Geometric Consistency Matters for Video Spatial Reasoning
Ting Huang, Zhenyu Zhang, Wenyuan Huang, Jian Yang, Hao Tang
TL;DR
Video spatial reasoning models struggle to efficiently and robustly aggregate consistent spatial evidence across changing viewpoints. ConsiSpace organizes evidence with geometry-consistent memory and trains cross-view consistency, improving average benchmark scores by 12.6 points over the strongest baselines.
Problem
Video spatial reasoning faces efficiency and robustness challenges because redundant or mismatched evidence can increase computation and destabilize spatial grounding across viewpoints.
Method
ConsiSpace combines geometry-consistent memory with geometry-guided evidence organization and post-SFT UC-SSRL to improve cross-view spatial consistency.
Results
12.6 points: ConsiSpace improves average scores over the strongest baselines across three video spatial-reasoning benchmarks.
Takeaways & Limitations
The results support geometric consistency as a practical principle for more robust video spatial reasoning.
Takeaways & Limitations
ConsiSpace relies on predicted pose and depth, while strict compute budgets may require additional memory compression or learned eviction strategies.
Abstract
from arXiv · showhide
Video spatial reasoning is essential for navigation-oriented perception and long-video question answering, where models must infer spatial relations across long horizons under changing viewpoints. However, existing multimodal large language models (MLLMs) remain largely semantic-centric, and often fail to reliably aggregate consistent spatial evidence from redundant video observations, leading to inefficient or unstable reasoning. To address these issues, we propose ConsiSpace, a geometry-consistency-aware framework for geometry-sensitive video spatial reasoning that turns spatial consistency into both an evidence organization principle and an explicit post-SFT learning signal. We build a geometry-consistent memory (GCM) including implicit evidence tokens and explicit geometric cues, and leverage efficient organization strategies to compactly preserve task-related spatial evidence. Furthermore, we utilize unified consistency self-supervised reinforcement learning (UC-SSRL) after supervised fine-tuning to improve cross-view stability, with answer-, metric-, and topology-consistency rewards. Extensive experiments on three spatial-reasoning benchmarks, VSI-Bench, OSI-Bench, and MMSI-Video-Bench, show consistent gains, improving the average score by 12.6 points over the strongest baselines.
1 Introduction
Video spatial reasoning requires models to infer spatial structure across partial observations and long video contexts, yet current MLLMs remain inefficient and unstable under viewpoint changes. ConsiSpace addresses these challenges with geometry-consistent evidence organization and post-SFT consistency learning, improving average performance by 12.6 points over the strongest baselines.
- Motivation: Video spatial reasoning requires inferring object locations, scene connectivity, directions, and spatial relations under partial observability or long video contexts.It supports navigation-oriented perception and video question answering.
- Motivation: Current MLLMs remain a bottleneck for reliable spatial reasoning because they emphasize semantics while weakly modeling viewpoint geometry and spatial structure.Recent spatial benchmarks expose this limitation despite rapid progress in MLLMs.
- Challenges: Redundant adjacent-frame evidence increases memory and computation without proportional gains, while redundant or mismatched evidence destabilizes spatial grounding and answers across viewpoints.These challenges are identified as efficiency and robustness problems under viewpoint changes.
- ConsiSpace: ConsiSpace uses 3D spatial consistency as both an evidence-organization principle and a post-SFT learning objective.Its Geometry-Consistent Memory stores visual-spatial evidence with explicit pose and depth cues and applies geometry-guided writing, fusion, and retrieval.
- Results: 12.6 points: ConsiSpace improves the average score over the strongest baselines across three spatial-reasoning benchmarks.The framework combines geometry-guided memory with UC-SSRL for post-SFT consistency learning and viewpoint stability.
2 Related Work
Prior work has improved multimodal understanding and long-video scalability, yet reliable spatial reasoning remains challenging for general-purpose MLLMs. ConsiSpace addresses an underexplored gap by using geometry as a consistency criterion for long-video evidence management.
- Multimodal large language models: General-purpose MLLMs are strong in semantic perception and language-conditioned reasoning but often lack explicit mechanisms for reliable spatial reasoning.Recent spatial benchmarks show that reliable spatial reasoning remains challenging.
- Multimodal large language models: Long-video MLLMs and memory-augmented agents improve temporal scalability through longer contexts, retrieval, or external memory.These approaches target temporal scalability, while spatial reliability remains difficult.
- Geometry-aware evidence management: Geometry priors are mostly used as input features or training signals, leaving their role in long-video evidence management less explored.ConsiSpace instead uses geometry to decide when to write, how to fuse, and what to retain during hierarchical retrieval.
3 Method
ConsiSpace organizes long-video spatial evidence with a geometry-consistent memory and refines cross-view reasoning through unified consistency self-supervised reinforcement learning. Its memory combines semantic and geometric representations with geometry-aware writing, fusion, and retrieval, while training rewards answer, metric, and topological stability.
- Overall framework: ConsiSpace combines dual encoders, geometry-consistent memory, query-conditioned fusion, and a video language model to stabilize spatial reasoning over redundant, viewpoint-dependent observations.UC-SSRL is applied after supervised fine-tuning to refine viewpoint stability with self-supervised consistency rewards.
- Evidence representations: Frozen SigLIP2 extracts visual tokens, while frozen VGGT produces spatial tokens and geometric cues including camera pose and per-frame depth.These representations provide semantic evidence and geometric consistency signals for memory writing, fusion, and retrieval.
- Geometry-Consistent Memory: The memory separates implicit evidence tokens from explicit camera-pose and depth cues, decoupling evidence retrieval from geometry-guided memory control.Implicit memory stores visual and spatial tokens; explicit memory stores aligned geometric cues with matching timestamps.
- Geometry-Consistent Memory: Geometry-gated writing suppresses redundant frame entries, while geometry-consistent fusion consolidates nearby, semantically similar evidence into a compact scene memory.Writes occur when predicted positional or angular changes exceed thresholds, and consistent entries are fused using semantic similarity.
- Geometry-Consistent Memory: Geometry-filtered retrieval first selects semantically relevant chunks, then reranks frames using semantic relevance and viewing-direction compatibility before adding topological and metric summary tokens.The resulting context concatenates the query token, summaries, retrieved evidence, and aligned geometric cues for video-language-model prediction.
- Unified consistency rewards: UC-SSRL samples paired views of the same question and uses complementary rewards for answer-level, metric, and relational consistency.Answer consistency penalizes KL divergence, metric consistency penalizes absolute distance-estimate differences, and topological consistency encourages symmetric relational agreement.
4 Experiments
ConsiSpace consistently improves video spatial reasoning across VSI-Bench, OSI-Bench, and MMSI-Video-Bench, while scaling more efficiently to longer inputs. Ablations show that geometry-consistent memory and UC-SSRL rewards are central to its accuracy, stability, and robustness.
- Benchmark Results: 76.6 Avg. on VSI-Bench exceeds SpaceMind by +7.0 points and Qwen3-VL-8B-Instruct by +19.2 points across numerical and multiple-choice tasks.ConsiSpace also reaches 53.0 Avg. on OSI-Bench, improving over VLM-3R by +12.7 points and Qwen3-VL-8B-Instruct by +21.8 points.
- Benchmark Results: 57.5 Avg. under Sufficient-Coverage and 58.1 Avg. under Uniform-50 outperform VLM-3R by +14.9 and +15.0 points, respectively, on MMSI-Video-Bench.UC-SSRL improves over SFT in both input settings, benefiting evidence-rich and fixed-budget inputs.
- Efficiency and Scaling: At 200 frames, VLM-3R requires 78.5GB memory and 10.80s, while ConsiSpace maintains higher accuracy with lower inference-time growth as input budgets increase.Measurements use the same A100 80GB GPU across systems.
- Ablations: Removing geometry-gated writing retains 2.3× more entries and removing geometry-consistent fusion retains 1.8× more, with both variants reducing accuracy.Geometry-filtered retrieval leaves entry count unchanged but degrades OSI and MMSI-Video performance, showing that filtering improves retrieval quality.
- Ablations: Each UC-SSRL reward improves over SFT, pairwise combinations add gains, and the full answer-, metric-, and topology-consistency objective performs best across VSI, OSI, and MMSI-Video.The rewards are complementary and refine cross-view stability from an SFT-initialized model.
- Robustness: Performance degrades gradually under perturbed depth, translation, and rotation geometry, while ConsiSpace remains consistently stronger than GeoThinker.The result indicates that GCM benefits from geometry cues without requiring perfect geometry estimates.
5 Conclusion
ConsiSpace is a geometry-consistent framework for video spatial reasoning that organizes spatial evidence through Geometry-Consistent Memory and promotes cross-view spatial agreement with Unified Consistency SSRL after supervised fine-tuning.
- 5 Conclusion: ConsiSpace builds Geometry-Consistent Memory on generic video encoders and VLMs to manage spatial evidence.Its evidence management uses geometry-gated writing, geometry-consistent fusion, and geometry-filtered retrieval.
- 5 Conclusion: Geometry-gated writing, geometry-consistent fusion, and geometry-filtered retrieval reduce redundancy and retrieve query-relevant spatial cues.
- 5 Conclusion: Unified Consistency SSRL encourages cross-view spatial agreement after supervised fine-tuning.
A Outdoor Data Synthesis
The paper introduces an automated pipeline that synthesizes nuScenes-10K, an outdoor spatial-reasoning instruction set designed to complement indoor VSI-590K data with outdoor geometric patterns.
- A Outdoor Data Synthesis: nuScenes-10K is automatically synthesized from nuScenes as an outdoor spatial-reasoning instruction set.The pipeline preserves supervision compatible with the video spatial reasoning formulation.
- A Outdoor Data Synthesis: The dataset adds long-range depth, large-scale scenes, and dynamic agents to complement indoor VSI-590K training data.
A.1 Data Sources and Representations · A.2 Automated Instruction and Answer Generation
The data pipeline uses calibrated nuScenes multi-camera driving clips with geometric representations aligned to the ConsiSpace interface. It automatically generates diverse spatial questions and geometry-derived answers across topological, metric, motion, and interaction categories.
- A.1 Data Sources and Representations: nuScenes sequences provide calibrated multi-camera imagery, ego poses, 3D object annotations, and optional LiDAR-based geometric verification.Driving clips are sampled as short temporal windows for multi-view video input.
- A.1 Data Sources and Representations: The outdoor samples reuse the main pipeline’s pose and depth extraction stack under the same frame-budget and input-interface settings.This keeps the representations consistent with ConsiSpace.
- A.2 Automated Instruction and Answer Generation: Question-answer pairs cover topological and relational families, including relative direction, relative ordering, and ego-heading-conditioned spatial relations.These questions target spatial relations across the sampled clips.
- A.2 Automated Instruction and Answer Generation: Metric questions ask about approximate distance, size comparison, and nearest or farthest objects, using numeric, binned, or multiple-choice answers.The answer formats support both quantitative and discretized evaluation.
- A.2 Automated Instruction and Answer Generation: Motion and interaction questions determine whether an agent approaches, recedes, crosses, or changes its relative state within the temporal window.The temporal formulation captures changes in agent relationships over each clip.
- A.2 Automated Instruction and Answer Generation: Templates provide controlled lexical diversity, while answers are computed from calibrated geometry by transforming 3D object centers into the ego frame.Relative directions use azimuth relative to the ego forward axis, and metric quantities use Euclidean distance with optional depth verification.
- A.2 Automated Instruction and Answer Generation: Multiple-choice questions use fixed option sets and retain only samples with unambiguous labels.This filtering ensures that generated choices have clear answer assignments.
A.3 Quality Control and Verification
The dataset undergoes multi-stage filtering and manual verification to improve QA reliability, producing 10,000 final pairs with 91.6% correctness on a random sample. No shared videos or annotations are found with OSI-Bench, and the final format supports mixed SFT training with VSI-590K.
- Filtering: 10,000 final QA pairs remain after filtering 22,518 generated candidates by geometry validity, answer parseability, consistency, MLLM verification, and de-duplication.The filtering pipeline targets label reliability through multiple validation stages.
- Overlap Check: No shared videos or annotations exist between nuScenes-10K and OSI-Bench, which use different viewpoints, annotations, and QA protocols.Only generic object-category overlap is observed between the datasets.
- Human Verification: 91.6% of 1,000 randomly sampled QA pairs are correct under manual verification.Reported errors mainly involve ambiguous references, occlusion or distance ambiguity, sparse annotations, and incorrect metric or relation answers.
- Training Compatibility: The final data format matches indoor instruction tuning, enabling mixed SFT training with VSI-590K.This preserves compatibility with the stated supervised fine-tuning setup.
B More Training and Implementation Details … B.5 Inference Protocol
The paper details a lightweight ConsiSpace implementation built with LoRA adaptation, geometry-consistent memory, and post-SFT UC-SSRL. It also specifies training configurations and standardized benchmark inference protocols for controlled evaluation.
- B.1 Backbone and Adaptation: ConsiSpace uses Qwen3-VL-8B-Instruct with LoRA on language-model linear layers while freezing the visual encoder, aligner, and geometry encoder.Training uses bfloat16 precision with DeepSpeed ZeRO-3, making UC-SSRL a post-SFT refinement stage rather than full-model retraining.
- B.1 Backbone and Adaptation: LoRA is configured with rank 16 and scaling factor α=32.Only LoRA parameters are trainable, keeping the trainable-parameter ratio small.
- B.2 SFT Settings: SFT instruction-tunes the model on indoor VSI-590K and outdoor nuScenes-10K using next-token prediction.The default configuration uses size 1, gradient accumulation 2, and 200 training steps.
- B.2 SFT Settings: SFT uses a maximum sequence length of 5,120 with right truncation and efficiency features including FlashAttention and padding-free batching.The implementation also uses gradient checkpointing and multi-process data loading.
- B.3 GCM Configuration: GCM retrieval uses fused hierarchical selection with top-K chunk retrieval and top-K frame retrieval both set to K=8.Each chunk contains 8 value tokens, each selected frame contributes 4 value tokens, and FAISS plus key caching reduce retrieval overhead.
- B.4 UC-SSRL Settings: UC-SSRL starts from the SFT checkpoint, updates only LoRA parameters, and uses the same batch size, step count, and optimizer setting as SFT.Its overall weight is 1×10−2, while paired contexts perturb temporal windows and retrieval contexts without changing the scene or question.
- B.5 Inference Protocol: Evaluation follows each benchmark’s official protocol, using MCA and NA for VSI-Bench and OSI-Bench and Sufficient-Coverage and Uniform-50 for MMSI-Video-Bench.Methods share decoding and frame-budget settings, while efficiency and scalability experiments use the same A100 80GB GPU setting.
C Visualization
Qualitative examples across three benchmarks show ConsiSpace handling diverse video spatial-reasoning settings. Across these examples, its answers are closer to ground truth than the baseline, supporting geometry-consistent evidence organization.
- Benchmark examples: Figs. 8–10 present qualitative examples from VSI-Bench, MMSI-Video-Bench, and OSI-Bench.The examples span multiple video spatial-reasoning benchmarks.
- Task coverage: The examples cover metric estimation, relative-direction reasoning, object counting, navigation, and motion-related questions.These tasks represent varied spatial-reasoning settings.
- Qualitative comparison: Compared with the baseline, ConsiSpace produces answers closer to the ground truth across different spatial-reasoning settings.The qualitative comparisons support the effectiveness of geometry-consistent evidence organization.
D Limitation
ConsiSpace remains limited by geometry estimation errors, threshold-based memory control, and the challenges of scaling to extremely long videos. Future work targets improved geometry estimation, adaptive memory control, and broader consistency learning.
- Future work: Future work will improve geometry estimation, develop adaptive memory control, and extend consistency learning to broader video spatial reasoning settings.These directions directly address the identified limitations.
- Geometry estimation errors: Geometry estimation errors under motion blur, reflective surfaces, or textureless regions can degrade memory writing, fusion, and retrieval quality.ConsiSpace uses predicted pose and depth cues for memory control.
- Threshold-based memory control: Threshold-based memory control remains a limitation, although sensitivity analysis shows stable performance across a range of thresholds.Adaptive gating and fusion may improve robustness across diverse scenes.
- Long-horizon scaling: Extremely long videos may require stronger memory compression or learned eviction strategies to maintain bounded compute under strict budgets.This is the paper’s long-horizon scaling limitation.