Source-linked AI summary
SpaceDG: Benchmarking Spatial Intelligence under Visual Degradation
Xiaolong Zhou, Yifei Liu, Ziyang Gong, Jiarui Li, Qiyue Zhao, Muyao Niu, Yuanyuan Gao, Le Ma, Xue Yang, Hongjie Zhang, Zhihang Zhong
TL;DR
Existing spatial benchmarks largely assume clean visual inputs, leaving MLLM spatial robustness under real-world degradations insufficiently measured. SpaceDG addresses this with a 3DGS-based dataset and benchmark, finding consistent degradation-related reasoning losses and substantial gains from degradation-aware fine-tuning.
Problem
Existing spatial benchmarks predominantly assume clean, high-resolution, well-illuminated images, leaving MLLM spatial intelligence under imperfect visual observations insufficiently evaluated.
Method
SpaceDG reconstructs geometry-consistent 3DGS scenes, synthesizes nine physically grounded degradations, and generates and verifies spatial QA for a large dataset and benchmark.
Results
Visual degradations consistently impair spatial reasoning across evaluated MLLMs, while degradation-based SFT substantially improves performance on clean and degraded inputs.
Takeaways & Limitations
Degradation-aware spatial knowledge and training improve robustness to imperfect visual inputs within the evaluated benchmark setting.
Takeaways & Limitations
SpaceDG is primarily restricted to indoor environments and currently supports only nine representative degradation types.
Abstract
from arXiv · showhide
Multimodal Large Language Models (MLLMs) have made rapid progress in spatial intelligence, yet existing spatial reasoning benchmarks largely assume pristine visual inputs and overlook the degradations that commonly occur in real-world deployment, such as motion blur, low light, adverse weather, lens distortion, and compression artifacts. This raises a fundamental question: how robust is the spatial intelligence of current MLLMs when visual observations are imperfect? To answer this question, we introduce SpaceDG, the first large-scale dataset for degradation-aware spatial understanding. It is constructed with a physically grounded degradation synthesis engine that embeds degradation formation process into 3D Gaussian Splatting (3DGS) rendering, enabling realistic simulation of nine degradation types. The resulting dataset contains approximately 1M QA pairs from nearly 1,000 indoor scenes. We further introduce SpaceDG-Bench, an human-verified benchmark with 1,102 questions spanning 11 reasoning categories and 9 visual degradation types, yielding over 10K VQA instances. Evaluating 25 open- and closed-source MLLMs reveals that visual degradations consistently and substantially impair spatial reasoning, exposing a critical robustness gap. Finally, we show that finetuning on SpaceDG markedly improves degradation robustness and can even surpass human performance under degraded conditions without any performance drop on clean images, highlighting the promise of degradation-aware training for robust spatial intelligence.
1 Introduction
Existing spatial benchmarks largely assume clean visual observations, leaving MLLM spatial intelligence under realistic degradation insufficiently evaluated. SpaceDG addresses this gap with a physically grounded dataset and benchmark, then shows broad degradation-related performance losses and benefits from degradation-aware training.
- Existing benchmarks predominantly evaluate MLLMs with clean, high-resolution, well-illuminated images despite degradations arising in real-world sensing pipelines.
- SpaceDG introduces the first large-scale dataset and benchmark dedicated to degradation-aware spatial understanding under imperfect visual observations.The benchmark is designed to introduce realistic degradations, preserve underlying 3D structure, support diverse reasoning tasks, and provide reliable ground truth.
- The data engine reconstructs multi-view images into 3DGS and simulates nine representative degradations across optical, meteorological, photometric, and digital categories.
- SpaceDG contains approximately 1M QA pairs from nearly 1,000 scenes, while SpaceDG-Bench provides a curated evaluation of 11 spatial question categories.
- Visual degradations consistently impair spatial reasoning across 25 evaluated MLLMs, and humans also experience clear performance drops under degraded conditions.
- Degradation-based SFT improves performance on both clean and degraded inputs, while fine-grained object perception is more affected than some geometric reasoning tasks.
2 Related works
Prior work has advanced spatial MLLMs and standardized robustness evaluation for common image corruptions, but these lines of research remain only partially connected. Existing robustness studies mainly target semantic or basic visual tasks, while 3DGS and degradation-synthesis research supplies relevant representations and corruption models.
- Recent spatial MLLMs progress from basic visual understanding toward fine-grained spatial reasoning, supported by large-scale spatial datasets and benchmarks.
- Visual degradation robustness has been studied using standardized corruptions, but existing MLLM evaluations mainly focus on semantic understanding, object recognition, or basic visual reasoning.
- 3DGS research improves scene representation for reconstruction and novel-view synthesis, alongside work modeling realistic degradations such as motion and defocus blur.
3 SpaceDG
SpaceDG combines geometry-consistent 3D scene reconstruction, physically grounded degradation synthesis, structured spatial QA generation, and human verification. Its dataset and benchmark cover varied views, spatial task groups, and degradation conditions for systematic evaluation.
- SpaceDG uses a 3DGS-based data engine with physically realistic degradation synthesis, diverse spatial tasks, multiple viewpoints, and varied visual degradations.
- Geometry-consistent 3DGS reconstruction provides calibrated camera poses and dense depth for rendering degradation variants without changing the underlying scene geometry.
- QA generation samples valid single-view and two-view observations, then computes physically grounded answers from camera extrinsics, 3D box centers, object extents, and relative directions.
- The engine organizes nine degradations into optical and dynamic, meteorological, photometric, and digital categories while preserving invariant 3D spatial ground truth.
- 971,090 QA instances cover 584 indoor scenes, while SpaceDG-Bench contains 1,102 manually verified questions from 320 disjoint scenes.
- The benchmark evaluates 11 question categories spanning camera-centric, object-centric, and camera-object relational reasoning in single-view and multi-view settings.
- A two-stage quality pipeline combines VLM filtering with human review to remove ambiguous questions, incorrect answers, and ill-formed options.
4 Experiments
Experiments evaluate 25 models on clean and degraded images, quantify degradation sensitivity, and compare model and human robustness. Degradations consistently reduce performance, while spatial fine-tuning and SpaceDG-based SFT improve robustness.
- Evaluation Setup: 25 models are evaluated on clean images and nine visual degradations using accuracy and mean relative accuracy tailored to answer format.The evaluation includes proprietary, open-source, spatially fine-tuned, and robotic-brain models.
- Model Performance: 6.4 percentage points: Gemini-3.1-Pro declines from 63.1% on clean images to 56.7% on degraded images.InternVL3.5-38B similarly drops from 52.9% to 47.7%.
- Human Baseline: 20.9%: human performance drops under degraded conditions from an 80.4% clean-image baseline.Humans are evaluated on a 900-question subset and substantially outperform MLLMs on clean images.
- Degradation-aware SFT: 73.2% clean and 66.1% degraded: SpaceDG-SFT-Qwen3 improves from 49.1% and 42.1% respectively, surpassing the 59.5% degraded human reference by 6.6 points.The model is fine-tuned for one epoch on SpaceDG.
- Robustness and Prompting: 5.5% versus 7.6%: spatially fine-tuned and robotic-brain models show smaller clean-to-degraded declines than general models.When degradation type and severity are explicitly provided, general models benefit consistently, whereas spatially fine-tuned models show little or no improvement.
- Degradation-wise Analysis: Low-light and haze produce the strongest degradation correlations, while over-exposure and distortion are weaker; object-centric tasks are most sensitive.The analysis uses absolute point-biserial correlation |r|, where larger values indicate larger score shifts between clean and degraded conditions.
5 Degradation-guided Spatial Reasoning
The paper tests whether degradation-guided reasoning helps MLLMs by prompting them to identify degradation type and severity before solving spatial questions. Structured CoT instead slightly harms performance, while explicit degradation information benefits general models more than spatially fine-tuned models.
- CoT Evaluation: 1.8% decrease: Gemini-3.1-Flash-Lite performs worse with the structured two-stage CoT prompt.The prompt first requests degradation identification and description, followed by standard spatial reasoning.
- Error Analysis: The error analysis identifies degradation attribution errors and spatial relation distortion among four degradation-induced reasoning failures.Misidentifying the corruption can create an incorrect reasoning premise, while degraded cues can bias orientation and relative-position judgments.
6 Conclusion
SpaceDG studies MLLM spatial intelligence under realistic visual degradations using a 3DGS-based dataset and human-verified benchmark. Degradations impair reasoning, especially fine-grained object perception, while SpaceDG fine-tuning improves clean and degraded performance.
- Conclusion: SpaceDG and SpaceDG-Bench evaluate MLLM spatial intelligence under nine realistic degradation types across diverse spatial reasoning categories.The dataset uses a physically grounded degradation synthesis engine built on 3DGS, and the benchmark is human-verified.
- Conclusion: Visual degradations consistently impair spatial reasoning, with fine-grained object-level perception especially vulnerable and some global geometric tasks relatively more robust.The conclusion reports a substantial robustness gap between clean and imperfect visual conditions.
- Conclusion: Supervised fine-tuning on SpaceDG substantially improves both clean and degraded performance, supporting degradation-aware training as a promising robustness direction.The stated conclusion does not report a performance drop on clean images.
A.1 Data Validation
Degradation-aware SFT improves robustness to both individual and compound corruptions while preserving clean-image performance.
- 73.2% vs. 73.1% clean-input performance shows nearly identical results for degradation-augmented and clean-image SFT.
- 64.6% to 66.1% average degraded-input performance demonstrates gains from degradation-augmented SFT.
- Held-out variants generalize to unseen corruptions and substantially outperform the no-SFT baseline.
- 62.4 average accuracy for SpaceDG-SFT-Qwen3-VL-8B-Instruct versus 37.0 for the base model under six compound-degradation settings.
A.3 Will supervised fine-tuning on degraded dataset affect the performance of general benchmarks?
The evaluation tests whether degraded-data fine-tuning harms general spatial capabilities, finding competitive performance on MMSI-Bench and MindCube.
- SpaceDG-SFT-8B is evaluated on MMSI-Bench and MindCube to assess general-benchmark performance after degraded-data fine-tuning.
- 30.0 and 37.0 are the scores achieved by SpaceDG-SFT-8B on MMSI-Bench and MindCube, respectively.
- The model outperforms comparable Qwen3-VL-8B and SpaceI-SFT-7B baselines while remaining competitive with Intern3-VL-8B and VST-SFT-7B.
- The findings suggest that degraded-dataset fine-tuning does not significantly compromise general capabilities.
A.4 Human-level Assessment
The paper validates a 900-question human-assessment subset and reports that humans perform well on clean spatial reasoning overall but struggle with precise metric estimation.
- 0.83% average absolute performance difference supports the reliability of SpaceDG-Bench-900 relative to the full 9,918-sample benchmark.
- 80.4% overall human accuracy is achieved on clean images, including 90.2% on MCA questions.
- 61.2% accuracy on NA questions and 48.1% on list-type NA questions indicate difficulty with precise metric estimation.
- The evaluation uses degradation-aware prompts that provide the degradation type and severity before spatial reasoning.
- Gemini-3.1-Flash-Lite first identifies the degradation type and then performs step-by-step spatial reasoning.
B.1 Optical and Dynamic Degradations
SpaceDG models optical, dynamic, meteorological, photometric, and digital degradations within a geometry-consistent 3DGS rendering pipeline.
- Optical and Dynamic Degradations: Nine realistic degradations are synthesized through camera-physics and motion-aware operations embedded in the 3DGS rasterizer.
- Optical and Dynamic Degradations: Defocus uses a thin-lens approximation based on rendered depth, focus depth, and aperture size.
- Optical and Dynamic Degradations: Distortion modifies pinhole projection with an equidistant polynomial model and recomputes the fisheye Jacobian for covariance projection.
- Optical and Dynamic Degradations: Motion blur integrates linear light over interpolated camera poses, using N = 80 sub-steps to preserve directional and rotational realism.
- Meteorological Degradations: Haze uses depth-dependent atmospheric scattering, while water droplets use procedural surface normals, refraction offsets, optical blur, and specular highlights.
- Photometric Degradations: Low-light and over-exposure model sensor effects through exposure changes, noise injection, and saturation clipping before encoding.
- Digital Degradations: JPEG compression removes high-frequency coefficients through DCT quantization, whereas low-resolution downsamples and upsamples images, truncating spatial detail.
- QA Construction: The QA pipeline combines view-invariant instance descriptions, covisibility-based view sampling, baseline constraints, and ambiguity filtering.
F Additional QA Examples
The paper provides complete SpaceDG QA examples for multiple degradation types, including questions, ground truth, model answers under clean conditions, and reasoning traces.
- The examples include the complete question and its ground-truth answer.
- Each example compares the model’s answer under clean conditions with the ground truth.
- The examples show generated reasoning processes and label responses that are incorrect.
G Limitations
SpaceDG is limited primarily by its indoor-scene scope and its support for only nine representative degradation types, leaving several real-world conditions for future work.
- The dataset is primarily restricted to indoor environments because it is built upon ScanNet++.
- The degradation engine currently supports only nine representative degradation types to preserve physical realism and geometric consistency.
- Rain, snow, lens flare, rolling-shutter artifacts, and complex compound corruptions remain outside the current coverage.