Source-linked AI summary
Exploring Spatial Intelligence from a Generative Perspective
Muzhi Zhu, Shunyao Jiang, Huanyi Zheng, Zekai Luo, Hao Zhong, Anzhou Li, Kaijun Wang, Jintao Rong, Yang Liu, Hao Chen, Tao Lin, Chunhua Shen
TL;DR
Spatial research has largely measured understanding rather than whether models can enforce 3D constraints while generating images. The paper introduces GSI-Bench, combining real and synthetic grounded-editing benchmarks with unified evaluation, and finds that GSI-Syn fine-tuning improves spatial editing and downstream understanding.
Problem
Existing spatial-intelligence research predominantly uses understanding-oriented supervision, leaving generative spatial intelligence and its transfer to spatial understanding underexplored.
Method
The paper introduces GSI-Bench, comprising GSI-Real and GSI-Syn, and operationalizes generative spatial intelligence through seven grounded spatial-editing operations and automated evaluation.
Results
Fine-tuning on GSI-Syn improves spatial compliance and transfers to real-world editing and downstream spatial-understanding tasks.
Takeaways & Limitations
Generative training can strengthen spatial reasoning in unified multimodal models, providing evidence for a generative route to advancing spatial intelligence.
Abstract
from arXiv · showhide
Spatial intelligence is essential for multimodal large language models, yet current benchmarks largely assess it only from an understanding perspective. We ask whether modern generative or unified multimodal models also possess generative spatial intelligence (GSI), the ability to respect and manipulate 3D spatial constraints during image generation, and whether such capability can be measured or improved. We introduce GSI-Bench, the first benchmark designed to quantify GSI through spatially grounded image editing. It consists of two complementary components: GSI-Real, a high-quality real-world dataset built via a 3D-prior-guided generation and filtering pipeline, and GSI-Syn, a large-scale synthetic benchmark with controllable spatial operations and fully automated labeling. Together with a unified evaluation protocol, GSI-Bench enables scalable, model-agnostic assessment of spatial compliance and editing fidelity. Experiments show that fine-tuning unified multimodal models on GSI-Syn yields substantial gains on both synthetic and real tasks and, strikingly, also improves downstream spatial understanding. This provides the first clear evidence that generative training can tangibly strengthen spatial reasoning, establishing a new pathway for advancing spatial intelligence in multimodal models.
1. Introduction
Existing spatial-intelligence research emphasizes understanding through recognition, question answering, and perception, while generation’s role remains underexplored. This paper introduces GSI-Bench to measure and improve generative spatial intelligence through grounded image editing.
- Spatial intelligence supports grounding language in 3D space and enables navigation, robotic manipulation, and 3D scene understanding.
- Existing spatial-intelligence benchmarks predominantly evaluate recognition, question answering, perception, or offline diagnostics rather than spatial generation.
- The paper asks whether generation can provide spatial intelligence, whether generative spatial intelligence can be measured reliably, and whether it transfers to understanding tasks.
- GSI-Bench operationalizes generative spatial intelligence through spatially grounded image editing across real-world and synthetic scenes.
- GSI-Real and GSI-Syn provide complementary real-world and synthetic benchmarks built with 3D grounding, controllable operations, automated processing, and human verification.
- Fine-tuning unified multimodal models on GSI-Syn enhances generative spatial intelligence and improves downstream spatial understanding.
2. Related Work
Prior work has mainly assessed spatial understanding, while unified multimodal models increasingly combine image understanding and generation. Recent systems add controllable editing and spatial manipulation, but generative spatial intelligence remains the paper’s distinct focus.
- Existing spatial benchmarks assess temporal video reasoning, sparse-view 3D modeling, and dimensions such as dynamic reasoning, spatial interaction, and perspective taking.
- Methodological work has sought to enhance spatial understanding in multimodal large language models.
- Unified multimodal models increasingly support both image understanding and generation, including controlled editing, multi-image conditioning, and pose or object manipulation.
3. Generative Spatial Intelligence
The paper defines generative spatial intelligence as actively enforcing spatial constraints during image generation rather than merely perceiving them. It formulates evaluation as grounded editing over structured 3D transformations and seven operation categories.
- 3.1. What is Generative Spatial Intelligence?: Generative Spatial Intelligence (GSI) is the ability to respect, reason about, and manipulate spatial constraints during image generation.
- 3.2. Task Formulation: Open-ended text-to-image prompts are difficult to evaluate quantitatively because they are ambiguous and lack unique ground-truth targets.
- 3.2. Task Formulation: The grounded-editing task maps an input image and spatial instruction to an output image satisfying the intended transformation while preserving realism and semantic consistency.
- 3.2. Task Formulation: Scenes encode object centers, sizes, rotations, and camera parameters, allowing spatial edits to be represented as structured 3D transformations.
- 3.2. Task Formulation: Spatial instructions specify target objects, actions, and a transformation from source to destination scene states.
- 3.3. Categories of Spatial Operations: The benchmark defines seven quantitatively grounded operations spanning object-, camera-, and scene-level transformations.
4. GSI-Bench Construction
GSI-Bench combines synthetic simulation and real-world reconstruction with geometric operation generation, validation, filtering, and human or multimodal quality control. The resulting datasets support physically grounded, scalable spatial-editing evaluation.
- 4.1. Synthetic Benchmark: GSI-Syn: GSI-Syn uses simulators, diverse actionable viewpoints, geometric checks, and physics execution to generate valid spatial-editing pairs.
- 4.1. Synthetic Benchmark: GSI-Syn: Failed executions are rolled back and resampled, while pixel-level filtering removes edits that are visually negligible.
- 4.1. Synthetic Benchmark: GSI-Syn: GSI-Syn provides diverse, physically valid, geometrically precise editing pairs at scale under controlled conditions.
- 4.2. Real-world Benchmark: GSI-Real: GSI-Real evaluates predicted edits by comparing their spatial consistency with specified 3D transformations instead of requiring ground-truth edited images.
- 4.2. Real-world Benchmark: GSI-Real: Real-scene construction reconstructs 3D object boxes and poses, validates projected transformations, uses MLLM gating, and applies human review to refine annotations.
- 4.3. Evaluation Protocol: The evaluation protocol measures instruction compliance, spatial accuracy, edit locality, and appearance consistency.
5. Fine-tuning Unified MLLMs for GSI
GSI-Syn’s automated synthesis pipeline supplies large-scale spatial editing data for fine-tuning unified multimodal models. Using BAGEL as the base model enables investigation of whether generative training improves spatial understanding and bridges simulation to real images.
- GSI-Syn’s automated synthesis pipeline constructs large-scale editing training data for fine-tuning unified multimodal models.The data supports investigation of generative effects on spatial understanding and sim-to-real transfer.
6. Experiments
GSI-Bench evaluates spatial editing across real and synthetic datasets, revealing substantial differences in models’ spatial reasoning and consistent benefits from synthetic fine-tuning. Fine-tuning improves both spatial editing and downstream spatial understanding.
- Benchmarks and Dataset Statistics: GSI-Real contains 441 samples from 211 indoor scenes, while GSI-Syn includes Room and Tabletop subsets with 593 and 600 samples, respectively.The suite also includes GSI-Syn-Bathroom with 200 samples for cross-view generalization.
- Model Comparisons: 43.52 average: Emu3.5 achieves the best GSI-Real result among evaluated open-source systems, while closed-source models remain comparable to Qwen and Emu3.5 there.Nano Banana and GPT-img score 33.52 and 34.70 on GSI-Real, versus Qwen’s 43.44 and Emu3.5’s 43.52.
- Qualitative Results: Removal generally outperforms other operations, whereas translation and precise manipulation expose failures in identity preservation, artifact avoidance, and spatial interpretation.BAGEL sometimes misinterprets translation as camera motion, and AnyEdit frequently leaves targets unchanged.
- Sim-to-Real Transfer: 7.83-point average gain: fine-tuning BAGEL on GSI-Syn raises GSI-Real performance from 28.46 to 36.28, with especially large gains in Edit Locality, Appearance Consistency, and Instruction Compliance.The corresponding gains are +9.22, +8.25, and +8.16, while Spatial Accuracy improves by +5.68.
- Downstream Spatial Understanding: +4.00% overall gain: synthetic generative fine-tuning also improves spatial understanding on SAT-Real, including Allocentric Perspective, Goal Aiming, and Egocentric Movement.The OmniSpatial evaluation additionally reports improvements, particularly in Spatial Interaction and Perspective Taking.
7. Conclusion
GSI-Bench evaluates generative spatial intelligence across seven spatial operation categories using real-world and synthetic data. Fine-tuning on GSI-Syn improves spatial compliance and transfers to real-world and spatial understanding tasks.
- GSI-Bench spans seven spatial operation categories through real-world and large-scale synthetic sets with automated 3D-grounding pipelines.The benchmark is designed to assess generative spatial intelligence in a scalable way.
- Current state-of-the-art models still struggle with spatially accurate generation.
- Fine-tuning on GSI-Syn improves spatial compliance and transfers to real-world and spatial understanding tasks.The reported transfer connects generative training with downstream spatial reasoning performance.