Source-linked AI summary
SpatialGuard: Harness-Guided Verifiable Spatial Reasoning for Text-to-Image Generation
Ziyun Qian, Zizhi Chen, Yizhou Liu, Mingyang Sun, Dingkang Yang, Lihua Zhang
TL;DR
Complex 3D text-to-image generation needs stable, verifiable spatial geometry rather than implicit prompt interpretation. SpatialGuard plans editable 3D layouts, realizes and validates them, and repairs failures through a harness-managed loop, achieving state-of-the-art performance while incurring higher inference cost.
Problem
Existing methods often lack a verifiable spatial intermediary before sampling, allowing relations, visibility, occlusion, and camera constraints to decay during generation.
Method
SpatialGuard uses a Spatial Layout Architect, Visual Realizer, Visual Alignment Critic, and Layout Harness to plan, realize, validate, and repair editable 3D layouts.
Results
SpatialGuard achieves new state-of-the-art performance and significantly outperforms existing text-to-image and layout-control baselines.
Takeaways & Limitations
SpatialGuard turns complex spatial generation into a verifiable process that preserves object relations, visibility, and camera constraints across iterations.
Takeaways & Limitations
The framework focuses on scenes expressible through object lists, relations, visibility constraints, and camera parameters, and has higher inference cost than a single direct call.
Abstract
from arXiv · showhide
Complex 3D spatial text to image generation requires models to convert natural language into stable visual geometry, not merely semantic appearance. Existing prompt-driven or layout-conditioned methods improve controllability, but often lack an optimizable and verifiable spatial intermediary before visual sampling. As a result, object relations, occlusion, visibility, and camera constraints can decay during multi-round generation. This paper presents SpatialGuard, a structured layout-guided framework for complex 3D spatial text-to-image generation. SpatialGuard parses prompts into image synthesis-oriented 3D layouts through a Spatial Layout Architect, realizes them as visual conditions and candidate images through a Visual Realizer, and uses a Visual Alignment Critic to validate consistency among prompt, layout, and image. To keep constraints stable across iterations, SpatialGuard introduces a Layout Harness that organizes rule constraints, tool invocation, shared knowledge, and feedback loops around the editable layout state. This design turns complex spatial generation from implicit prompt following into a verifiable process of planning, realization, validation, and repair. Comprehensive experiments show that SpatialGuard achieves state-of-the-art performance in complex 3D spatial layout generation and improves spatial faithfulness over existing text-to-image and layout control baselines.
1 Introduction
SpatialGuard addresses unstable spatial reasoning in complex 3D text-to-image generation by externalizing spatial intent into an editable, verifiable layout and maintaining constraints across iterative generation. Its closed-loop framework combines layout planning, visual realization, alignment validation, and repair, achieving state-of-the-art performance.
- Complex 3D spatial generation supports applications including game-asset previewing, film storyboard design, and embodied-intelligence simulation.
- Prompt-driven methods often infer spatial structure implicitly, satisfying local semantics while struggling with global consistency in relation-dense, view-sensitive scenes.
- SpatialGuard requires a verifiable layout before sampling and cross-round constraint enforcement so spatial requirements remain encoded, optimized, checked, and updated.
- The Spatial Layout Architect parses text into an editable 3D layout, the Visual Realizer converts it into visual conditions and images, and the Visual Alignment Critic feeds deviations back into planning.
- The Layout Harness uses rule constraints, tool invocation, shared knowledge, and feedback loops to mitigate spatial-constraint memory decay across rounds.
- SpatialGuard achieves new state-of-the-art performance and significantly outperforms existing text-to-image and layout-control baselines in complex 3D spatial generation.
2 Related Work
Related work advances spatial controllability through reasoning, layout conditioning, compositional planning, and agentic harnesses. SpatialGuard builds on these directions by framing generation around structured execution and verification.
- Recent text-to-image models improve prompt adherence through reasoning, reward optimization, autoregressive modeling, and spatial evaluation signals.
- Janus-Pro-R1 and T2I-R1 connect visual comprehension with generation using reinforcement learning and chain-of-thought-style approaches.
- Compositional methods explore visual planning, multi-agent collaboration, layered reasoning, object integration, orientation control, and 9 DoF object-pose control.
- Agent harness research organizes execution around tools, state, orchestration, context management, verification, observability, recovery, and governance.
- SpatialGuard’s overview combines layout parsing, visual realization, structured validation, executable repair, and harness-managed feedback loops.
3 Methodology
SpatialGuard converts textual spatial intent into an editable 3D layout, realizes that layout as visual conditions and candidate images, and validates the results in a repairable loop. Its Layout Harness preserves constraints, tools, shared state, and feedback across rounds.
- Spatial Layout Architect: SpatialGuard parses prompts into a structured 3D layout that explicitly represents object positions, sizes, orientations, relations, and camera parameters.The Spatial Layout Architect decomposes spatial phrases into 3D and screen-space requirements, making relations such as front-right checkable after projection.
- Visual Realizer: The Visual Realizer renders the current layout, derives instance masks, aligns object regions with prompt tokens, and synthesizes a candidate image.Rendered layouts provide global geometry, masks preserve object boundaries, and token alignment attaches appearance phrases to their targets.
- Visual Alignment Critic: The Visual Alignment Critic checks the prompt, layout, and candidate image, recording passed or failed constraints and producing repair feedback for the next revision.Failures are localized to concrete layout fields such as position, size, azimuth, occlusion, visibility, or camera configuration rather than reduced to a scalar score.
- Layout Harness: The Layout Harness maintains rule constraints, executable tools, shared knowledge, and feedback loops around the editable layout state.Shared knowledge records object manifests, predicates, prior layouts, validation records, and repair actions, while tools apply operations such as translation, scaling, support, occlusion, visibility, and camera adjustment.
- Closed-loop generation: Together, these components turn spatial generation from implicit prompt following into persistent execution that externalizes, checks, repairs, and remembers spatial constraints.Failed constraints return through executable repair actions; when no failed item remains, the current image is returned as the final output.
4 Experiments
Experiments evaluate spatial layout faithfulness through multi-model quantitative scoring, qualitative comparisons, and component ablations. SpatialGuard consistently preserves object completeness, relations, visibility, and layout better than baselines, while the ablations support the value of coordinated modules.
- Evaluation Protocol: Evaluation uses three vision language models, shared prompts, three images per method, and seven spatial faithfulness dimensions scored from 1 to 10.Evaluators judge visible evidence while ignoring appearance quality unless it prevents spatial judgment.
- Quantitative Evaluation: 9.37 Overall: SpatialGuard ranks first on all seven dimensions, exceeding HunyuanImage-2.1 at 7.90 by 1.47 points.The largest gains over the best baseline occur on Depth, Support, Scale, and Relation.
- Quantitative Evaluation: SpatialGuard’s largest advantages target spatial structure, with gains of 2.18 on Depth, 1.81 on Support, 1.48 on Scale, and 1.42 on Relation.Presence improves by 0.67, indicating a smaller benefit for simple object inclusion than for multi-constraint structure.
- Qualitative Evaluation: Qualitative comparisons show baselines missing objects, truncating entities, or violating directional relations, whereas SpatialGuard maintains explicit layout, relation, and visibility constraints.Examples include omitted lamps and bookshelves, a partial bookshelf, and a broken back-right relation.
- Qualitative Evaluation: Across compound scenes, SpatialGuard retains entities and readable relative positions after rendering, while validation localizes conflicts before acceptance.This helps preserve completeness, direction, depth, and visibility when correcting one object could disturb another relation.
- Ablation Studies: Ablations show that the full 9.37 Overall score exceeds the best component-removed configuration, whose Overall score is 9.01, supporting module interaction.The Architect, Realizer, Critic, and Harness contribute differently across presence, geometry, visible violations, and cross-round preservation.
5 Conclusion
SpatialGuard externalizes spatial intent into an editable 3D layout and organizes generation as planning, realization, validation, and repair. Its Layout Harness maintains constraints across iterations, and experiments report gains in spatial faithfulness over strong baselines.
- 5 Conclusion: SpatialGuard externalizes spatial intent into an editable 3D layout state instead of relying on implicit prompt interpretation.The framework organizes generation as planning, realization, validation, and repair.
- 5 Conclusion: The Layout Harness stabilizes object relations, visibility, and camera constraints through rule constraints, tool invocation, shared knowledge, and feedback loops.These mechanisms keep constraints recoverable across iterations.
- 5 Conclusion: Experiments show gains in spatial pose, relation grounding, and measurement accuracy over strong text-to-image baselines.Qualitative and ablation results attribute these gains to verifiable layout execution.
Limitations
SpatialGuard targets spatially grounded scenes, while abstract or ambiguous artistic prompts may need additional interpretation rules. Its planning, validation, and repair stages increase inference cost, and final visual quality remains dependent on the underlying image generator.
- The framework focuses on scenes expressible through object lists, relations, visibility constraints, and camera parameters.
- Highly abstract artistic intent or intentionally ambiguous spatial descriptions may require additional interpretation rules.
- Planning, validation, and repair before final image production make inference more costly than a single direct text-to-image call.
- Final visual quality still depends on the capability of the underlying image generator.
- Future work may extend the tool library to finer physical interactions and improve efficiency through lighter validation strategies.