Source-linked AI summary
SpaceControl: Introducing Test-Time Spatial Control to 3D Generative Modeling
Elisabetta Fedele, Francis Engelmann, Ian Huang, Or Litany, Marc Pollefeys, Leonidas Guibas
TL;DR
Precise geometric control remains difficult because text can be ambiguous and images are hard to edit for fine-grained shape specification. SPACECONTROL applies training-free spatial guidance to 3D generators, and the paper reports stronger geometric faithfulness than baselines while maintaining visual realism.
Problem
Text prompts can be ambiguous and images difficult to edit, limiting their usefulness for specifying precise 3D geometry.
Source: §1 · p. 2Method
SPACECONTROL uses user-specified 3D geometry as training-free latent-space guidance for pretrained generative models.
Source: §1 · p. 2Results
SPACECONTROL outperforms training-based and guidance-based baselines in geometric faithfulness while maintaining high visual realism, supported by quantitative evaluation and a user study.
Source: §1 · p. 2Takeaways & Limitations
The interface supports interactive superquadric editing and real-time generation of detailed textured assets for creative workflows.
Source: §1 · p. 2The paper defines evaluation in terms of faithfulness to control geometry and realism as retaining the original model's quality.
Source: §4.1 · p. 5
Abstract
from arXiv · showhide
Generative methods for 3D assets have recently achieved remarkable progress, yet providing intuitive and precise control over the object geometry remains a key challenge. Existing approaches predominantly rely on text or image prompts, which often fall short in geometric specificity: language can be ambiguous, and images are difficult to manipulate. In this work, we introduce SpaceControl, a training-free test-time method for explicit spatial control of 3D asset generation. Our approach accepts a wide range of geometric inputs, from coarse primitives to detailed meshes, and integrates seamlessly with modern generative models without requiring any additional training. A control parameter lets users trade off between geometric fidelity and output realism. Extensive quantitative evaluation and user studies demonstrate that SpaceControl outperforms both training-based and optimization-based baselines in geometric faithfulness while preserving high visual quality. Finally, we present an interactive interface for real-time superquadric editing and direct 3D asset generation, enabling seamless use in creative workflows. Project page: https://spacecontrol3d.github.io/.
1 INTRODUCTION
Precise geometric control remains a challenge: text and images are difficult to use for fine-grained geometry, while existing spatial-control methods involve training or optimization tradeoffs. SpaceControl introduces training-free control from varied 3D inputs and reports stronger geometric faithfulness while maintaining visual realism.
1 INTRODUCTION
- 3D asset generation supports virtual worlds and applications including gaming, simulation, virtual reality, and digital design.Source: §1 · p. 1
- Most controllable 3D generation approaches use text or images: text ambiguity limits precise geometry, while images are difficult to edit and unintuitive for fine-grained control, preventing direct object-geometry manipulation.The work introduces spatial control: using 3D geometry as three-dimensional sketches to steer detailed 3D asset synthesis.Source: §1 · p. 2
- Existing spatial-control methods either fine-tune models or use inference-time guidance with substantial optimization overhead; asset-enrichment methods often assume fine-grained input geometry.These tradeoffs limit generalization or usefulness when artists begin with coarse sketches.Source: §1 · p. 2
- SPACECONTROL introduces training-free latent-space guidance for text- or image-conditioned 3D generators, accepting inputs from simple primitives to detailed meshes.It requires no additional training.Source: §1 · p. 2
- SPACECONTROL outperforms training-based and guidance-based baselines in geometric faithfulness while maintaining high visual realism, with evaluations and a user study supporting the result.The paper also presents an interface for interactive superquadric editing and real-time textured asset generation.Source: §1 · p. 2
2 RELATED WORK
Related work traces improvements in 3D generation to latent-space modeling and separating geometry from appearance, while noting that spatial control remains less mature than control for images. Prior 3D approaches use projected-view optimization or category-specific fine-tuning, motivating direct test-time control in 3D.
2.1 3D GENERATIVE MODELS
- Separating geometry from appearance in latent-space 3D generators has improved generation quality and efficiency and enables isolated geometry modification through spatial conditioning.Source: §2.1 · pp. 2–3
2.2 CONTROLLABLE GENERATIVE MODELS
- Control modalities can be added through fine-tuning or training-free inference-time guidance, but analogous mechanisms for 3D object generation remain less mature than those for images.Source: §2.2 · p. 3
CONTROLLING IMAGE GENERATIVE MODELS
- Image-generation control includes fine-tuned network additions such as ControlNet and training-free methods such as SDEdit, which conditions denoising from inputs like stroke paintings.Source: CONTROLLING IMAGE GENERATIVE MODELS · p. 3
CONTROLLING 3D GENERATIVE MODELS
- Prior 3D spatial-control methods either optimize 3D representations from projected views or fine-tune models on specific categories; the paper positions SPACECONTROL as direct test-time guidance in 3D.The cited approaches are described as requiring lengthy optimization or class-specific fine-tuning, respectively.Source: CONTROLLING 3D GENERATIVE MODELS · p. 3
3 PRELIMINARIES
The preliminaries introduce rectified flow models and Trellis, the two-stage generator that SpaceControl builds on, and superquadrics as compact spatial-control primitives.
3 PRELIMINARIES
- The preliminaries review rectified flows, Trellis, and superquadrics as foundations for the proposed spatial-control method.The overview shows control encoded into a latent before structure denoising, followed by appearance generation and decoding.Source: §3 · p. 3, Figure 2 · p. 4
3.1 RECTIFIED FLOW MODELS
- Rectified flow linearly interpolates clean data with Gaussian noise, then uses a time-dependent velocity field and discretized steps to denoise.Trellis uses a predicted vector field for this iterative process.Source: §3.1 · pp. 3–4
3.2 TRELLIS
- Trellis separates asset generation into coarse geometric-structure generation and appearance generation, with outputs decoded as 3D Gaussians, radiance fields, or meshes.Its first stage denoises a latent into an occupancy grid; the second augments active voxels with point-wise features and denoises them.Source: §3.2 · p. 4
- A canonical superquadric uses three scale and two exponent parameters; adding translation and rotation gives 11 parameters for world-space control.The paper presents superquadrics as compact spatial-control primitives.Source: §3.2 · p. 5
4 METHOD
SpaceControl guides a pretrained Trellis model with user-specified geometry while aiming to preserve both control faithfulness and asset realism. Its control strength trades off between those goals.
4.1 SETUP
- The method targets generated assets that align with the input geometry while retaining the original model’s quality.The user supplies geometric conditioning alongside a text or image prompt.Source: §4.1 · p. 5
4.2 APPROACH
- SpaceControl voxelizes user geometry, encodes it with Trellis’ pretrained encoder, and noises the latent to a chosen timestep before structure denoising.The control can be coarse or detailed, and the procedure requires neither architectural changes nor additional training.Source: §4.2 · p. 5
- After structure generation, active voxels receive point-wise noisy latent features that the Appearance Flow Model denoises using text or image conditioning.Image conditioning can guide appearance even though structure generation is conditioned on text.Source: §4.2 · p. 6
- Higher τ0 preserves more injected spatial structure but can reduce realism; lower τ0 favors Trellis-like realism with less spatial faithfulness.At lower τ0, initialization is closer to noise and the model performs more denoising steps.Source: §4.2 · p. 6
5 EXPERIMENTS
SPACECONTROL outperforms the baselines in spatial faithfulness while achieving comparable visual-quality metrics, and user studies favor it for overall appearance and alignment. Analysis shows that users can adjust conditioning strength, use images to preserve appearance across edits, and achieve fine-grained spatial alignment.
5.1 COMPARING WITH STATE-OF-THE-ART METHODS
- SPACECONTROL significantly outperforms the baselines in Chamfer Distance across experiments while achieving comparable CLIP-I, FID, and P-FID scores.Table 1 reports results with τ0 = 6.Source: §5.1 · p. 7, Table 1 · p. 7
5.2 ANALYSIS EXPERIMENTS
- Increasing τ0 strengthens spatial conditioning and brings generations into closer alignment with the input control.The parameter allows users to choose different adherence strengths.Source: §5.1 · p. 8, §5.2 · p. 8
5.1 COMPARING WITH STATE-OF-THE-ART METHODS
- In a 52-volunteer study, SPACECONTROL was preferred over Spice-E and SPICE-E-T for overall appearance and alignment to spatial control.Participants evaluated randomly selected samples from ShapeNet and Toys4K.Source: §5.1 · p. 8
- Qualitative comparisons show SPACECONTROL producing plausible, spatially faithful objects where baselines sometimes misplace parts or generate low-quality shapes.Examples include pose-specific errors and anatomically incorrect placements.Source: Figure 5 · p. 6, §5.1 · p. 8
5.2 ANALYSIS EXPERIMENTS
- Optional image conditioning primarily affects texture, helping maintain visual consistency across edits while having only minor influence on geometry.The image-conditioning analysis illustrates this role alongside spatial guidance.Source: §5.2 · pp. 8–9, Figure 7 · p. 9
- For conditioning shapes with non-axis-oriented rotations, SPACECONTROL is reported to align precisely while preserving generated-mesh quality.The comparison is presented as an example of fine-grained spatial alignment.Source: §5.2 · p. 9
6 DISCUSSION AND CONCLUSION
The discussion presents SPACECONTROL as a training-free framework for precise spatial conditioning in high-quality asset generation. It identifies possible extensions to adaptive adherence, localized control, and scene generation, and reports that the code and experimental resources are publicly available.
6 DISCUSSION AND CONCLUSION
- SPACECONTROL is presented as a training-free framework operating directly in 3D space for precise spatial conditioning and high-quality asset generation.The approach builds on Trellis and uses open-source ShapeNet and Toys4K datasets; the code is publicly available.Source: §6 · p. 9
- Future work could investigate adaptive τ0 scheduling and localized adherence to vary geometric fidelity across an object's regions.The current τ0 setting is manually navigated by users.Source: §6 · p. 9
- The authors identify scene generation from geometric primitives, using scene abstraction and functional scene graphs, as a potential extension.They describe generating realistic spatial layouts and respecting semantic relationships between objects.Source: §6 · p. 9
A.1 FINE-GRAINED SPATIAL EDITING
Superquadrics support localized spatial edits, while optional image conditioning helps preserve visual appearance across generations. The examples include both text-prompted and image-conditioned editing.
A.1 FINE-GRAINED SPATIAL EDITING
- With image-conditioned generation, visual appearance can remain consistent across edits as the spatial control changes.The figure contrasts text-prompted examples with image-conditioned examples.Source: §A.1 · p. 13, Figure 8 · p. 13
- Superquadrics support intuitive, localized shape modifications that are more direct than relying on text- or image-only generative models.The figure illustrates fine-grained editing of generated assets.Source: Figure 8 · p. 13
- The editing examples show SPACECONTROL with both natural-language prompts and image conditioning.The figure presents text-conditioned examples at the top and image-conditioned examples at the bottom.Source: Figure 8 · p. 13
A.2 COARSE AND FINE-GRAINED SPATIAL CONTROL WITH SUPERQUADRICS
Superquadrics support both precise geometric sculpting and coarse spatial sketching. Together with adjustable control strength, they let users choose between specifying geometric detail and drafting a spatial setting.
A.2 COARSE AND FINE-GRAINED SPATIAL CONTROL WITH SUPERQUADRICS
- Superquadrics provide fine-grained control for sculpting precise geometry, as illustrated by a motorbike, staircase, and helicopter.These examples use detailed spatial controls.Source: §A.2 · p. 14, Figure 9 · p. 14
- Superquadrics also provide coarse control when used to draft a 3D sketch, as illustrated by a duck and drumkit.The examples show spatial setting rather than carefully designed geometric details.Source: §A.2 · p. 14, Figure 9 · p. 14
- Combining superquadrics with flexible control strength lets users choose between carefully designing geometric details and sketching the desired spatial setting.The examples use different control strengths.Source: §A.2 · p. 14
A.3 FINE-GRAINED ALIGNMENT WITH STATE-OF-THE-ART METHODS
The appendix shows alignment outputs at different control strengths and illustrates spatial conditioning for a scene of independently generated objects. It also describes placing those objects without time-consuming post-processing.
A.3 FINE-GRAINED ALIGNMENT WITH STATE-OF-THE-ART METHODS
- The same chair prompt and spatial control are shown with different τ0 values, alongside inputs, generated assets, and overlays for inspecting alignment.The figure places controls in the first row, generations in the second, and overlays in the third.Source: Figure 10 · p. 15, §A.3 · p. 15
- A scene sketch of geometric primitives can directly condition object generation, avoiding time-consuming post-processing to align generated shapes.Each object is generated independently using a unit-cube-scaled superquadric as spatial control, then automatically placed by undoing the transformation.Source: §A.3 · p. 15, Figure 11 · p. 15
- The scene example uses a different-colored superquadric for each object and shows the input scene beside the generated assets.Generated objects are independently conditioned and then placed by undoing their transformations.Source: Figure 11 · p. 15
A.4 LOCAL CONTROL
The appendix describes local semantic appearance control, an interactive editing interface, the user-study comparison interface, and a training-based SPICE-E-T baseline.
A.4 LOCAL CONTROL
- Local semantic control assigns prompts to superquadrics and combines global and local attention for appearance, demonstrated with a white chair and red seat.The baseline demonstrates part-level semantic control without conceptual modification.Source: §A.4 · p. 16, Figure 12 · p. 16
B INTERACTIVE USER INTERFACE
- The interface lets users edit or add and delete superquadrics, select control strength, and condition generation with text and optionally images.Users can toggle between the input primitives and meshes before generating again.Source: B INTERACTIVE USER INTERFACE · p. 17, Figure 13 · p. 17
C USER STUDY
- The user-study interface presents a control shape and two competing methods for comparison on fidelity, realism, and overall preference.Participants choose which generated object is more faithful, which model looks more realistic, and which they like best.Source: C USER STUDY · p. 17, Figure 14 · p. 17
D SPICE-E-T
- SPICE-E-T adapts Trellis’s structure generator with a shape-conditioning cross-attention layer and trains it for 120,000 iterations on ABO.The modified flow transformer is compared with the original Trellis flow transformer.Source: D SPICE-E-T · p. 18, Figure 15 · p. 18