PaperlayerView exact revision on arXiv

Source-linked AI summary

SpaceFlow: Locally Controllable 3D Generation

Neil De La Fuente, Joan Lafuente, Mukhammadali Sayfiddinov, Felicia Scharitzer, Marc Pollefeys, Ata Celen, Sayan Deb Sarkar, Elisabetta Fedele

arXiv:2610.12399v1cs.CVcs.AIcs.GR

TL;DR

3D generators lack unified local control over geometry and appearance, despite the need for part-specific guidance. SpaceFlow routes geometric constraints and appearance cues to regions during generation; evaluations report a balance of geometric fidelity and generative freedom alongside stronger localized appearance results.

  • Problem

    Existing approaches lack a unified mechanism for locally controlling geometry and appearance, although creators need part-specific guidance over where and how strictly instructions apply.

    Source: §1 · p. 2

  • Method

    SpaceFlow uses editable geometric inputs with per-region control levels and routes each part’s appearance cue only to its corresponding generated part.

    Source: §3.5 · p. 6, §5 · p. 12

  • Results

    Regional metrics and a user study support a balance of geometric fidelity and generative freedom, while localized text routing on fixed geometry outperforms evaluated TRELLIS and GuideFlow3D variants in prompt faithfulness and color/material accuracy.

    Source: §1 · p. 2

  • Takeaways & Limitations

    SpaceFlow lets users specify where spatial constraints and appearance cues apply during 3D asset generation.

    Source: §5 · p. 12

    Appearance routing inherits the granularity of PartField clusters, so cues can reach the wrong voxels when cluster boundaries do not match material boundaries.

    Source: S8 Limitations · p. 27

Abstract

from arXiv · show

Current 3D generation methods lack explicit local control: geometric adherence is often defined by a global control strength, and appearance cannot be specified locally. We present SpaceFlow, a training-free pipeline for locally controllable 3D generation from text descriptions and a collection of geometric primitives. Each primitive serves as a proxy for an object part and is assigned a local control level, enabling users to specify whether regions should strictly follow the input shape or allow generative completion. During structure generation, we enforce these spatial constraints within the generative flow process. For appearance synthesis, the generated structure is segmented and matched to the primitives. Each generated part is conditioned only on its assigned text or image cue, thereby limiting cross-part leakage. Regional geometry metrics demonstrate that SpaceFlow preserves the specified geometry in high-control regions and enables plausible shape variation in low-control areas. A user study further indicates that the resulting balance between geometric fidelity and generative freedom remains competitive in overall quality. When evaluating appearance on fixed geometry, text-conditioned routing achieves state-of-the-art prompt faithfulness and color/material accuracy. Qualitative results additionally show localized routing of image cues. The project page is available at SpaceFlow3D.github.io.

1 Introduction

SpaceFlow addresses the lack of unified local control over geometry and appearance in 3D generation. It combines region-specific geometric constraints and appearance cues, with evaluations supporting its balance of geometric fidelity and generative freedom.

1 Introduction

  • Creators need iterative, part-specific guidance, but existing methods lack a unified mechanism for local geometric and appearance control.Proxy-guided geometry applies constraints globally; appearance methods either leave geometry unconstrained or use a single global cue.Source: §1 · p. 2
  • SpaceFlow assigns each editable part a geometric control level, making high-control regions adhere more closely to input geometry while low-control regions leave more room for the generative prior.Parts can range from coarse primitives to detailed meshes; the paper uses editable superquadric primitives.Source: §1 · p. 2
  • For appearance, SpaceFlow assigns a distinct cue to each part and restricts cross-attention to that cue while retaining global self-attention for coherence and smooth blending.Source: §1 · p. 2
  • Regional metrics and a user study support SpaceFlow’s balance of geometric fidelity and generative freedom; on fixed geometry, localized text routing beats evaluated TRELLIS and GuideFlow3D variants in prompt faithfulness and color/material accuracy.Source: §1 · p. 2

3D Asset Generation

  • Recent 3D generators produce high-fidelity assets across varied representations, but condition on global prompts and lack fine-grained part control.Source: §2 · p. 3

Spatially Controllable 3D Generation

  • Existing spatial-control methods either require fine-tuning or apply test-time constraints uniformly, limiting region-specific choices between preserving geometry and generative completion.Source: §2 · p. 3

Appearance Control for 3D Assets

  • Prior appearance methods often texture existing assets post hoc, while GuideFlow3D transfers one appearance source globally rather than routing independent cues to semantic parts.Source: §2 · p. 3

Part-Aware Generation and Editing

  • Unlike part-aware generators, SpaceFlow uses primitives and part assignments as control signals without requiring the generated asset itself to be represented as parts.Source: §2 · p. 3

3 Method

SpaceFlow assigns geometric control and appearance cues locally to primitive-defined parts. Its pipeline guides structure generation with per-part control levels, then matches generated regions to primitives to route their appearance cues.

3.1 Problem Formulation

  • Each primitive receives a high or low control strength and optional appearance cue, setting its local trade-off between geometric fidelity and generative freedom.High control promotes strict input adherence; low control relaxes constraints and relies on the generative prior.Source: §3.1 · p. 4

3.2 Preliminaries

  • TRELLIS generates assets in two stages, while SpaceControl encodes input geometry and uses a global flow timestep to trade geometric adherence against generative freedom.TRELLIS first establishes sparse voxel structure, then synthesizes local geometry and appearance latents.Source: §3.2 · p. 4

3.3 Method Overview

  • SpaceFlow guides sparse-structure generation with primitive control levels, then segments the result, matches parts to input primitives, and routes local appearance cues.Figure 3 illustrates independent control of the hourglass frame and sand: high control preserves corresponding input geometry, while low control allows variation.Source: §3.3 · p. 5, Figure 3 · p. 5

3.4 Local Geometric Control

  • Spatial guidance partitions primitives by control level, preserves high-control latents during denoising, and uses masked blending and iterative resampling across the flow interval.The low-control mask is built from a bounding box around low-control primitives; the interval endpoints are τlow and τhigh.Source: §3.4 · p. 5

3.5 Local Appearance Routing and Guidance

  • Appearance routing maps voxels to conditions by clustering PartField descriptors and matching semantic clusters to input primitives, then restricts cross-attention to each region’s cue.A soft self-attention bias prioritizes propagation within parts, and test-time contrastive optimization encourages coherent appearance within semantic parts.Source: §3.5 · pp. 6–7

4 Experimental Results

Across geometric and appearance evaluations, SpaceFlow combines local control with generative freedom, while comparisons test its regional fidelity, user preference, and part-level appearance routing.

4 Experimental Results

  • The evaluation uses 83 geometric and appearance conditionings, with superquadrics marking high- and low-control regions and local text prompts assigned to primitives.The experiments analyze geometric control and localized appearance conditioning separately.Source: §4 · p. 7

4.1.1 Quantitative Results

  • SpaceFlow follows high-control input geometry, completes low-control regions to match prompts, and is preferred overall to both uniform-guidance baselines.Table 1 reports head-to-head win rates against SpaceControl with uniform global control.Source: Table 1 · p. 7, §4.1.1 · p. 7

4.1.2 Geometric and Feature-Space Evaluation

  • Regional metrics show stronger high-control adherence than τ=3, positive geometry and feature-space control selectivity, and comparable CLIP similarity across methods.SpaceFlow’s high-control Chamfer Distance approaches the strict τ=10 baseline; uniform guidance shows virtually no regional difference.Source: §4.1.2 · pp. 8–9, Table 2 · p. 8

4.1.3 Human Evaluation

  • In 337 user-study trials, SpaceFlow had a 48% win rate against τ=3 and a 71% win rate against τ=10 in overall preference.The study included 29 evaluators.Source: §4.1.3 · p. 9, Figure 6 · p. 9

4.3 Qualitative Results

  • Qualitative comparisons show high-control regions retaining specified geometry, low-control regions gaining text-aligned details, and textures localized to their target parts.Figure 5 contrasts SpaceFlow with uniform-guidance geometry baselines and unrouted appearance baselines on fixed geometry.Source: Figure 5 · p. 10, §4.3 · p. 11

4.2 Appearance Control

  • For text-conditioned appearance, SpaceFlow leads TRELLIS and GuideFlow3D in prompt faithfulness and overall preference, while both baselines are favored on texture detail.Absolute ratings show the highest overall score, with localized prompt fidelity and material accuracy compensating for lower unconstrained texture uniformity.Source: Table 3 · p. 9, §4.2 · p. 11, Table 4 · p. 11
  • For image-conditioned appearance, SpaceFlow scores 6.53 in style fidelity, 7.07 in local routing accuracy, 6.33 in texture detail, and 6.73 overall.The framework is evaluated standalone because baseline methods cannot process per-part image conditions.Source: §4.2 · p. 11

5 Conclusion

SpaceFlow is an inference-time framework for local geometry and appearance control in 3D asset generation. Evaluations report localized geometric control and stronger prompt faithfulness than unrouted appearance baselines.

5 Conclusion

  • Editable geometric inputs define controlled regions with local geometry strengths and text or image appearance cues.Spatial guidance varies by region, while appearance cues are matched to generated parts and routed only to their corresponding parts.Source: §5 · p. 12
  • Vision-language assessments, user studies, and regional metrics demonstrate localized geometric control that overcomes uniform global guidance limitations.Localized appearance routing consistently outperforms existing unrouted baselines in prompt faithfulness.Source: §5 · p. 12
  • The framework lets users specify where each spatial constraint and appearance cue should apply.Source: §5 · p. 12

Supplementary Material

The supplementary material lists implementation and pipeline details, evaluation protocols, additional analyses, and qualitative examples.

Supplementary Material

  • It documents the 83-asset evaluation dataset, vision-language-model evaluation and validation, user-study protocol, and additional analyses.Analyses include sensitivity to local control strength, spatial feature-distance visualization, and primitive-to-part assignment visualizations.Source: Supplementary Material · p. 16
  • It includes additional image- and text-conditioned appearance results and an interactive HTML gallery of rotatable 3D inputs and generated outputs.Source: Supplementary Material · p. 16
  • It provides implementation details, structure-generation and appearance-generation pipeline descriptions, and the mathematical formulation of superquadric primitives.Source: Supplementary Material · p. 16

S1 Structure Generation and Guidance Architecture

Conditioned Flow applies spatially varying geometric constraints during generation: high-control regions blend toward target geometry, while low-control regions remain freer to generate. The method builds a latent mask for these regions and iteratively refines the latent state.

S1 Structure Generation and Guidance Architecture

  • Conditioned Flow blends the predicted latent with the high-control target inside constrained regions while leaving low-control regions to generate freely.The blended latent is taken through repeated resampling before the next generation step.Source: S1 Structure Generation and Guidance… · pp. 16–17, Figure S1 · p. 17
  • A padded bounding box around low-control superquadrics is voxelized, downsampled, and binarized to form the mask that locates latent-flow blending.The described process uses 7% diagonal padding, 64^3 voxelization, and downsampling to 16^3 latent resolution.Source: S1 Structure Generation and Guidance… · p. 16
  • Each appearance latent voxel attends only to its assigned text or image cue, with unassigned voxels using the global condition.The routing volume partitions latent queries by part before outputs are recombined.Source: Figure S2 · p. 17

S2 Appearance Generation Pipeline

The appearance pipeline maps generated structure parts to input primitives, then routes local cues to their assigned regions. It also applies test-time guidance to encourage coherent appearance within semantic parts and separation between them.

S2 Appearance Generation Pipeline

  • Table S2 summarizes appearance-generation and self-similarity guidance settings for text- and image-conditioned evaluation.Text-conditioned appearance is evaluated pairwise and with standalone VLM scores; image-conditioned appearance is evaluated standalone and illustrated qualitatively.Source: S2 Appearance Generation Pipeline · p. 18
  • Generated structure voxels are clustered into semantic regions and matched to their closest input primitive, producing a routing volume that stays part-consistent despite low-control geometry drift.PartField features are extracted from the decoded mesh; small primitives with too few latent cells can have their masks recovered and dilated on the latent grid.Source: S2 Appearance Generation Pipeline · pp. 18–19, Figure S3 · p. 18
  • Each latent voxel attends to its assigned global or part-specific condition, while a self-similarity objective encourages coherent appearance within parts and separation across parts.The routing volume selects the condition embedding, and same-condition voxel pairs receive a self-attention bias scaled by β.Source: S2 Appearance Generation Pipeline · pp. 18–19
  • Test-time guidance takes an AdamW step on appearance latents using a supervised contrastive objective, retains the lowest-loss state, and decodes it into the textured asset.Setting the guidance weight to zero recovers part-routed sampling alone.Source: S2 Appearance Generation Pipeline · p. 19

S3 Superquadric Primitives

Superquadrics provide compact primitives for representing a range of shapes, and deformable superquadrics serve as editable spatial control regions rather than final surface meshes. Their geometry is described through an inside-outside function and deformation parameters.

S3 Superquadric Primitives

  • Superquadrics represent a continuous range of shapes, including ellipsoids, cylinders, rounded boxes, and cuboids, with parameters governing roundness and squareness.The cited passage introduces a scale vector and cross-section shape parameters.Source: S3 Superquadric Primitives · p. 19
  • A superquadric’s inside-outside function classifies points as inside when F(x) < 1, on the surface when F(x) = 1, and outside when F(x) > 1.The point coordinates are expressed in the primitive’s local frame using its rotation and translation.Source: S3 Superquadric Primitives · pp. 19–20
  • Deformable superquadrics use tapering and axis-specific bending parameters, and serve as compact editable spatial controls rather than final surface meshes.The forward map tapers the canonical primitive before bending it about the y, x, and z axes.Source: S3 Superquadric Primitives · p. 20
  • Table S3 reports object-category distribution for an evaluation benchmark of 83 assets spanning seven broad semantic categories, designed to evaluate multi-part geometric control and local appearance conditioning.Source: Table S3 · p. 21

S4 Dataset Details

The supplementary material describes an 83-asset benchmark and its validation, then details VLM-judge evaluations for structure and appearance. The judge calibration probes report sensitivity to mismatched controls and agreement on identical renders.

S4 Dataset Details

  • The benchmark contains 83 assets with high- and low-control superquadric regions and local appearance prompts, combining authored and semi-automated layouts that humans validated.Validation checked control contrast, spatial validity, and coherent mapping of appearance prompts; the benchmark spans seven categories and 2–18 parts per asset.Source: S4 Dataset Details · p. 21

S5 Vision-Language-Model Evaluation

S5.1 Structure Control Evaluation

  • Structure-control comparisons covered all 83 assets, contrasting SpaceFlow’s local guidance with SpaceControl’s uniform τ=3 and τ=10 baselines.Judges assessed control and prompt adherence, realism, and overall preference; untextured, scale- and alignment-matched shapes isolate geometry from appearance quality.Source: S5.1 Structure Control Evaluation · p. 22

S5.2 Appearance Evaluation: Pairwise Evaluation

  • Text-guided appearance comparisons pit part-level routing against TRELLIS and GuideFlow3D on identical SpaceFlow-generated base geometry.Using the same geometry isolates appearance generation in the pairwise comparison.Source: S5.2 Appearance Evaluation: Pairwise… · p. 22

S5.3 Appearance Evaluation: Standalone Prompts

  • Standalone appearance scoring uses 1–10 scales, assessing text prompts for faithfulness, color/material accuracy, detail, and overall quality, and image prompts for reference-style fidelity, routing accuracy, detail, and quality.The text- and image-conditioned criteria are shown in separate figures.Source: S5.3 Appearance Evaluation: Standalone… · pp. 22–23

S5.4 VLM-Judge Configuration and Validation

  • Mismatched geometric controls reduced adherence scores by 4.43 points, while identical renders received 100% “Equal” ratings.Pairwise preferences were aggregated by majority vote over three passes, with 0.898 average inter-pass agreement.Source: S5.4 VLM-Judge Configuration and… · p. 23, Table S4 · p. 23

S6 User Study Details

The supplementary evaluation describes a pairwise user-study protocol and ablations of local geometric control, spatial feature variation, and primitive-to-part appearance routing. Varying low-control strength reveals a continuum between prompt-driven shape freedom and adherence to the input geometry.

S6 User Study Details

  • The user study presents participants with a prompt, a color-coded control shape, and two randomized anonymous outputs, asking which object looks better overall.Participants could inspect the input shape and candidate samples interactively from any viewpoint.Source: S6 User Study Details · p. 23, Figure S5 · p. 24

S7 Additional Ablation Study

S7.1 Sensitivity to Local Control Strength

  • Increasing low-control strength progressively constrains generated shapes toward the input geometry, while low strength permits greater prompt-driven variation.The evaluation varies τ_low while holding τ_high = 10; some high-control-region deviation at low τ_low reflects both cross-boundary geometric pull and the metric’s boundary assignment effect.Source: S7.1 Sensitivity to Local Control… · pp. 24, 26, Figure S6 · p. 25, Figure S7 · p. 25

S7.2 Spatial Feature-Distance Visualization

  • Spatial feature deviations concentrate mainly in low-control regions, while high-control regions remain closer to a uniformly high-control reference.The comparison uses voxel-wise cosine distances between PartField features and nearest spatial matches in the τ = 10 reference.Source: S7.2 Spatial Feature-Distance… · p. 26, Figure S8 · p. 26

S7.3 Primitive-to-Part Assignment Visualizations

  • The routing visualization maps generated-structure regions to local appearance cues, with gray regions assigned to the global condition.The routing volume is derived from the generated structure in 323 voxel space.Source: S7.3 Primitive-to-Part Assignment… · p. 26

S8 Limitations

The limitations concern conflicts between text and constrained geometry, and errors when appearance-routing clusters do not match material boundaries. Additional examples show image-cue transfer and text-conditioned results across varied objects.

S8 Limitations

  • When a text prompt conflicts with the input geometry, the generator lacks a joint prior for the two modalities and may produce unnatural distortions around constrained regions.The example requests a person over a bench structure while retaining high-control geometry.Source: S8 Limitations · pp. 26–27, Figure S10 · p. 27
  • Appearance routing can misapply cues when PartField cluster boundaries do not align with material boundaries, causing localized texture leakage or discontinuity.Examples include wooden handle appearance bleeding onto a metal pan rim and competing textures covering part of a chair cushion; the errors arise in routing rather than geometry generation.Source: S8 Limitations · pp. 27–28, Figure S11 · p. 28

S9 Additional Qualitative Results

S10 Implementation Details

The implementation uses frozen pretrained components and provides example prompts and evaluation instructions. The supplied passages also specify the hardware used for experiments.

S10 Implementation Details

  • SpaceFlow is training-free and uses publicly released TRELLIS-XL components with frozen encoders, flow transformers, and decoders.CLIP encodes text conditions and DINOv2 encodes reference images.Source: S10 Implementation Details · p. 30
  • Experiments were conducted on a single NVIDIA GeForce RTX 5060 Ti GPU, using inference settings reported in Tables S1 and S2.Source: S10 Implementation Details · p. 30
  • The examples include prompts for localized color and material requests, while the evaluation templates instruct judges to assess shape fidelity or appearance transfer under specified viewing conditions.The pairwise geometry template randomizes A/B ordering; appearance instructions focus scores on color, material, and pattern transfer to correct parts.Source: S10 Implementation Details · pp. 31–32, Figure S15 · p. 33

Related papers

Loading 2610.12399v1…