SceneTransporter: Optimal Transport-Guided Compositional Latent Diffusion for Single-Image Structured 3D Scene Generation

• Open-world scenes expose structural mispartition, where instances fail to form disjoint parts, and geometric redundancy, where multiple latents describe the same area.Source: §1 · p. 2
• A debiased clustering probe identifies missing structural constraints in the assignment mechanism as the core failure in part-level generators.Source: §1 · p. 2
• SceneTransporter frames generation as OT-guided correlation assignment, using entropic OT for exclusive one-to-one cross-attention routing and edge-regularized grouping.Source: §1 · p. 2
• SceneTransporter achieves state-of-the-art open-world 3D scene generation with improved instance-level coherence and geometric fidelity.Source: §1 · p. 2

3D Scene Generation

• Retrieval methods are limited by database scope and often single-view alignment errors, while 2D-prior distillation faces geometric inconsistency and native 3D latent models require large-scale 3D datasets and generalize poorly in the wild.Both generative avenues output a single, unstructured monolithic mesh.Source: §2 · p. 3

3D Structured Scene Generation

• Divide-and-conquer pipelines accumulate stage errors and lack global consistency, while MIDI cannot reconstruct occluded parts and compositional generators generalize poorly to complex open-world scenes.Source: §2 · p. 3

Attention control in diffusion models

• Attention control is established for 2D diffusion, but irregular 3D latent representations often lack canonical spatial indices, making attention manipulation harder and less explored.Source: §2 · p. 3

3.1 PRELIMINARIES: COMPOSITIONAL 3D GENERATORS

• A compositional generator denoises part-specific latent token blocks conditioned on image features, then decodes each part and fuses the resulting meshes.The generator combines a Transformer VAE with a latent-space denoising network.Source: §3.1 · pp. 3–4

3.2 PROBING LATENT STRUCTURE WITH DEBIASED CLUSTERING

• CCA-based debiased clustering suppresses shared latent structure before regrouping residuals, revealing that raw part tokens fail to form stable instances while debiased tokens succeed.The probe identifies missing structural constraints in the assignment mechanism as the bottleneck.Source: §3.2 · p. 4

3.3 OPTIMAL-TRANSPORT–GUIDED CORRELATION ASSIGNMENT

• SceneTransporter solves a global entropic OT assignment between image patches and 3D parts, enforcing per-part capacity budgets and equal mass contribution from each patch.The transport plan gates cross-attention, regulating how much evidence each patch contributes to each part.Source: §3.3 · pp. 5–6
• An edge-regularized assignment cost smooths affinities within image regions while discouraging their spread across boundaries, supporting coherent, boundary-respecting assignments.The edge map is downsampled to the patch grid; positive edge weighting suppresses cross-boundary transport without semantic masks.Source: §3.3 · pp. 6–7

4.1 EXPERIMENTAL SETUP

• The evaluation compares PARTCRAFTER, PARTPACKER, and MIDI using released code and checkpoints, geometry-fidelity metrics, and inter-part overlap measures.Lower top-20 mean and maximum IoU indicate better disentanglement; inference uses a dual-volume latent and OT-gated attention in the first half of DiT blocks.Source: §4.1 · p. 7

4.2 COMPARISONS WITH STATE-OF-THE-ARTS.

• On 74 diverse Web scene images, SceneTransporter achieves the highest geometry fidelity and second-lowest inter-part overlap, while running slower than PartPacker but faster than MIDI and PartCrafter.PartCrafter’s lowest overlap is attributed largely to discarding background and ground regions, which compromises scene completeness.Source: §4.2 · p. 7
• Figure 4 compares generated part colors across methods; SceneTransporter produces coherent object-level parts where PartPacker exhibits fragmentation and feature entanglement.MIDI degrades on outdoor cases and requires instance masks at inference, while PartPacker performs well on separated objects but deteriorates in complex layouts.Source: §4.2 · pp. 7–8, Figure 4 · p. 8
• In a 30-participant study, SceneTransporter receives the highest preference for geometry quality, layout coherence, and segmentation plausibility.The study uses forced rankings from 1 to 4; the caption says higher scores indicate better performance.Source: §4.2 · p. 8, Table 2 · p. 8

4.3 ABLATION STUDY

• OT plan-gated cross-attention yields focused, non-overlapping assignments and clean geometries, whereas standard cross-attention produces diffuse maps and corrupted geometry.The figure contrasts the OT-gated and standard cross-attention rows, including soft maps, hard assignments, decoded geometries, and fused meshes.Source: §4.3 · p. 8, Figure 6 · p. 9
• After roughly t≈540/600, the OT plan’s global partition changes little; later denoising mainly makes local refinements.Buildings, furniture, and trees are routed into single volumes in the late stage.Source: §4.3 · p. 9, Figure 7 · p. 9
• Edge regularization separates adjacent but distinct objects, producing crisper boundaries and fewer mixed parts without additional instance-mask supervision.The examples include a sofa beside a side table and wooden posts adjoining a fence.Source: Figure 5 · p. 8, §4.3 · p. 9

5 CONCLUSION

• SceneTransporter is a framework for structured 3D scene generation from a single image.Source: §5 · p. 9
• The method reframes generation as global correlation assignment and uses an Optimal Transport layer to impose structural constraints on the generative process.The conclusion says these constraints address structural mispartition and geometric redundancy in existing models.Source: §5 · p. 9
• Experiments report state-of-the-art performance with improved geometric fidelity and instance-level coherence on complex open-world scenes.Source: §5 · p. 9

A THE USE OF LARGE LANGUAGE MODELS

• LLMs assisted with grammar, clarity, tone, and restructuring; human authors directed, reviewed, and edited outputs and retained responsibility for the paper.Source: A THE USE OF LARGE LANGUAGE MODELS · p. 14

• The probe stacks dual-volume latents from Tang et al. (2025) and applies mild whitening before examining their organization.Source: B METHOD DETAILS · p. 14
• CCA identifies correlated directions shared by the whitened volumes; directions above threshold τ define a shared subspace whose component is removed from each volume.The resulting debiased tokens down-weight high-variance global modes while preserving object-specific variation.Source: B METHOD DETAILS · p. 14
• A flexible Gaussian mixture clusters the debiased tokens, reassigns low-confidence tokens to high-confidence centroids, and decodes groups independently with the frozen VAE decoder.Source: B METHOD DETAILS · p. 14

C.1 HYPERPARAMETERS

• Table 3 provides the hyperparameters needed to replicate the experiments.Source: §C.1 · p. 14
• Table 3 lists hyperparameters for OT plan-gated cross-attention.Source: Table 3 · p. 15
• The listed OT plan-gated cross-attention hyperparameters are used in all experiments.Source: Table 3 · p. 15

• Thirty computer-vision and graphics researchers and graduate students rank each method from 1 to 4 on three criteria.The four methods are shown under randomly permuted labels, with meshes available for interactive rotation and zooming.Source: §C.2 · p. 15
• The study evaluates geometry fidelity, scene layout relative to the reference, and plausibility of object-level part decomposition.The segmentation criterion considers clear instances with minimal overlaps, missing regions, or mixed parts.Source: §C.2 · p. 15
• SceneTransporter achieves the highest average rank across geometry quality, layout coherence, and segmentation plausibility.Higher average ranks indicate stronger human preference.Source: §C.2 · p. 15

• All methods train on synthetically rendered scenes, then undergo zero-shot evaluation on natural images from DL3DV-10K.The synthetic training data provides consistent part-level annotations.Source: §D.1 · p. 15
• Directly applying the synthetic-trained model to raw photographs causes a noticeable performance drop from the appearance domain gap; recent image-editing models such as GPT-5 transfer style toward graphics-engine renderings while preserving scene layout and object identities.Source: §D.1 · p. 15
• Test-time style transfer yields more faithful, coherent reconstructions, with better object separation, image-aligned part boundaries, and layout matching across indoor and outdoor scenes.Figure 8 presents qualitative results on style-transferred real-world images.Source: §D.1 · p. 16, Figure 8 · p. 16

• Each OT-gated layer minimizes transport cost plus entropic regularization subject to prescribed row and column marginals.The transport plan A is in R^2×M; ε controls regularization, and H(A) is the negative-entropy regularizer.Source: §D.2 · p. 16
• Sinkhorn updates use dual potentials for row and column constraints, then normalize the transport plan toward the prescribed marginals.A fixed point corresponds to a stationary solution of the entropic OT objective.Source: §D.2 · pp. 16–17
• Dual and plan residuals drop 3–5 orders of magnitude within 3–5 Sinkhorn iterations, then flatten; marginal violation stays small under 40 iterations.Figure 9 plots residuals and marginal-constraint violation for three representative OT-gated layers; the analysis reports stable convergence across layers and denoising steps.Source: §D.2 · p. 17

D.3 ADDITIONAL ABLATION STUDIES

• Defaults εt=0.10, γedge=8.0, and λedge=0.8 balance transport smoothness, boundary separation, and intra-part coherence; extremes degrade metrics or segmentation.Low γedge can benefit global geometry while increasing boundary leakage; γedge≥10.0 harms all metrics.Source: §D.3 · p. 18
• Using OT in half the DiT blocks gains geometry fidelity and part disentanglement at moderate runtime; applying it to all blocks brings diminishing returns and significant latency.The half-block configuration is the default; one-third coverage is insufficient for effective part separation.Source: §D.3 · p. 18, Table 4 · p. 19
• The full configuration performs best across metrics; removing OT gating lowers Geometry Fidelity and IoU, while OT modules raise inference time from 47.4 s to 55.0 s with negligible memory consumption.Table 4 reports the ablation metrics; omitting edge regularization improves on baseline but leaves IoU below the full configuration.Source: §D.3 · pp. 17–18, Table 4 · p. 19

D.4 FAILURE CASES AND RESIDUAL DIAGNOSTIC

• Dense clusters of tiny similar objects sharing few patches may merge into neighbors and be under-counted; residual cross-attention recovers instances while preserving global layout and part separation.Figure 10 compares the baseline, OT-guided routing, and the residual-attention variant.Source: §D.4 · p. 19, Figure 10 · p. 19
• Short denoising schedules or small token budgets can produce nonsmooth surfaces or floating parts, usually as local artifacts that leave overall layout intact.More diffusion steps or tokens alleviate these cases.Source: §D.4 · p. 19
• Strongly out-of-distribution real-image appearances, such as unusual lighting or textures, can degrade geometry and part grouping; style transfer improves results on many challenging inputs.Source: §D.4 · p. 20