SPOON: Towards Coherent Compositional 3D Scene Generation from Uncalibrated Multi-view Images

• Image-conditioned 3D generators provide strong priors for high-quality object geometry, making compositional scene generation increasingly practical.The remaining challenge is arranging those assets coherently from multi-view observations.Source: §1 · pp. 1–2
• Some approaches entangle scene-space placement with object generation, requiring canonical-space generative priors to accommodate incomplete observations under arbitrary transformations.3D-Fixer conditions object completion on partial scene-coordinate geometry as a spatial anchor.Source: §1 · p. 2
• Separately estimating object poses from individual views can leave scale ambiguity and inaccurate hypotheses under occlusion, truncation, or limited visibility.Independently plausible hypotheses may become inconsistent in a shared scene frame.Source: §1 · p. 2
• SPOON uses reconstruction-derived scene geometry as a shared reference to guide pose generation, route hypotheses by geometric reliability, and jointly reconcile object layouts with camera poses.The framework preserves generated object geometry while reducing object–camera inconsistencies.Source: §1 · p. 2
• Experiments on ARSG-110K and MIDI-3D-Front show consistent improvements in scene composition and object placement across varying numbers of input views.SPOON frames this as scene-level pose reasoning through its Guide–Route–Reconcile paradigm.Source: §1 · p. 2

2.1 3D SHAPE GENERATION

• Large-scale 3D collections and varied generator representations have improved the geometric fidelity and structural complexity of generated assets.These developments make extending object-level generative models toward compositional scenes increasingly feasible.Source: §2.1 · p. 3

2.2 COMPOSITIONAL 3D SCENE GENERATION

• Multi-view generation methods such as ReconViaGen, UniRecGen, and Mix3R transfer reconstruction-model knowledge to generators, primarily to enhance object shape using observations.The section frames evidence integration and shared-frame layout recovery as two central design questions for compositional scenes.Source: §2.2 · p. 3
• Compositional approaches expose layout as an explicit variable, while some pose-aware or transformed-shape formulations risk broadening the modeled shape distribution and degrading mesh geometry.Examples include predicting transformations before generation, assembling independently generated assets, and refining composition with photometric or physics-aware objectives.Source: §2.2 · p. 3

3.1 JOINT SHAPE AND POSE MODELING

• SAM3D first jointly generates low-resolution canonical object geometry and relative camera pose, then generates texture and fine geometry.Separately modeling geometry and layout keeps the data distribution simple, though it may sacrifice layout-estimation accuracy.Source: §3.1 · pp. 3–4
• At the endpoint, the shape latent decodes to a canonical object representation, while the pose state yields a camera-frame transformation placing it relative to each view.The model evolves shape and pose states using predicted velocity fields.Source: §3.1 · pp. 3–4

3.2 MULTI-VIEW FLOW FUSION

• Multi-view velocity fusion aggregates shape velocities into a shared shape trajectory, while pose trajectories remain view-conditioned and evolve independently.The resulting shared canonical shape integrates complementary evidence across observations without merging the camera-relative pose hypotheses.Source: §3.2 · p. 4

4 METHOD

• Given unposed RGB images and instance masks, SPOON reconstructs shared camera geometry and combines canonical object representations with view-specific pose hypotheses.The method addresses the possibility that individually plausible placements may not form a scene-coherent configuration.Source: §4 · p. 4

4.1 GEOMETRY-GUIDED OBJECT POSE GENERATION

• Reconstruction-derived geometry aligns world-space object points with the canonical object surface, providing a pose reference for geometry-guided generation.The aligned pose becomes the clean endpoint for pose rectified flow; the flow update retains its noise endpoint while steering the clean endpoint toward the geometry-derived pose.Source: §4.1 · pp. 4–5

4.2 GEOMETRY-AWARE POSE ROUTING

• SPOON estimates pose reliability from geometric observability, using frame completeness and visible coverage to select a view for world-frame placement.The aggregated object point cloud is reprojected into each image, including geometry beyond the image boundary, to capture truncation and visible coverage.Source: §4.2 · p. 6

4.3 MULTI-VIEW POSE RECONCILIATION

• SPOON jointly refines object layouts and camera poses using multi-view observations and differentiable Gaussian splatting.The optimization uses rendered-image discrepancy with photometric and depth supervision; depth from the 3D reconstruction model acts as a regularization term.Source: §4.3 · pp. 6–7

5.1 EXPERIMENTAL SETUP

• The evaluation uses ground-truth scene configurations and object geometry, measuring scene- and object-level Chamfer Distance and F-Score plus object-bounding-box volumetric IoU.Predicted object poses are retained for the geometry metrics; lower CD and higher F-Score and IoU indicate better performance.Source: §5.1 · p. 7

5.2 COMPARISON WITH STATE-OF-THE-ART METHODS

• On ARSG-110K, versus MV-SAM3D, SPOON reduces CDS and CDO by 12.7% and 17.7%, and improves FSS and FSO by 5.7% and 12.3%; on MIDI-3D-Front, it reduces CDS and CDO by 16.9% and 20.0%.Object-layout IoU improves by 13.2% on ARSG-110K and 14.6% on MIDI-3D-Front.Source: §5.2 · p. 8
• On ARSG-110K, the IoU gain over MV-SAM3D rises from 13.2% with 4 views to 17.6% with 8 views, while CDS reduction grows from 12.7% to 25.5%.On MIDI-3D-Front, SPOON maintains stable scene-level CD and layout IoU as the number of views increases.Source: §5.2 · p. 9
• Qualitative comparisons show more accurate object orientations and spatial arrangements, with fewer misplaced objects and more coherent scene layouts.Figure 3 compares MIDI-3D-Front results; Figure 4 shows ARSG-110K comparisons.Source: Figure 3 · p. 8, Figure 4 · p. 8, §5.2 · p. 9

5.3 ABLATION STUDIES

• Ablations show that removing any Guide–Route–Reconcile stage degrades scene reconstruction and layout accuracy; geometry-grounded initialization, reliable hypothesis selection, and reconciliation each contribute.Jointly refining object placements and camera poses improves scene-level reconstruction and layout accuracy.Source: §5.3 · p. 9, Table 4 · p. 9, Table 3 · p. 9
• Pose reliability is not adequately captured by mask size or a single geometric cue; combining frame completeness and visible coverage supports more reliable selection across views.Boundary-truncated observations yield less reliable hypotheses, while the reliability score favors a placement more consistent with the scene layout.Source: §5.3 · p. 9, Figure 5 · p. 9, Table 2 · p. 9

• SPOON frames compositional 3D generation as scene-level pose reasoning, using reconstructed multi-view geometry as a shared reference.Source: §6 · p. 9
• Its Guide–Route–Reconcile paradigm guides pose generation, routes reliable hypotheses, and reconciles object–camera configurations.Source: §6 · p. 9
• Experiments on ARSG-110K and MIDI-3D-Front show consistent improvements in object placement and scene composition across different numbers of input views.Source: §6 · p. 9

• The appendix provides further methodological details.Source: A APPENDIX · p. 13
• It provides implementation details and additional analyses and qualitative results.Source: A APPENDIX · p. 13
• It includes further discussion of limitations and future directions.Source: A APPENDIX · p. 13

• Large Language Models are used only for minor language editing.Source: §A.1 · p. 13
• The language editing is intended to improve grammar and readability.Source: §A.1 · p. 13
• The authors develop all methodological design, experiments, equations, and results.Source: §A.1 · p. 13

• SPOON fuses multi-view shape velocities into a shared canonical trajectory, guides view-specific pose trajectories when a stable surface is available, routes hypotheses by geometric reliability, then jointly refines object and camera poses.The final reconciliation uses differentiable multi-view rendering.Source: §A.2 · pp. 13–14
• The surface bridge centers and isotropically normalizes both surfaces, aligns their up axes, then estimates a shared yaw rotation that preserves relative camera configuration across views.The resulting bridge maps reconstruction world coordinates to the object-canonical frame and transfers reconstruction-derived camera poses into that frame.Source: §A.2 · pp. 13–14
• The bridge is accepted only when normalized fitting RMSE is below 0.15 and at least four valid views support alignment; otherwise, SPOON disables geometry guidance and uses the original pose flow.TrimRMS uses the lowest-error 0.85 fraction of nearest-neighbor residuals to reduce the influence of unmatched regions and reconstruction outliers.Source: §A.2 · p. 14

• Pose guidance uses strength αP = 0.8 from t = 0.6 onward; routing uses α = 2 and completeness threshold τf = 0.5, while reconciliation combines photometric and depth losses.AdamW learning rates are 3 × 10−3 for object transformations and 1 × 10−4 for camera poses; depth loss is used as regularization.Source: §A.3 · p. 14
• With four-view input, SPOON takes 326.5s per five-object scene versus 195s for MV-SAM3D; guidance adds 10s, routing 1.5s, and reconciliation 120s.The timing evaluation uses a single NVIDIA H100 GPU.Source: §A.3 · p. 14
• For scene evaluation, one global Sim(3) aligns predicted camera centers to ground-truth centers and is applied uniformly, with no additional per-object alignment.Single-view baselines instead follow their official coordinate and scale calibration protocols.Source: §A.3 · pp. 14–15

A.4 ADDITIONAL RESULTS

• Geometry guidance steers objects away from inaccurate orientations toward poses more consistent with the observed scene layout.The comparison is qualitative and illustrated in Figure 6.Source: §A.4 · p. 15, Figure 6 · p. 15
• Across α ∈ {1, 2, 3}, all three metrics remain nearly unchanged, indicating robustness to moderate variations in the completeness weighting.Table 5 evaluates the routing exponent on MIDI with four input views and multi-view reconciliation disabled.Source: §A.4 · p. 15, Table 5 · p. 15
• On MIDI-3D-Front and ARSG-110K, qualitative comparisons show more consistent orientations and relative placements that better agree with reference observations.The comparisons appear in Figures 7 and 8, respectively.Source: §A.4 · p. 15, Figure 7 · p. 16, Figure 8 · p. 17

A.5 LIMITATION AND FUTURE WORK

• SPOON generates individual objects independently rather than through an explicit multi-instance process; the authors identify joint multi-instance generation as future work.They suggest it could enable stronger coupling between object generation and scene-level spatial organization.Source: §A.5 · p. 15
• Geometry-grounded reasoning depends on the underlying multi-view reconstruction and may be affected by inaccurate or incomplete geometry.Source: §A.5 · p. 16