Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene Reconstruction

1 INTRODUCTION

• Holistic scene generation absorbs placement into one representation, while compositional methods preserve object fidelity by separating canonical-space geometry from scene-space layout.Under fixed token, voxel, or latent budgets, holistic representations cover the scene with fewer effective degrees of freedom per object, often under-resolving fine structures, small objects, and boundaries.Source: §1 · p. 2
• Sparse, unbounded translation, rotation, and scale are difficult to regress under occlusion, perspective ambiguity, long-tailed configurations, and scarce scene-level supervision.SAM3D's artist-assisted, large-scale data engine still uses sparse pose variables and has limited object alignment accuracy.Source: §1 · p. 2
• Coord Cube densifies prediction with scene-space locations for sampled canonical points and improves over raw pose regression, but its unbounded coordinates constrain robustness under limited scene data.The object transformation is recovered by aligning predicted scene-space points with their canonical coordinates.Source: §1 · p. 2
• CCM maps each visible object pixel to a surface coordinate in bounded canonical space; paired with a scene-space PCM, it enables dense correspondence alignment to recover transformations.Bounded canonical coordinates provide a stable target for scalable object-level supervision, and dense correspondences improve robustness to local prediction errors.Source: §1 · pp. 2–3
• On Blend Swap (2026), Mira-Scene improves 3D-IoU from 0.520 to 0.727 and 2D-IoU from 0.672 to 0.783 over SAM3D using substantially less, publicly sourced training data.Experiments span indoor, outdoor, synthetic, and in-the-wild scenes.Source: §1 · p. 3
• A multimodal diffusion transformer jointly generates canonical geometry and pixel-aligned CCMs through modality-specific streams and shared attention, promoting shape-layout consistency.The design preserves the distinct structures of 3D geometry and coordinate maps, with shared positional encoding supporting cross-modal interaction.Source: §1 · p. 3

2.1 PROBLEM FORMULATION

• Each object is represented by canonical geometry and a transformation mapping it into the input image’s camera coordinate frame, while the model predicts geometry and CCM rather than directly regressing that transformation.The explicit transformation is recovered afterward by aligning the CCM with a scene-space PCM.Source: §2.1 · p. 3

2.2 PIXEL-ALIGNED LAYOUT REPRESENTATION

• CCM assigns visible object pixels bounded canonical surface coordinates, which pair with PCM scene points to recover object placement through geometric alignment.Because CCM coordinates are in canonical object space, rendered assets can supervise them without ground-truth scene layouts.Source: §2.2 · pp. 3–4

2.3 GEOMETRY-LAYOUT CO-GENERATION

• A Geometry Expert generates 3D voxel geometry while a Layout Expert generates pixel-space CCMs; shared self-attention and a shared 3D positional basis connect the streams.The training objective uses separate rectified-flow losses for the geometry latent grid and CCM.Source: §2.3 · pp. 4–5, Figure 4 · p. 4

2.4 SCENE ASSEMBLY

• RANSAC filters noisy CCM predictions and depth outliers before Umeyama alignment estimates the object transformation from valid canonical-to-scene correspondences.The correspondence set uses pixels where the object mask, CCM prediction, and PCM are all valid.Source: §2.4 · p. 5

2.5 TRAINING PIPELINE

• Training first learns geometry and CCM priors from isolated-object renderings, then fine-tunes on occluded 3D-FRONT scene views for amodal reconstruction.Pre-training uses 60K objects and 1M object-centric views; fine-tuning uses 20K scene views.Source: §2.5 · p. 5, Figure 5 · p. 6

3.1 EXPERIMENTAL SETUP

• The evaluation uses 3D-Future Scene and BlendSwap benchmarks, compares four scene-generation baselines under shared image and mask inputs, and measures geometry and layout separately.BlendSwap covers indoor and outdoor settings as well as realistic and cartoon-style appearances; in-the-wild inputs are also evaluated qualitatively.Source: §3.1 · p. 6

3.2 SCENE GENERATION RESULTS

• Mira-Scene’s advantage is most pronounced in scene layout, improving 3D-IoU by 39.8% on BlendSwap and 16.4% on 3D-Future Scene relative to SAM3D, the strongest baseline on this metric.Source: §3.2 · p. 7

3.3 3D–2D CORRESPONDENCE ANALYSIS

• CCM directly maps 2D pixels to 3D points, reducing correspondence ambiguity; in Mesh+GT, Mira-Scene has substantially better CD and F-scores than CUPID, though CUPID has slightly higher 2D mask IoU.The Table 2 metrics compare correspondence-induced and target rendered-depth point clouds.Source: §3.3 · p. 8, Table 2 · p. 8

3.4 ABLATION

• CCM with PCM raises 3D-IoU from 0.379 to 0.727 and 2D-IoU from 0.381 to 0.783 over Coord Cube; it also outperforms Raw and Coord Cube under matched training data.Coord Cube only slightly improves over Raw, indicating that densifying unbounded scene-space prediction alone does not resolve its learning difficulty.Source: §3.4 · pp. 8–9, Table 3 · p. 9
• Removing joint attention reduces 2D-IoU from 0.757 to 0.535 and increases CD from 0.017 to 0.070, while removing shared positional encoding causes a smaller but consistent alignment drop.The full model achieves the best consistency across all reported metrics.Source: §3.4 · p. 9, Table 4 · p. 9

3.5 APPLICATIONS

• Reconstructed object assets can be removed, repositioned, reconfigured, rigged, or animated while preserving a coherent spatial layout, and exported to editing tools, simulators, and physics engines.The paper identifies perception, planning, manipulation, and physical simulation as downstream uses.Source: §3.5 · p. 9

4 DISCUSSION AND CONCLUSIONS

• Mira-Scene replaces sparse pose regression with a dense, bounded CCM representation and recovers placement through CCM–PCM alignment, reducing dependence on scarce scene-level 3D supervision.The model uses separate geometry and layout streams with shared attention and a shared positional basis to improve their consistency.Source: §4 · p. 9
• Across indoor, outdoor, synthetic, and in-the-wild inputs, Mira-Scene achieves competitive object geometry and significantly more accurate scene layouts than strong baselines.The authors state that learning from abundant object-level 3D assets provides potential for further scaling.Source: §4 · p. 9
• The authors used generative AI tools to polish the manuscript and to create some training-image backgrounds and qualitative-evaluation inputs.They state that they take responsibility for the final content, including AI-assisted text and generated data.Source: AI USE STATEMENT · p. 10

A.1 QUALITATIVE COMPARISON WITH SCENE GENERATION METHODS

• Scene reconstructions pair a single-image input with generated object assets and a coherent scene layout; input-view and novel-view overlays expose layout errors as objects misalign or drift.The appendix includes indoor, outdoor, realistic, and stylized examples and compares Mira-Scene with SAM3D, MIDI, SceneGen, and Gen3DSR.Source: Figure 9 · p. 17, §A.1 · p. 17

A.2 QUALITATIVE COMPARISON WITH CUPID

• The CUPID comparison overlays CCM-induced points on Mira-Scene's generated mesh to visualize geometry–correspondence consistency.The last column of Figure 10 is the consistency visualization.Source: §A.2 · p. 18

A.3 ADDITIONAL ABLATION VISUALIZATIONS

• Compared with CUPID, Mira-Scene produces more plausible geometry and better CCM–mesh consistency.Figure 10 presents this qualitative comparison.Source: Figure 10 · p. 18
• The architecture ablation visualization indicates that joint attention is important for geometry–CCM consistency, while shared positional encoding provides additional alignment improvement.Figure 11 shows the architecture variants.Source: Figure 11 · p. 18

A.4 LAYOUT REPRESENTATION UNDER MATCHED TRAINING DATA

• CCM with PCM outperforms Raw and Coord Cube under matched training data, and additional object-level supervision further improves layout accuracy.This supports both a representation benefit and a benefit from object-level data.Source: §A.4 · p. 18, Table 5 · p. 18

A.5 QUALITATIVE COMPARISON WITH SCENEMAKER

• SceneMaker results show object substitution and inaccurate scale, pose, or relative layout, with representative errors marked in Figure 12.The comparison uses at most five selected objects per scene, following SceneMaker’s reported setting.Source: §A.5 · p. 18, Figure 12 · p. 18

A.6 ADDITIONAL VISUAL QUALITY EVALUATION

• 18.8 vs. 18.6 PSNR: Mira-Scene scores slightly higher than SAM3D.The PSNR evaluation uses SceneGen’s evaluation implementation with RGB values normalized to [0, 1].Source: §A.6 · p. 19, Table 6 · p. 19
• Table 6 also reproduces CD and 3D-IoU results from Table 1 for reference.The additional evaluation focuses on visual quality using PSNR.Source: Table 6 · p. 19, §A.6 · p. 19

A.7 CORRESPONDENCE ERRORS AND ALIGNMENT ROBUSTNESS

• 13.4% relative 3D-IoU gain for RANSAC+Umeyama over Umeyama; IRLS with Huber adds 1.0% relative, but the authors retain the simpler RANSAC+Umeyama.The correspondence-error distribution is long-tailed: median 0.046, 95th percentile 0.395, and 99th percentile 0.882.Source: §A.7 · p. 19, Table 7 · p. 19

A.8 LAYOUT ACCURACY UNDER OCCLUSION

• 3D-IoU falls from 0.752 for mostly visible objects to 0.635 for heavily occluded objects, while substantial overlap with ground-truth boxes remains.Scene-level fine-tuning uses 3D-FRONT views to pair partial observations with complete objects for amodal reconstruction and layout recovery under occlusion.Source: §A.8 · p. 19, Table 8 · p. 19

A.9 ADDITIONAL SCENE RECONSTRUCTIONS

• Additional reconstructions span indoor, outdoor, real, and stylized scenes, showing object decompositions, compositional scenes, and novel-view renderings.The examples complement the paper’s quantitative comparison and illustrate varied scene layouts and visual styles.Source: §A.9 · p. 20, Figure 13 · p. 20, Figure 14 · p. 21

A.10 APPLICATIONS

• Separate object assets support individual-object editing and export to interactive tools, embodied-AI simulators, and physics engines.Figure 15 presents scene editing, embodied AI, and physical simulation as applications.Source: §A.10 · p. 22, Figure 15 · p. 22

B.1 SINGLE-IMAGE 3D GENERATION

• Holistic scene models leverage object priors but allocate limited representation to each object, potentially under-resolving small, occluded, or thin-structured objects (Ling et al., 2025a; Wang et al., 2026b; Lin et al., 2025).Object-level generation models have improved speed and object fidelity (Hong et al., 2023; Zhang et al., 2023; Wu et al., 2024; Zhang et al., 2024; Li et al., 2025b; Xiang et al., 2025b;a; Lai et al., 2025; Li et al., 2025c).Source: §B.1 · p. 22

B.2 3D LAYOUT GENERATION

• Semantic layout primitives offer controllability but are not tied to pixel-level geometry; sparse pose variables provide little intermediate structure and can be sensitive to ambiguity and outliers.Mira-Scene instead aligns generated canonical geometry and CCM with scene-space points, enabling layout learning from object-level assets without scene-level layout annotations and yielding over-determined alignment.Source: §B.2 · p. 23

B.3 MULTI-MODAL GENERATIVE MODELS

• Mira-Scene uses separate geometry and layout expert streams with shared self-attention and a shared geometry-layout positional space to co-generate canonical 3D geometry and pixel-aligned 2D CCMs.The design preserves the distinct spatial structures of the outputs while allowing information exchange between them.Source: §B.3 · p. 24

C.1 ARCHITECTURE AND IMAGE CONDITIONING

• Geometry and layout use separate transformer experts: geometry operates on sparse 8 × 8 × 8 shape latents, while the layout expert predicts pixel-aligned CCMs at 296 × 296 crop resolution.Geometry is conditioned through cross-attention to full-image, mask, and object-crop features; layout concatenates cropped RGB with noisy CCM and also uses full-image and mask features through cross-attention.Source: §C.1 · p. 24, Figure 16 · p. 24

C.2 ROBUST GEOMETRIC ALIGNMENT

• Robust alignment estimates object transformations from valid CCM–PCM correspondences using RANSAC hypotheses, adaptive inlier thresholds, and iterative refinement.The pipeline constructs a camera-space PCM from monocular depth, filters invalid and depth-edge pixels, and refines the best-supported transform for up to five rounds before applying it to the canonical geometry.Source: §C.2 · p. 25

C.3 RECTIFIED FLOW BACKGROUND

• The rectified-flow background describes a linear interpolation between a data sample and Gaussian noise, with a learned time-dependent vector field transporting noisy samples back toward the data distribution.The vector field is learned using a conditional flow-matching objective.Source: §C.3 · p. 26

D.1 DATA CONSTRUCTION DETAILS

• Object-level pretraining uses 60K Objaverse assets rendered into 1M views, with visible canonical surface coordinates providing direct supervision for geometry and CCMs.A 20K background-completed subset reduces the gap to scene images; separate scene-level fine-tuning uses 20K 3D-FRONT views to adapt to occlusion, perspective variation, and amodal completion.Source: §D.1 · p. 27

D.2 TRAINING SETTINGS

• Training uses rectified flow matching; geometry receives 10% condition dropout and classifier-free guidance at inference, while layout uses deterministic image conditioning without classifier-free guidance.The geometry stream is initialized from TRELLIS-image-large and pretrained for 100K steps; fine-tuning uses 8 NVIDIA A100-80G GPUs.Source: §D.2 · p. 27

D.3 INPUT PREPROCESSING AND INFERENCE

• At test time, the model uses a single RGB image and instance masks to jointly predict per-object geometry and CCMs, then estimates camera-space points and recovers placement by robust alignment.Final meshes use SAM3D’s second-stage mesh-generation and refinement module; the stated contribution focuses on geometry-layout co-generation and correspondence-based layout recovery.Source: §D.3 · p. 27

D.4 AUTOMATIC VLM–SAM3 INSTANCE SEGMENTATION

• The VLM-guided SAM3 pipeline verifies instance membership, deduplicates masks, and recycles unselected candidates to recover instances omitted by the initial object list.In the illustrated six-chair case, recycling adds three chairs from cached candidates without additional SAM3 calls; quantitative comparisons use ground-truth instance masks.Source: §D.4 · p. 28, Figure 17 · p. 28

D.5 DETAILS OF LAYOUT REPRESENTATIONS

• With geometry and co-generation architecture fixed, the layout variants use 8 Raw, 4096 Coord Cube, and 1369 CCM tokens.The variants differ in layout targets and how the object-to-scene transformation is recovered.Source: §D.5 · pp. 28–29, Table 10 · p. 29
• Raw pose directly predicts an object-to-scene transformation from scale, rotation, and translation, requiring scene-level pose supervision.Source: §D.5 · p. 29
• Coord Cube predicts scene locations for uniformly sampled canonical-grid points and recovers the transform by alignment; its denser target tolerates small local errors but remains unbounded and scene-supervised.Source: §D.5 · p. 29
• CCM predicts canonical coordinates at visible crop pixels; restored pixels pair with PCM scene points for robust geometric alignment.Source: §D.5 · p. 29

E.1 EVALUATION DATASETS

• The evaluation samples 50 noisy-mask 3D-Future Scene examples, uses higher-quality and more diverse BlendSwap as its primary benchmark, and includes web- or generated in-the-wild images qualitatively.Source: §E.1 · p. 30

E.2 METRIC DEFINITIONS AND COMPUTATION

• Geometry reports CD, FS@0.1, and EMD; layout reports 3D-IoU on aligned scene boxes and 2D-IoU from calibrated-camera silhouettes, alongside ADD-S and ICP-Rot.Source: §E.2 · p. 30

E.3 NORMALIZATION AND ALIGNMENT

• Predictions are converted to a common evaluation frame before robust ICP, which uses yaw-sweep initialization and coarse-to-fine registration.The registration strategy adds initialization steps to reduce sensitivity to local minima.Source: §E.3 · pp. 30–31

• The four settings are CUPID-(Mesh+GT), Ours-(Mesh+GT), Ours-(CCM+GT), and Ours-(CCM+Mesh).Source: §E.4 · p. 31
• Mesh+GT compares rendered posed-mesh depth points with ground-truth depth points, while CCM+GT compares CCM-induced points with ground-truth depth points.Source: §E.4 · p. 31
• CCM+Mesh compares CCM-induced points with depth points from the final generated and posed mesh to measure layout–geometry consistency.Source: §E.4 · p. 31

• Independent per-object reconstruction can produce intersections, particularly when objects are close or generated geometry is inaccurate.The paper gives a cat’s tail intersecting a chair as an example.Source: F LIMITATIONS AND FUTURE WORK · p. 31, Figure 18 · p. 31
• The method requires RGB images and object masks; noisy masks in complex scenes can cause floaters in reconstructed results.The authors identify reducing mask dependence as future work.Source: F LIMITATIONS AND FUTURE WORK · p. 31
• Mira-Scene relies on monocular geometry estimation for global scene geometry, and thin, sheet-like point clouds on some cartoon images can affect result quality.Source: F LIMITATIONS AND FUTURE WORK · p. 31
• The current training set is substantially smaller than SAM3D’s and other 3D generation models’, and the authors report relatively weaker single-object generation quality.Source: F LIMITATIONS AND FUTURE WORK · p. 31