WorldRecon: Generative Multi-View Mesh Reconstruction for Large-Scale 3D World

• Existing reconstruction methods can leave occluded regions incomplete and produce noisy geometry that is difficult to convert into clean meshes with detailed surfaces and materials.Source: §1 · p. 2
• A single high-resolution generative representation struggles to scale when an environment’s spatial extent and geometric complexity exceed its capacity.Source: §1 · p. 2
• WorldRecon uses coarse camera and point-cloud estimates to partition scenes into overlapping blocks, condition block generation on local geometry and relevant images, and propagate context between neighbouring blocks.Its causal generation order carries sparse structure, geometry, and material context across overlaps.Source: §1 · p. 2
• Experiments on 3D-FRONT and ScanNet++ report globally accurate and complete indoor scenes with fine geometric and material detail in synthetic and real-world settings.Source: §1 · p. 2

• Traditional Structure-from-Motion and Multi-View Stereo recover poses and geometry through triangulation, while learned MVS uses differentiable cost volumes and pose-free models regress pointmaps [Schonberger & Frahm (2016); Pan et al. (2024); Furukawa & Hernández (2015); Schӧnberger et al. (2016); Yao et al. (2018; 2019); Wang et al. (2021a); Wang et al. (2024); Leroy et al. (2024); Wang et al. (2025; 2026)].NeRF, implicit surfaces, and Gaussian Splatting offer alternative scene representations, but these approaches can leave occluded regions incomplete or over-smooth.Source: §2 · pp. 2–3
• Generative multi-view reconstruction ranges from diffusion-based view or depth synthesis to feed-forward primitives and direct mesh generation, but scene-level methods SCube and GenRecon rely on accurate camera poses [Liu et al. (2024); Gao et al. (2024); Wu et al. (2024a); Guo et al. (2025); Xu et al. (2025); Jin et al. (2024); Tang et al. (2024b); Ziwen et al. (2025); Xiang et al. (2025; 2026); Zhang et al. (2024); Chang et al. (2025); Huang et al. (2026); Ren et al. (2024); Schmid et al. (2026)].Source: §2 · p. 3
• Scene-generation methods scale through blocks, but are typically conditioned on text or layouts rather than visual guidance and 3D spatial grounding [Chen et al. (2025); Wu et al. (2024b); Meng et al. (2025); Li et al. (2026); Engstler et al. (2025)].WorldRecon instead grounds blocks in a coarse predicted point cloud and a few multi-view images.Source: §2 · p. 3

3 WORLDRECON METHOD

• A reconstruction model supplies coarse camera estimates and a global point cloud that are partitioned into overlapping blocks, grounding blockwise generation while the generative prior completes detailed surfaces.Source: §3 · p. 3, Figure 2 · p. 4

3.1 DATA CURATION

• Training uses about 117k canonicalized blocks sliced from 3D-FRONT and SAGE-10k scenes, with randomized RGBD views rendered under varied lighting.Each block retains up to 16 views by in-block pixel coverage and is paired with noisy point clouds from augmented depth or VGGT reconstructions, preparing for test-time point-cloud conditions.Source: §3.1 · p. 4

3.2 RECONSTRUCTION-BASED BLOCK GENERATION

• Block generation fuses multi-view image and camera conditions, while the Sparse Structure stage also receives point-cloud tokens as a coarse geometric prior.Cascaded stages generate sparse voxel positions, geometry latents, and aligned material latents; all three are finetuned from TRELLIS.2 for block-level generation.Source: §3.2 · pp. 4–5
• Dedicated inpainting variants take a binary mask and known latent values so each new block can complete its unknown region from previously generated neighbours.Training masks cover one- or two-neighbour overlap layouts; inference leaves an editable transition band near block boundaries.Source: §3.2 · p. 5

3.3 WORLD GENERATION PIPELINE

• At inference, VGGT estimates coarse cameras and a point cloud; blocks are selected by visible-point coverage and generated in a voxel-priority order with at most two previously generated neighbours.Source: §3.3 · p. 6
• Sparse-structure overlap is clamped to its forward-flow trajectory during denoising, and geometry and material latents are integrated into shared world representations with boundary-weighted blending.Decoded block meshes are transformed into the shared frame and merged sequentially.Source: §3.3 · p. 6

4.1 EXPERIMENTAL SETUP

• Evaluation uses 30 held-out scenes per dataset, 32 unposed RGB inputs, and 50 held-out RGB, depth, and normal views; blocks are 4 m cubes with up to 4 views and 8192 points.Source: §4.1 · pp. 6–7

4.2 QUANTITATIVE RESULTS

• On 3D-FRONT, WorldRecon achieves the best overall performance, substantially outperforming all baselines across 3D geometry metrics and leading on evaluated 2D depth and normal metrics.It completes regions missing from the initial VGGT-Ω reconstruction.Source: Table 1 · p. 7, §4.2 · p. 7
• On ScanNet++, WorldRecon reduces CD by 43.1% and improves F-score by 12.0% over the strongest baselines.The comparison evaluates 30 real-world scenes; the reported results hold despite training solely on synthetic scenes.Source: Table 2 · p. 7, §4.2 · pp. 7–8

4.1 EXPERIMENTAL SETUP

• Metrics cover 3D geometry and rendered 2D depth and normal maps; five baselines span TSDF fusion, depth guidance, per-scene optimization, and chunked feature projection.Source: §4.1 · p. 7

4.3 QUALITATIVE RESULTS

• Across synthetic and real-world examples, WorldRecon produces more complete, geometrically coherent reconstructions with sharper boundaries, thin structures, and local details despite noisy initial estimates.The comparison reports fewer artifacts than baselines.Source: §4.3 · p. 8, Figure 3 · p. 8

4.4 ABLATION STUDY

• Reconstruction-based sparse-structure generation uses the point cloud as a spatial prior, while geometry-and-material generation uses multi-view observations for finer geometry and more faithful materials.These components are added progressively to pretrained TRELLIS.2.Source: §4.4 · p. 9
• Neighbour-aware inpainting propagates context between adjacent blocks and improves depth and normal metrics, but small boundary seams may remain and slightly degrade 3D metrics.Source: §4.4 · p. 9
• Stage-wise world latent clamping and blending maintains cross-block continuity, eliminates visible boundary discontinuities, and yields the best performance across reported metrics.Source: §4.4 · p. 9

5 CONCLUSION

• WorldRecon combines coarse geometry and camera estimates with block generation, neighbour-aware latent inpainting, and causal propagation to produce coherent, detailed large-scale scene meshes.The conclusion reports state-of-the-art indoor reconstruction and strong sim-to-real generalization despite synthetic-only training.Source: §5 · p. 9
• Training uses only indoor scenes, leaving outdoor generalization unestablished; single-layer 4 m-high blocks also restrict reconstruction to single-story environments.The authors identify more diverse training environments and vertical block generation as future work.Source: §5 · pp. 9–10

APPENDIX

• The appendix covers data curation, method and implementation details, evaluation protocols, additional qualitative results, and failure-case analysis.Its sections address block-level training data, reconstruction-based conditioning, inpainting, training, and world generation.Source: APPENDIX · p. 15

A DATA CURATION DETAILS

• Scenes from 3D-FRONT and SAGE-10k are rendered as multi-view RGB and depth observations for block-level training data.Camera views are sampled near eye level with varied lighting and material augmentation.Source: A DATA CURATION DETAILS · p. 15, §A.1 · p. 15

A.1 SCENE RENDERING

• Images are rendered at 1024×1024 resolution with a 90° field of view, while camera intrinsics and camera-to-world transforms are stored.Material augmentation varies architectural surfaces; furniture retains its original materials.Source: §A.1 · p. 15

A.2 SCENE SLICING

• Scenes are covered by overlapping axis-aligned 4 m cubic blocks with uniformly distributed overlap and the minimum number needed to cover each horizontal extent.Training scenes are under 4 m tall; meshes are clipped at block boundaries, with original materials preserved and new cut surfaces uncapped.Source: §A.2 · pp. 15–16

A.3 MULTI-VIEW CONDITION EXTRACTION

• Block-local RGBA conditions use views ranked by in-block pixel coverage, retain up to 16 valid views, and transform camera translations into the canonical block frame.A view is valid when at least 25% of its pixels fall inside the block; the in-block visibility mask is its alpha channel.Source: §A.3 · p. 16

A.4 POINT-CLOUD PREPARATION

• Training uses perturbed depth-derived clouds and filtered, aligned VGGT-Ω reconstructions, each cropped and sparsified to 8192 points.When both sources are available, they are registered as separate training instances; Table S1 summarizes reported training configurations.Source: §A.4 · p. 16, Table S1 · p. 17

B.1 RECONSTRUCTION-BASED CONDITIONING

• Image patches and camera-ray Plücker coordinates are fused into a fixed-length multi-view condition shared across generation stages.The camera token is prepended to each image’s patch tokens before cross-attention fuses the views.Source: §B.1 · p. 17
• The Sparse Structure stage additionally conditions on an 8192-point block cloud encoded with Fourier features, a learned projection, and modality tokens.The point-cloud stream is dropped with probability 0.1 during training.Source: §B.1 · p. 17

B.2 NEIGHBOUR-AWARE LATENT INPAINTING

• Dedicated inpainters receive masked known latents from previously generated neighbours, with mask value 0 for known locations and 1 for locations to generate.The formulation applies across Sparse Structure, Geometry, and Material stages; the Material stage can also receive a Geometry latent.Source: §B.2 · p. 17

B.3 TRAINING DETAILS

• Sparse Structure and Geometry are fully fine-tuned, while Material denoisers use rank-8, scale-16 LoRA modules alongside trainable multi-view fusion and camera-tokenization modules.Training uses AdamW, BF16, adaptive gradient clipping, and 0.1 unconditional condition dropout.Source: §B.3 · pp. 17–18

B.4 WORLD GENERATION PIPELINE

• The global pipeline filters and merges VGGT-Ω depth-derived points, then retains the largest connected component as the scene scaffold.It aligns gravity to positive Z and dominant wall directions to X and Y using point-cloud orientation estimates.Source: §B.4 · p. 18
• The aligned scaffold is covered by centred 4 m blocks at a 3 m stride, and up to four conditioning views are selected to maximize joint voxel coverage.Blocks with fewer than 100 reconstructed points are discarded; each retained crop yields an 8192-point condition.Source: §B.4 · p. 18

C.1 BASELINES

• All baselines use VGGT-Ω geometric predictions where required, including camera parameters, per-view depth and confidence, and an unprojected point cloud.Baseline pipelines include TSDF fusion, depth refinement, per-scene SDF optimization, and chunk-based feature projection.Source: §C.1 · p. 19

• VGGT-Ω’s uncalibrated reconstruction has no absolute scale or canonical horizontal orientation, so evaluation estimates a Sim(3) transform from corresponding camera centres.The transform is applied to the complete predicted mesh and used for all 2D and 3D metrics.Source: §C.2 · p. 19
• Dataset-specific axis conventions are converted before metrics are computed.The shared alignment transform is used for both 2D and 3D metrics.Source: §C.2 · p. 19
• For synthetic scenes, evaluation clips prediction triangles to the ground-truth axis-aligned bounding box enlarged by 20%.For ScanNet++, evaluation uses the observed scanner surface represented by a 1.1 cm voxel grid and dilated by 15 cm.Source: §C.2 · p. 19

C.3 METRICS

• 3D evaluation samples 200k points per mesh surface and reports Chamfer distance, F-score@10cm, and normal consistency.F-score matches points within 0.1 m; normal-consistency correspondences farther than 0.2 m contribute zero.Source: §C.3 · p. 19
• 2D evaluation renders aligned predictions and references from 50 held-out views, excluding frames used as VGGT-Ω input.Synthetic views are 512×512 and ScanNet++ views are 640×480; depth and normal errors are pooled over valid view–pixel pairs.Source: §C.3 · pp. 19–20
• The 2D metrics include depth and normal errors, perceptual and semantic similarity for depth and normal renderings, and completeness.Completeness measures valid reference foreground pixels with valid predicted geometry; LPIPS, CLIP similarity, and completeness are averaged across views.Source: §C.3 · pp. 19–20

D.1 LARGE SCENE GENERATION

• WorldRecon reconstructs an expansive nine-room scene with high fidelity, demonstrating scalability to large indoor environments.Figure S1 presents top-down and detailed interior views.Source: §D.1 · p. 20, Figure S1 · p. 20

D.2 COMPARISON WITH AGENTIC BASELINE.

• The agent iteratively scripts Blender scenes from geometric primitives and numerical material parameters, without image textures or imported assets.The setup uses fixed RGB views and does not enforce the original cost ceiling.Source: §D.2 · p. 20
• The two recorded agent reconstructions consumed approximately 2.12M and 1.71M tokens, with estimated Standard API-equivalent costs of $2.93 and $2.26.WorldRecon runs on a single GPU without external LLM calls.Source: §D.2 · p. 20
• With the same 32 views, WorldRecon more faithfully recovers furniture arrangement, architectural structure, and surface appearance than the GPT-6 Astra agent.The agent produces simplified object geometry and misplaced furniture, and reconstructs an incorrect global layout in the multi-room scene.Source: §D.2 · pp. 20–21, Figure S2 · p. 21

D.3 FAILURE CASES AND LIMITATIONS

• Sparse or occluded viewpoints lead the VGGT-Ω scaffold to misestimate several camera poses, leaving regions unreconstructed or projecting views to wrong locations.Some regions are filtered out because of low confidence.Source: §D.3 · p. 22
• WorldRecon still has missing geometry in unobserved regions when input coverage is incomplete, although it faithfully reconstructs well-covered areas despite substantial pose errors.It can plausibly complete missing point-cloud regions from sparse image cues.Source: §D.3 · p. 22