Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Image

Motivation

• Single-image reconstruction lacks triangulation and evidence about occluded regions; depth alone also omits appearance and unseen content needed for novel-view synthesis.Existing monocular scene reconstructors often require dataset-specific training and can be slow or memory-intensive.Source: §1 · p. 1

Setting

• Unlike object reconstruction, scene reconstruction cannot use a reservoir of background pixels to represent hidden content, while binocular scene methods require two views.Flash3D instead targets the more generally applicable monocular setting without requiring camera extrinsics.Source: §1 · p. 1

Generalization

• Flash3D builds on a monocular depth predictor as a foundation model to improve generalization to new datasets, including domains where it was not specifically trained.The paper reports greater accuracy than models trained specifically on the target domains.Source: §1 · p. 2

Representation

• Flash3D predicts multiple deterministic Gaussians per pixel, anchoring only the first to estimated depth and shifting others to model occluded or truncated scene regions.Unlike the cited layered representation [37] [78] [80] and pixelSplat [9], its Gaussians are not restricted to specific depth ranges.Source: §1 · p. 2

Empirical results

• Flash3D achieves state-of-the-art novel-view synthesis accuracy on RealEstate10K, NYU and KITTI (PSNR), and sometimes exceeds binocular methods in extrapolation.The reported NYU and KITTI results use the same frozen model transferred to those datasets.Source: §1 · p. 2

Efficiency

• Flash3D uses 1/64th of the GPU resources of prior works such as MINE while remaining efficient to evaluate and train.The authors connect modest training resources with opening the research area to more researchers.Source: §1 · p. 2

Monocular feed-forward reconstruction

• Monocular feed-forward scene methods use multiplane images [78] [79] [80] or neural radiance fields [89], while Flash3D uses Gaussian Splatting for scenes rather than objects.The paper reports better speed and generalization than prior monocular feed-forward reconstructors; SynSin [88] requires a rendering network to improve novel views.Source: §2 · pp. 2–3

Few-view feed-forward reconstruction

• Few-view methods such as pixelSplat [9], latentSplat [87] and MVSplat [11] reconstruct scenes from image pairs, whereas Flash3D addresses the harder monocular setting.The cited few-view approaches use cross-view attention and Gaussian mixtures to represent scene geometry.Source: §2 · p. 3

Iterative reconstruction

• Iterative reconstruction methods fit a 3D model through repeated optimization and rendering, while Flash3D produces its reconstruction in one feed-forward pass.Diffusion-based methods including ZeroNVS, ReconFusion [90] and Cat3D [17] still require per-scene optimization for consistent 3D structure.Source: §2 · p. 3

Monocular depth prediction

• Monocular depth prediction estimates metric or relative depth per pixel and generalizes across datasets; Flash3D uses UniDepth [49] in its experiments.The method is described as agnostic to which depth predictor is used.Source: §2 · p. 3

3. Method

• The method learns a network that maps an RGB scene image to a representation of the scene’s 3D geometry and photometry.The method section introduces monocular depth prediction as a prior for this reconstruction.Source: §3 · p. 3

3.1. Background: Scene reconstruction from a single image

• The scene is represented as 3D Gaussians whose mixture defines opacity and directional radiance, rendered from a camera viewpoint with differentiable Gaussian Splatting.Training uses input-image, target-image, and relative-pose triplets, minimizing the difference between rendered and target images.Source: §3.1 · p. 4

3.2. Monocular feed-forward multi-Gaussians

• The network takes an image and monocular depth map and predicts per-pixel Gaussian opacity, displacement, scale, rotation, and colour parameters.Gaussian means combine the pixel’s depth-based position with a predicted displacement; the depth comes directly from the depth predictor.Source: §3.2 · p. 4
• Flash3D predicts multiple Gaussians per pixel because scenes have no idle background pixels to repurpose for content around occlusions or beyond the field of view.The first Gaussian is tied to the predicted depth; subsequent layers are placed behind it using nonnegative depth offsets.Source: §3.2 · pp. 4–5
• The encoder pads the image and depth map so the network can predict Gaussians outside the input border, which supports synthesis when the camera retracts.The output dimensions are larger than the inputs.Source: §3.2 · p. 5

4.1. Experiment settings

• Flash3D is trained on RealEstate10k and evaluated on unseen KITTI and NYU datasets, using PSNR, SSIM, and LPIPS for quantitative comparisons.The cross-domain evaluation uses 1,079 KITTI test images and 250 NYU source images.Source: §4.1 · p. 5, §4.2 · p. 5

4.2. Cross-domain novel view synthesis

• Flash3D performs on par with KITTI-trained state of the art and achieves higher PSNR on KITTI, while significantly outperforming prior work on unseen NYU.The cross-domain table evaluates datasets not used to train Flash3D; the authors report that its model was trained only on an indoor dataset.Source: Table 1 · p. 5, §4.2 · pp. 5–6

4.3. In-domain novel view synthesis

• Flash3D achieves state-of-the-art RealEstate10k novel-view synthesis across source-target distances and sharper, more accurate reconstructions than MINE [37], despite using 1 vs. 64 GPUs.Source: Table 2 · p. 6, §4.3 · p. 7

4.2. Cross-domain novel view synthesis

• Flash3D is sharper than MINE [37] in qualitative comparisons, represents occluded regions better, and explains regions outside the source camera frustum.The figure also contrasts Flash3D with a depth predictor used alone, which cannot represent occluded regions.Source: Figure 3 · p. 6

4.4. Comparison to few-view novel view synthesis

• With one source view, Flash3D cannot outperform two-view methods on interpolation but surpasses all previous two-view methods on extrapolation.Source: §4.4 · p. 7, Table 4 · p. 7

4.5. Ablation study and analysis

• Removing the pretrained depth network significantly reduces performance and degrades wall and bed geometry; without it, two Gaussian layers perform worse than one.Source: §4.5 · p. 7, Figure 4 · p. 8
• A single depth-aligned Gaussian layer leaves holes at disocclusions, while depth-only unprojection worsens them; learned Gaussian shapes can partially mitigate holes at depth discontinuities.Source: §4.5 · p. 7, Figure 4 · p. 8
• The first layer is most opaque at boundaries and complex geometry, while the second predicts farther wall regions for large baselines; ambiguous views can still render blurry.Source: §4.5 · pp. 7–8

• Flash3D achieves state-of-the-art results for monocular scene reconstruction after 16 hours of training on a single GPU.Source: §5 · p. 8
• Using a monocular depth estimator as a foundation for full 3D scene reconstruction is the formulation the authors associate with strong generalization.Source: §5 · p. 8
• Analyses reveal interactions between the pretrained network and learned modules, while ablations verify the importance of each component.Source: §5 · p. 8

• RealEstate10k evaluation uses 3,205 frames, with target views 5 or 10 frames ahead or randomly sampled within ±30 frames of the source.The videos exceed 65,000, and COLMAP is used to recover sparse point clouds and camera trajectories.Source: §6 · p. 12
• The NYUv2 benchmark uses 80 raw sequences, three random source frames per video, and target frames sampled within ±30 frames.COLMAP recovers camera trajectories; images are undistorted and resized to 256 × 384.Source: §6 · p. 12
• KITTI evaluation uses the Tulsiani test split and crops the outer 5% of images.The network uses the dataset's metric-scale cameras and scenes without additional preprocessing.Source: §6 · p. 12

7.1. Depth unprojection

• The depth-unprojection baseline places fixed-opacity, isotropic Gaussians at predicted depths; the analysis also shows first-layer Gaussians on visible regions and second-layer or padding Gaussians in hidden or newly revealed regions.The baseline tests fixed Gaussian scale and scale proportional to depth, with hyperparameters selected by gradient-based optimization.Source: §7.1 · pp. 12–13, Figure 6 · p. 13

7.2. Splatter Image

• Splatter Image is reimplemented with the method's U-Net and ResNet-50 backbone for comparison, trained for 350,000 steps over six GPU-days.The training used two NVIDIA A6000 GPUs.Source: §7.2 · p. 13

7.3. MINE

• MINE evaluation reuses the authors' N = 64 model and reproduces its reported RealEstate10K results; NYU evaluation uses its RealEstate10k-trained model.Source: §7.3 · p. 13

7.4. Two-view methods

• Two-view comparisons select one input as the source view and evaluate at 256 × 256 without border cropping, treating source-to-target baseline as the indicative performance factor.Source: §7.4 · p. 13

7.5. Probability distribution of Gaussian

• Continuous depth offsets outperform pixelSplat-style depth probabilities for additional Gaussian layers, while the ablation table identifies its cross-domain evaluation setting.The comparison is limited to K > 1 Gaussian layers because depth probabilities without pretrained depth have slower coverage and worse monocular performance.Source: §7.5 · p. 13, Table 6 · p. 14

7.6. Off-the-Shelf Depth Models

• DepthAnything V2 and Metric3D V2 produce comparable framework results without hyperparameter adjustment; the depth-unprojection table compares fixed-size and depth-proportional Gaussian scales.Source: §7.6 · p. 13, Table 5 · p. 14

8.1. Architecture

• A shared ResNet-50 encoder feeds multiple decoders for appearance parameters and depth offsets, except the first-layer offset decoder, which is replaced by pretrained depth.Source: §8.1 · p. 14

8.2. Optimisation

• Optimization uses a weighted L1 and SSIM photometric loss, with α = 0.85, and does not use sparse depth supervision.Adam training uses batch size 16, learning rate 0.0001, and 40,000 steps.Source: §8.2 · p. 14

8.3. Scale alignment

• Depth outliers can distort scale alignment and shift rendered novel views, affecting PSNR; test-time RANSAC alignment is used for Flash3D and transfer-set MINE evaluation.Source: §8.3 · p. 14

9. Limitations

• A deterministic, regressive model is incentivized to produce blurry renderings when baselines are very large, regions are occluded, or the camera moves backward.The limitation is tied to ambiguity in these settings.Source: §9 · p. 14
• Some occluded surfaces remain missing as holes in reconstructed 3D models, even when multiple Gaussians are predicted.Many such regions are filled, but some are still missed.Source: §9 · p. 14
• Overestimated depth can cause reconstruction failures because non-negative depth offsets cannot recover structure closer to the camera than the estimated surface.This makes the method dependent on the quality of a third-party depth model within the inference domain.Source: §9 · pp. 14–15

10. Broader impacts

• Reduced compute and time for acquiring 3D assets in the wild may support house-sale visualization, cultural-heritage preservation, and safe autonomous driving.These are examples of potential positive impacts identified by the authors.Source: §10 · p. 15
• Potential misuse includes illegal or unethical tracking and surveillance, as well as privacy-invasive reconstruction of someone’s body without consent.Source: §10 · p. 15
• Incorrect 3D predictions may cause harm in autonomous driving and robotics through crashes or suboptimal performance.Source: §10 · p. 15