MeshSplat: Generalizable Sparse-View Surface Reconstruction via Gaussian Splatting

Motivation

• MeshSplat addresses sparse-view reconstruction by learning geometric priors in a feed-forward pipeline rather than relying on insufficient per-scene geometric optimization.Sparse views provide limited multiview constraints, while geometric labeling is expensive.Source: §1 · pp. 1–2
• 2DGS serves as a bridge between novel view synthesis and surface reconstruction because its thin-surface representation supports consistent intersection planes and mesh extraction.Unlike 3DGS, 2DGS is described as naturally suited to thin surfaces and surface representation.Source: §1 · p. 2, Figure 2 · p. 2

Method

• MeshSplat predicts pixel-aligned 2DGS from sparse images, synthesizes novel views for supervision, and extracts meshes without direct 3D ground-truth supervision.The predicted Gaussian position and orientation are transformed from depth and normal maps, while other attributes come from a convolutional Gaussian head.Source: §1 · p. 2
• A Weighted Chamfer Distance Loss aligns depth-derived point clouds across views, assigning higher weights to overlapping regions identified by confidence maps.This regularizes 2DGS positions in the feed-forward setting, where depth errors directly shift thin Gaussians.Source: §1 · p. 2
• An uncertainty-based normal prediction network converts per-view normals into Gaussian orientations and uses monocular normal estimates to provide geometric supervision.Orientation errors directly affect the surface normals and can distort reconstructed surfaces.Source: §1 · p. 2

Contributions

• MeshSplat is presented as the first Gaussian-splatting method for generalizable sparse-view surface reconstruction and is reported to achieve state-of-the-art performance and cross-dataset generalization.The listed contributions emphasize learning generalizable geometric priors through 2DGS and improving position and orientation prediction.Source: §1 · p. 2

2.1. 3D Scene Representation

• Per-scene NeRF- and 3DGS-based methods lack cross-scene generalization, while sparse views provide insufficient geometric constraints for accurate surface reconstruction.Generalizable alternatives improve cross-scene prediction but remain inadequate for mesh extraction or surface reconstruction under sparse inputs.Source: §2.1 · p. 3, §2.2 · p. 3

2.2. Generalizable 3D Reconstruction

• Existing generalizable reconstruction methods either rely on inefficient NeRF rendering, predict point maps unsuitable for surface reconstruction, or generate RGB views without directly producing geometry.These limitations distinguish MeshSplat’s joint novel-view and mesh-reconstruction objective from prior generalizable frameworks.Source: §2.2 · p. 3

3.2. Cost Volume Construction

• MeshSplat extracts multi-view features with a CNN and Multi-View Transformer, constructs plane-swept cost volumes, and predicts coarse depth for pixel-aligned Gaussian positions.Feature correlations across depth candidates form cost volumes whose softmax-weighted depth estimates support back-projection into 3D positions.Source: §3.1 · p. 3, §3.2 · pp. 3–4

3.1. Overall Framework

• MeshSplat predicts 2DGS that can render novel views and reconstruct the scene mesh.Source: §3.1 · p. 3

3.3. Gaussian Prediction Network

• A Gaussian prediction network refines depth for positions, predicts normals and rotation quaternions for orientations, and decodes remaining 2DGS attributes from convolutional inputs.The normal network processes refined cost volumes, feature maps, and RGB images at multiple scales, while quaternion rotations produce Gaussian normals.Source: §3.3 · pp. 4–5

3.4. Weighted Chamfer Distance Loss

• The Weighted Chamfer Distance Loss weights cross-view point-cloud alignment by confidence maps derived from maximum cost-volume responses, emphasizing reliable matching regions.This regularization improves cost-volume generation and yields more accurate 2DGS positions while avoiding uniform penalties on occluded or unmatched pixels.Source: §3.4 · p. 5

3.5. Uncertainty-Guided NLL Normal Loss

• An uncertainty-guided normal NLL loss supervises predicted normals with monocular pseudo-ground-truth maps from Omnidata, sampling pixels according to predicted uncertainty.The normal regularization addresses orientation degradation and improves the alignment of 2DGS orientations with scene surfaces.Source: §3.5 · p. 5

3.6. Training and Inference

• MeshSplat trains with photometric, Weighted Chamfer, and normal losses, then reconstructs 2DGS during inference and follows 2DGS [17] to extract the scene mesh.The photometric term combines MSE and LPIPS losses between ground-truth and rendered RGB images.Source: §3.6 · p. 5

4.2. Main Results

• MeshSplat significantly outperforms compared methods on all reported metrics across Re10K and Scannet sparse-view surface reconstruction.The comparison transfers four generalizable novel-view-synthesis methods to mesh reconstruction and applies TSDF fusion except for SparseNeuS.Source: §4.2 · p. 6, Table 1 · p. 6
• MeshSplat reconstructs smoother and more complete meshes than baselines with holes and uneven surfaces on Re10K and Scannet.The visual comparisons show accurate reconstruction of complex scenes despite sparse-view inputs.Source: §4.2 · p. 6, Figure 4 · p. 7, Figure 5 · p. 7
• Zero-shot models trained on Re10K provide significant improvements on Scannet and Replica, demonstrating generalizability across diverse datasets.The cross-dataset evaluation uses Re10K-trained models without dataset-specific retraining.Source: §4.2 · p. 6, Table 2 · p. 6, Figure 6 · p. 8
• MeshSplat surpasses MVSplat by a large margin on all reported depth and normal prediction metrics, indicating more accurate scene-geometry recovery.The method also estimates depth and normal maps accurately while exposing low-confidence texture-less and non-overlapping regions.Source: §4.2 · p. 6, Table 3 · p. 6

4.3. Ablations

• Ablations show that 2DGS is more suitable than 3DGS for mesh reconstruction, while WCD loss and the normal prediction network improve surface fidelity and reduce geometric artifacts.The proposed components play complementary roles in improving reconstruction quality on Scannet.Source: Table 4 · p. 6, §4.3 · pp. 7–8
• Normal-map ablations show that unregularized 2DGS suffers stronger normal degradation than 3DGS, whereas normal prediction regularization makes its orientations more accurate.The comparison uses normals rendered by Gaussian Splatting to assess the necessity of the normal prediction network.Source: §4.3 · p. 8, Figure 8 · p. 8

• Weakly textured areas can produce discontinuous depth maps even when rendered RGB images remain reliable, likely because feature matching is ambiguous.Source: §5 · p. 8
• MeshSplat cannot reconstruct surfaces that are not visible from the input views; generative approaches are suggested as a possible remedy.Source: §5 · p. 9

• MeshSplat uses 2DGS in an end-to-end sparse-view reconstruction network, with WCD loss for coarse point maps and a normal prediction network for Gaussian orientations.Source: §6 · p. 9
• Experiments demonstrate that MeshSplat outperforms existing methods in generalizable sparse-view mesh extraction.Source: §6 · p. 9

• UG-NLL supervises predicted normal maps using an angular von Mises-Fisher distribution for the actual normal at each pixel.Source: §7 · p. 12
• Higher κ_i indicates lower uncertainty in a pixel's normal prediction when computing the negative log-likelihood.Source: §7 · p. 12
• UG-NLL samples the top β · N pixels with lowest κ at each scale, then adds (1−β) · N randomly selected pixels.Source: §7 · p. 12

• The experiments use Re10K, ScanNet, and Replica, with cross-dataset evaluation from Re10K to ScanNet and Replica.Source: §8 · p. 12
• Replica contributes 8 testing scenes for evaluation under the stated protocol.Source: §8 · p. 12

Network Architecture

• The backbone follows MVSplat [8], while the cost volume samples D = 128 depth candidates between the near and far planes.The normal prediction network uses 10 convolutional layers, and normal supervision is applied at 1/4, 1/2, and full resolution.Source: §9 · p. 12

Model Efficiency Analysis

• Table 5 reports rendering time and model sizes for MeshSplat and baseline methods, marking methods that render only one image per forward pass with an asterisk.The table provides the scope of the model-efficiency comparison, but the supplied passage does not include its numerical entries.Source: Table 5 · p. 12

Training Strategy

• Depth refinement uses U-Net components with self-attention and cross-attention layers, while uncertainty-guided sampling uses β = 0.7 and N = 0.4 × H × W.Training uses Adam with a maximum learning rate of 2 × 10−4 and specified loss weights.Source: §9 · p. 12

Others

• TSDF fusion uses dataset-specific camera ranges, voxel sizes, and truncation thresholds for Re10K, Scannet, and Replica.Re10K uses dnear = 1.0 and dmax = 100.0, whereas Scannet and Replica use dnear = 0.5 and dmax = 15.0.Source: §9 · p. 12

Model Efficiency Analysis

• MeshSplat renders significantly faster than NeRF-based methods, with only a few extra parameters and little extra rendering time compared with MVSplat.These comparisons are reported in the model-efficiency analysis conducted on a single NVIDIA 3090 GPU.Source: §10 · p. 12

Additional Mesh Visualization

• Figures 9, 10, and 12 show additional visualizations of reconstructed meshes, and the passage directs readers to the corresponding captions for further details.Source: §10 · p. 12

Additional Depth and Normal Visualization

• Figures 13 and 14 visualize rendered depth and normal maps, including outputs from 2DGS and predictions from the Gaussian prediction network.The visualizations are provided for the Scannet dataset and context images.Source: §10 · p. 13

11. Video Demo

• The video demo is used to further show MeshSplat’s mesh reconstruction results.The paper directs readers to the supplementary materials for details.Source: §11 · p. 13

Additional Visualizations

• Additional figures visualize extracted meshes in Re10K, Scannet, and cross-dataset generalization experiments.These visualizations are provided in Figures 9, 10, and 12.Source: Figure 9 · p. 14, Figure 10 · p. 14, Figure 12 · p. 15

Ground Truth and Rendered Maps

• The supplementary visual materials include COLMAP ground-truth dense point clouds for Re10K and rendered RGB, depth, and normal maps for Scannet.Figures 11 and 13 cover these visualizations, respectively.Source: Figure 11 · p. 15, Figure 13 · p. 16

Predictions and Failure Cases

• Figure 14 shows predicted depth and normal maps from the Gaussian prediction network, while Figure 15 presents MeshSplat failure cases.These figures provide complementary prediction and failure-case visualizations.Source: Figure 14 · p. 16, Figure 15 · p. 16