Source-linked AI summary
3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting
Xiaoyang Lyu, Yang-Tian Sun, Yi-Hua Huang, Xiuzhe Wu, Ziyi Yang, Yilun Chen, Jiangmiao Pang, Xiaojuan Qi
TL;DR
3DGS renders scenes efficiently but produces noisy, incomplete point clouds and struggles to reconstruct accurate surfaces. 3DGSR jointly aligns an implicit SDF with 3D Gaussians using opacity coupling and volumetric geometric consistency; experiments report competitive or better reconstruction and rendering than prior methods, with faster training than neural implicit methods.
Problem
Despite its efficient, high-quality novel-view synthesis, 3DGS produces noisy, incomplete point clouds and struggles to accurately reconstruct surfaces needed for geometry-related applications.
Source: §1 · p. 1Method
Results
Across multiple datasets, 3DGSR performs on par with or surpasses neural implicit reconstruction methods and outperforms GS-based methods in surface reconstruction and novel-view synthesis, while training significantly faster than neural implicit methods.
Source: §1 · p. 2Takeaways & Limitations
3DGSR produces high-quality surface reconstructions while maintaining the efficiency and rendering qualities of 3DGS.
Source: §6 · p. 9Tight coupling may slightly compromise rendering quality and produce visible reconstruction artifacts; differences in surface and color detail frequencies may contribute.
Source: §4.1 · p. 4
Abstract
from arXiv · showhide
In this paper, we present an implicit surface reconstruction method with 3D Gaussian Splatting (3DGS), namely 3DGSR, that allows for accurate 3D reconstruction with intricate details while inheriting the high efficiency and rendering quality of 3DGS. The key insight is incorporating an implicit signed distance field (SDF) within 3D Gaussians to enable them to be aligned and jointly optimized. First, we introduce a differentiable SDF-to-opacity transformation function that converts SDF values into corresponding Gaussians' opacities. This function connects the SDF and 3D Gaussians, allowing for unified optimization and enforcing surface constraints on the 3D Gaussians. During learning, optimizing the 3D Gaussians provides supervisory signals for SDF learning, enabling the reconstruction of intricate details. However, this only provides sparse supervisory signals to the SDF at locations occupied by Gaussians, which is insufficient for learning a continuous SDF. Then, to address this limitation, we incorporate volumetric rendering and align the rendered geometric attributes (depth, normal) with those derived from 3D Gaussians. This consistency regularization introduces supervisory signals to locations not covered by discrete 3D Gaussians, effectively eliminating redundant surfaces outside the Gaussian sampling range. Our extensive experimental results demonstrate that our 3DGSR method enables high-quality 3D surface reconstruction while preserving the efficiency and rendering quality of 3DGS. Besides, our method competes favorably with leading surface reconstruction techniques while offering a more efficient learning process and much better rendering qualities. The code will be available at https://github.com/CVMI-Lab/3DGSR.
1 INTRODUCTION
3DGSR integrates an implicit SDF with 3D Gaussians for joint alignment and optimization, aiming to improve surface reconstruction while retaining 3DGS rendering quality and efficiency. The introduction motivates this design by noisy, incomplete 3DGS geometry and reports favorable comparisons with neural implicit and GS-based methods.
1 INTRODUCTION
- 3DGS renders high-quality novel views efficiently, but its geometry is often a noisy, incomplete point cloud rather than an accurate surface.Source: §1 · p. 1
- Chamfer-L1 error is 0.72 for 3DGSR versus 1.40 for 2DGS, whose reconstruction is described as oversmoothed.Source: Figure 2 · p. 2
- Unstructured Gaussian centers, elliptical shapes, weak geometric constraints, and limited views make accurate surface extraction difficult and can produce noisy points.Source: §1 · p. 2
- Prior GS surface methods can smooth away detail and decouple surface reconstruction from Gaussian learning, causing alignment issues and accumulated errors.Source: §1 · p. 2
- 3DGSR jointly aligns an SDF with Gaussians, using SDF-derived opacity and loose constraints on Gaussian distribution and orientation to preserve geometry and appearance flexibility.Source: §1 · p. 2
- Because Gaussian-based supervision is sparse, volumetric depth or normal consistency regularizes SDF regions without Gaussians and in turn helps regularize the Gaussians.The method samples a few rays per iteration and uses a single-layer MLP for this rendering supervision.Source: §1 · p. 2
- Across multiple datasets, 3DGSR performs on par with or surpasses neural implicit methods and outperforms GS-based methods in surface reconstruction and novel view synthesis.The paper also reports faster learning and better novel-view synthesis than neural implicit methods.Source: §1 · p. 2
2 RELATED WORK
Related work covers neural implicit reconstruction and Gaussian-splatting approaches, contrasting their reconstruction and rendering properties with 3DGSR.
2 RELATED WORK
- The paper situates its approach among prior multi-view reconstruction work spanning neural implicit representations and 3D Gaussian splatting.Source: §2 · p. 2
- Neural implicit methods include NeRF for novel-view synthesis and SDF-based approaches such as NeuS and Neuralangelo for surfaces, but can require extensive training time.Source: §2 · p. 3
- GS surface methods use Poisson reconstruction, TSDF fusion, or ray-traced volume rendering, yet may miss fine geometry and high-quality rendering.SuGaR, 2DGS, and GOF use different extraction strategies, including mesh refinement, TSDF fusion, and marching tetrahedra.Source: §2 · p. 3
3 PRELIMINARIES
The preliminaries describe 3D Gaussians by their centers and covariances, then explain how covariance projection supports image-space splatting and attribute rendering.
3 PRELIMINARIES
- Each 3D Gaussian is characterized by a center and covariance, with covariance factorized using scale and rotation for optimization.Source: §3 · p. 3
- For novel-view rendering, Gaussians are projected onto the image plane using elliptical weighted average splatting.Source: §3 · p. 3
- Point-based alpha blending combines ordered Gaussian splats to render pixel color, with opacity modulated by the projected covariance.Source: §3 · p. 3
- The same rendering framework estimates depth from Gaussian centers and derives pseudo normals by projecting depth into 3D and fitting a local plane.Source: §3 · p. 3
4 METHOD
3DGSR embeds an implicit SDF in 3D Gaussians and jointly aligns them for surface modeling. It combines Gaussian-derived supervision with volumetric depth and normal consistency to regularize the continuous SDF.
4.1 Alignment of Surfaces and Gaussians
- Tight coupling maps SDF values to Gaussian opacity, while loose coupling places Gaussians near the SDF surface and aligns their orientations with its normals.Tight coupling can compromise rendering quality and produce artifacts; differing surface and color detail frequencies may contribute.Source: §4 · p. 3, §4.1 · pp. 4–5
4 METHOD
- Compared with previous neural implicit surface methods, 3DGSR avoids color-prediction branches and volumetric-rendering losses for optimization, making training more than 20 times faster (12 hours vs. 30 minutes).Source: §4 · p. 4
4.1 Alignment of Surfaces and Gaussians
- Loose coupling can reduce the number of Gaussian points by 20% without compromising, or even enhancing, rendering quality.The authors use loose coupling by default.Source: §4.1 · p. 5
4.2 Regularization with Volumetric Rendering
- Because Gaussian-only supervision is sparse, volumetric rendering aligns SDF-derived depth and normals with Gaussian-rendered attributes along training rays, supplying dense supervision that removes floating artifacts.The method regularizes samples along all training rays, including locations not occupied by Gaussians.Source: §4.2 · p. 5
4.3 Training Objectives and Implementation Details
- The final objective combines photometric, depth, normal, alignment, and Eikonal losses; volumetric rendering uses 1,024 rays per training step.Synthetic scenes initialize the implicit function as a sphere; real-world scenes initialize Gaussian positions from SfM points and also apply sparse SfM depth supervision.Source: §4.3 · pp. 5–6
5 EXPERIMENTS
Experiments compare 3DGSR on surface reconstruction and novel-view synthesis across synthetic and real datasets. Results favor 3DGSR overall, while ablations examine volumetric constraints, reconstruction strategy, coupling, and sparse depth supervision.
5.1 Experimental Settings
- The evaluation covers NeRF-synthetic, DTU, and Tanks & Temples, and benchmarks surface reconstruction and novel-view synthesis against neural and Gaussian-based methods.Image synthesis is measured with PSNR; reconstruction uses Chamfer distance and F1 score.Source: §5.1 · p. 6
5.2 Main Results and Comparisons
- On DTU, 3DGSR preserves fine details and intricate structures better than Relightable 3D Gaussian and 2DGS.The qualitative comparison is shown in Fig. 6.Source: Figure 6 · p. 6
- On NeRF-synthetic, 3DGSR outperforms all competitors in novel-view synthesis and surface reconstruction; on DTU it has the best average surface quality.The reported DTU comparison includes Gaussian-based and neural implicit reconstruction methods.Source: §5.2 · p. 6, Table 1 · p. 7, Table 2 · p. 7
- DTU training takes 30 minutes for 3DGSR, over 30 hours for Neuralangelo, and 18.8 minutes for 2DGS; both Gaussian methods render at 175 fps.Surface regularization is used only during training, and the extracted surface can be reused after optimization.Source: §5.2 · pp. 7–8
5.3 Ablation Studies
- Adding volumetric constraints changes Chamfer-L1 from 3.63 to 1.37, while PSNR changes from 32.83 to 33.86; the constraints provide dense depth and normal supervision for the continuous SDF.Without them, noisy structures appear in regions not occupied by Gaussians, and tight coupling alone yields a coarse result.Source: §5.3 · p. 8
- Marching cubes on the learned implicit SDF outperforms Poisson reconstruction from Gaussians with SDF-derived normals.The authors attribute this to the SDF's continuous surface representation and tolerance to noisy supervision.Source: §5.3 · p. 8
- Loose coupling improves both reconstruction and novel-view synthesis compared with tight coupling.The authors describe it as balancing the different modeling aspects.Source: §5.3 · p. 8
- Sparse depth worsens 2DGS Chamfer-L1 from 0.80 to 0.85 but improves 3DGSR from 0.81 to 0.70, especially where image supervision is sparse or camera poses are inaccurate.The comparison is reported in Table 6; the cues have less impact where observations are sufficient.Source: §5.3 · pp. 8–9, Table 6 · p. 8
6 CONCLUSION
3DGSR combines an implicit SDF aligned with 3D Gaussians and volumetric geometry-consistency supervision. Experiments report high-quality surfaces alongside preserved rendering capabilities and efficiency, with limitations on unbounded scenes, noisy-pose real data, and transparent objects.
6 CONCLUSION
- 3DGSR integrates an implicit SDF with 3D Gaussians and uses volumetric depth-normal consistency to supervise SDF regions not occupied by Gaussians.Gaussian optimization also supplies supervision for learning the SDF.Source: §6 · p. 9
- Experiments show high-quality surface reconstruction without compromising 3D Gaussians' rendering capabilities and efficiency.Source: §6 · p. 9
- The method has limited capability on unbounded scenes, worse results on real data with noisy pose estimates, and may struggle to reconstruct transparent objects.Source: §6 · p. 9