Source-linked AI summary
High-quality Surface Reconstruction using Gaussian Surfels
Pinxuan Dai, Jiamin Xu, Wenxiang Xie, Xinguo Liu, Huamin Wang, Weiwei Xu
TL;DR
High-quality geometric reconstruction remains difficult for 3D Gaussian representations because thickness, normal ambiguity, and alpha-blending artifacts can misalign reconstructed surfaces. The paper introduces Gaussian surfels with consistency guidance and volumetric cutting, and reports a balance of reconstruction quality and computational cost, while noting limitations under strong specular reflections and weak textures.
Problem
3D Gaussian representations struggle with high-quality surface reconstruction because non-zero thickness, normal ambiguity, and alpha-blending artifacts hinder accurate geometric alignment.
Method
The method flattens 3D Gaussian points into Gaussian surfels, uses normal-depth consistency and monocular normal priors, cuts erroneous volumetric regions, and extracts meshes with Poisson reconstruction.
Results
The method achieves a balance between high-quality surface reconstruction and computational cost, with rapid convergence to high-quality reconstructions and reconstruction speed comparable to INSR and NeuS2.
Takeaways & Limitations
Gaussian surfels provide a point-based representation that closely aligns with surfaces while retaining efficient optimization for detailed reconstruction, including open surfaces.
Takeaways & Limitations
Strong specular reflections can remain inaccurately reconstructed, and very weak textures may produce a global shift relative to ground-truth surfaces.
Abstract
from arXiv · showhide
We propose a novel point-based representation, Gaussian surfels, to combine the advantages of the flexible optimization procedure in 3D Gaussian points and the surface alignment property of surfels. This is achieved by directly setting the z-scale of 3D Gaussian points to 0, effectively flattening the original 3D ellipsoid into a 2D ellipse. Such a design provides clear guidance to the optimizer. By treating the local z-axis as the normal direction, it greatly improves optimization stability and surface alignment. While the derivatives to the local z-axis computed from the covariance matrix are zero in this setting, we design a self-supervised normal-depth consistency loss to remedy this issue. Monocular normal priors and foreground masks are incorporated to enhance the quality of the reconstruction, mitigating issues related to highlights and background. We propose a volumetric cutting method to aggregate the information of Gaussian surfels so as to remove erroneous points in depth maps generated by alpha blending. Finally, we apply screened Poisson reconstruction method to the fused depth maps to extract the surface mesh. Experimental results show that our method demonstrates superior performance in surface reconstruction compared to state-of-the-art neural volume rendering and point-based rendering methods.
1 INTRODUCTION
The paper identifies geometric weaknesses in 3D Gaussian Splatting and introduces Gaussian surfels, losses, and volumetric cutting to improve surface alignment and reconstruction quality.
- Motivation: 3D Gaussian Splatting struggles with geometric reconstruction because its primitives have non-zero thickness, ambiguous normal directions, and edge artifacts from alpha blending.These issues hinder alignment with actual surfaces and can reduce accuracy on fine details.
- Gaussian surfels: Gaussian surfels flatten each 3D Gaussian into a 2D ellipse by setting its z-scale to 0, making the local z-axis the surface normal.This avoids selecting a changing minimal scale and provides clearer optimization guidance.
- Optimization: Because covariance derivatives along the local z-axis become zero, a self-supervised normal-depth consistency loss guides that axis toward normals computed from rendered depth.The loss compensates for the inability of photometric loss alone to update the local z-axis.
- Geometric refinement: A volumetric cutting method removes voxels according to their distance from Gaussian surfels to reduce blurred edges and floating geometries caused by depth-rendering artifacts.The method addresses errors that remain when rays interact with front surfaces near depth discontinuities.
- Pipeline: The pipeline jointly optimizes Gaussian surfels with photometric, depth-normal consistency, and normal-prior losses before cutting rendered depth maps and extracting a mesh with Poisson meshing.The contributions also include monocular normal priors for regions affected by specular reflections.
2 RELATED WORK
Prior work spans neural volume rendering, implicit and hybrid representations, and point-based rendering. This paper instead uses anisotropic Gaussian surfels with opacity to prioritize fast surface reconstruction while preserving fine details.
- Classical reconstruction: Multi-view stereo methods reconstruct geometry using voxel-grid optimization, feature-point growing, or depth-map estimation and merging.These categories form the classical background for multi-view surface reconstruction.
- Neural volume rendering: Neural rendering methods obtain detailed reconstructions through differentiable image rendering and can model complex materials with sophisticated rendering processes.NeRF and later explicit or hybrid representations pursue photorealistic rendering and faster optimization.
- Implicit surfaces: NeRF variants using density thresholding can produce high-frequency surface noise, motivating occupancy grids and signed-distance-function representations.Voxel grids and hash encodings were also introduced to accelerate implicit surface reconstruction.
- Point-based rendering: Point-based rendering represents geometry with unstructured samples, while surface splatting uses extended primitives such as ellipsoids or circular and elliptic surfels.These representations address holes and discontinuities associated with directly rendering point samples.
- Gaussian representations: Recent methods approximate volume rendering with alpha-blended splats, including isotropic spheres, 2D Gaussian discs, and 3D Gaussian Splatting.3DGS provides fast reconstruction and real-time rendering, while the present method emphasizes surface alignment.
- Positioning: Unlike 3DGS and SuGaR, which emphasize realistic rendering, the proposed method targets super-fast surface reconstruction while preserving fine details.It uses Gaussian surfels with opacity and anisotropic covariance for better surface alignment.
3 METHOD
The method represents surfaces with Gaussian surfels, optimizes them using photometric and geometry-aware losses, and reconstructs a mesh from fused depth and normal maps. It addresses depth-rendering errors near discontinuities with volumetric cutting before screened Poisson meshing.
- Gaussian Surfels: Gaussian surfels represent each primitive as an anisotropic ellipse with opacity and view-dependent appearance, combining 3DGS optimization with surfel-like surface alignment.The representation uses Gaussian kernels with centers, rotations, two-axis scales, opacities, and spherical-harmonic coefficients.
- Gaussian Surfels: Setting the third scale to zero flattens each 3D Gaussian into a 2D ellipse, while its local z-axis directly defines the surfel normal.The covariance is formed from the scaling and rotation matrices, and the normal is computed from the third rotation-matrix column.
- Optimization: Depth is rendered by intersecting each pixel ray with the Gaussian ellipse rather than blending the center depth, improving consistency for surfels on slanted planes.Depth and normals are produced through Gaussian splatting and alpha blending, with blending weights normalized for depth and normal rendering.
- Optimization: The method optimizes Gaussian surfels with photometric, monocular-normal, opacity, mask, and depth-normal consistency losses.The total loss combines Lp, Ln, Lo, Lc, and Lm with trade-off weights for opacity, consistency, and masking.
- Optimization: The consistency loss links rendered depth and normals bidirectionally, correcting ellipse orientation when depth is reliable and smoothing depth when normals are reliable.The rendered depth is converted into 3D points, whose neighboring-point normals are compared with rendered normals.
- Gaussian Point Cutting and Meshing: Rendered depth errors near discontinuities are reduced by volumetric cutting, after which fused depth maps and normals are meshed using screened Poisson reconstruction.Alpha blending can let foreground Gaussians influence background depth; the supplied method section identifies this as a remaining source of erroneous depth.
4 EXPERIMENTS
Experiments evaluate Gaussian surfels for geometry and rendering on DTU, BlendedMVS, DeepBlending, and MipNeRF360, including sparse-input comparisons and loss ablations. The method outperforms point-based baselines in surface reconstruction, improves sparse rendering over 3DGS, and balances quality with computational cost.
- Geometry comparisons: Gaussian surfels significantly outperform 3DGS and SuGaR on Chamfer distance across DTU and BlendedMVS.Compared with NeuS2, they obtain a larger Chamfer distance on DTU but often produce less noise than NeuS2 and more details than NeuS.
- Efficiency and quality: The method converges rapidly to high-quality reconstructions, with reconstruction speed comparable to INSR and NeuS2.Its performance is contrasted with NeuS, whose large MLP can over-smooth reconstructed surfaces.
- Geometry comparisons: Compared with 3DGS and SuGaR, Gaussian surfels reconstruct noise-free surfaces and capture intricate details more effectively.SuGaR still exhibits ellipsoid-like artifacts and holes on surfaces.
- Rendering comparisons: Gaussian surfels exceed NeuS2 in rendering quality under both common and sparse DTU settings.In sparse inputs, Gaussian surfels and NeuS2 avoid floaters, while Gaussian surfels recover better visual details than NeuS2.
- Rendering comparisons: In the common setting, Gaussian surfels perform slightly worse than vanilla 3DGS, but their precise geometry improves rendering generality in sparse settings.The sparse-setting improvement is illustrated in Figure 7.
- Ablations: Removing depth-normal consistency significantly reduces reconstruction quality, while normal-prior, opacity, mask, and volumetric-cutting losses improve specific reconstruction properties.The normal prior reduces highlighted-area fluctuations, opacity sharpens LEGO studs, and mask loss plus volumetric cutting reduce noise; removing the normal prior can slightly improve rendering quality.
5 CONCLUSION
The paper concludes that Gaussian surfels can be efficiently optimized to balance high-quality surface reconstruction with computational cost. It also identifies unresolved difficulties with strong specular reflections and very weak textures.
- Conclusion: Gaussian surfels achieve a balance between high-quality surface reconstruction and computational cost.The conclusion describes the representation as efficiently optimized.
- Limitations: Strong specular reflections can still prevent accurate reconstruction despite monocular normal priors.The authors propose view-dependent feature modeling as future work for handling specular reflections.
- Limitations: Very weak textures may produce a global shift relative to ground-truth surfaces.The authors suggest depth sensors or additional shape priors as possible mitigation.