PaperlayerView exact revision on arXiv

Source-linked AI summary

TopoSurfel: Closing the Loop between Gaussian Surfels and Meshes for Surface Reconstruction

Chuanjin Fan, Wenjie Chang, Bohao Liao, Yujia Chen, Wenfei Yang, Tianzhu Zhang

arXiv:2608.20687v1cs.CVcs.GR

TL;DR

3D Gaussian Splatting makes high-quality mesh extraction difficult because its representation is discrete and lacks explicit geometric structure. TopoSurfel closes the loop between Gaussian surfels and continuous meshes with a differentiable proxy mesh that guides surfel evolution, achieving accurate, coherent reconstruction while preserving comparable novel-view rendering fidelity.

  • Problem

    3D Gaussian Splatting’s discrete, unstructured representation makes reliably producing high-quality meshes difficult, especially without explicit correspondence to continuous manifold structure.

  • Method

    TopoSurfel dynamically extracts a parameter-free differentiable proxy mesh and uses it to guide Gaussian surfel evolution through geometric consistency and scene-aware re-initialization.

  • Results

    TopoSurfel achieves accurate and coherent surface reconstruction while maintaining rendering fidelity comparable to current state-of-the-art methods.

  • Takeaways & Limitations

    A mesh-based geometric prior can improve surface quality while preserving novel-view synthesis fidelity across object-centric and large-scale scenes.

  • Takeaways & Limitations

    Related geometry-recovery pipelines may lack global topological consistency and struggle to correct errors during optimization in occluded or weak-texture regions.

Abstract

from arXiv · show

3D Gaussian Splatting has achieved remarkable success in novel view synthesis. However, extracting high-fidelity surfaces directly from 3DGS remains challenging due to its discrete and unstructured nature. Existing 3DGS-based reconstruction methods typically rely on multi-view geometric consistency or local constraints. Without an explicit structured geometric prior during optimization, these methods often struggle to resolve structural ambiguities, leading to artifacts and floaters, particularly in textureless or occluded regions. To address this limitation, we propose TopoSurfel, a novel framework that closes the loop between Gaussian surfels and continuous meshes. Unlike recent methods that incorporate mesh extraction into the differentiable pipeline by introducing auxiliary neural networks or extra per-Gaussian parameters, we dynamically extract a continuous proxy mesh via a non-trainable differentiable iso-surfacing process. Leveraging this differentiable connection, we introduce a mesh-guided surfel evolution strategy, including normal alignment and geometry-aware density control, to effectively suppress floaters and fill surface holes. Furthermore, to address the initialization challenges in large-scale environments, we propose a spatially aware hybrid re-initialization strategy that ensures robust reconstruction across complex scenes. Extensive experiments demonstrate that TopoSurfel achieves competitive geometric reconstruction accuracy while maintaining high-quality mesh-based novel view synthesis. The code for our method is available at https://github.com/Fan-Treasure/TopoSurfel.

1 Introduction

TopoSurfel addresses the difficulty of extracting high-quality meshes from unstructured 3D Gaussian representations by incorporating a differentiable proxy mesh into Gaussian surfel training. It combines mesh-guided surfel refinement with scene-aware re-initialization for stable reconstruction across object-centric and large-scale scenes.

  • Motivation: 3D Gaussian Splatting’s discrete, unstructured volumetric representation makes reliable high-quality mesh extraction difficult.This limits its direct use for high-fidelity surface reconstruction despite strong novel view synthesis performance.
  • Method: TopoSurfel incorporates a differentiable proxy mesh into Gaussian surfel training without extra learnable parameters, providing a structured surface prior.The framework dynamically connects Gaussian surfels with continuous mesh extraction and rendering.
  • Method: The framework extracts a continuous mesh using geometry-driven differentiable Poisson reconstruction and differentiable marching cubes.The resulting mesh acts as a global explicit geometric prior for surfel optimization.
  • Method: Mesh-guided surfel evolution combines normal alignment with geometry-aware density control to refine surfel structure using the extracted mesh prior.Explicit point-to-face distances suppress floaters far from surfaces and help fill locally under-covered regions.
  • Method: A stable warm-up stage and scene-aware hybrid re-initialization scheme stabilize reconstruction across object-centric and large-scale scenes.The re-initialization strategy is designed to better handle background reconstruction in large-scale environments.

2 Related Work

Related work progresses from implicit neural rendering and surface fields to explicit 3D Gaussian representations and hybrid geometric priors. However, existing Gaussian–geometry methods often couple discrete Gaussians and continuous surfaces indirectly, leaving geometry recovery decoupled from optimization and limiting topological consistency.

  • Neural Rendering and Implicit Surfaces: NeRF-based methods enable novel view synthesis through volume rendering, but their implicit density representation can produce noisy geometry.Follow-up methods improve efficiency, multi-view fusion, or robustness to unbounded scenes.
  • Neural Rendering and Implicit Surfaces: SDF- and occupancy-based methods enable high-quality surface extraction by replacing density fields while retaining volume-rendering integration.NeuS, VolSDF, IDR, and UNISURF are foundational examples, although implicit methods suffer from slow training and rendering and require complex topology extraction.
  • 3D Gaussian Splatting Reconstruction: 3DGS provides high-quality real-time rendering through an explicit point-based representation, but its unstructured nature makes accurate continuous surface extraction difficult.One research direction refines the rendering primitives to improve geometric alignment.
  • 3D Gaussian Splatting Reconstruction: Hybrid methods combine explicit Gaussians with SDF or opacity fields, but commonly treat geometry recovery as decoupled post-processing or one-way guidance.This design often lacks global topological consistency and struggles to correct geometric errors during Gaussian optimization.
  • Explicit Geometric Proxies: Differentiable mesh and isosurface methods introduce explicit geometric proxies into optimization, yet Gaussian–mesh approaches such as MILo and MeshSplatting still couple discrete Gaussians and continuous surfaces indirectly.MILo uses virtual corner subdivision and an intermediate implicit field before mesh extraction, illustrating the indirect coupling.

3 Method

TopoSurfel uses a two-stage pipeline that first stabilizes and reinitializes Gaussian surfels, then closes the loop with a differentiable proxy mesh. The mesh supplies explicit topology and geometric guidance for surfel evolution through parameter-free reconstruction and normal alignment.

  • Warm-up: TopoSurfel first performs warm-up optimization, progressively flattening volumetric Gaussians into surfel-like primitives with photometric, local geometric, and scale regularization.The shortest Gaussian axis is continuously compressed during warm-up.
  • Spatially-Aware Re-initialization: After warm-up, TSDF fusion extracts a coarse mesh and initializes one Gaussian surfel per triangle using its normal and tangent directions.The resulting surfel set discretely approximates the mesh, with flat shapes and appearance initialized from triangle colors.
  • Hybrid Re-initialization for Large-scale Scenes: For large-scale scenes, hybrid re-initialization preserves background surfels while resetting mesh-covered core regions, avoiding degradation from partial TSDF meshes.TSDF resolution and depth truncation can limit extraction to the central object.
  • Differentiable Proxy Mesh Construction: The mesh branch converts opacity-filtered surfels into a weighted oriented point cloud, generates a continuous Poisson field, and extracts explicit topology with differentiable marching cubes.The construction introduces no extra learnable parameters, and differentiability propagates gradients back to surfel parameters.
  • Explicit Mesh Extraction via DiffMC: DiffMC makes mesh vertex coordinates continuous functions of the scalar field, replacing standard marching cubes’ discrete, non-differentiable topology decisions.The output is an explicit proxy mesh M = (V, F) containing vertices and faces.
  • Mesh-Guided Normal Alignment: At densification steps, KNN nearest-surface correspondences guide normal alignment: reliable mesh normals are used near consistent faces, otherwise accumulated viewing directions determine orientation.The operation flips only the surfel normal orientation while preserving subsequent gradient-based refinement.

4 Experiments

TopoSurfel is evaluated on standard geometry and rendering benchmarks, where it achieves competitive or best reconstruction quality while preserving comparable novel-view fidelity. Ablations further show the importance of mesh-guided optimization, hybrid initialization, and a 512^3 differentiable proxy for balancing accuracy and scalability.

  • Geometry Evaluation: TopoSurfel evaluates geometry on DTU and Tanks and Temples, recovering cleaner, more complete surfaces and suppressing holes, fragmentation, and floaters in difficult regions.On DTU it achieves competitive quality and outperforms most baselines; on TNT it attains the best geometry accuracy.
  • Rendering Evaluation: TopoSurfel remains comparable to current state-of-the-art methods in Mip-NeRF 360 novel-view synthesis while improving surface quality through a mesh-based geometric prior.The evaluation uses PSNR, SSIM, and LPIPS, and the method preserves rendering fidelity despite its geometry-focused design.
  • Scalability Analysis: A 512^3 DPSR grid improves DTU Chamfer distance from 0.54 to 0.51 and TNT F1-score from 0.45 to 0.52, whereas 720^3 offers limited DTU gain and causes out-of-memory failures on three TNT scenes.The 720^3 grid improves DTU Chamfer distance only to 0.50 while substantially increasing memory and training time; three TNT scenes exceed a 24-GB GPU.
  • Ablation Study: Mesh-guided normal alignment outperforms view-based or random normal flipping, which can break proxy meshes near boundaries and under sparse viewpoints.Incorrect normal directions affect both geometry and rendering performance, while random flips further degrade results.
  • Ablation Study: Hybrid initialization preserves reconstruction stability in large unbounded scenes, whereas reinitializing only from the local TSDF mesh degrades surfel distribution, background depth, and rendering quality.The hybrid strategy combines foreground mesh surfels with pretrained background surfels after the warm-up stage.
  • Ablation Study: Geometry-aware density control reduces floaters and holes compared with gradient-only 3DGS control, producing less rough and fragmented extracted meshes near boundaries.Classical view-space-gradient control can create floaters outside objects and leave holes in occluded or sparse-view regions; Table 7 reports corresponding F1-Score and PSNR changes.

5 Conclusion

TopoSurfel bridges discrete Gaussian surfels and continuous meshes by dynamically extracting a parameter-free proxy mesh to guide surfel evolution. It achieves accurate, coherent surface reconstruction with efficient rendering, while differentiable iso-surfacing resolution remains bounded by GPU memory capacity.

  • Conclusion: TopoSurfel bridges discrete Gaussian surfels and continuous meshes through a dynamically extracted, parameter-free proxy mesh.The proxy mesh guides surfel evolution during reconstruction.
  • Conclusion: The framework achieves accurate and coherent surface reconstruction while maintaining efficient rendering.
  • Conclusion: Differentiable iso-surfacing resolution is currently bounded by GPU memory capacity, limiting the method’s scalability.The supplied passage states that this memory bound can limit the method’s ext…

Supplemental Material

The supplemental material provides implementation details, additional ablations, and additional experimental results. It is organized into implementation details and additional ablation studies.

  • The supplement includes implementation details, additional ablation studies, and additional experimental results.
  • Implementation Details: Section A covers mesh-guided normal orientation, hybrid re-initialization for large-scale scenes, the training and mesh extraction pipeline, and main hyperparameters.
  • Additional Ablations: Section B reports additional ablations on DPSR mesh extraction and comparisons between DiffMC and FlexiCubes.

A Implementation Details · A.1 Normal Orientation Pseudocode · A.2 Hybrid Re-initialization for Large-Scale Scenes

The implementation uses a two-stage normal-flip strategy combining camera-view consistency with local mesh orientation. For large-scale scenes, hybrid re-initialization applies the mesh prior only to mesh-covered surfels while preserving background surfels for continued training.

  • A.1 Normal Orientation Pseudocode: Normal orientation first accumulates visible-camera view directions to establish a provisional surfel orientation.
  • A.1 Normal Orientation Pseudocode: Surfels sufficiently close to the proxy mesh align their normals with the nearest mesh-face normal.
  • A.1 Normal Orientation Pseudocode: Surfels farther from the proxy mesh retain the view-consistent orientation, preserving each surfel’s shortest-axis geometry.
  • A.2 Hybrid Re-initialization for Large-Scale Scenes: The hybrid scheme addresses partial TSDF coverage in large-scale scenes, where depth truncation and finite resolution typically limit reconstruction to the foreground object.
  • A.2 Hybrid Re-initialization for Large-Scale Scenes: Background surfels keep their current parameters and continue training instead of being discarded by reinitialization from the partial mesh.This preserves the mesh prior on the main object while retaining the background scene representation.
  • A.2 Hybrid Re-initialization for Large-Scale Scenes: Mesh coverage is determined by nearest-surface distance, with surfel i covered when d_i < βD_scene.D_scene denotes scene scale, and β is a fixed coefficient, making the threshold proportional to scene extent.
  • A.2 Hybrid Re-initialization for Large-Scale Scenes: Mesh-covered surfels are reinitialized from the mesh using the triangle frame described above.

A.3 Pipeline Implementation · A.4 Hyperparameters

TopoSurfel uses a two-stage training pipeline that converts surfels into a differentiable proxy mesh for geometric supervision, then exports the final mesh with TSDF fusion. The implementation specifies staged optimization schedules, differentiable DPSR–DiffMC extraction, and hyperparameters controlling normal alignment and hybrid re-initialization.

  • A.3 Pipeline Implementation: The warm-up stage trains Gaussian surfels for 10,000 iterations with photometric supervision and scale regularization.Gaussian opacities are reset every 3,000 iterations; densification and pruning run every 100 iterations from iteration 500 through iteration 9,000.
  • A.3 Pipeline Implementation: Mesh-guided normal alignment uses a proxy mesh and camera-view reference directions to produce consistently oriented surfel normals.The procedure applies the sign of the dot product between each surfel normal and a reference view direction, subject to mesh-validity and distance conditions.
  • A.4 Hyperparameters: Table 8 defines the implementation hyperparameters, including shared initial Gaussian learning rates and controls for normal alignment and hybrid re-initialization.τcos and γ control angular and distance conditions for normal alignment, while β controls hybrid re-initialization.
  • A.3 Pipeline Implementation: After TSDF-based re-initialization, mesh-guided training runs for 10,000 iterations with proxy-mesh supervision activated at iteration 500.Surfel-to-mesh correspondences are updated by KNN nearest-surface search every 100 iterations, while densification and pruning stop at iteration 7,000.
  • A.3 Pipeline Implementation: At each iteration, retained surfels are converted into weighted oriented point clouds using one center point and four tangent-plane offsets.The center uses the surfel opacity as its weight, offsets use smaller weights, and all sampled points inherit the surfel normal.
  • A.3 Pipeline Implementation: DPSR constructs a continuous scalar field from normalized weighted oriented points on a voxel grid with resolution 512 during training.World-space points are normalized to [0, 1]^3, scattered by trilinear interpolation, and processed by DPSR’s frequency-domain Poisson solver.
  • A.3 Pipeline Implementation: DiffMC extracts the differentiable proxy mesh from the DPSR field using a fixed zero iso-surface level and continuous edge interpolation.The proxy mesh is rendered during training to provide geometric supervision.
  • A.3 Pipeline Implementation: The final reconstructed mesh is exported with TSDF fusion rather than using the proxy mesh output.This is more memory efficient because higher DPSR resolutions increase memory costs; training at 720^3 can exceed a 24-GB GPU budget on some scenes.

B Additional Ablations · B.1 Final Mesh Extraction with DPSR · B.2 DiffMC versus FlexiCubes

The ablations examine DPSR-based final mesh extraction and FlexiCubes-based differentiable extraction during training. DPSR can smooth fine structures at default resolution, while FlexiCubes offers greater expressiveness but incurs substantial resource costs.

  • B.1 Final Mesh Extraction with DPSR: B.1 Final Mesh Extraction with DPSR: DPSR followed by DiffMC is competitive but not ideal for final mesh output.This alternative uses the same differentiable pipeline as training.
  • B.1 Final Mesh Extraction with DPSR: B.1 Final Mesh Extraction with DPSR: At the default training resolution of 512, DPSR smooths thin structures and small surface details.The smoothing slightly reduces geometric accuracy on the Tanks and Temples benchmark.
  • B.1 Final Mesh Extraction with DPSR: B.1 Final Mesh Extraction with DPSR: The observed smoothing slightly hurts geometric accuracy on Tanks and Temples.The passage attributes this effect to DPSR-based extraction at the default training resolution.
  • B.2 DiffMC versus FlexiCubes: B.2 DiffMC versus FlexiCubes: FlexiCubes introduces additional weights and vertex offsets to make surface parameterization more flexible.Its extra expressiveness can sometimes produce cleaner topology.
  • B.2 DiffMC versus FlexiCubes: B.2 DiffMC versus FlexiCubes: FlexiCubes’ extra expressiveness increases computation and memory cost.This cost is reported in connection with comparisons from NeuManifold.
  • B.2 DiffMC versus FlexiCubes: B.2 DiffMC versus FlexiCubes: At reconstruction resolution 400, FlexiCubes is already close to the hardware limit on a 24 GB GPU.This makes scaling the extraction resolution further difficult in the reported setting.

B.3 Sampling and Weighting Strategies · B.4 Normal Estimation without Gaussian Flattening

B.3 shows that surfel sampling and weighting trade reconstruction quality against efficiency, with five-point KNN-density weighting achieving the best F1-score but higher training cost. B.4 demonstrates that removing Gaussian Flattening destabilizes normals and degrades the mesh-guided supervision loop, reducing reconstruction and rendering quality.

  • B.3 Sampling and Weighting Strategies: Single-point sampling retains only the surfel center, while opacity weighting assigns each surfel weight w_i = α_i.Opacity-area weighting instead uses w_i = α_iA_i.
  • B.3 Sampling and Weighting Strategies: Opacity-area weighting incorporates surfel area, using w_i = α_iA_i to account for each surfel’s spatial extent.The formulation is evaluated against single-point opacity weighting.
  • B.3 Sampling and Weighting Strategies: KNN-density weighting scales the default five-point weights inversely with local density, assigning α_i/ρ_i to the center and 0.5α_i/ρ_i to each offset sample.The four offset samples each receive the same density-adjusted weight.
  • B.3 Sampling and Weighting Strategies: Five-point KNN-density weighting achieves the highest F1-score of 0.53, but increases training time from 98 to 142 minutes.Single-point opacity weighting reaches 0.49, while opacity-area weighting reaches 0.51; the full model retains five-point opacity weighting for balance.
  • B.4 Normal Estimation without Gaussian Flattening: Without Gaussian Flattening Loss, covariance-eigenvector normal estimation leaves the five-point sampling unchanged but removes the constraint that Gaussians collapse toward local tangent planes.This variant follows the normal-estimation idea used in GOF and RaDe-GS.
  • B.4 Normal Estimation without Gaussian Flattening: Removing Gaussian Flattening makes normals less stable in topologically complex regions, producing a noisier and less consistently oriented proxy mesh.The degraded proxy mesh can feed incorrect geometric constraints back to surfels through the mesh-guided loop.
  • B.4 Normal Estimation without Gaussian Flattening: The degraded mesh-guided supervision lowers final reconstruction accuracy and harms rendering quality because noisy geometric guidance propagates incorrect constraints to the surfels.The mesh-guided loop depends on the proxy mesh for geometric supervision.

B.5 Stage-wise Ablation … C.1 Our Reconstructed Meshes on DTU

Stage-wise ablation shows that stable warm-up initialization and the surfel-mesh optimization loop are important for reconstruction quality, while hyperparameter sensitivity is concentrated in the mesh-guided normal alignment threshold. Qualitative results further show reconstructed meshes on DTU and geometrically consistent meshes with rendered appearance on Mip-NeRF 360 scenes.

  • B.5 Stage-wise Ablation: The warm-up stage provides stable initialization, whereas directly entering closed-loop optimization reduces F1-score from 0.52 to 0.39 while saving only five minutes.The initial TSDF mesh after warm-up reaches an F1-score of 0.28.
  • B.5 Stage-wise Ablation: Removing the surfel-mesh optimization loop reduces training time from 98 to 64 minutes but also decreases reconstruction quality.
  • B.6 Hyperparameters: The mesh-guided normal alignment threshold 𝛾 has the clearest effect among the tested hyperparameters because reducing it makes fewer surfels trust the local mesh prior.
  • B.6 Hyperparameters: Lower 𝛾 causes more surfels to rely on accumulated view directions, producing less accurate normals in difficult regions and a clear reconstruction-quality drop.
  • B.6 Hyperparameters: Reconstruction accuracy remains stable across a reasonable range of settings because densification, pruning, and opacity learning compensate for moderate changes in the hybrid re-initialization threshold 𝛽.
  • C.1 Our Reconstructed Meshes on DTU: Figure 14 provides a complete qualitative view of the meshes reconstructed by the method across DTU scenes, a controlled benchmark with accurate ground-truth geometry.
  • C Additional Results: On Mip-NeRF 360, reconstructed meshes remain geometrically consistent with rendered appearance across unbounded indoor and outdoor scenes with complex backgrounds and illumination changes.

C.3 Detailed Results on NeRF-Synthetic · C.4 Visualization of Proxy Mesh Evolution · C.5 Failure Case

The supplementary results detail NeRF-Synthetic mesh-based rendering and texture optimization, visualize proxy-mesh refinement during closed-loop training, and identify reflective metallic surfaces as a failure case. Together, these sections show progressive structural improvement while documenting a limitation of reconstruction under weak geometric cues.

  • C.3 Detailed Results on NeRF-Synthetic: The reconstructed NeRF-Synthetic mesh is unwrapped into a 2048 × 2048 texture atlas initialized from vertex colors.Only texture parameters are optimized for 3,000 iterations with photometric loss, a 0.005 learning rate, and a runtime within 3 minutes.
  • C.3 Detailed Results on NeRF-Synthetic: Table 10 reports detailed per-scene mesh-based novel view synthesis results on NeRF-Synthetic, with reconstructed meshes and post-training UV maps shown in Fig. 15.
  • C.4 Visualization of Proxy Mesh Evolution: After warm-up, the differentiable proxy mesh is initialized but still contains incomplete structures at the start of mesh-guided training.
  • C.4 Visualization of Proxy Mesh Evolution: As optimization proceeds, spurious components are suppressed, surface holes are reduced, and architectural details such as window boundaries and facade structures become more stable.
  • C.5 Failure Case: Highly reflective metallic surfaces remain challenging because view-dependent appearance and weak geometric cues interfere with surfel optimization and mesh extraction.These effects can lead to incomplete surface recovery and less stable reconstruction in such regions.
  • C.5 Failure Case: Fig. 18 provides a comprehensive visualization across multiple Mip-NeRF 360 scenes, including ground-truth images, rendered images, reconstructed meshes, surfel normals, and mesh normals.
Loading 2608.20687v1…