Source-linked AI summary
Deep Geometric Prior for Surface Reconstruction
Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, Daniele Panozzo
TL;DR
Surface reconstruction requires continuous, feature-preserving representations of noisy discrete point clouds. The paper uses untrained overfitted neural networks as local chart maps, fits them with Wasserstein distance, and enforces consistency across overlapping patches to form a manifold atlas. The resulting deep geometric prior reports strong quantitative and visual performance, while remaining substantially more expensive than alternative methods.
Problem
Surface reconstruction asks how to estimate a continuous surface from a possibly noisy point cloud while preserving critical features and approximating the sampled surface well.
Method
The method overfits deep ReLU networks to local point-cloud patches using 2-Wasserstein distance and makes overlapping local charts consistent to construct a manifold atlas.
Results
The reconstruction method produces superior quantitative and visual quality to commonly used surface-reconstruction methods and outperforms existing methods in the reported experiments.
Takeaways & Limitations
The resulting explicit parametrization can be resampled at arbitrary resolutions and used for texture mapping or computing analytic normals and curvature values.
Takeaways & Limitations
The method is currently substantially more expensive than alternative approaches.
Abstract
from arXiv · showhide
The reconstruction of a discrete surface from a point cloud is a fundamental geometry processing problem that has been studied for decades, with many methods developed. We propose the use of a deep neural network as a geometric prior for surface reconstruction. Specifically, we overfit a neural network representing a local chart parameterization to part of an input point cloud using the Wasserstein distance as a measure of approximation. By jointly fitting many such networks to overlapping parts of the point cloud, while enforcing a consistency condition, we compute a manifold atlas. By sampling this atlas, we can produce a dense reconstruction of the surface approximating the input cloud. The entire procedure does not require any training data or explicit regularization, yet, we show that it is able to perform remarkably well: not introducing typical overfitting artifacts, and approximating sharp features closely at the same time. We experimentally show that this geometric prior produces good results for both man-made objects containing sharp features and smoother organic objects, as well as noisy inputs. We compare our method with a number of well-known reconstruction methods on a standard surface reconstruction benchmark.
1. Introduction
Surface reconstruction seeks a continuous surface representation from discrete, possibly noisy point clouds while preserving critical features and fitting the samples well. The proposed method builds a consistent manifold atlas with overfitted neural-network charts and reports superior quantitative and visual quality against established methods.
- Surface reconstruction must represent discrete point clouds continuously while remaining robust to noise and preserving critical surface features.
- Existing approaches include implicit scalar functions, globally consistent meshes, unions of primitive shapes, and fitted surface patches.
- The method constructs an atlas of consistent 2D parametrizations by overfitting neural networks to local point-cloud regions.
- Wasserstein distance supplies a robust training loss and correspondences for minimizing disagreement between neighboring patches.
- The method is compared with 12 competing methods on a standard shape-reconstruction benchmark and produces superior quantitative and visual quality.
2. Related work
Related work spans geometric reconstruction, learning-based point-cloud processing, deep image priors, and manifold-based parameterizations. The proposed approach differs by using untrained networks to fit local chart maps from parametric domains to surface points.
- Most geometric deep-learning architectures process point clouds for classification or segmentation, while PU-Net directly addresses point-cloud upsampling.
- The method's untrained networks take square-domain parametric coordinates as inputs and produce surface points as outputs.
- Geometric reconstruction methods include indicator functions, point-set surfaces, partition-of-unity techniques, and scattered point meshing.
- EAR computes normals away from singularities and resamples toward them using a bilateral filter before reconstruction.
- Manifold-based prior work constructs global parameterizations from range images, whereas this method computes chart maps by fitting.
- Deep image priors show that untrained networks can be overfitted to data for high-quality upsampling and hole filling without training data.
3. Method
The method fits overparameterized neural networks as local surface charts, then jointly optimizes overlapping charts to form a consistent manifold atlas. It uses Wasserstein-based fitting and an implicit optimization bias to reconstruct surfaces without explicit regularization.
- Local parametrization: The method represents each local surface neighborhood with a neural-network chart mapping a square parametric domain into R3.A sufficiently small neighborhood is characterized through a differentiable map, which is approximated by an MLP.
- Local parametrization: Local charts are fitted by matching parametrically sampled outputs to neighborhood point clouds using the Earth Mover’s or 2-Wasserstein distance.The exact assignment formulation costs O(n3), so the method uses a Sinkhorn-regularized distance that can be computed in near-linear time.
- Local parametrization: The MLP uses half-rectified activations and is overparameterized so its trainable parameter count T is much larger than the 3n fitting constraints.This produces a piece-wise linear approximation and supports gradient-descent fitting to zero loss under the stated overparameterized conditions.
- Optimization behavior: Gradient descent empirically favors low-complexity, non-oscillatory solutions despite the model’s capacity to realize arbitrary point correspondences as λ approaches infinity.The authors identify this as an implicit regularization effect and note that large λ may add regularization in practice.
- Building a Global Atlas: Overlapping charts are fitted in two phases: independent local fitting followed by joint optimization enforcing agreement at corresponding overlap samples.The consistency term supports smooth transitions in noisy point clouds without requiring exact interpolation through every noisy sample.
- Building a Global Atlas: An atlas is built by Poisson-disk sampling patch centers, extracting local neighborhoods, and filtering points likely to belong to different surface sheets.The neighborhoods use ball intersections, larger fitting radii to reduce boundary effects, and a normal-based heuristic when needed.
4. Experiments
Experiments evaluate the method from local patch fitting through globally consistent surface reconstruction, using benchmark metrics and comparisons against competing methods. The results show tight local fits, improved patch consistency, strong benchmark performance, and favorable behavior on noisy inputs and sharp features.
- Single Patch Fitting: A single overfit patch tightly fits a complex point cloud while providing an explicit local surface parametrization.The parametrization supports computing curvature and normals, and applying textures to the surface.
- Single Patch Fitting: Gradient-descent optimization follows a trajectory in which the fitted parametrization shows no apparent distortions, supporting a low-complexity bias.The observation is presented as evidence for implicit regularization through optimization.
- Global Consistency: Consistency terms significantly improve agreement among overlapping patches during joint optimization.The pipeline first fits charts locally, then jointly fits overlapping chart pairs.
- Surface Reconstruction Benchmark: The benchmark evaluates reconstruction precision with drec→GT and recall with dinp→rec using dense ground-truth and reconstructed point clouds.The precision measure does not penalize failure to cover undersampled input regions, while copying noisy input points gives dinp→rec ≡ 0.
- Surface Reconstruction Benchmark: The method outperforms all tested techniques on the benchmark and is on par with the state-of-the-art EAR method for the five evaluated models.The comparison uses a standard benchmark setup and quantitative and visual reconstruction quality.
- Noise and Sharp Features: On noisy inputs, the method smooths noise while preserving sharp features, whereas EAR interpolates noise and produces spurious points and visible artifacts.The broader sharp-feature experiment contrasts Screened Poisson Surface Reconstruction, which is robust to noise but misses sharp features, with EAR's opposite behavior.
5. Discussion
The paper frames surface reconstruction as a low-dimensional interpolation problem and introduces a deep geometric prior based on local ReLU charts and Wasserstein consistency. The approach is effective but substantially more expensive than alternatives.
- The authors study neural networks’ capacity to reconstruct complex surface structures while remaining robust to noise.
- The method constructs local piece-wise linear charts with a vanilla ReLU network and enforces consistent transitions using Wasserstein distances.
- The resulting deep geometric prior outperforms existing surface-reconstruction methods that rely on domain-specific geometric assumptions.
- The approach is currently substantially more expensive than alternative methods.
A.1. Effect of the parameter λ
The reconstruction quality depends on the Sinkhorn regularization parameter λ: smaller values provide a better Wasserstein approximation and improve reconstruction of the original points.
- Smaller λ values yield better Wasserstein-distance approximations and produce better reconstructions of the original points.
A.2. Kinect reconstruction
The method reconstructs a raw Kinect V2 point cloud despite substantial noise and highly non-uniform sampling, producing a smooth approximation of the object’s geometry.
- The technique reconstructs point clouds with large quantities of noise and highly non-uniform sampling from a raw Kinect V2 scan.
- The resulting reconstruction is smooth and approximates the geometry of the original object despite the challenging input.
A.3. Surface Reconstruction Benchmark
The paper evaluates its method on all five models in a standard Surface Reconstruction Benchmark using cumulative error distributions for reconstructed and input-to-reconstruction distances.
- The benchmark reports cumulative histograms for results on all 5 models shown in Figure 4.
- Figures 15 and 16 show the percentages of vertices whose drec→GT and dinp→rec errors fall below specified thresholds.
A.4. Surface Reconstruction Benchmark Statistics
The benchmark reports two one-sided reconstruction distances across competing methods, using cumulative error histograms to assess fitted-surface quality from complementary directions.
- The benchmark compares drec→GT and dinp→rec across multiple reconstruction methods and models.Table 2 reports distance from reconstruction to input, while Table 1 reports distance from input to reconstruction.
- The plotted methods include Apss, EAR, Fourier, IMLS, MPU, MPUsmooth, Poisson, Screen Poisson, RBF, Scattered, SPSS, Wavelet, and Our.The method labels are repeated across the benchmark visualizations.
- Figure 15 plots the percentage of fitted vertices reaching each error level against ground-truth distance.The y-axis uses a log scale, and the x-axis represents the error level.