Source-linked AI summary
Points2Surf: Learning Implicit Surfaces from Point Cloud Patches
Philipp Erler, Paul Guerrero, Stefan Ohrhallinger, Michael Wimmer, Niloy J. Mitra
TL;DR
Surface reconstruction from point clouds remains difficult because classical methods degrade on noisy or partial scans, while data-driven methods struggle with geometric and topological variation. Points2Surf learns implicit surfaces from raw scans using coarse global sign information and detailed local patch distances. On unseen classes, it reports lower reconstruction error than SPR and learning-based baselines, with longer computation and some small-scale topological noise.
Problem
Classical reconstruction degrades on noisy or partial scans, while data-driven alternatives struggle to generalize to new shapes with substantial geometric and topological variation.
Method
Points2Surf learns implicit surfaces directly from raw point clouds by combining a global inside/outside function with a patch-based local absolute-distance function.
Results
On unseen classes, Points2Surf reduced reconstruction error by 30% over SPR, 470% over DeepSDF, and 270% over AtlasNet.
Takeaways & Limitations
The method captures geometric and topological details and generalizes to unseen shapes more robustly than current methods.
Takeaways & Limitations
Distance-dependent global subsampling can make neighboring patch outputs inconsistent, producing bumpy surfaces; multi-scale reconstruction is proposed to reduce this.
Abstract
from arXiv · showhide
A key step in any scanning-based asset creation workflow is to convert unordered point clouds to a surface. Classical methods (e.g., Poisson reconstruction) start to degrade in the presence of noisy and partial scans. Hence, deep learning based methods have recently been proposed to produce complete surfaces, even from partial scans. However, such data-driven methods struggle to generalize to new shapes with large geometric and topological variations. We present Points2Surf, a novel patch-based learning framework that produces accurate surfaces directly from raw scans without normals. Learning a prior over a combination of detailed local patches and coarse global information improves generalization performance and reconstruction accuracy. Our extensive comparison on both synthetic and real data demonstrates a clear advantage of our method over state-of-the-art alternatives on previously unseen classes (on average, Points2Surf brings down reconstruction error by 30% over SPR and by 270%+ over deep learning based SotA methods) at the cost of longer computation times and a slight increase in small-scale topological noise in some cases. Our source code, pre-trained model, and dataset are available on: https://github.com/ErlerPhilipp/points2surf
1 Introduction
Surface reconstruction from raw scans must handle noise, uneven sampling, partial observations, varying topology, and broad shape variation. Points2Surf addresses these challenges with separate global and local learned functions, improving reconstruction on unseen classes.
- Scanning workflows need surfacing algorithms that tolerate noisy, unevenly sampled point clouds, varied topologies, and diverse scanned shapes.
- SPR can produce overly smooth surfaces, incorrectly close holes and tunnels, and degrade when point-normal estimates are erroneous.
- Category-specific data-driven methods handle noisy and partial scans but generally fail to generalize across new surfaces with varying shape and topology.
- Points2Surf learns implicit surfaces directly from raw point clouds and handles fine-scale features, varying topologies, and new objects.
- Points2Surf decomposes signed-distance learning into a coarse global inside/outside function and a detailed local patch-based absolute-distance function.
- 30% lower reconstruction error than SPR and 470% lower than DeepSDF and 270% lower than AtlasNet were reported on unseen classes.
2 Related Work
Prior surface-reconstruction methods either avoid learned priors and struggle with noisy or partial inputs, or use global learned representations that improve robustness while limiting generality and accuracy. Points2Surf instead separates coarse global sign information from detailed local distance information.
- Surface-reconstruction methods are broadly divided into data-driven approaches that learn priors and methods that do not use data-driven priors.
- Non-data-driven methods may struggle with noisy point clouds, preserve unsuitable connectivity, or fail on partial inputs depending on their construction.
- Data-driven methods learn priors from large datasets, often encoding a surface as a single latent feature vector inferred from a point cloud.
- A single latent vector improves robustness to noise and missing parts but limits the variety and generality of representable shapes.
- Points2Surf separates coarse global sign information from detailed local absolute distance, using separate feature vectors to retain geometry and global shape information.
3 Method
Points2Surf represents surfaces with an implicit SDF learned from point clouds, factorizing estimation into local absolute distance and global sign information. Its architecture combines local and global encoders, a shared decoder, and Marching Cubes reconstruction.
- SDF representation: Points2Surf approximates an SDF from a point cloud using a latent encoding and a neural decoder.The implicit representation is continuous, typically produces watertight meshes, and can encode arbitrary topology.
- Local and global factorization: The method estimates absolute distance from a local neighborhood, which improves local accuracy and robustness for noisy or sparsely sampled scans.Larger neighborhoods are used because the closest point alone is unreliable under noise and sparse sampling.
- Local and global factorization: The method estimates inside/outside sign from a global point-cloud subsample because local patches cannot reliably determine a watertight surface’s interior and exterior.The global task is coarse and can generalize across significant shape variations.
- Architecture and sampling: The architecture encodes a query-centered local patch and a coarse global subsample into feature vectors that a decoder maps to distance and sign outputs.The local patch uses nearest neighbors, while the global subsample uses a distance-dependent density gradient with clamping; the experiments choose nd = 300 and ns = 1000.
- Surface extraction: Marching Cubes converts samples of the estimated SDF into the reconstructed surface.The method can reduce computation by evaluating only a subset of grid samples.
- Architecture and sampling: Both point subsets are normalized around the query point, and a QSTN predicts a shared rotation from the global subset without direct rotation supervision.The estimated distance is rescaled to the original size after inference.
- Architecture and training: Training jointly regresses absolute distance and classifies the sign using an L2-based distance loss and binary cross entropy.The tanh transform emphasizes smaller distances, while sign classification lets logit magnitude express confidence.
4 Results
Points2Surf is evaluated against classical and learning-based baselines on synthetic and real scans, including unseen object classes and datasets. Results show stronger generalization and detail preservation, with longer reconstruction times and some increased topological noise.
- Evaluation setup: The evaluation compares Points2Surf with SPR, DeepSDF, and AtlasNet using qualitative and quantitative tests, ablations, and timing measurements.The comparisons cover several datasets, including ABC, Famous, Thingi10k, and real-world scans.
- Generalization: Points2Surf generalizes better than DeepSDF and AtlasNet from CAD models to more organic shapes.Their reconstruction errors increase more when moving from ABC to Famous than the error for Points2Surf.
- Geometric and topological quality: Points2Surf better preserves holes, concavities, and fine details than competing methods on noisy and real-world scans.Global information helps recover small holes and large concavities, while the local prior preserves details that SPR tends to smooth away.
- Real-world data: On real-world data, Points2Surf preserves holes and details better than SPR, at the cost of a slight increase in topological noise.DeepSDF and AtlasNet produce inaccurate reconstructions on unseen categories, while SPR tends to over-smooth shapes and close holes.
- Timing: A smaller model reduces reconstruction time by roughly 68% but increases ABC var-noise Chamfer distance by roughly 55%.The reported time decreases from 11.5 to 3.7 minutes per mesh, while Chamfer distance rises from 150.6 to 234.3.
5 Conclusion
Points2Surf reconstructs raw point-cloud surfaces while capturing geometric and topological details and generalizing robustly to unseen shapes. Its patch-based outputs can exhibit bumpiness, motivating multi-scale reconstruction as a future direction.
- Points2Surf reconstructs surfaces directly from raw point clouds while capturing geometric and topological details.
- The method generalizes to unseen shapes more robustly than current methods.
- Distance-dependent global sub-sampling can make neighboring patch outputs inconsistent, producing bumpy surfaces.
- Multi-scale reconstruction is proposed to reduce bumpiness by letting coarser levels provide consistency for finer levels.The authors also expect this direction to reduce computation times.