Source-linked AI summary
Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces
Baorui Ma, Zhizhong Han, Yu-Shen Liu, Matthias Zwicker
TL;DR
Reconstructing continuous surfaces from point clouds requires accurate SDFs, but learning them without signed-distance ground truth is challenging. Neural-Pull trains a network to differentiably pull query locations to nearby surface points using predicted signed distances and gradients. The method reports more accurate and flexible surface and single-image reconstruction than prior state-of-the-art methods, with longer training as a stated limitation.
Problem
Learning SDFs directly from raw point clouds without ground-truth signed distances remains a challenge for continuous surface reconstruction.
Method
Neural-Pull pulls sampled query locations to nearest surface points using network-predicted signed distances and gradients, updating both during differentiable training.
Results
Neural-Pull reports more accurate and flexible shape reconstruction than state-of-the-art methods for surface reconstruction and single-image reconstruction.
Takeaways & Limitations
Directly predicting signed distances strengthens 3D shape representation for surface and single-image reconstruction.
Takeaways & Limitations
The method takes more time in training than the compared method used in previous experiments.
Abstract
from arXiv · showhide
Reconstructing continuous surfaces from 3D point clouds is a fundamental operation in 3D geometry processing. Several recent state-of-the-art methods address this problem using neural networks to learn signed distance functions (SDFs). In this paper, we introduce \textit{Neural-Pull}, a new approach that is simple and leads to high quality SDFs. Specifically, we train a neural network to pull query 3D locations to their closest points on the surface using the predicted signed distance values and the gradient at the query locations, both of which are computed by the network itself. The pulling operation moves each query location with a stride given by the distance predicted by the network. Based on the sign of the distance, this may move the query location along or against the direction of the gradient of the SDF. This is a differentiable operation that allows us to update the signed distance value and the gradient simultaneously during training. Our outperforming results under widely used benchmarks demonstrate that we can learn SDFs more accurately and flexibly for surface reconstruction and single image reconstruction than the state-of-the-art methods.
1. Introduction
Neural-Pull learns signed distance functions directly from raw point clouds without ground-truth signed distances by differentiably pulling query locations onto surfaces. It improves reconstruction accuracy across surface and single-image reconstruction benchmarks.
- SDFs represent high-resolution shapes with arbitrary topology, motivating their use for 3D shape analysis.
- Existing point-cloud methods estimate unsigned distances and require extra constraints to infer signs.
- Neural-Pull learns SDFs directly from raw point clouds without ground-truth signed distance values.
- The method pulls query locations onto the surface using predicted signed distances and network gradients, updating both simultaneously.
- Neural-Pull significantly improves state-of-the-art accuracy in surface reconstruction and single-image reconstruction across widely used benchmarks.
2. Related Work
Related work learns implicit shape functions from 3D ground truth, 2D supervision, or raw point clouds. Point-cloud approaches avoid signed-distance or occupancy ground truth but rely on additional constraints for distance-field learning.
- Learning from 3D Ground Truth Globally: Global implicit-function methods learn occupancy or signed-distance values from ground-truth 3D supervision.
- Learning from 3D Ground Truth Locally: Local implicit representations improve scalability, generality, or memory efficiency by focusing on localized shape information.
- Learning from 2D Supervision: Differentiable-rendering methods learn implicit functions from multiple images as 2D supervision.
- Learning from Point Clouds: Raw point-cloud learning is more challenging without ground-truth signed-distance or occupancy values.
- Learning from Point Clouds: Point-cloud methods use geometric regularization, sign-agnostic losses, gradient constraints, or analytical gradients as additional constraints.
3. Method
Neural-Pull learns an SDF directly from raw point clouds by differentiably pulling sampled query locations to nearest surface points using predicted signed distances and gradients. Its loss, sampling strategy, and convergence analysis support learning signed rather than unsigned distance fields.
- Overview: Neural-Pull learns SDFs directly from 3D point clouds, without ground truth signed distance values, by pulling surrounding space onto the represented surface.The network can use a point cloud alone or multiple point clouds with conditions.
- Pulling Query Points: A query location is pulled to its nearest surface point using the network’s predicted signed distance and gradient, whose normalized direction determines movement along or against the gradient.The gradient is obtained from the network during back-propagation, making the operation differentiable.
- Loss Function: The training loss minimizes squared distance between each pulled query location and its nearest neighbor in the point cloud.Queries are sampled around points using isotropic Gaussians, with variance set by the squared distance to each point’s 50-th nearest neighbor.
- Convergence to SDF: Unlike unsigned distance fields, an SDF changes distance sign across the surface while retaining a consistent gradient direction near the surface.The paper states that a continuous MLP trained with the proposed loss can converge to an SDF when the local sign condition is satisfied on the surface.
- Optimization Visualization: A 2D circle visualization shows that the loss correctly pulls sampled queries onto the surface and learns distance magnitudes and signs.The authors use consistent colors to track pulled queries and visualize unsigned distances and their signs separately.
- Training: Training samples 2 × 10^4 point-cloud points, 25 queries per point, and batches of 5000 queries, using Adam for 2500 epochs with geometric network initialization.The initialization approximates the signed distance function of a sphere, with negative values inside and positive values outside.
4. Experiments and Analysis
Neural-Pull is evaluated across surface and single-image reconstruction benchmarks, where it generally outperforms prior methods and reconstructs detailed, smooth shapes. Additional analyses examine noise robustness, latent representations, convergence, and training-time trade-offs.
- Surface Reconstruction: The experiments evaluate Neural-Pull on three surface-reconstruction benchmarks using L2-CD, L1-CD, normal consistency, and F-score.The benchmarks include ABC, FAMOUS, and a ShapeNet subset.
- Surface Reconstruction: Neural-Pull significantly improves L2-CD surface-reconstruction accuracy across the FAMOUS and ABC datasets.The authors attribute this to better inference of the surface learned through the pulling process.
- Surface Reconstruction: Visual comparisons show more complete, detailed reconstructions, especially around thin structures and sharp corners, while Points2Surf better reconstructs sharp edges in some ABC examples.Neural-Pull also produces smoother planes than Points2Surf in the reported comparison.
- ShapeNet and Single Image Reconstruction: On ShapeNet, Neural-Pull achieves the best reported L1-CD performance and improves reconstruction performance across almost all single-image shape classes.The single-image evaluation also reports stronger normal consistency and EMD performance under sparse-point settings.
- Analysis: Reconstructions from real images indicate generalization from synthetic-data training, while latent-space interpolation produces novel shapes.The paper also reports smoothly converging loss curves and evidence of learning signed distance functions from noisy point clouds.
5. Conclusion
The paper introduces Neural-Pull for learning SDFs from point clouds by pulling query space onto surfaces. It reports more accurate and flexible reconstruction for 3D surfaces and single images than prior state-of-the-art methods.
- Conclusion: Neural-Pull learns signed distance functions from 3D point clouds by pulling sampled query locations onto their nearest surface points.The network uses predicted signed distances and gradients to move queries along or against the gradient.
- Conclusion: The method directly predicts signed distances without ground-truth signed distance values, improving 3D shape representation during testing.Training pulls query locations using a stride given by the predicted signed distance.
- Conclusion: Reported results show more accurate and flexible surface and single-image reconstruction than state-of-the-art methods.The conclusion summarizes the method’s outperforming performance across both applications.