Source-linked AI summary
Differentiable Manifold Reconstruction for Point Cloud Denoising
Shitong Luo, Wei Hu
TL;DR
Noisy point clouds can distort underlying structures, while prior methods mainly predict point displacements rather than explicitly recovering the surface. This paper learns a manifold from adaptively subsampled low-noise points and neighborhood features, then resamples reconstructed local surfaces to denoise the cloud. Experiments report superiority over state-of-the-art methods under synthetic and real-world noise, with especially strong results at high noise levels.
Problem
Point-cloud noise distorts underlying structures, while previous denoisers mainly infer point displacements instead of explicitly recovering the underlying surface.
Method
An autoencoder-like network adaptively pools lower-noise points, reconstructs local patch manifolds from neighborhood features, and resamples them into a denoised point cloud.
Results
The method significantly outperforms state-of-the-art denoising methods under synthetic and real-world noise, especially at high noise levels.
Takeaways & Limitations
The network captures intrinsic surface structure and supports either supervised or unsupervised end-to-end training.
Takeaways & Limitations
The patch-manifold output has no natural one-to-one correspondence with the input points, requiring a constructed correspondence for unsupervised training.
Abstract
from arXiv · showhide
3D point clouds are often perturbed by noise due to the inherent limitation of acquisition equipments, which obstructs downstream tasks such as surface reconstruction, rendering and so on. Previous works mostly infer the displacement of noisy points from the underlying surface, which however are not designated to recover the surface explicitly and may lead to sub-optimal denoising results. To this end, we propose to learn the underlying manifold of a noisy point cloud from differentiably subsampled points with trivial noise perturbation and their embedded neighborhood feature, aiming to capture intrinsic structures in point clouds. Specifically, we present an autoencoder-like neural network. The encoder learns both local and non-local feature representations of each point, and then samples points with low noise via an adaptive differentiable pooling operation. Afterwards, the decoder infers the underlying manifold by transforming each sampled point along with the embedded feature of its neighborhood to a local surface centered around the point. By resampling on the reconstructed manifold, we obtain a denoised point cloud. Further, we design an unsupervised training loss, so that our network can be trained in either an unsupervised or supervised fashion. Experiments show that our method significantly outperforms state-of-the-art denoising methods under both synthetic noise and real world noise. The code and data are available at https://github.com/luost26/DMRDenoise
1 INTRODUCTION
Point-cloud noise distorts underlying structures and challenges downstream 3D vision tasks. The paper addresses this by explicitly reconstructing the underlying manifold from low-noise sampled points and neighborhood features.
- Motivation: Noise in acquired point clouds deforms underlying structures and affects downstream understanding tasks.The paper highlights irregular and unordered point-cloud characteristics as an additional denoising challenge.
- Motivation: Existing deep-learning denoisers generally infer each noisy point’s displacement from the underlying surface rather than recovering the surface explicitly.
- Approach: The proposed paradigm explicitly learns the underlying manifold from differentiably subsampled points and embedded neighborhood features.
- Approach: An autoencoder-like network extracts local and non-local features, adaptively pools lower-noise points, and reconstructs local patch manifolds.
- Training: The method uses an unsupervised training loss, allowing either unsupervised or supervised training.
- Results: Experiments report significant superiority over state-of-the-art denoising methods, especially at high noise levels.
2 RELATED WORK
Prior point-cloud denoising methods include surface-fitting, sparsity-based, graph-based, and neural approaches. Their reported weaknesses include outlier sensitivity, geometric shrinking, and degradation under high noise.
- Non-deep-learning methods: Local-surface-fitting methods project noisy points onto fitted smooth surfaces but are often sensitive to outliers.
- Non-deep-learning methods: Sparsity-based methods reconstruct normals through sparse regularization, but performance can degrade at high noise through over-smoothing or over-sharpening.
- Non-deep-learning methods: Graph-based methods represent point clouds as graph signals and denoise them with graph-filtering or graph-regularized optimization.
- Deep-learning methods: Deep-learning denoisers generally predict and apply per-point displacement from noisy points toward the clean surface.
- Deep-learning methods: Neural Projection requires clean point clouds and point normals during training, whereas PointCleanNet avoids normals but is sensitive to outliers and point-cloud shrinking.
- Deep-learning methods: Total Denoising introduced an unsupervised deep-learning approach using denser surroundings as a prior, but remains sensitive to outliers and shrinking.
- Point-cloud consolidation: Point-cloud consolidation methods such as PointProNet, EC-Net, and 3PU are generally prone to failure when noise levels are high.
3 METHOD
The method section presents a framework for learning the underlying manifold of noisy point clouds. It proceeds from key ideas to differentiable reconstruction and then to training losses and further analysis.
- Overview: The method learns the underlying manifold for point-cloud denoising.
- Overview: The section first introduces the key ideas before detailing differentiable manifold reconstruction.
- Overview: It concludes by presenting loss functions and further analysis of the method.
3.1 Overview
The overview describes an autoencoder-like denoising network that maps a noisy point cloud to a clean point cloud. Its encoder extracts local and non-local geometry and differentiably downsamples points with less noise.
- Network overview: Given a noisy point cloud P ∈ R^N×3, the network produces a clean point cloud P̂ ∈ R^N×3.
- Representation Encoder: The representation encoder combines feature extraction with a differentiable pooling unit.
- Representation Encoder: The feature extractor encodes both local and non-local geometry at each input point.
- Representation Encoder: The pooling operator identifies and samples points that are closer to the underlying surface.
3.2 Representation Encoder with Differentiable Pooling
The representation encoder extracts multi-scale local and non-local point features, then uses adaptive differentiable pooling to retain points likely closer to the underlying surface. A differentiable gate and pre-filtering prepare the retained subset for manifold reconstruction.
- Feature Extraction Unit: The feature extraction unit stacks dynamic graph convolution layers with dense connections to encode local, non-local, and multi-scale geometry.Parallel units use different k-NN values, and their outputs are concatenated into the final feature matrix.
- Differentiable Pooling Operator: The pooling operator computes an MLP score for each point and retains the top-M points, favoring points closer to the underlying surface.The score function is trained end-to-end to assign higher scores to less-perturbed points.
- Differentiable Pooling Operator: A feature gate multiplies retained point features by sigmoid-transformed scores, making the selection operation differentiable for backpropagation.The gated feature matrix Y is formed from the retained features X[i] and their score vector s[i].
- Differentiable Pooling Operator: Pre-filtering further reduces noise variance in the sampled point set before decoding.The pre-filtering displacement ΔS moves sampled points closer to the underlying surface.
- Architecture: The implementation uses standard neural-network building blocks, including addition, concatenation, linear or identity mappings, MLPs, and matrix multiplication.These operations are shown as architectural components in the framework illustration.
3.3 Manifold Reconstruction Decoder
The decoder reconstructs local patch manifolds from retained points and their neighborhood features, then resamples those manifolds to form the denoised point cloud. Each patch is generated by mapping a 2D parameter domain into a local 3D surface.
- Patch Manifold Reconstruction: The decoder transforms each retained point and its embedded neighborhood feature matrix into a local surface called a patch manifold.The feature vector encodes the geometry of the neighborhood surface surrounding the retained point.
- Manifold Parameterization: A 2D rectangular domain [−1, 1]^2 is mapped into an arbitrarily shaped 3D manifold parameterized by the neighborhood feature.Uniform samples from the rectangle are transformed into 3D points through the learned mapping.
- Patch Manifold Reconstruction: Each parameterized manifold is translated to center on its corresponding retained point, producing one local patch per point.The decoder defines patch manifold M_i around each point p_i in the retained set.
- Resampling: Sampling two points from each of M=|Ŝ| patches produces a denoised point cloud with the same size as the input.Because the sampled set is half the input size, two samples per patch constitute an upsampling process.
- Implementation: The patch manifold mapping is implemented with an MLP chosen for its ability to approximate arbitrarily shaped manifolds.The MLP parameterizes the local surface from the sampled point’s neighborhood feature.
- Resampling: The reconstructed point cloud is obtained by sampling the learned patch manifolds rather than preserving point-to-point correspondence with the retained set.The method reconstructs a clean point cloud from the parameterized manifolds.
3.4 Loss Functions
The method supports supervised and unsupervised training through losses that separately address subsampling and final reconstruction, while its bijective EMD formulation avoids point clustering.
- Supervised training loss: Supervised training uses dual losses: Lsample measures subsampled-set quality, while Lrec measures final reconstruction quality.Lsample explicitly reduces noise in the subsampled set but is not required for convergence.
- Supervised training loss: Chamfer distance is used for Lsample because the subsampled and ground-truth point clouds contain different numbers of points.The subsampled set has fewer points than the ground-truth cloud.
- Supervised training loss: Earth Mover’s distance is used for Lrec because denoising produces point clouds with equal numbers of points.The loss compares the denoised point cloud with the ground truth through a bijection.
- Supervised training loss: The one-to-one correspondence in LEMD avoids the clustering effect reported in prior denoising methods.Previous methods often introduce a repulsion loss to alleviate clustering.
- Unsupervised training loss: The network also supports unsupervised training using a loss tailored to manifold reconstruction and dense-neighborhood points as clean-surface proxies.The formulation requires constructing a bijection between noisy input points and reconstructed output points before computing the unsupervised loss.
3.5 Analysis
The method can be viewed both as neural local-surface fitting and as complementary filtering: pooling removes noise while manifold reconstruction restores detail.
- Interpretation: The patch manifold is analogous to a locally fitted surface, while the decoder uses neural networks to infer its shape.This connects the method to local-surface-fitting denoising while replacing explicit approximation with learned reconstruction.
- Interpretation: Differentiable pooling acts like a low-pass filter for noise, and manifold reconstruction restores details from neighborhood features to avoid over-smoothing.The two stages are interpreted as complementary filtering operations.
4 EXPERIMENTAL RESULTS
Experiments show that the proposed denoiser performs strongly across synthetic, simulated LiDAR, and real-world noise, while ablations support the value of differentiable pooling and dual loss functions. Its manifold-reconstruction design is especially favorable under point-to-surface evaluation and produces cleaner qualitative results at higher noise levels.
- Quantitative Results: The supervised method significantly outperforms prior deep-learning and non-deep-learning denoisers, while the unsupervised version remains competitive at higher noise levels.The unsupervised version outperforms TotalDenoising at higher noise levels and surpasses non-deep-learning methods at 2%, 2.5%, and 3% noise.
- Quantitative Results: The denoiser generalizes effectively from Gaussian training noise to unseen simulated LiDAR noise and performs much better than previous methods.The LiDAR evaluation uses realistic scanner noise generated with Blensor and a Velodyne HDL-64E2 model.
- Quantitative Results: Point-to-surface distance better reflects the method’s manifold-reconstruction objective than point-to-point Chamfer distance.Resampling on the reconstructed manifold can increase point-to-point distances even when surface reconstruction quality improves.
- Qualitative Results: Visual comparisons show cleaner, more pleasing surfaces and greater robustness to outliers, especially at higher noise levels.On Paris-rue-Madame, the method produces cleaner and smoother results than PCNet while preserving details.
- Ablation Studies: Differentiable pooling samples lower-noise points, providing a better initialization for patch-manifold reconstruction.The intermediate subsampled point set demonstrates the pooling layer’s intended role before reconstruction.
- Ablation Studies: All evaluated components contribute positively, with differentiable pooling and dual loss functions improving manifold reconstruction and denoising output.The ablation evaluation uses 50K-point clouds with 2% Gaussian noise.
5 CONCLUSION
The paper learns and reconstructs the underlying manifold of noisy point clouds from differentiably subsampled points, then resamples patch manifolds to produce clean point clouds. Experiments show superiority over state-of-the-art methods under synthetic and real-world noise.
- The method learns the underlying manifold of a noisy point cloud from differentiably subsampled points.
- Adaptive differentiable pooling selects points closer to the underlying surfaces, while neighborhood features support local patch-manifold inference.
- Resampling each inferred patch manifold reconstructs a clean point cloud that captures intrinsic structure.
- The network supports end-to-end training in either supervised or unsupervised fashion.
- Experiments demonstrate superiority over state-of-the-art methods under both synthetic and real-world noise.