Source-linked AI summary
Total Denoising: Unsupervised Learning of 3D Point Cloud Cleaning
Pedro Hermosilla, Tobias Ritschel, Timo Ropinski
TL;DR
3D point-cloud denoising lacks clean or paired training data, while total noise makes naïve unsupervised image-denoising extensions ambiguous. The paper combines a self-mapping network with spatial locality and optional bilateral appearance priors, and reports unsupervised performance that can outperform supervised methods with the same number of training examples. Its evidence is limited to distortions where a surface remains conceivable, with appearance offering benefits but also introducing reliability caveats.
Problem
Scanner noise and limited clean geometry make paired supervised learning unsuitable for denoising the rapidly growing supply of acquired 3D point clouds.
Method
The method combines a point-cloud-to-itself network with spatial locality and an optional bilateral appearance prior, using color only to sample the prior.
Results
The unsupervised approach can outperform supervised methods, including with the same number of training examples.
Takeaways & Limitations
The approach enables denoising from natural noisy data without clean examples or noisy pairs, and appearance can improve results without serving as network input.
Takeaways & Limitations
The evaluation considers distortions where a surface remains conceivable, while uneven sampling and outliers remain future work; color can also be unreliable around texture and shadow edges.
Abstract
from arXiv · showhide
We show that denoising of 3D point clouds can be learned unsupervised, directly from noisy 3D point cloud data only. This is achieved by extending recent ideas from learning of unsupervised image denoisers to unstructured 3D point clouds. Unsupervised image denoisers operate under the assumption that a noisy pixel observation is a random realization of a distribution around a clean pixel value, which allows appropriate learning on this distribution to eventually converge to the correct value. Regrettably, this assumption is not valid for unstructured points: 3D point clouds are subject to total noise, i. e., deviations in all coordinates, with no reliable pixel grid. Thus, an observation can be the realization of an entire manifold of clean 3D points, which makes a naïve extension of unsupervised image denoisers to 3D point clouds impractical. Overcoming this, we introduce a spatial prior term, that steers converges to the unique closest out of the many possible modes on a manifold. Our results demonstrate unsupervised denoising performance similar to that of supervised learning with clean data when given enough training examples - whereby we do not need any pairs of noisy and clean training data.
1. Introduction
The paper targets unsupervised 3D point-cloud denoising from noisy scans alone, addressing total noise and the lack of clean-noisy pairs. Its evaluation reports that the approach can outperform supervised learning with sufficient training data.
- Motivation: Noisy 3D scans are abundant, but scanner noise and scarce clean CAD geometry make paired supervised denoising impractical.The paper therefore seeks to denoise acquired point clouds using only noisy data.
- Problem: 3D point clouds suffer total noise, because both the domain and range can be corrupted rather than only signal values.This setting differs from common image noise, where pixel positions remain clean.
- Approach: Unsupervised denoising is learned from noisy examples alone, without requiring paired noisy and clean training data.The approach extends unsupervised denoising ideas to unstructured 3D point clouds.
- Results: The evaluation reports that the unsupervised approach can outperform supervised learning given enough training examples, including some cases with the same training-data magnitude.The claim covers simulated noise of different kinds and real point clouds.
2. Related Work
Prior work developed unsupervised image denoising and supervised point-cloud denoising, but deep point-cloud methods still generally require clean-noisy pairs. The paper positions its approach as removing that pairing requirement for irregular 3D data.
- Image denoising: Noise2Noise, Noise2Void, and Noise2Self reduce image-denoising supervision from clean-noisy pairs to paired or single noisy images.Noise2Void and Noise2Self regress held-out pixels from surrounding context using blind spots.
- 3D point-cloud denoising: 3D point clouds are harder than images because their irregular structure complicates denoising while preserving fine spatial detail.Classical filters and transferred image operators address this difficulty but can sacrifice details or require tuning.
- Deep methods: Deep point-cloud denoising methods such as PointProNet are supervised and rely on clean-noisy training pairs, often generated from synthetic point clouds.The paper’s approach does not require such pairs.
- Noise and learning: Denoising auto-encoders use noise as an augmentation strategy for robustness rather than targeting a clean signal from noisy input.Their training target is not itself noisy, distinguishing them from denoising methods discussed here.
3. Denoising Theory
Image denoising can exploit distributions around known pixel locations, but unstructured point clouds lack that correspondence and may yield multimodal clean-point explanations. The paper extends unsupervised denoising to colored or positional point clouds while using appearance to address ambiguities.
- 3.1. Regular Domains: Noise2Noise learns from two noisy realizations, while Noise2Void and Noise2Self use blind spots to regress a held-out pixel from the remaining image context.The latter methods do not require two images.
- 3.1. Regular Domains: For pixel noise, each observation is sampled around a clean value at a known pixel position, enabling supervised and unsupervised estimators to target distribution statistics.Noise2Noise converges to the mean, median, or mode under L2, L1, or L0 loss, respectively.
- 3.1. Regular Domains: These image-denoising formulations assume noise affects the range while pixel-domain positions remain certain.That separation of domain and range semantics does not hold for total noise in point clouds.
- 3.2. Unstructured Domains: The paper represents clean points as x, noisy points as y, and the noise model as p for XYZ or XYZRGB point clouds.Colored point clouds extend positional coordinates with appearance information.
- 3.2. Unstructured Domains: Color can help unsupervised colored-point-cloud denoising during training even when color is absent at test time.The paper presents this as a way to overcome challenges in training unsupervised point-cloud denoisers.
4. Unsupervised 3D Point Cloud Denoising
Unpaired self-denoising fails for point clouds because total noise makes each observation compatible with multiple clean surface points. The method resolves this ambiguity with spatial and appearance priors, while its learned mode surface remains limited by noise convolution.
- 4.2. Unpaired: A noisy point can correspond to multiple clean surface points, so self-regression converges toward a multimodal distribution rather than a unique denoised point.Unlike images, unstructured points lack a reliable grid or point identity for determining which hidden surface point generated an observation.
- 4.3. Manifold of Modes vs. Clean Surface: The mode manifold is close to the clean surface under small noise but deviates from it as noise increases.Figure 4 compares the mode manifold with the ground-truth surface for small and large noise.
- 4.3. Manifold of Modes vs. Clean Surface: The method cannot recover details removed by convolution and assumes that among multiple explanations, the closest one is correct.The mode structure need not be a connected surface for the solution to apply, but the learned target remains the convolved surface rather than clean geometry.
- 4.4. Unique Modes: The spatial prior q(z|y) selects the unique closest mode by encoding how likely an observed point y is to arise from a clean point z.The prior combines spatial and, when available, appearance proximity; its Gaussian form does not assume Gaussian noise.
- 4.4. Unique Modes: Color can preserve sharp appearance edges by separating points that spatial proximity alone would merge or round.Color is used during training and can improve the prior even when it is unavailable as a test-time network input.
- 4.6. Implementation: The approach learns the same closest mode targeted by supervised training on the convolved clean-surface and noise distribution, without paired noisy-clean data.The network uses an unstructured encoder-decoder, while training optimizes convergence toward the prior’s mode.
5. Evaluation
The evaluation compares unsupervised denoising with classical and supervised baselines on synthetic and real point clouds, including noise, data-scale, and ablation studies. The full method performs similarly to or better than supervised approaches across several settings, while spatial and appearance priors improve results.
- Setup: Synthetic experiments use 15 object classes with 5 training and 2 testing models per class, while real-data evaluation is qualitative because ground truth is unavailable.Synthetic point clouds contain 22 million training points and 10 million testing points, with Gaussian and simulated sensor noise.
- Supervision scalability: Our full method performs better than supervised learning across a range of training-data magnitudes, with the methods crossing at around 22 million points.Beyond that point, the unsupervised method requires more training data to match supervised performance.
- Denoising performance: Across SIMPLE-noise evaluations, the full method is similar to supervised learning, whereas removing the spatial or color prior worsens performance and scaling.Supervised learning scales best, but the full unsupervised method remains close in performance.
- Denoising performance: For advanced sensor noise, the full method outperforms all unsupervised baselines, while supervised learning can perform better at 12 million training points.Removing the suggested priors increases error, indicating that both priors remain relevant under this noise model.
- Denoising performance: On the evaluated PointCleanNet dataset, the method slightly outperforms the supervised competitor without modifying its architecture or supervision.The comparison uses the PointCleanNet dataset and retains the paper’s unified evaluation setup.
- Ablation: The spatial prior is consistently important across noise types, noise magnitudes, and training-data amounts, while appearance consistently improves results to near or beyond supervised quality.Including color in the prior slightly reduces error, even though color is not used as a network input.
6. Conclusions
The paper presents unsupervised 3D point-cloud denoising without clean examples or noisy pairs, using a self-mapping network with spatial locality and bilateral appearance priors.
- The method denoises 3D point clouds without clean examples or noisy pairs, scaling with natural data rather than clean CAD models with synthetic noise.The paper reports that it can outperform supervised methods even with the same number of training examples.
- A self-mapping network combined with spatial locality and a bilateral appearance prior enables the unsupervised denoising method.Appearance is optional and can improve results without being provided to the network during training or testing.
- Joint denoising of color and position, and using color as network input, remain future work.
A. Error distribution
The denoised point distribution peaks at the real surface, while training initially favors the unsupervised method before additional data narrows the gap.
- The point-distance distribution peaks at zero distance from the real surface and decreases with distance, matching the supervised setting.
- With the smallest dataset, unsupervised training converges to lower evaluation error than supervised training, although more data eventually reduces the gap until the curves cross.Both losses decrease similarly during training.
C. Prior Kernel
The prior is evaluated with Gaussian, Wendland, and Inverse Multi-Quadric kernels across parameter settings, with Gaussian achieving the best performance.
- Gaussian, Wendland, and Inverse Multi-Quadric kernels are compared as candidate functions for the spatial prior.The kernels are adjusted to the range [-1, 1].
- The Gaussian kernel obtains the best performance among the tested prior functions and parameter settings.The ablation evaluates three α values independently for each kernel.
D. Noise levels
A network trained at one noise level retains performance across other levels, while iterative application improves results because the learned surface approximation is imperfect.
- Performance drops only marginally when a network trained at one noise level is tested across all noise levels.The experiment compares training on a single noise level with evaluation at 0.5%, 1.0%, 1.5%, and all levels.
- Iterative denoising improves results because the network’s approximation of the surface during training is not exact.Figure 10 visualizes each point’s distance to the real surface after each denoising step.
F. Additional Qualitative Results
Additional qualitative evaluations show that the method removes noise from sparse KITTI point clouds and achieves quality similar to supervised training. It also processes large Paris-rue-Madame models and supports iterative application.
- The network successfully removes noise from sparse KITTI point clouds.Qualitative results demonstrate robustness on sparse point clouds.
- Applying the network iteratively produces a resulting point cloud from a noisy input.
- The method achieves quality similar to a network trained with supervised data across the evaluated simulated datasets.The comparison includes variants using a prior without color information and no prior.
- 375 K-800 K points are processed in Paris-rue-Madame models using a workstation with a single Nvidia RTX 2080.The method processes each model in parallel.