Source-linked AI summary
PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Clouds
Marie-Julie Rakotosaona, Vittorio La Barbera, Paul Guerrero, Niloy J. Mitra, Maks Ovsjanikov
TL;DR
Raw point clouds are contaminated by noise and outliers, complicating downstream geometry processing. POINTCLEANNET learns to remove outliers and denoise remaining points, consistently outperforming state-of-the-art approaches across models and noise settings while requiring no user-specified parameters.
Problem
Raw point clouds contain noise and outliers, so cleaning is typically needed before surface reconstruction and other geometry-processing applications.
Method
POINTCLEANNET trains a PCPNet-based two-stage network on synthetically corrupted surface patches to remove outliers and estimate per-point correction vectors without assuming a noise or outlier model.
Results
POINTCLEANNET consistently outperforms state-of-the-art denoising approaches across models and varied noise settings, including medium to large noise levels.
Takeaways & Limitations
The framework can be readily integrated into geometry-processing workflows because it requires neither additional parameters nor noise/device specifications from the user.
Takeaways & Limitations
Its uniform output distribution is established for uniformly distributed noisy points, while handling non-uniform noisy points remains a future direction.
Abstract
from arXiv · showhide
Point clouds obtained with 3D scanners or by image-based reconstruction techniques are often corrupted with significant amount of noise and outliers. Traditional methods for point cloud denoising largely rely on local surface fitting (e.g., jets or MLS surfaces), local or non-local averaging, or on statistical assumptions about the underlying noise model. In contrast, we develop a simple data-driven method for removing outliers and reducing noise in unordered point clouds. We base our approach on a deep learning architecture adapted from PCPNet, which was recently proposed for estimating local 3D shape properties in point clouds. Our method first classifies and discards outlier samples, and then estimates correction vectors that project noisy points onto the original clean surfaces. The approach is efficient and robust to varying amounts of noise and outliers, while being able to handle large densely-sampled point clouds. In our extensive evaluation, both on synthesic and real data, we show an increased robustness to strong noise levels compared to various state-of-the-art methods, enabling accurate surface reconstruction from extremely noisy real data obtained by range scans. Finally, the simplicity and universality of our approach makes it very easy to integrate in any existing geometry processing pipeline.
1. Introduction
POINTCLEANNET addresses noisy and outlier-contaminated point clouds with a parameter-free, two-stage learned cleanup method. It removes outliers, denoises the remaining points, and is evaluated on synthetic and real data.
- Motivation: Raw point clouds from scanners and reconstruction algorithms commonly contain noise and outliers that impair downstream processing.Cleaning is typically an initial step before surface reconstruction, shape matching, and model retrieval.
- Design goals: The method is parameter-free and does not require information about the scanned surface type, device characteristics, or noise model.It is designed to automatically preserve high-curvature features.
- Method: POINTCLEANNET adapts PCPNet into a two-stage network that first classifies outliers and then estimates correction vectors for denoising.Training uses synthetically corrupted surface patches with varying noise and outlier levels.
- Evaluation: POINTCLEANNET is evaluated qualitatively and quantitatively on synthetic datasets with ground truth and real-world datasets.The evaluation covers varying shape and noise conditions.
- Results: The approach outperforms various state-of-the-art denoising methods across shape and medium-to-high noise variations.The comparison includes methods with manually tuned parameters.
2. Related Work
Prior point-cloud cleanup methods use statistical, surface-fitting, filtering, sparse-representation, or learning-based strategies. POINTCLEANNET instead targets general-purpose denoising and outlier removal across varied artifacts without user intervention.
- Outlier removal: Classical outlier methods use robust local statistics but often require critical parameters or additional information such as color.These requirements make application across general noise models difficult.
- Surface fitting and filtering: MLS, jet-fitting, bilateral, and related surface methods project points onto estimated surfaces to denoise while preserving features.They are robust for small noise but can oversmooth or oversharpen under high noise.
- Non-local methods: Non-local and dictionary-based methods consolidate similar patches or use shape-part dictionaries to construct coherent noise-free point clouds.These methods are related to patch-based representations of geometry.
- Learning-based methods: Learning-based denoising methods reduce reliance on manually setting parameters by learning models from data.This trend spans images, meshes, and point-cloud processing.
- POINTCLEANNET: POINTCLEANNET learns to classify outliers and displace noisy points, handling varied artifacts in dense point clouds without user intervention.Its goal is a general-purpose method for potentially high levels of structured noise.
3. Overview
The paper models cleaning as converting an observed noisy point cloud into a higher-quality one through outlier removal and per-point displacement. It uses local PCPNet-based processing while designing losses for surface proximity and regular point distribution.
- Problem formulation: The observed cloud combines perfect surface samples, additive noise, and outlier points, without assumptions about the noise or outlier models.The desired output is closer to the clean surface and better suited for further processing.
- Cleaning pipeline: Cleaning consists of removing estimated outliers first and then applying per-point displacement vectors to denoise the remaining points.The output point cloud is represented by the displaced non-outlier samples.
- Architecture: The approach adapts the local PCPNet architecture because the task depends mainly on local neighborhoods and dense global processing has higher memory and capacity demands.Local estimates can nevertheless become inconsistent across neighboring patches.
- Objectives: The losses seek points close to the scanned surface and distributed regularly on it.The method does not attempt to recover tangent-direction noise that cannot be inferred from the noisy cloud.
- Objectives: The overview describes separate loss functions for the outlier-classification and denoising stages.These loss functions largely determine the properties of the denoised point cloud.
4. Cleaning Model
The cleaning model processes noisy point clouds locally through outlier removal followed by displacement-based denoising. It uses PCPNet-inspired, permutation-invariant architecture and iterative refinement, with inflation to counteract observed shrinkage.
- Cleaning Model: The model takes a noisy point cloud P′ and produces a cleaned point cloud ˜P by estimating local corrections toward the unknown surface.Each point is processed using its local neighborhood within radius r.
- Cleaning Model: The first stage estimates outlier probabilities and removes points with probability greater than 0.5, yielding ˆP.The second stage then operates on the remaining points.
- Cleaning Model: The second stage estimates displacement vectors d_i for remaining points, producing denoised points ˜p_i = ˆp_i + d_i.Both stages use deep neural networks with a PCPNet-based architecture.
- Cleaning Model: PCPNet supplies a local, order-invariant architecture suited to shape details and densely sampled point clouds, unlike global PointNet processing.The architecture is based on PointNet and PCPNet, with local patches normalized before processing.
- Cleaning Model: A quaternion spatial transformer canonicalizes each patch, while shared pointwise features, symmetric aggregation, and a regressor estimate displacement vectors and outlier probabilities.The final displacement vectors are rotated back to world space.
- Cleaning Model: Iterating denoising improves residual noise, but repeated iterations can shrink the point cloud, so an inflation step is applied after each iteration.Residual noise arises because neighboring points use different local neighborhoods and therefore receive slightly different predictions.
5. Training Setup
POINTCLEANNET trains local networks with paired noisy and clean point clouds, using losses that encourage surface proximity and regular point distribution. The method also uses iterative application and a specialized PCPNet adaptation for denoising and outlier removal.
- Training data: Training uses paired noisy and clean point clouds, with known ground-truth outlier labels but no exact point correspondences.Ground-truth point clouds can come from higher-quality scans of the same scene.
- Loss functions: The outlier-removal network uses L1 loss between estimated outlier probabilities and ground-truth labels; binary cross-entropy performed worse in practice.
- Loss functions: The denoising loss combines surface proximity with regularity, because surface-only optimization can cause tangent drift and filament-like point clusters.The regularity term discourages excessive clustering and promotes centered, more regular cleaned points.
- Loss functions: The alternative nearest-neighbor loss uses fixed targets that can be precomputed, making it simpler and more efficient, but slightly reducing performance.Its fixed targets also prevent tangent drift.
- Network design: POINTCLEANNET applies a PCPNet-based network iteratively; with the proper loss and an inflation term, it stabilizes and achieves higher accuracy.A straightforward implementation might not converge, whereas the adapted iterative design improves noise reduction.
- Evaluation: The qualitative evaluation compares denoising errors across shapes of increasing complexity, with distance-to-surface encoded by color.DGCNN uses sparser sampling because it can only handle small point clouds.
6. Results
The evaluation uses separate datasets and corruption settings for denoising and outlier removal, with additional tests on alternative noise sources. The study also releases the training datasets.
- Datasets: The main dataset contains 28 shapes, split into 18 training shapes and 10 test shapes, with 100K uniformly sampled points per shape.
- Datasets: Denoising data uses Gaussian noise at standard deviations from 0.25% to 2.5% of the shape bounding-box diagonal, including clean points.
- Datasets: Outlier-removal training varies outlier density from 10% to 90% and includes 432 example shapes across six densities and four noise levels.
- Datasets: The outlier-removal test set contains 30% outliers and also evaluates uniformly distributed outliers inside an enlarged shape bounding box.
- Additional evaluation: Additional experiments use simulated non-uniform noise and noise from real acquisition devices, and the training datasets are available on the project page.
6.2. Evaluation Metric
The evaluation metric is chosen to capture both proximity to the ground-truth surface and regularity of point distribution, with RMSD used for non-uniformly distributed ground truth data.
- Chamfer measure: The Chamfer measure evaluates cleaned point clouds using both distance to the target surface and regularity of point coverage.
- Chamfer measure: Its first term measures approximate cleaned-point distance to the target surface, while its second term rewards even coverage and penalizes gaps.
- Normalization: All point clouds are scale-normalized to unit bounding-box diagonal so point distances are comparable across shapes.
- Alternative metric: For simulated scanner noise with non-uniform clean-point distributions, evaluation uses only root mean square distance-to-surface.
6.3. Evaluating Denoising
The denoising evaluation compares POINTCLEANNET with several classical and learning-based methods across noise levels. POINTCLEANNET performs better and more consistently at mid to high noise, while multiple iterations reduce residual error in detailed regions.
- Baseline behavior: Jet fitting depends on neighborhood size, edge-aware resampling has larger errors near details, and bilateral filtering performs poorly under strong noise.
- Method behavior: Local patch processing and repeated denoising iterations produce lower residual error, especially in detailed surface regions.
- Quantitative results: POINTCLEANNET performs noticeably better than the other methods under mid to high noise levels, especially when using multiple iterations.
- Quantitative results: POINTCLEANNET remains more stable as noise changes, whereas most other methods perform well only at a specific noise level.
6.4. Evaluating Outlier Removal
The outlier-removal evaluation emphasizes recall alongside precision and compares POINTCLEANNET with jet fitting and edge-aware filtering. POINTCLEANNET performs strongly across noise levels, shapes, and surface detail without user parameter tuning.
- Evaluation criterion: For dense-data cleaning, the evaluation prioritizes correctly classifying outliers, so recall is treated as more important than precision.
- Evaluation criterion: The Fβ score balances recall and precision, with F1 and F2 used to compare POINTCLEANNET against jet fitting.
- Quantitative results: POINTCLEANNET outperforms edge-aware and jet-fitting techniques for joint outlier removal and denoising at medium to large noise levels.
- Quantitative results: At smaller noise levels, POINTCLEANNET still outperforms some tuning variations of the related techniques without requiring user parameter tuning.
- Qualitative results: Qualitatively, POINTCLEANNET is more consistent across shapes and detail levels, while edge-aware filtering struggles near details and edges and jet fitting leaves scattered outliers.
6.5. Performance under Different Noise Types
POINTCLEANNET remains competitive across structured, anisotropic, scanner-specific, Kinect, and misaligned-scan noise settings, with retraining often producing the strongest results. Qualitative evaluations also show improved preservation of surface structure and scan merging compared with competing methods.
- Anisotropic noise: On anisotropic noise, POINTCLEANNET performs on par with state-of-the-art methods at low and medium noise levels despite training only on other noise.At high noise, non-data-driven methods with parameters tuned to the noise strength outperform it.
- Velodyne structured noise: The Velodyne evaluation tests distance bias and per-ray Gaussian noise using RMSD distance-to-surface measurements on non-uniform scans.Both a version trained on the original dataset and a version retrained on Velodyne are evaluated.
- Velodyne structured noise: The Velodyne-retrained model significantly outperforms all other methods, while the original-trained model remains competitive.Qualitatively, jet and bilateral methods retain structured noise, and jet medium introduces artefacts in detailed regions.
- Kinect datasets: POINTCLEANNET achieves the best performance on Kinect v1 and performs well against other methods on Kinect v2.The comparisons use Chamfer measures and include Poisson reconstructions and normalized distances to the ground-truth surface.
- Misaligned scans: On misaligned scans, the retrained model has the best RMSD performance and POINTCLEANNET merges scans while bilateral filtering retains or introduces artefacts.The dataset uses scans from six angles with independently rotated scans at θ values of 1° and 2°.
- Real-world data: On real image-based reconstruction data containing 1.4M-point inputs, POINTCLEANNET produces high-quality qualitative results despite training on a different dataset.No ground truth is available for this evaluation, so only qualitative results are shown.
7. Conclusion, Limitations and Future Work
The conclusion presents POINTCLEANNET as a simple, broadly usable cleaning framework while identifying supervised paired-data training and distribution mismatch as boundaries. Future work targets unified processing, greater efficiency, uniformity, and point upsampling.
- Conclusion: POINTCLEANNET removes outliers and denoises remaining points by displacing them toward the underlying scanned surface.The framework requires no additional test-time parameters or user-provided noise or device specifications.
- Conclusion: Across varied models and noise settings, POINTCLEANNET consistently outperforms state-of-the-art denoising approaches supplied with manually tuned parameters.The authors present its universality and ease of use as supporting integration into geometry-processing workflows.
- Limitations: POINTCLEANNET currently requires paired noisy-clean data for training and may fail when training and test noise characteristics differ substantially.The authors identify learning from unpaired data as a future direction.
- Future work: Future work includes combining outlier removal and denoising in one network instead of two separate parts.
- Future work: Additional directions include patch-level denoising, uniformity control for non-uniform inputs, and upsampling where point coverage is insufficient.The authors note the challenge of keeping points on the surface when adding uniformity terms.
Appendix A:
The appendix figure compares POINTCLEANNET with state-of-the-art methods on the Velodyne dataset.
- Figure 17 provides a qualitative comparison with state-of-the-art methods on the Velodyne dataset.