PaperlayerView exact revision on arXiv

Source-linked AI summary

PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling

Xu Yan, Chaoda Zheng, Zhen Li, Sheng Wang, Shuguang Cui

arXiv:2003.00492v3cs.CV

TL;DR

Raw point clouds contain noise and outliers, making robust processing difficult because existing sampling can be outlier-sensitive and restricted to original points. PointASNL combines differentiable adaptive sampling with local-nonlocal feature learning, and the paper reports state-of-the-art robust classification and segmentation across its evaluated datasets.

  • Problem

    Noise and outliers in raw point clouds, together with limitations of existing sampling methods, create a need for robust point-cloud processing.

  • Method

    PointASNL combines adaptive sampling that adjusts initial sampled coordinates with a local-nonlocal module capturing neighborhood and long-range dependencies.

  • Results

    PointASNL achieves state-of-the-art robust performance for classification and segmentation across the evaluated datasets.

  • Takeaways & Limitations

    Combining local neighbors, global context, and adaptive spatial redistribution improves robustness to noisy outliers in point-cloud processing.

Abstract

from arXiv · show

Raw point clouds data inevitably contains outliers or noise through acquisition from 3D sensors or reconstruction algorithms. In this paper, we present a novel end-to-end network for robust point clouds processing, named PointASNL, which can deal with point clouds with noise effectively. The key component in our approach is the adaptive sampling (AS) module. It first re-weights the neighbors around the initial sampled points from farthest point sampling (FPS), and then adaptively adjusts the sampled points beyond the entire point cloud. Our AS module can not only benefit the feature learning of point clouds, but also ease the biased effect of outliers. To further capture the neighbor and long-range dependencies of the sampled point, we proposed a local-nonlocal (L-NL) module inspired by the nonlocal operation. Such L-NL module enables the learning process insensitive to noise. Extensive experiments verify the robustness and superiority of our approach in point clouds processing tasks regardless of synthesis data, indoor data, and outdoor data with or without noise. Specifically, PointASNL achieves state-of-the-art robust performance for classification and segmentation tasks on all datasets, and significantly outperforms previous methods on real-world outdoor SemanticKITTI dataset with considerate noise. Our code is released through https://github.com/yanx27/PointASNL.

1. Introduction

PointASNL addresses noisy, unstructured point clouds with adaptive sampling and local-nonlocal feature learning. The proposed network is designed to reduce outlier influence while capturing both neighborhood and long-range dependencies.

  • Raw 3D point clouds are challenging because sparsity and disorder complicate processing, while sensor and reconstruction data inevitably contain noise or outliers.
  • PointASNL is an end-to-end robust point-cloud model whose two main modules adjust sampled points and capture neighbor and long-range dependencies.The paper reports robustness verification on classification and segmentation tasks.
  • Existing sampling methods are limited by Euclidean-distance sensitivity, predefined uniformity, feature-space-only sampling, or confinement to the original point subset.These limitations allow a sampled outlier to influence downstream processing.
  • PointASNL uses a differentiable adaptive sampling module to adjust initial sampled coordinates toward intrinsic geometry and beyond the original cloud.The adjustment is intended to reduce outlier influence while improving feature learning and noise robustness.
  • Its local-nonlocal module combines local Euclidean features with long-range dependencies computed across the entire point cloud.The point nonlocal cell uses a weighted sum of influences from the entire cloud rather than only nearby neighbors.

2. Related Work

Related work spans volumetric, projection-based, point-based, and robust sampling approaches. The cited methods differ in how they represent point clouds, aggregate context, and address outliers or sampling constraints.

  • Volumetric-based and Projection-based Methods: Volumetric and projection-based methods improve computational efficiency but convolution-based representations may lack nonlocal geometric feature capture.
  • Point-based Learning Methods: Point-based methods directly process sparse, unstructured clouds, using pointwise global aggregation or hierarchical neighborhood grouping.
  • Outlier Removal and Sampling Strategy: Prior robust statistics methods for outlier removal face non-trivial parameterization issues, motivating learned sampling strategies.

3. Our Method

PointASNL combines adaptive sampling with local-nonlocal feature learning in a hierarchical architecture for classification and segmentation. Adaptive sampling updates FPS-selected points in spatial and feature space, while L-NL jointly captures local neighborhoods and global context.

  • Network Architecture: AS and L-NL are combined hierarchically to form PointASNL architectures for classification and segmentation.Classification uses three layers with two sampling levels, while segmentation uses a deeper encoder-decoder with interpolation and skip connections.
  • Adaptive Sampling: FPS is sensitive to outliers and restricts sampled points to the original cloud, motivating adaptive point updates.The AS module first obtains relatively uniform FPS samples, then learns shifts for each sampled point.
  • Adaptive Sampling: AS reweights K neighbors with softmax-normalized point-wise MLP outputs and shifts both coordinates and features by weighted sums.The learned weights are applied across neighbor coordinates and feature channels.
  • Local-Nonlocal Feature Learning: The local-nonlocal module combines a point local cell for neighborhood aggregation with a point nonlocal cell for multi-scale correlations across the entire point cloud.The PL cell can use methods such as PointNet++ or PointConv, while PNL integrates global information for sampled query points.
  • Local-Nonlocal Feature Learning: The PNL cell uses adaptively updated sampled points as queries and entire-layer points as keys, then fuses global context with local features.A nonlinear convolution aligns the global-context channels with the local-cell output before fusion.

4. Experiment

PointASNL is evaluated across synthetic classification, indoor segmentation, outdoor segmentation, noisy inputs, and ablation settings. The experiments show robust performance across these settings, with particularly strong indoor and outdoor segmentation results.

  • Classification: PointASNL is evaluated on ModelNet10 and ModelNet40 for synthetic point-cloud classification.ModelNet40 contains 9,843 training models and 2,468 test models across 40 classes; ModelNet10 contains 3,991 training and 908 testing objects across 10 classes.
  • Classification with Noise: Under noisy-point testing, PointASNL is reported as highly robust, especially after adding adaptive sampling.Adaptive sampling preserves the sampled point-cloud shape and distribution despite randomly injected noise.
  • Indoor Scene Segmentation: PointASNL outperforms all compared methods under the same training strategy on both S3DIS and ScanNet segmentation.On ScanNet, its result is reported as 8% higher than PointConv under the same experimental setting.
  • Outdoor Scene Segmentation: On SemanticKITTI, PointASNL outperforms other state-of-the-art methods by a large margin and achieves the best result in 13 of 19 categories.The qualitative results remain strong in scenes containing substantial noise, including noise caused by unmanned collection.
  • Ablation Study: Combining local and global information improves classification and segmentation, while adding adaptive sampling raises segmentation results to 93.2% and 63.5% in model D.The ablation also reports reductions in classification and segmentation error of 23.1% and 12.6% when PointNet++ is used in the proposed point-local cell.

5. Conclusion

PointASNL combines adaptive sampling with local-nonlocal learning for robust 3D point-cloud processing. The authors report improved robustness and performance across several benchmarks.

  • PointASNL combines adaptive sampling and a local-nonlocal module for robust 3D point-cloud processing.The architecture integrates local neighbors with global context interaction.
  • Adaptive sampling differentiably fine-tunes the spatial distribution of sampled points, improving network robustness.
  • Combining local neighbors with global context interaction improves performance on several benchmarks.

Supplementary Material

The supplementary material adds experiments and documents the network architecture in separate sections.

  • The supplementary material provides additional experiments in Section B and network architecture details in Section C.

B. Additional Experiment

The supplementary part-segmentation experiment evaluates PointASNL on ShapeNet using a standardized 2048-point setup. PointASNL performs comparably to state-of-the-art point-based methods.

  • The ShapeNet dataset contains 16,881 shapes spanning 16 classes and 50 parts.The experiment uses randomly selected 2048-point inputs and appends the object label's one-hot encoding at the final layer.
  • PointASNL achieves a comparable part-segmentation result to state-of-the-art point-based methods.It is only slightly lower than RS-CNN, which uses a different sampling and voting strategy.

B.2. Selection of Adaptive Sampling

The adaptive-sampling study compares initial sampling and neighbor-update strategies, while visualizations illustrate how local and global context are combined. FPS and group-feature weighting improve the reported inference accuracy.

  • Selection of Adaptive Sampling: The ablation varies initial sampling between FPS and random sampling, and compares group-feature weighting with neighbor averaging.The number of neighbors is controlled across layers for fair comparison.
  • Selection of Adaptive Sampling: For each sampled point, local neighbors and K globally highest-response points are selected, with K equal to the number of local neighborhoods.
  • Selection of Adaptive Sampling: 87.9% accuracy results from random sampling with averaged coordinates and features, compared with 91.5% using FPS.
  • Selection of Adaptive Sampling: Group-feature weighting can substantially improve the model's inference ability, although adding it with random sampling causes accuracy loss.
  • Selection of Adaptive Sampling: Global points match sampled points by characteristics in the first layer and by semantic information in the second layer.This includes sampled points located at junctions between different semantics.

B.4. Visualization of Adaptive Sampling

Adaptive sampling preserves the sampled-point manifold under heavy noise by reducing noise while maintaining shape. The reported comparisons also indicate further gains from richer preprocessing, deeper structure, and ensembling.

  • Visualization: Adaptive sampling maintains the sampled-point manifold when the input point cloud contains substantial noise.It reduces noise in sampled points while preserving the sampled manifold’s shape.
  • Comparison setting: The manuscript’s main comparison uses the same model structure and training strategy as PointNet++ rather than richer preprocessing or deeper structures.The authors report that other preprocessing or deeper structures can further improve PointASNL.
  • Further improvement: 66.6% is achieved on the ScanNet benchmark through ensemble learning with models from different training epochs.

B.6. Concrete Results

The paper reports detailed benchmark results for S3DIS and SemanticKITTI and points readers to the online ScanNet benchmark. Additional visualizations illustrate performance in complicated scenes.

  • Benchmark results: Detailed results are provided for S3DIS and SemanticKITTI as benchmarks for future work.
  • Benchmark results: ScanNet class scores are available through its online benchmark website.
  • Qualitative results: Additional visualizations illustrate the model’s performance in complicated scenes.

C. Network Architectures

PointASNL uses hierarchical encoder layers with adaptive sampling and local/nonlocal feature cells, trained end-to-end with classification or segmentation losses. The design combines sampled-point efficiency with global contextual modeling and adds only around 30% extra time.

  • Network configuration: Each encoder layer specifies sampled-point count, neighbor count, AS neighbors, and an MLP shared by local and nonlocal cells.The AS and local cell use the same k-NN query.
  • Training objective: The model uses cross-entropy loss, category weighting for semantic segmentation, and first-layer Repulsion Loss to prevent sampled points from clustering.Repulsion Loss accelerates training but does not improve performance.
  • Training objective: PointASNL is trained end-to-end by minimizing a joint loss containing cross-entropy, Repulsion Loss, and weight decay terms.The loss balance uses α = 0.01 for cross-entropy and Repulsion Loss.
  • Training objective: The Repulsion Loss constrains sampled-point movement by penalizing points that are too close to neighbors and encouraging relatively constant density.
  • Efficiency: Using sampled points as nonlocal queries makes the AS and nonlocal cells efficient, adding only around 30% extra time.
Loading 2003.00492v3…