PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling

Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, Andrew Markham

arXiv:2107.02389v1cs.CVcs.AIcs.ROeess.SP

TL;DR

Large-scale point-cloud segmentation is difficult because existing sampling and preprocessing methods limit efficient operation on massive, irregular data. RandLA-Net combines random sampling with local feature aggregation to process large point clouds directly, achieving up to 200× faster inference while surpassing state-of-the-art benchmark methods. The paper demonstrates high efficiency and strong semantic-segmentation performance across multiple large-scale datasets.

  • Problem

    Existing approaches often rely on expensive sampling or computationally heavy preprocessing and therefore are limited in training and operating on small-scale point clouds.

  • Method

    RandLA-Net uses simple random sampling and a local feature aggregation module that progressively expands receptive fields to preserve complex geometric structures.

  • Results

    Up to 200× faster than existing approaches, RandLA-Net surpasses state-of-the-art semantic-segmentation methods on multiple large-scale benchmarks.

  • Takeaways & Limitations

    Random sampling combined with lightweight local aggregation can support efficient and effective semantic segmentation of large-scale point clouds.

  • Takeaways & Limitations

    KNN neighbour search does not scale linearly and may become a major bottleneck for point clouds larger than 10^6 points.

Abstract

from arXiv · show

We study the problem of efficient semantic segmentation of large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing approaches are only able to be trained and operate over small-scale point clouds. In this paper, we introduce RandLA-Net, an efficient and lightweight neural architecture to directly infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Although remarkably computation and memory efficient, random sampling can discard key features by chance. To overcome this, we introduce a novel local feature aggregation module to progressively increase the receptive field for each 3D point, thereby effectively preserving geometric details. Comparative experiments show that our RandLA-Net can process 1 million points in a single pass up to 200x faster than existing approaches. Moreover, extensive experiments on five large-scale point cloud datasets, including Semantic3D, SemanticKITTI, Toronto3D, NPM3D and S3DIS, demonstrate the state-of-the-art semantic segmentation performance of our RandLA-Net.

1 INTRODUCTION

Large-scale point-cloud segmentation requires efficient processing of irregular, unstructured data without expensive sampling or preprocessing. RandLA-Net addresses this with random sampling and local feature aggregation, achieving efficient large-scale inference and strong benchmark performance.

  • Motivation: Large-scale point clouds are irregular, unstructured, and unordered, preventing direct application of standard 2D convolutional networks.Efficient segmentation is important for real-time systems such as autonomous driving and augmented reality.
  • Motivation: Over 200 seconds are required for farthest-point sampling to sample 10% of 1 million points, creating a scaling bottleneck.Existing local feature learners and preprocessing pipelines can also be computationally expensive or memory inefficient.
  • Approach: RandLA-Net uses random sampling to progressively downsample large point clouds while avoiding voxelization, block partitioning, and graph construction.The architecture is designed to process large-scale 3D point clouds directly in a single pass.
  • Results: Up to 200× faster than existing approaches, RandLA-Net also surpasses state-of-the-art methods on Semantic3D, SemanticKITTI, and Toronto-3D.The reported gains combine computational efficiency with benchmark segmentation performance.
  • Approach: The local feature aggregation module progressively increases each point’s receptive field to preserve complex local geometric structures despite random sampling.The paper identifies this module as a countermeasure to randomly discarding key information, especially for sparse objects.

2 RELATED WORK

Related work develops point-based feature learners and diverse sampling strategies, but many approaches incur substantial computational or memory costs. RandLA-Net’s design emphasizes simple random sampling and lightweight operations for large-scale processing.

  • Projection and Voxel-Based Networks: Projection and voxel-based methods can lose geometric details during projection or combine voxelization with point-level processing.These approaches are presented as alternatives to direct point-based processing.
  • Point-Based Networks: Point-based networks learn per-point local features using neighboring pooling, graph message passing, kernel convolutions, or attention-based aggregation.These methods explicitly model local spatial relationships and point features, but their operations can be computationally demanding.
  • Sampling Methods: The sampling comparison evaluates complexity and time for sampling 10% of 1 million points, with K denoting the number of nearest neighbours.The table distinguishes FPS, IDIS, PDS, RS, GS, CRS, and PGS.
  • Learning for Large-Scale Point Clouds: SPG constructs superpoint graphs, whereas FCPN and PCT combine voxelization and point networks; these pipelines introduce costly preprocessing or block partitioning.The cited approaches process massive point clouds through intermediate representations rather than the paper’s direct single-pass strategy.
  • Sampling Methods: Farthest-point, inverse-density, Poisson-disk, generator-based, and policy-based methods represent heuristic or learning-based point-sampling strategies.The paper compares these representative methods with random sampling for large-scale point-cloud processing.

3 PROPOSED METHODS

RandLA-Net combines fast random sampling with local feature aggregation to process large-scale point clouds efficiently while preserving geometric details. Its aggregation module encodes local geometry, selectively pools neighboring features, and expands receptive fields through stacked residual units.

  • Overview: Random sampling reduces point density efficiently, while local feature aggregation retains prominent features during progressive downsampling.The approach targets large-scale point clouds that must be downsampled across neural-network layers without losing useful point features.
  • Sampling Strategy: Random sampling is computationally efficient and requires no extra memory, unlike several alternative sampling strategies.The paper’s empirical comparison identifies FPS, IDIS, and GS as computationally expensive, CRS as memory-intensive, and PDS as lower-performing.
  • Local Feature Aggregation: The local feature aggregation module uses local spatial encoding, attentive pooling, and dilated residual blocks.These three neural units are applied in parallel or stacked to capture local geometry and aggregate neighboring features.
  • Local Feature Aggregation: LocSE encodes relative positions of K nearest neighbors and concatenates them with neighboring point features before aggregation.Neighboring points are gathered with KNN based on point-wise Euclidean distance, and the encoded positions augment their corresponding features.
  • Local Feature Aggregation: Attentive pooling learns feature-specific attention scores and computes a weighted sum to emphasize important local features.The attention function uses a shared MLP followed by softmax, producing a soft mask over neighboring features.
  • Local Feature Aggregation: Stacking two LocSE and attentive-pooling units expands each point’s effective neighborhood to up to K^2 points while balancing efficiency and effectiveness.The dilated residual block uses skip connections; additional units could expand reach further but would reduce efficiency and risk overfitting.

4 EXPERIMENTS

Experiments evaluate RandLA-Net’s efficiency, benchmark accuracy, and architectural choices. The results show strong large-scale performance, with local aggregation and spatially aware modules supporting robustness to random sampling.

  • 4.1 Efficiency of Sampling Approaches: Random sampling has superior time and memory efficiency for large-scale point clouds of approximately 10^6 points.FPS, IDIS, PDS, GS, CRS, and PGS become extremely time-consuming or memory-costly at this scale.
  • 4.3 Semantic Segmentation on Benchmarks: RandLA-Net clearly outperforms existing methods on Semantic3D reduced-8 in both mIoU and OA, leading six of eight classes.The exceptions are low vegetation and scanning artefact.
  • 4.3 Semantic Segmentation on Benchmarks: RandLA-Net surpasses all point-based SemanticKITTI approaches by large margins, improving 15% over the second-best method.It also outperforms most projection-based methods while using fewer parameters.
  • 4.3 Semantic Segmentation on Benchmarks: RandLA-Net improves Toronto3D mIoU from 70.50% to 77.71%, achieving the best result among the compared methods.This comparison uses only 3D coordinates for fair evaluation.
  • 4.4 Ablation Studies: Removing chained spatial embedding and attentive pooling causes the greatest ablation impact by reducing information propagation from roughly K to K^2 neighbouring points.Removing local spatial encoding or attention also diminishes performance, showing that the modules complement one another.
  • 4.4.3 Ablation of Dilated Residual Block: A single aggregation unit limits the receptive field, whereas three units tend to overfit because of larger receptive fields and more trainable parameters.The study therefore identifies an intermediate aggregation depth as preferable.
  • 4.4 Ablation Studies: RandLA-Net’s performance is comparable across random, farthest-point, and policy-gradient sampling, but drops significantly with inverse-density sampling.Inverse-density sampling selects lower-density points that tend to be outliers and noisy in practice.
  • 4.5 Neighbour Searching: Increasing K from 4 to 16 raises mIoU from 46% to 57%, after which larger neighbourhoods gradually reduce the score.This indicates that both very small and extremely large neighbourhoods are suboptimal.

5 DISCUSSION

The discussion identifies scalability and observability boundaries for RandLA-Net. KNN can bottleneck extremely large inputs, while random sampling cannot recover information from missing or occluded regions.

  • Scalability: KNN neighbour search does not scale linearly and may become a major bottleneck for point clouds exceeding 10^6 points.The paper suggests voxel-local search or K-random path selection using KD-Trees or OcTrees as possible alternatives.
  • Missing and Occluded Regions: Random sampling focuses on observed points and cannot produce potentially informative features in missing or occluded regions.The authors propose generating new random point sets as a direction for learning better local patterns.

6 CONCLUSION

The paper presents a lightweight architecture that efficiently and effectively segments large-scale point clouds. It reduces computation and memory through random sampling while using local feature aggregation to preserve useful wide-neighbourhood features.

  • Random sampling significantly reduces the network’s memory footprint and computational cost compared with expensive sampling strategies.
  • A local feature aggregation module preserves useful features from a wide neighbourhood.
  • Extensive experiments on multiple benchmarks demonstrate high efficiency and state-of-the-art performance.
  • The framework could be extended to end-to-end 3D instance segmentation and real-time dynamic point-cloud processing.

APPENDIX A DETAILS FOR THE EVALUATION OF SAMPLING.

The appendix describes four sampling strategies and their implementations for selecting M point features from a large-scale point cloud containing N points. It also details learned sampling with Gumbel-based differentiable selection and policy-gradient optimization.

  • Farthest Point Sampling uses a GPU operator following PointNet++’s implementation.
  • Inverse Density Importance Sampling ranks points by inverse estimated density and selects the top M points.Density is approximated from distances to the nearest t points, with t set to 16.
  • Poisson Disk Sampling uses Fast Poisson Disk Sampling with a specified radius r to select M points from N input points.
  • Random Sampling is implemented with numpy.random.choice() for selecting point features.
  • A learned sampler produces scores s with an MLP, adds Gumbel noise, and uses temperature τ to obtain differentiable sampled feature vectors.As τ approaches zero, the sampling approaches a discrete distribution with probability s(i) for input row P(i).
  • Policy-Gradient Sampling selects the top M scores after adding zero-mean Gaussian exploration noise, and REINFORCE optimizes the sampling policy using segmentation accuracy as reward.Control variates are used to alleviate the high-variance problem of policy gradients.

B.1 Evaluation on ScanNet

The ScanNet evaluation shows satisfactory performance, but RandLA-Net does not achieve the best mean IoU. Its weaker results on several small object categories are attributed to information loss from aggressive random sampling.

  • RandLA-Net achieves satisfactory but not best results on the ScanNet online test set.
  • Its mean IoU is lower than KPConv and SparseConvNet because it performs worse on small categories such as refrigerators and cabinets.
  • Aggressive random sampling may lose major information from small objects.
  • ScanNet is mainly composed of relatively small indoor scenes.

B.2 Evaluation on DALES

The DALES dataset contains large-scale aerial point clouds without color information, and RandLA-Net achieves satisfactory overall segmentation performance. Its mean IoU is 80.0%, while KPConv performs slightly better.

  • DALES contains 505 million labeled points from aerial laser systems covering urban areas spanning 10 km^2.
  • The dataset has eight semantic categories and provides no color information.
  • 80.0% mean IoU is achieved by RandLA-Net on DALES.
  • RandLA-Net performs best on categories including trucks and vegetation, while its overall performance is slightly lower than KPConv.

B.3 Ablation of Voting Scheme

Removing voting reduced RandLA-Net’s overall mIoU by 2%–4% on Toronto-3D, NPM3D, and SemanticKITTI, indicating that voting improves segmentation performance but adds computation.

  • 2%–4%: RandLA-Net’s overall mIoU decreased without voting on Toronto-3D, NPM3D, and SemanticKITTI.The comparison uses only the per-point result from the last inference instead of voting across multiple runs.
  • Voting boosts segmentation performance across Toronto-3D, NPM3D, and SemanticKITTI.
  • Voting may incur extra computation burdens during inference.

B.4 Ablation of Nearest-Neighbor Interpolation

Nearest-neighbor interpolation performed best among the tested upsampling strategies because sparse encoded points can carry different semantic meanings, making multi-point interpolation detrimental.

  • Nearest-neighbor interpolation is selected for simplicity and efficiency.The framework can also integrate weighted interpolation strategies or other learnable layers.
  • Random upsampling is unsuitable because skip connections require features at the same spatial location to remain related.
  • 57.1%: nearest upsampling outperformed trilinear and attentive upsampling in mIoU.The compared alternatives were inferior after the input point clouds had been heavily down-sampled through encoding.
  • Multi-point interpolation can be detrimental when sparse encoded points have different semantic meanings.Selecting only the nearest point is described as more robust in this setting.

B.5 Ablation of Preprocessing

RandLA-Net remains comparable across large-point-cloud preprocessing choices, but excessive block partitioning harms geometric structure and performance; random sampling also introduces run-to-run variation.

  • Preprocessing: 61.63% to 59.10%: performance decreased when block size was reduced to 1m×1m.The paper attributes this decline to excessive block partitions breaking geometrical structures and hindering robust representation learning.
  • Preprocessing: Comparable performance was achieved with grid subsampling and 6m×6m block random sampling, both using 40960 points.
  • Sampling sensitivity: 62.44%: RandLA-Net’s average Area 5 S3DIS mIoU across five runs, with a standard deviation of 0.75.Variations mainly affected minor categories such as sofa, column, board, and door, which contain relatively few points.
  • Model variants: Deeper or wider variants increase model size and training parameters, sacrificing computation efficiency.
  • Model variants: Adding channels slightly improved segmentation, whereas adding layers slightly decreased it.The deeper model had nearly 20 times more parameters, making optimization harder and increasing overfitting risk.
  • Generalization: 1.24M parameters: RandLA-Net has fewer parameters than PointConv (21.7M), KPConv (14.3M), and PointCNN (11.5M).The paper links this lightweight design and random sampling to good generalization without overfitting.
Loading 2107.02389v1…