PaperlayerView exact revision on arXiv

Source-linked AI summary

RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds

Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, Andrew Markham

arXiv:1911.11236v3cs.CVcs.LGeess.IV

TL;DR

Large-scale 3D point-cloud segmentation is difficult because existing approaches depend on expensive sampling or preprocessing and often operate only on small clouds. RandLA-Net uses random sampling with local feature aggregation to directly process massive point clouds, achieving high efficiency and state-of-the-art results on two large-scale benchmarks.

  • Problem

    Existing semantic-segmentation methods struggle to process massive point clouds because expensive sampling, graph construction, voxelization, or limited receptive fields hinder scalability.

  • Method

    RandLA-Net combines fast random point sampling with local spatial encoding, attentive pooling, and stacked receptive-field expansion to preserve geometric structures efficiently.

  • Results

    RandLA-Net is up to 200× faster than existing approaches and surpasses state-of-the-art semantic-segmentation methods on Semantic3D and SemanticKITTI.

  • Takeaways & Limitations

    A lightweight architecture using random sampling and local feature aggregation can efficiently and effectively segment large-scale point clouds.

Abstract

from arXiv · show

We study the problem of efficient semantic segmentation for large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing approaches are only able to be trained and operate over small-scale point clouds. In this paper, we introduce RandLA-Net, an efficient and lightweight neural architecture to directly infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Although remarkably computation and memory efficient, random sampling can discard key features by chance. To overcome this, we introduce a novel local feature aggregation module to progressively increase the receptive field for each 3D point, thereby effectively preserving geometric details. Extensive experiments show that our RandLA-Net can process 1 million points in a single pass with up to 200X faster than existing approaches. Moreover, our RandLA-Net clearly surpasses state-of-the-art approaches for semantic segmentation on two large-scale benchmarks Semantic3D and SemanticKITTI.

1. Introduction

RandLA-Net targets efficient, direct semantic segmentation of large-scale 3D point clouds, where existing methods face scaling, computation, and memory challenges. It combines random sampling with local feature aggregation to preserve geometric structure while achieving faster, state-of-the-art segmentation.

  • Motivation: Existing local feature learners generally cannot scale from small point clouds to massive scenes because graph construction, kernelisation, or limited receptive fields create computational and structural constraints.Prior methods often operate on roughly 4k points or 1×1 meter blocks, while large scenes may contain millions of points.
  • Approach: RandLA-Net directly processes large-scale point clouds in a single pass without voxelization, block partitioning, graph construction, or other pre/post-processing.The architecture is designed to fit large point clouds within current GPU memory and computational limits.
  • Approach: Random sampling is selected because it is fast and scales efficiently, although it can discard prominent features by chance.The method addresses this risk with a local feature aggregation module operating over progressively smaller point sets.
  • Approach: The local feature aggregation module uses local spatial encoding, attentive pooling, and stacked dilated residual blocks to enlarge each point’s effective receptive field.These components preserve local geometry and retain useful features using shared MLPs for memory and computational efficiency.
  • Results: RandLA-Net is up to 200× faster than existing approaches and surpasses state-of-the-art semantic segmentation methods on Semantic3D and SemanticKITTI.The paper reports these gains as the overall outcome of combining random sampling with the local feature aggregator.

2. Related Work

Prior point-cloud networks use projection, voxelization, or sophisticated point-based local-feature modules, but large-scale processing remains limited by geometric information loss, computation, or memory demands. RandLA-Net is distinguished by random sampling, progressively larger receptive fields, and end-to-end operation without additional preprocessing.

  • Projection and Voxel Based Networks: Projection-based methods can lose geometric details, while voxel-based networks incur heavy computation costs on large-scale point clouds.Both approaches adapt 3D data to grid-like processing, but their limitations become more pronounced for massive scenes.
  • Point Based Networks: Point-based networks learn local features through neighboring pooling, graph message passing, kernel convolution, or attention-based aggregation.These categories include sophisticated modules designed to improve per-point representation learning.
  • Point Based Networks: Most point-based networks cannot directly scale to large scenarios because their local-feature modules require substantial computation and memory.RandLA-Net instead relies only on random sampling and progressively enlarges receptive fields with its local feature aggregator.
  • Learning for Large-scale Point Clouds: SPG, FCPN, and PCT process large point clouds using graph partitioning or voxelization, but these preprocessing steps are computationally expensive.RandLA-Net is presented as end-to-end trainable without additional pre- or post-processing.

3. RandLA-Net

RandLA-Net combines scalable random sampling with local feature aggregation to process large point clouds while retaining geometric information. Its aggregation module encodes local geometry, selects informative neighboring features, and expands each point’s receptive field through stacked operations.

  • Efficient sampling: Random sampling is constant-time, memory-efficient, and processes 10^6 points in 0.004 seconds, but may discard useful features.FPS, IDIS, GS, CRS, and PGS are described as computationally expensive, memory-intensive, or difficult to learn for large-scale clouds.
  • Local feature aggregation: Local spatial encoding concatenates relative point positions with neighboring features to explicitly represent local geometric structures.The encoded relative positions include coordinate differences and Euclidean distance, and the resulting vectors augment neighboring point features.
  • Local feature aggregation: RandLA-Net uses K-nearest neighbors based on Euclidean distance to gather local neighborhoods for each point.The neighborhood is formed before local spatial encoding and attentive pooling.
  • Local feature aggregation: Attentive pooling aggregates neighboring features by learning weights that automatically select important local information instead of hard max or mean pooling.The learned attention scores act as a soft mask over the neighboring features.
  • Dilated residual block: Two chained LocSE and attentive-pooling units expand each point’s effective neighborhood from K to approximately K^2 points.The dilated residual block increases receptive-field size through feature propagation while using feed-forward MLPs for computational efficiency.

4. Experiments

The experiments evaluate sampling efficiency, end-to-end inference efficiency, benchmark segmentation performance, and the contribution of local feature aggregation. RandLA-Net combines strong efficiency with competitive or state-of-the-art segmentation results across large-scale datasets.

  • Efficiency of Random Sampling: For ∼10^6-point clouds, random sampling has superior overall time and memory efficiency, whereas FPS, IDIS, GS, CRS, and PGS are extremely time-consuming or memory-costly.At ∼10^3 points, the approaches have similar resource consumption; the gap emerges at large scale.
  • Efficiency of RandLA-Net: 185 seconds averaged over 4071 frames, or roughly 22 FPS, is RandLA-Net’s inference time on SemanticKITTI Sequence 08 for clouds up to 10^6 points.PointNet and KPConv cannot process extremely large-scale clouds in a single pass because of memory-inefficient operations.
  • Semantic Segmentation on Benchmarks: RandLA-Net outperforms all existing methods on Semantic3D in both mIoU and OA, with superior performance on six of eight classes.The exceptions are low vegetation and scanning art.
  • Semantic Segmentation on Benchmarks: On SemanticKITTI, RandLA-Net surpasses point-based approaches by a large margin and outperforms projection-based methods, while using 40× fewer parameters than RangeNet++.Its computational efficiency also avoids costly pre/post projection steps.
  • Semantic Segmentation on Benchmarks: On S3DIS, RandLA-Net achieves on-par or better performance than state-of-the-art methods while taking entire rooms as input instead of small blocks.The baselines commonly use sophisticated operations or samplings on small point-cloud blocks.
  • Ablation Study: Removing chained spatial embedding and attentive pooling has the greatest impact, while removing local spatial encoding or attention also diminishes performance.Two chained blocks propagate information from approximately K^2 points rather than K, which is especially important under random sampling.

5. Conclusion

The conclusion presents RandLA-Net as a lightweight architecture that efficiently and effectively segments large-scale point clouds. It attributes this combination to random sampling and local feature aggregation, and reports high efficiency with state-of-the-art performance across multiple benchmarks.

  • Conclusion: RandLA-Net demonstrates efficient and effective semantic segmentation of large-scale point clouds with a lightweight network architecture.The conclusion contrasts this with approaches relying on expensive sampling strategies.
  • Conclusion: Random sampling reduces memory footprint and computational cost, while local feature aggregation preserves useful features from a wide neighbourhood.These components form the basis of the proposed framework’s efficiency and feature preservation.
  • Conclusion: Experiments on multiple benchmarks demonstrate high efficiency and state-of-the-art performance.The paper identifies end-to-end 3D instance segmentation and real-time dynamic point-cloud processing as future extensions.

A. Details for the Evaluation of Sampling.

The evaluation compares six sampling strategies, including random sampling, using implementations and procedures designed for progressively downsampling large-scale point clouds. It also describes the score-based, continuous-relaxation, and policy-gradient alternatives.

  • Sampling Setup: The evaluation samples K point features from a large-scale point cloud P containing N points or feature vectors.The implementation details introduce the sampling procedures evaluated in Section 4.1.
  • Sampling Approaches: FPS selects points using the PointNet++ GPU implementation, while IDIS ranks points by inverse local density and retains the top K.For IDIS, density uses distances to the nearest t points, with t set to 16.
  • Sampling Approaches: Random sampling generates K indices with numpy.random.choice() and gathers the corresponding coordinates and per-point features.GS instead orders points with a trained ProgressiveNet and keeps the first K points.
  • Sampling Approaches: CRS predicts a categorical score distribution with an MLP, adds Gumbel noise, and uses continuous relaxation to calculate sampled feature vectors.The sampled vector is a weighted combination of input rows, with annealing temperature τ controlling the relaxation toward discrete sampling.
  • Sampling Approaches: PGS predicts point scores with an MLP, samples top-scoring vectors as sequential actions, and optimizes the sampling policy with REINFORCE using segmentation accuracy as reward.Control variates are used to alleviate the high-variance problem of policy gradients.
  • RandLA-Net Architecture: RandLA-Net’s architecture is organized around local feature aggregation, random sampling, shared MLPs, up-sampling, and dropout.Figure 7 defines (N, D) as the number of points and feature dimension, respectively.

B. Details of the Network Architecture

RandLA-Net uses an encoder-decoder architecture to process large-scale point clouds, progressively reducing points while increasing feature dimensions, then restoring pointwise predictions through interpolation and skip connections.

  • The network uses four encoding and four decoding layers with skip connections to learn and reconstruct per-point features.The input first passes through a shared MLP; final predictions use three fully connected layers and dropout.
  • Each encoding layer combines local feature aggregation with random sampling, retaining 25% of points at every stage.The point count follows N → N/4 → N/16 → N/64 → N/256.
  • Feature dimensions increase from 8 to 32 to 128 to 256 to 512 as the point cloud is downsampled.
  • Decoder layers use nearest-neighbor interpolation, concatenate encoder features through skip connections, and apply shared MLPs.
  • The final prediction head maps features through (N, 64) → (N, 32) → (N, nclass), followed by dropout with ratio 0.5.

C. Additional Ablation Studies on LocSE

LocSE ablations show that explicitly encoding complete spatial information, especially relative positions, is important for semantic segmentation performance.

  • The experiments compare encoding the center point coordinates, neighboring coordinates, both sets, and additional spatial combinations.
  • Explicitly encoding all spatial information produces the best mIoU among the LocSE ablations on SemanticKITTI.
  • Relative position p_i − p_k is important because it makes the network aware of local geometric patterns.

D. Additional Ablation Studies on Dilated Residual Block

The dilated residual block balances receptive-field growth and model capacity: one aggregation unit is insufficient, while three can overfit without improving accuracy.

  • The standard block stacks two LocSE and attentive-pooling units to progressively increase each point’s receptive field.
  • One aggregation unit causes a significant segmentation-performance drop because its receptive field is limited.
  • Three aggregation units do not improve accuracy because their larger receptive fields and additional trainable parameters tend to overfit.
  • Attention weights are larger in early encoding layers and become smoother and more stable in later layers.

F. Additional Results on Semantic3D

Additional qualitative results of RandLA-Net are presented on the Semantic3D reduced-8 dataset.

  • Figure 9 provides further qualitative RandLA-Net results on the Semantic3D reduced-8 dataset.

G. Additional Results on SemanticKITTI

On SemanticKITTI, RandLA-Net’s validation failures concentrate in visually similar colorless classes and severely underrepresented categories. Qualitative results mark these errors explicitly.

  • Other-vehicle points are often misclassified as car when partial point clouds lack color cues.The two classes are difficult to distinguish under these inputs.
  • RandLA-Net struggles with minority classes including bicycle, motorcycle, bicyclist, and motorcyclist.The reported imbalance includes 7000 times more vegetation points than motorcyclist points.
  • Figure 10 presents qualitative SemanticKITTI validation results and marks failure cases with red boxes.
Loading 1911.11236v3…