PaperlayerView exact revision on arXiv

Source-linked AI summary

3D-MiniNet: Learning a 2D Representation from Point Clouds for Fast and Efficient 3D LIDAR Semantic Segmentation

Iñigo Alonso, Luis Riazuelo, Luis Montesano, Ana C. Murillo

arXiv:2002.10893v5cs.CV

TL;DR

LIDAR semantic segmentation needs accurate predictions under the computational and temporal constraints of robotic applications. 3D-MiniNet learns a 2D representation from raw 3D points, segments it with an efficient 2D FCNN, and re-projects the labels to 3D; it achieves state-of-the-art results on SemanticKITTI and KITTI while being faster and more parameter-efficient than prior methods.

  • Problem

    Robotic applications need fast and efficient LIDAR semantic segmentation, but point-based operations are computationally expensive and projection-based approaches do not exploit raw 3D information.

  • Method

    3D-MiniNet learns local and context information in a 2D representation from grouped 3D points, applies a fast 2D fully convolutional network, and re-projects and refines the predicted labels in 3D.

  • Results

    3D-MiniNet achieves state-of-the-art results on SemanticKITTI and KITTI while being faster and more parameter-efficient than previous methods.

  • Takeaways & Limitations

    The learned projection preserves local and context information while enabling efficient 2D segmentation, and ablations assess how the proposed components contribute to performance.

Abstract

from arXiv · show

LIDAR semantic segmentation, which assigns a semantic label to each 3D point measured by the LIDAR, is becoming an essential task for many robotic applications such as autonomous driving. Fast and efficient semantic segmentation methods are needed to match the strong computational and temporal restrictions of many of these real-world applications. This work presents 3D-MiniNet, a novel approach for LIDAR semantic segmentation that combines 3D and 2D learning layers. It first learns a 2D representation from the raw points through a novel projection which extracts local and global information from the 3D data. This representation is fed to an efficient 2D Fully Convolutional Neural Network (FCNN) that produces a 2D semantic segmentation. These 2D semantic labels are re-projected back to the 3D space and enhanced through a post-processing module. The main novelty in our strategy relies on the projection learning module. Our detailed ablation study shows how each component contributes to the final performance of 3D-MiniNet. We validate our approach on well known public benchmarks (SemanticKITTI and KITTI), where 3D-MiniNet gets state-of-the-art results while being faster and more parameter-efficient than previous methods.

I. INTRODUCTION

3D-MiniNet targets fast, efficient LIDAR semantic segmentation by learning a 2D representation from 3D points and processing it with a 2D FCNN. It reports state-of-the-art benchmark performance with fewer parameters and faster execution than prior methods.

  • Accurate, fast LIDAR semantic segmentation is important for autonomous robots performing tasks including SLAM, autonomous driving, and inventory operations.
  • Direct 3D point-based operations are computationally expensive, while projection-based methods are more efficient but do not exploit raw 3D information.
  • 3D-MiniNet learns a 2D representation from the 3D point cloud and computes segmentation with a fast 2D fully convolutional neural network.
  • State-of-the-art results on SemanticKITTI and KITTI are achieved with faster execution and greater parameter efficiency than prior work.
  • The paper extends MiniNet-v2 to 3D LIDAR semantic segmentation and introduces a novel projection module.
  • The approach uses a novel projection module whose context, local, spatial, and fusion submodules learn a richer 2D representation, with an ablation study evaluating their contributions.

II. RELATED WORK

Related work spans point-based methods, 3D representations, and efficient 2D semantic-segmentation architectures. These approaches address unordered point clouds and computational efficiency through specialized operators, representations, and lightweight networks.

  • Fully Convolutional Neural Networks adapted classification CNNs for semantic segmentation by producing per-pixel predictions at the input resolution.
  • Efficient segmentation architectures reduce computation through low-resolution processing, quick downsampling, lightweight decoders, and efficient convolutions.
  • Point-cloud segmentation is commonly approached with point-based methods, 3D representations, or projection-based methods.

1) Point-based Methods:

Point-based and projection-based strategies operate on different representations of raw point clouds, while the proposed overview combines grouped 3D points, learned features, 2D prediction, and post-processing. The central design tension is preserving 3D information while retaining efficient 2D computation.

  • Point-based Methods: Point-based methods operate directly on unordered point clouds using shared MLPs, pooling, hierarchical grouping, or neighborhood-based operations.
  • 3D representations: Voxel and lattice representations encode 3D space before applying 3D or sparse convolutions, but voxel-based 3D-FCNN processing can be slow for real-time applications.
  • Projection-based Methods: Spherical projection maps 3D points into a 2D representation so fast 2D image operations can be used for LIDAR semantic segmentation.
  • Projection-based Methods: Projection-based methods are faster but can lose the ability to learn 3D features, motivating a context feature extractor based on point operations.
  • Proposed approach: The proposed pipeline groups input points through fast neighbor search, predicts one label per point, and refines the segmentation with post-processing.

III. 3D-MININET: LIDAR POINT CLOUD SEGMENTATION

The approach accelerates LIDAR segmentation by finding 3D neighbors through a spherical 2D projection, forming point groups, augmenting their features, and processing them with a 2D representation.

  • 3D-MiniNet uses fast point-neighbor search, a 3D-to-2D learning module, and KNN post-processing to produce a segmented point cloud.
  • 3D point operations are costly, so the method converts raw 3D points into a 2D representation for processing with a 2D CNN.
  • The spherical projection maps each 3D point (x, y, z) to a 2D coordinate (u, v) on a W × H grid.
  • A sliding k × k window creates non-overlapping groups of N = k^2 projected points, using all spherical-projection points with P × N = W × H.
  • Each point is augmented with features relative to its group mean and its 3D Euclidean distance from the mean point.

B. 3D-MiniNet

The projection learning module transforms grouped raw points into a 2D tensor by extracting local, contextual, spatial, and fused features before efficient 2D segmentation.

  • The projection module takes groups of N points with C2 features and transforms them into a 2D representation for efficient segmentation.
  • It extracts and fuses local, context, and spatial features before the final feature-fusion step.
  • Local Feature Extractor: The local extractor applies four shared linear layers followed by BatchNorm and LeakyReLU in a PointNet-like operation.
  • Context Feature Extractor: The context extractor uses max pooling and three 3 × 3 sliding-window groupings with dilation rates 1, 2, and 3 to expand the receptive field.
  • Spatial Feature Extractor: The spatial extractor applies a 1 × N convolution to capture each point’s relationship with neighboring points in the spherical projection.
  • Feature Fusion: Self-attention fuses the extracted features and learns which features are more important while reducing the feature space.

2) 2D Segmentation Module (MiniNet Backbone):

The MiniNet backbone converts the learned tensor into a 2D semantic segmentation using an efficient FCNN with depthwise-separable convolutions and a fine-grained branch.

  • The backbone receives a W/4 × H/4 × C6 tensor and produces the 2D semantic segmentation with an efficient FCNN.
  • Its encoder uses depthwise-separable convolutions, including multi-dilation convolutions, to process the learned representation efficiently.
  • A second convolutional branch extracts fine-grained high-resolution low-level features from the spherical projection.
  • The predicted 2D labels are re-projected into 3D, while points absent from the spherical projection initially lack semantic labels.

C. Post-Processing

Post-processing assigns labels to points not directly covered by the spherical projection and refines the segmentation using depth-aware KNN search.

  • Non-projected 3D points are handled using a post-processing method based on K Nearest Neighbors.
  • KNN selection uses relative depth rather than relative Euclidean distance and is narrowed by 2D spherical-coordinate distances.
  • The implementation is GPU-based and runs in 7ms while maintaining a high frame rate.
  • The evaluation uses SemanticKITTI, with over 43,000 scans and 19 evaluated semantic classes, and KITTI, with 8,057 training and 2,791 validation scans.

B. Settings

The experiments use dataset-specific spherical projections, fast neighbor-search settings, cross-entropy optimization with class balancing, and geometric data augmentation.

  • Projection and neighbor search: SemanticKITTI uses a 2048 × 64 spherical projection, while KITTI uses 512 × 64 to match prior work.A 4 × 4 window with stride 4 and no zero-padding produces 8192 and 2048 point groups, respectively.
  • Post-processing: The K-nearest-neighbor post-processing uses a 7 × 7 2D search window and K=7.
  • Training protocol: Training runs for 500 epochs with SGD, an initial learning rate of 4 · 10^-3, and a 0.99 per-epoch decay.Batch sizes are 3, 6, and 8 for 3D-MiniNet, 3D-MiniNet-small, and 3D-MiniNet-tiny, respectively, due to memory constraints.
  • Training protocol: Optimization uses cross-entropy loss with median-frequency class balancing and smoothed weights using exponent i=0.25.The loss is defined over labeled points and classes, with CNN probabilities obtained through soft-max.
  • Data augmentation: Training augmentation randomly rotates, shifts, reflects, and drops points from the whole 3D point cloud.The rotation angle, axis shifts, and dropped-point proportion are sampled from the specified Gaussian or uniform distributions.

V. RESULTS

The ablation study evaluates how projection-module components affect mIoU, speed, and parameter count, showing benefits from combining complementary point and context features.

  • Projection-module ablation: The projection module is the main novelty and is evaluated with the 3D-MiniNet-small configuration.
  • Projection-module ablation: The ablation measures mIoU, speed, and learning parameters, beginning with spherical projection using convolution as a downsampling baseline.
  • Projection-module ablation: Data augmentation improves generalization because the projection is neither rotation nor shift invariant.
  • Projection-module ablation: Combining convolutions with local MLP operations increases performance by learning different feature types, although MLP operations require more execution time.
  • Projection-module ablation: The attention module improves performance with almost no extra computational effort, while context extraction provides a larger boost than the other feature extractors.Adding group-relative point features also improves performance without reducing frame-rate or adding learning parameters.

B. Benchmarks results

Across SemanticKITTI and KITTI evaluations, 3D-MiniNet provides a strong speed–performance trade-off, with efficient configurations achieving competitive or state-of-the-art results and post-processing improving point coverage.

  • Quantitative results: The quantitative comparison evaluates mIoU, processing speed, and parameter count across point-based, 3D-representation, and projection-based methods.
  • Quantitative results: 3D-MiniNet-small-KNN achieves state-of-the-art results with fewer parameters and higher speed than RandLANet, SPLATNet, and RangeNet53-KNN.
  • Quantitative results: 3D-MiniNet-tiny runs at 98 fps with mIoU of 46.9%, compared with 29% for SqueezeSeg at 90 fps.
  • Quantitative results: Post-processing improves results and is crucial for assigning labels to points omitted from the spherical projection.
  • KITTI results: On KITTI, 3D-MiniNet achieves state-of-the-art performance and exceeds SqueezeSeg by +10-20 mIoU.The KITTI scans have lower resolution, and 3D-MiniNet also outperforms LuNet and DBLiDARNet.
  • KITTI results: KNN post-processing is not evaluated on KITTI because the dataset provides only 2D labels.
  • Qualitative results: Qualitative results show strong performance for cars and traffic signs, while building–fence distinctions remain difficult.

VI. CONCLUSIONS

3D-MiniNet learns a 2D representation from raw 3D point clouds before applying a fully convolutional network for semantic segmentation. It achieves state-of-the-art results on SemanticKITTI and KITTI while being faster and more efficient than previous methods.

  • 3D-MiniNet learns a 2D representation from a 3D point cloud and computes segmentation with a fast 2D fully convolutional neural network.
  • Unlike predefined projection methods, 3D-MiniNet learns the projection from raw 3D points using local and context information.
  • 3D-MiniNet’s ablation study evaluates how each component contributes to learning the representation.
  • 3D-MiniNet achieves state-of-the-art results on the SemanticKITTI and KITTI public benchmarks while being faster and more efficient than previous methods.
Loading 2002.10893v5…