PaperlayerView exact revision on arXiv

Source-linked AI summary

PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation

Yang Zhang, Zixiang Zhou, Philip David, Xiangyu Yue, Zerong Xi, Boqing Gong, Hassan Foroosh

arXiv:2003.14032v2cs.CV

TL;DR

Online single-scan LiDAR segmentation must provide fine-grained labels under near-real-time constraints despite uneven point distributions and increasingly fine-grained classes. PolarNet uses a polar BEV representation with learned per-cell features and ring convolutions, improving mIoU across three datasets with lower computational cost. The method’s scope remains constrained by extreme rarity and ambiguity of the SemanticKITTI motorcyclist class.

  • Problem

    Online single-scan LiDAR segmentation requires fine-grained point labels under restrictive real-time processing conditions and uneven spatial point distributions.

  • Method

    PolarNet quantizes points into polar BEV grids, encodes each cell with a trainable PointNet, and processes the resulting grid with ring convolutions.

  • Results

    PolarNet outperforms state-of-the-art methods by 2.1%, 4.5% and 3.7% mIoU on SemanticKITTI, A2D2 and Paris-Lille-3D, respectively, using merely 1/3 of their parameters and MACs.

  • Takeaways & Limitations

    Polar BEV yields universal improvement across different segmentation networks compared with spherical projection and Cartesian BEV, while maintaining fewer parameters, higher throughput, and lower inference latency.

  • Takeaways & Limitations

    SemanticKITTI’s motorcyclist class is extremely rare and difficult to distinguish from person or bicyclist because the motorcycle is often largely occluded.

Abstract

from arXiv · show

The need for fine-grained perception in autonomous driving systems has resulted in recently increased research on online semantic segmentation of single-scan LiDAR. Despite the emerging datasets and technological advancements, it remains challenging due to three reasons: (1) the need for near-real-time latency with limited hardware; (2) uneven or even long-tailed distribution of LiDAR points across space; and (3) an increasing number of extremely fine-grained semantic classes. In an attempt to jointly tackle all the aforementioned challenges, we propose a new LiDAR-specific, nearest-neighbor-free segmentation algorithm - PolarNet. Instead of using common spherical or bird's-eye-view projection, our polar bird's-eye-view representation balances the points across grid cells in a polar coordinate system, indirectly aligning a segmentation network's attention with the long-tailed distribution of the points along the radial axis. We find that our encoding scheme greatly increases the mIoU in three drastically different segmentation datasets of real urban LiDAR single scans while retaining near real-time throughput.

1. Introduction

PolarNet addresses online fine-grained LiDAR segmentation by reshaping the input representation around LiDAR’s uneven ring structure. Its polar BEV encoding and learned grid features improve segmentation across three datasets while retaining low computational cost.

  • Motivation: Online LiDAR segmentation must support automatic, fast point-wise labeling as large point-cloud datasets and semantic labels expand.The task is especially relevant to self-driving applications, where processing resources are restrictive.
  • Motivation: Cartesian BEV grids distribute LiDAR points unevenly, blurring dense near-sensor details while leaving distant cells too sparse for reliable labeling.Mixed labels within cells can also suppress minority-point predictions at voxel level.
  • Method: PolarNet partitions scans into polar BEV grids so CNN perception fields track LiDAR ring structures and account for imbalanced spatial point distributions.The representation is designed to balance points across grid cells and improve feature representativeness.
  • Method: The method learns fixed-length local feature vectors per grid cell with a simplified PointNet rather than handcrafting local features.It redesigns BEV quantization by assigning points according to top-down polar coordinates.
  • Results: PolarNet improves mIoU over state-of-the-art methods by 2.1%, 4.5% and 3.7% on SemanticKITTI, A2D2 and Paris-Lille-3D, respectively.These results use merely 1/3 of the compared method’s parameters and MACs.
  • Results: The work analyzes polar-grid performance across different segmentation backbones and compares it with spherical projection and Cartesian BEV.The proposed PolarNet network is reported to surpass state-of-the-art methods with low computational cost.

2. Related Works

Prior LiDAR and point-cloud methods use projection, graph, set-function, or volumetric representations to extract context from irregular points. LiDAR applications additionally require sparse-data processing with real-time performance on restrictive embedded hardware.

  • Point cloud applications and methods: General point-cloud applications extract local or global context from irregularly distributed points using parameterized or non-parameterized aggregation.Examples include object recognition, indoor segmentation, and outdoor-scene reconstruction.
  • Point cloud applications and methods: PointNet processes points individually and aggregates their context with a set function, while graph-based methods model points through KNN connectivity.Other approaches voxelize points before applying 3D volume segmentation or detection.
  • LiDAR applications and methods: LiDAR point clouds are spatially sparse, and self-driving systems often impose restrictive processing resources that require real-time embedded-hardware performance.These constraints motivate specialized representations and efficient segmentation methods.
  • LiDAR applications and methods: Spherical projections support lightweight 2D semantic segmentation networks such as SqueezeSeg and SqueezeSegV2, which achieve real-time performance.Depth-map and viewing-frustum approaches also project 3D points onto 2D image grids.
  • LiDAR applications and methods: Online LiDAR semantic-segmentation datasets are relatively rare because point-wise LiDAR annotation is difficult and less intuitive than RGB-image annotation.The paper identifies the Audi, Paris-Lille-3D, and SemanticKITTI datasets as the three available datasets at the time described.

3. Approach

PolarNet represents single-scan LiDAR with polar BEV grids that better match the scan’s ring structure and uneven point distribution. It encodes each cell with a fixed-length learned feature, processes the resulting ring matrix with ring convolutions, and decodes predictions back to points.

  • 3.4. Learning the Polar Grid: The model quantizes points, encodes cells into a ring matrix, predicts on that matrix with a ring CNN, and decodes quantized predictions to the point domain.This end-to-end pipeline is summarized in the model overview.
  • 3.2. Bird’s-eye-view Partitioning: Polar partitioning replaces Cartesian quantization by assigning points according to azimuth and radius around the sensor.The representation is designed around LiDAR’s ring-like top-down structure.
  • 3.3. Polar Bird’s-eye-view: 0.7 ± 1.4 points per polar grid cell versus 0.7 ± 3.2 for Cartesian BEV indicates a more even point distribution.The comparison is reported on the SemanticKITTI validation split with the same number of grid cells.
  • 3.3. Polar Bird’s-eye-view: 99.3% of points in polar cells share one label versus 98.75% in Cartesian cells, raising the majority-label mIoU upper bound from 97.3% to 98.5%.The authors connect this higher upper bound to reduced representation-induced misclassification, especially for small objects.
  • 3.4. Learning the Polar Grid: A simplified PointNet followed by max-pooling transforms the points in each grid cell into a fixed-length representation.The learned feature represents the entire vertical column because the input is not quantized along the z-axis.
  • 3.4. Learning the Polar Grid: Ring convolution treats the polar feature matrix as connected at its azimuthal boundaries, allowing information and gradients to pass across the wraparound.A ring CNN replaces ordinary 2D convolutions for polar-grid processing.

4. Experiments

The experiments evaluate PolarNet on three LiDAR datasets with substantially different scan characteristics and spatial coverage. Dataset-specific Cartesian and polar spatial ranges are selected to include more than 99% of points on average.

  • 4.1. Datasets: The evaluation uses SemanticKITTI, A2D2, and Paris-Lille-3D datasets.These datasets provide distinct urban LiDAR scan settings for validation.
  • 4.1. Datasets: SemanticKITTI contains 43,551 scans, averages 104,452 points per scan, and includes 19 classes with severe class imbalance.Vegetation has 4.82 × 10^7 times more points than motorcyclist.
  • 4.1. Datasets: A2D2 provides 38-class annotations from five asynchronous LiDAR sensors whose partial, non-horizontal scanlines create heterogeneous reconstructed views.The dataset’s generated scans may be incomplete because sensors are not always available.
  • 4.1. Datasets: Paris-Lille-3D supplies three aggregated point clouds from continuous scans collected with one tilted rear-mounted Velodyne HDL-32E.Individual scans are extracted every 50 ms using point timestamps and scanner trajectory.
  • 4.1. Datasets: Cartesian and polar BEV ranges are fixed separately for each dataset to include more than 99% of points per scan on average.The selected spatial extents differ across SemanticKITTI, A2D2, and Paris-Lille-3D.

4.2. Baselines and Metric

The experiments compare PolarNet with spherical, point-based, and other point-cloud segmentation approaches using both segmentation quality and computational measures. Reported metrics include accuracy, per-class IoU, mIoU, latency, FPS, MACs, and parameter count.

  • Baselines and Metric: SqueezeSeg converts LiDAR segmentation into 2D spherical-image segmentation, while RangeNet++ adds KNN-based postprocessing.The related methods establish spherical projection and point-label refinement baselines.
  • Baselines and Metric: PointNet processes points individually before max-pooling a global representation, whereas PointNet++ adds hierarchical pooling and context representation.These methods represent point-based alternatives to grid-based processing.
  • Baselines and Metric: TangentConv applies tangent convolutions to surface geometry, while RandLA uses a local feature aggregation module for large-scale point-cloud segmentation.Both are included as alternative point-cloud segmentation approaches.
  • Baselines and Metric: mIoU is the mean over semantic classes of class intersection over union.For each class, IoU divides prediction-ground-truth intersection by their union.
  • Baselines and Metric: The evaluation also reports single-scan latency, maximum FPS, average MACs per scan, and model parameter count.Latency and throughput are averaged over the validation split on the same GPU.

4.3. SemanticKITTI Segmentation Experiment

On SemanticKITTI, PolarNet’s BEV approaches improve segmentation across most classes, particularly for irregular and spatially sparse classes, while retaining lower computational demands than the state-of-the-art comparison. Performance remains weak for visually ambiguous rare classes such as motorcyclist.

  • 4.3. SemanticKITTI Segmentation Experiment: PolarNet’s BEV approach outperforms the state-of-the-art method with fewer parameters and lower latency on SemanticKITTI.Table 1 compares the proposed approaches with multiple baselines on the test split.
  • 4.3. SemanticKITTI Segmentation Experiment: Improvements occur in most classes, especially those that are irregular and sparsely distributed in space.The authors relate this pattern to polar BEV’s scale- and range-preserving properties.
  • 4.3. SemanticKITTI Segmentation Experiment: “Other-ground” and “motorcyclist” receive particularly low performance because they are visually difficult to distinguish from similar classes.The motorcycle is often largely occluded, complicating distinction from person or bicyclist.
  • 4.3. SemanticKITTI Segmentation Experiment: Motorcyclist constitutes 0.004% of training points, and only one instance appears in the official validation sequence.This makes the class both rare and difficult to evaluate robustly.

4.4. A2D2 Segmentation Experiment

On A2D2, PolarNet outperforms other baselines in both mIoU and speed, although the dataset remains challenging for LiDAR-only segmentation.

  • 23% mIoU is achieved by PolarNet using only LiDAR data on A2D2, compared with 54% on SemanticKITTI.The authors characterize A2D2 as challenging because multiple classes receive near-zero IoU for both baselines and proposed methods.
  • PolarNet doubles IoU for multiple A2D2 classes, including bicycle, pedestrian, small-vehicle, and traffic-light.Improvements are also reported for sidebars, signal corpus, parking area, and dash-line.
  • PolarNet outperforms other baselines on both mIoU and speed in the A2D2 test results.

4.5. Paris-Lille-3D Segmentation Experiment

On Paris-Lille-3D, PolarNet improves mIoU over DarkNet53, with gains varying substantially across semantic classes and spatial distributions.

  • 3.7% higher mIoU is achieved by PolarNet than DarkNet53 on Paris-Lille-3D.
  • PolarNet greatly improves barrier segmentation because barriers are mostly far away from the vehicle.
  • Cartesian Unet performs better on trash can, which has very few samples in training and validation.
  • Polar BEV is evaluated across segmentation backbones and projection methods, including spherical projection, Cartesian BEV, and polar BEV.The comparison examines how projection choices affect models’ segmentation performance.

4.7. Augmenting LiDAR Segmentation

The ablation study finds that fixing the BEV volume space provides the largest validation mIoU improvement among the tested training and representation settings.

  • 2.8% mIoU improvement comes from fixing the BEV volume space, the largest gain among the tested settings.The authors describe this change as making scale invariant in each scan.
  • The baseline is a polar BEV Unet with grid size [256, 256, 32].
  • The study evaluates ring convolution, nine input features, flip augmentation, fixed volume space, and tuned grid size.

4.8. mIoU vs. Distance to Sensor

mIoU decreases as point distance from the sensor increases, while polar BEV performs particularly well at close range where points are more evenly distributed and more numerous.

  • mIoU decreases simultaneously as distance from the sensor increases because distant points are rarer and more spatially separated.
  • The distance analysis sorts validation predictions by sensor distance and computes mIoU across distance ranges.
  • Polar BEV achieves higher close-range mIoU than Cartesian BEV through more evenly distributed points across its grid representation.Closer points constitute the majority of points in a scan.

5. Conclusion

PolarNet introduces polar bird’s-eye-view grids with trainable PointNet encoding to address LiDAR’s long-tailed spatial distribution. It improves mIoU across three datasets while using fewer parameters, higher throughput, and lower inference latency.

  • PolarNet quantizes LiDAR points into polar BEV grids and encodes each grid with a trainable PointNet.The representation is designed for online, single-scan semantic segmentation and addresses long-tailed spatial point distributions.
  • PolarNet significantly improves mIoU over state-of-the-art methods on SemanticKITTI, A2D2, and Paris-Lille-3D.
  • The network achieves these improvements with fewer parameters, more throughput, and lower inference latency.
  • Polar BEV produces universal improvement across different segmentation networks compared with spherical projection and Cartesian BEV.This supports the representation as a general LiDAR point-cloud encoding for online semantic segmentation.
Loading 2003.14032v2…