PaperlayerView exact revision on arXiv

Source-linked AI summary

Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds

Yifan Zhang, Qingyong Hu, Guoquan Xu, Yanxin Ma, Jianwei Wan, Yulan Guo

arXiv:2203.11139v1cs.CVcs.RO

TL;DR

Efficient LiDAR detection is challenging because point-based pipelines spend resources on task-agnostic samples even though foreground points matter most. IA-SSD uses learnable instance-aware downsampling, contextual centroid perception, and an encoder-only single-stage architecture, achieving competitive accuracy and 80+ FPS on KITTI. Its reported scope includes three detection benchmarks, while the supplied limitation evidence concerns comparison-result reporting for pedestrians and cyclists.

  • Problem

    Existing point-based LiDAR detectors use task-agnostic random or farthest-point sampling, despite foreground points being more important for object detection.

  • Method

    IA-SSD is a single-stage point-based detector using class-aware and centroid-aware learnable downsampling, contextual centroid perception, and an encoder-only architecture.

  • Results

    IA-SSD demonstrates competitive accuracy and superior efficiency across three detection benchmarks, reaching 83 FPS on KITTI with a single RTX2080Ti GPU.

  • Takeaways & Limitations

    Instance-aware sampling preserves important foreground information, while contextual points improve centroid prediction and detection performance.

  • Takeaways & Limitations

    For pedestrians and cyclists, fair comparison requires reporting published results, best reproduced results, and OpenPCDet implementation results.

Abstract

from arXiv · show

We study the problem of efficient object detection of 3D LiDAR point clouds. To reduce the memory and computational cost, existing point-based pipelines usually adopt task-agnostic random sampling or farthest point sampling to progressively downsample input point clouds, despite the fact that not all points are equally important to the task of object detection. In particular, the foreground points are inherently more important than background points for object detectors. Motivated by this, we propose a highly-efficient single-stage point-based 3D detector in this paper, termed IA-SSD. The key of our approach is to exploit two learnable, task-oriented, instance-aware downsampling strategies to hierarchically select the foreground points belonging to objects of interest. Additionally, we also introduce a contextual centroid perception module to further estimate precise instance centers. Finally, we build our IA-SSD following the encoder-only architecture for efficiency. Extensive experiments conducted on several large-scale detection benchmarks demonstrate the competitive performance of our IA-SSD. Thanks to the low memory footprint and a high degree of parallelism, it achieves a superior speed of 80+ frames-per-second on the KITTI dataset with a single RTX2080Ti GPU. The code is available at \url{https://github.com/yifanzhang713/IA-SSD}.

1. Introduction

3D LiDAR object detection must recognize and localize objects in geometrically complex, non-uniform point clouds. IA-SSD addresses efficiency and point-selection limitations with task-oriented sampling, contextual centroid perception, and an efficient single-stage design.

  • 3D LiDAR detection predicts 3D bounding boxes with location, size, orientation, and class labels despite complex geometry and non-uniform density.
  • Projection- and voxel-based pipelines introduce quantization error, while direct point-based methods retain raw points but incur high computation, memory cost, and limited detection performance.
  • Heuristic sampling can discard important foreground points before bounding-box regression, particularly limiting detection of small objects such as pedestrians.
  • IA-SSD uses class-aware and centroid-aware learnable sampling to preserve foreground points, plus contextual centroid perception for instance-center regression.
  • IA-SSD is an efficient bottom-up single-stage detector that reaches 83 FPS and supports up to 100 parallel inference frames on one RTX2080Ti GPU on KITTI.
  • Experiments report accurate, efficient multi-class LiDAR detection across several large-scale datasets, alongside analysis of memory footprint and inference speed.

2. Related Work

3D detection methods use voxel, point, or joint point-voxel representations, each balancing quantization, structural information, memory access, and efficiency. IA-SSD is a single-stage point-based detector designed to improve accuracy and runtime through instance-aware sampling and contextual centroid perception.

  • Voxel-based detectors regularize point clouds into voxel grids, but their computational and memory costs can grow cubically with input resolution.
  • Single-stage voxel-based detectors are efficient but can lose spatial resolution and structural information, especially for small objects with sparse points.
  • Voxelization introduces quantization loss, while compensatory modules can reduce final detection efficiency and optimal voxel resolution is difficult to determine.
  • Point-based methods learn geometry directly from unstructured clouds and generate object proposals using point-wise features aggregated by symmetric functions.
  • Point-based detectors avoid voxelization but face insufficient learning capacity, irregular sparse data access, poor memory locality, and limited efficiency.
  • Point-voxel methods combine point and voxel representations to address point-based access limitations and voxel-based quantization loss.
  • IA-SSD targets simultaneous accuracy and runtime improvements using instance-aware sampling and contextual instance centroid perception in a single-stage point-based pipeline.

3. The Proposed IA-SSD

IA-SSD uses task-oriented instance-aware downsampling to preserve informative foreground points while reducing computation, then predicts instance centers and proposals from contextual features. Its design combines class-aware and centroid-aware sampling with contextual centroid perception in an encoder-only pipeline.

  • IA-SSD Architecture: The IA-SSD pipeline applies Set Abstraction layers, instance-aware downsampling, centroid perception, centroid-based aggregation, and proposal generation before 3D non-maximum suppression.The final head predicts bounding boxes with class, location, scale, and orientation information.
  • Motivation: Task-agnostic downsampling can discard foreground points, leaving too few points for reliable detection of small objects such as pedestrians and cyclists.Random sampling loses instance recall substantially, while D-FPS and Feat-FPS retain more points early but still fail at the final encoding layer.
  • Instance-aware Downsampling: Class-aware sampling uses point-wise semantic supervision to retain the top k points with the highest foreground scores during inference.Additional MLP branches estimate semantic categories from latent point features, and the selected points become representatives for subsequent encoding layers.
  • Instance-aware Downsampling: Centroid-aware sampling weights points nearer instance centers more heavily through a soft point mask, favoring representative points for center estimation.The mask is multiplied with the foreground loss, and inference retains the top k highest-scoring points without requiring bounding boxes.
  • Contextual Instance Centroid Perception: Contextual centroid prediction estimates offsets to instance centers and aggregates votes from representative points inside expanded bounding boxes.The method uses surrounding context near objects rather than only points or shifted points within the original bounding boxes.

4. Experiments

Experiments evaluate IA-SSD across KITTI, Waymo, and ONCE for detection accuracy, efficiency, and component effectiveness. The results show competitive or superior performance alongside fast, memory-conscious inference, with instance-aware sampling and contextual centroid perception benefiting especially small objects.

  • Efficiency: 85 FPS on a single NVIDIA RTX 2080Ti with an Intel I9-10900X CPU demonstrates IA-SSD’s inference efficiency.
  • KITTI: IA-SSD achieves the best cyclist detection performance on KITTI and outperforms PointRCNN and 3DSSD for car detection.Car mAP improvements over PointRCNN are (1.91%, 4.68%, 4.4%), and over 3DSSD are (0.51%, 0.75%, 0.55%).
  • Waymo: On Waymo, IA-SSD achieves significantly better pedestrian and cyclist detection than strong baselines, but slightly underperforms voxel methods on vehicles.The authors attribute the vehicle gap to the complicated 3D-size distribution of instances.
  • ONCE: On ONCE, IA-SSD yields competitive performance among all baselines, supporting its application to large-scale, complex LiDAR scenes.
  • Ablation Experiments: Instance-aware sampling achieves the best detection performance across all three KITTI categories, especially for pedestrians and cyclists.Centroid-aware sampling performs better for pedestrians and cars but can miss distal geometric details on elongated cyclists.
  • Ablation Experiments: Replacing contextual centroid perception with vanilla or original center assignment decreases detection performance, particularly for small objects.The study also finds category-dependent effects from extending bounding-box context by 2× size or +1.0m.

5. Conclusion

The paper concludes that IA-SSD provides efficient and accurate point-based LiDAR detection through instance-aware foreground selection and contextual centroid perception. It also identifies class-imbalance sensitivity in semantic point predictions as a limitation.

  • IA-SSD uses learning-based instance-aware downsampling to select sparse foreground instance points and contextual centroid perception to exploit geometry around bounding boxes.
  • Experiments on three detection benchmarks demonstrate superior efficiency and accuracy for IA-SSD.
  • Limitations: Instance-aware sampling depends on semantic prediction for each point and is susceptible to class-imbalance distributions.

A. Details of The Proposed IA-SSD

IA-SSD uses a lightweight encoder-only architecture with point-based sampling, centroid perception, and dual-branch detection heads. Its architecture scales sampled points and contextual boundaries for larger datasets.

  • The lightweight backbone contains three Set Abstraction layers using two radii for spherical neighbor queries.
  • The architecture defines sampled-point counts, grouping radii, grouping-point counts, and feature dimensions through the SA notation.
  • The contextual instance centroid perception module uses an MLP mapping 256-dimensional features to 3-dimensional outputs.
  • Centroid-based instance aggregation combines Set Abstraction with an MLP that maps 1536 dimensions to 512.
  • The final detection head has separate classification and regression branches, ending in outputs of 3 and 30 dimensions respectively.
  • For Waymo and ONCE, sampled-point counts increase to 16384, 4096, 2048, and 1024, while the centroid-perception boundary increases to 2.0m.

B. Additional Implementation Details

Training uses scene-level and object-level augmentation, followed by end-to-end optimization. During inference, IA-SSD processes raw point clouds in one pass and applies 3D-NMS.

  • Data augmentation: Scene-level augmentation randomly flips scenes with 50% probability, rotates them around the z-axis, and scales them by factors from 0.95 to 1.05.
  • Data augmentation: Object-level augmentation copies 20 cars, 15 pedestrians, and 15 cyclists from other scenes, with at least 5 points per sampled instance.
  • Training and inference: The model is trained end-to-end for up to 80 epochs using Adam, one-cycle learning, batch size 8, and learning rate 0.01.
  • Training and inference: During inference, IA-SSD generates proposals for all objects from raw point clouds in one forward pass before 3D-NMS filtering.

C. Additional Experimental Results

Additional experiments link foreground-point retention to detection quality, especially for sparse small objects, while evaluating efficiency, benchmark performance, and qualitative robustness across datasets.

  • Higher instance recall correlates with better final detection performance, especially for pedestrians and cyclists with few points.
  • Instance-aware sampling at early encoding layers can hurt performance, whereas applying it at the last two layers significantly improves detection.
  • Instance-aware sampling uses less time and memory than Feat-FPS, contributing to a higher inference frame rate.
  • IA-SSD achieves superior detection performance over other point-based baselines on the KITTI validation set using one multi-class model.
  • IA-SSD maintains satisfactory real-time performance on the large-scale Waymo and ONCE datasets.
  • Visualizations show that instance-aware sampling preserves more foreground points, particularly for small, sparse, and distant instances.
  • Contextual centroid perception uses information outside ground-truth boxes to support regression when downsampled point clouds are sparse.
  • Qualitative results report satisfactory KITTI performance, including multi-class detection in a single pass, and promising results in challenging Waymo and ONCE scenes.

D. Potential Negative Societal Impact

The paper reports low-cost object detection for autonomous driving using open-sourced datasets and identifies no known potential negative societal impact.

  • IA-SSD is presented as a low-cost object-detection solution for autonomous-driving scenarios trained and evaluated on open-sourced datasets.
  • The authors report no known potential negative impact on society.

E. Video Illustration

The paper supplements its evaluation with a video demo and qualitative visualizations of IA-SSD across KITTI, Waymo, and ONCE scenes.

  • A video demo illustrates IA-SSD’s detection performance directly in 3D point clouds.The demo is available at https://youtu.be/3jP2o9KXunA.
  • Figure 4 compares downsampled point clouds from D-FPS, F-FPS, and instance-aware sampling.Raw points are white, representative points gold, and positive representative points red.
  • Figure 5 visualizes contextual centroid perception on KITTI validation scenes using representative points, predicted centroids, and offsets.Representative points and predicted centroids are gold and red, respectively; offsets inside and around objects use red and gold.
  • Figures 6–9 show qualitative IA-SSD detections on KITTI, Waymo, and ONCE, including challenging scenes and projected 3D boxes.Ground-truth and predicted boxes use dataset-specific colors, while KITTI test visualizations also mark centroids and 256 representative points.
Loading 2203.11139v1…