Source-linked AI summary
Fast Point R-CNN
Yilun Chen, Shu Liu, Xiaoyong Shen, Jiaya Jia
TL;DR
3D LiDAR detection must handle sparse, irregular point clouds while balancing localization fidelity and computational cost. Fast Point R-CNN combines voxel-based convolutional proposals with raw-point refinement using attention-fused coordinates and convolutional features. On KITTI, it reports state-of-the-art BEV and 3D detection with 15FPS detection speed.
Problem
Sparse, irregular LiDAR point clouds are difficult for CNNs, while voxelization and direct whole-cloud processing respectively lose localization detail or require heavy computation.
Method
A two-stage detector uses economical voxel-based convolutions for initial predictions and a lightweight PointNet to refine them from raw points and fused convolutional features.
Results
15FPS detection achieves state-of-the-art performance for BEV and 3D detection on KITTI.
Takeaways & Limitations
Combining voxel and raw point-cloud representations provides a fast detector that is on par with existing solutions while maintaining higher detection speed.
Takeaways & Limitations
Direct point-based processing over large autonomous-driving point clouds requires huge GPU memory and computation, limiting high detection speed.
Abstract
from arXiv · showhide
We present a unified, efficient and effective framework for point-cloud based 3D object detection. Our two-stage approach utilizes both voxel representation and raw point cloud data to exploit respective advantages. The first stage network, with voxel representation as input, only consists of light convolutional operations, producing a small number of high-quality initial predictions. Coordinate and indexed convolutional feature of each point in initial prediction are effectively fused with the attention mechanism, preserving both accurate localization and context information. The second stage works on interior points with their fused feature for further refining the prediction. Our method is evaluated on KITTI dataset, in terms of both 3D and Bird's Eye View (BEV) detection, and achieves state-of-the-arts with a 15FPS detection rate.
1. Introduction
Fast Point R-CNN addresses the computational and localization challenges of LiDAR 3D detection with a two-stage framework combining voxel and raw point-cloud representations. It achieves high-quality BEV and 3D detection at 15FPS.
- Challenges: LiDAR point clouds are sparse and irregular, making them difficult for CNNs, while voxelization introduces quantization artifacts and weakens precise localization.Coarse grids and successive convolutional or subsampling operations can discard fine-grained information.
- Challenges: Direct point-cloud processing preserves localization information but is computationally heavy for large autonomous-driving scenes.Processing entire point clouds can require substantial GPU memory and computation, limiting detection speed.
- Approach: The proposed framework combines voxel representation and raw dense point-cloud input in a fast two-stage detector.VoxelRPN uses economical convolutional layers to produce a small number of initial predictions, followed by refinement on raw points.
- Approach: Attention fuses each interior point’s coordinates with convolutional features for box refinement, retaining localization and context information.The second stage uses a lightweight PointNet, and the fused features make each point aware of contextual information.
2. Related Work
Related work represents LiDAR point clouds mainly through voxelization or raw points, while other detectors fuse LiDAR with images or use view-based representations. These approaches trade computational tractability, localization fidelity, and multimodal information differently.
- 3D Data Representation: Point-cloud representation methods primarily use voxelization or directly process raw points, with PointNet pioneering learned representations from raw point sets.Some methods combine PointNet descriptors within 3D grids and then apply 3D convolution.
- 3D Object Detection: A series of 3D detectors has achieved promising results on the KITTI benchmark.
- Joint Image-LiDAR Detection: Joint image-LiDAR detectors fuse RGB images with LiDAR views or BEV features to improve prediction quality, especially for small objects.Examples include fusion across BEV, front-view, and image representations.
- LiDAR-based Detection: LiDAR-only detectors commonly voxelize point clouds and apply 2D or 3D convolutions, but deep layers may lose coordinate information.Encoding techniques have been proposed to preserve additional geometric information.
3. Our Method
Fast Point R-CNN combines voxel processing for efficient initial detection with raw-point refinement to recover localization information lost through voxelization and convolution. Its two-stage design uses VoxelRPN proposals and RefinerNet feature fusion to improve box predictions efficiently.
- Motivation: The voxel representation enables CNN processing but introduces quantization artifacts and weakens precise localization through discretization and downsampling.Direct processing of entire autonomous-driving point clouds preserves positional information but is computationally heavy.
- RefinerNet: RefinerNet uses points inside each VoxelRPN prediction together with convolutional context features to refine localization.The design avoids reliance on RGB images and combines raw coordinates with features whose receptive field captures local geometric structure.
- RefinerNet: An attention module fuses each point’s high-dimensional coordinate feature with convolutional features before a lightweight PointNet aggregates information for each box.The fused representation makes point features aware of contextual information from the first stage.
- Results: The framework improves box-prediction accuracy, particularly for the Z dimension and boxes with higher IoUs in both 3D and BEV.The reported refinement gains are obtained with a lightweight RefinerNet.
4. Experiments
Experiments on KITTI evaluate Fast Point R-CNN for 3D and BEV detection, using standard difficulty splits, augmentation, and implementation settings. The method achieves state-of-the-art performance while maintaining 15FPS efficiency and outperforming VoxelNet in both detection tasks.
- Dataset and Evaluation: Experiments use the KITTI dataset for 3D and BEV detection, with ablation studies conducted on a training-validation split.The dataset provides 7,481 training images and point clouds and 7,518 test examples; the split uses 3,712 training and 3,769 validation examples.
- Dataset and Evaluation: KITTI evaluation separates easy, moderate, and hard examples by occlusion, truncation, and 2D-box height, ranking methods primarily by AP0.7 on moderate examples.
- Implementation Details: The voxelized input covers a fixed KITTI point-cloud range, while category-specific settings remove fourth-layer downsampling for smaller pedestrians and cyclists.Car processing uses four anchor orientations, and NMS filters VoxelRPN predictions before RefinerNet.
- Implementation Details: Training uses multiple scene and object augmentations, including flipping, scaling, rotation, translation, and point-cloud mixup with surrounding context.Mixup crops a region extending 0.3 meters beyond each ground-truth box to preserve context information.
- Main Results: 15FPS on an NVIDIA Tesla P40 accompanies state-of-the-art KITTI performance on the moderate subset.The comparison covers both 3D and BEV detection.
- Main Results: Fast Point R-CNN outperforms VoxelNet in both 3D and BEV detection, while the complete system is nearly twice as fast as VoxelNet.VoxelRPN alone also outperforms VoxelNet in 3D object detection.
5. Ablation Studies
Ablations show that preserving 3D information, increasing voxel resolution, augmenting data, and refining with fused coordinate and convolution features improve detection quality. RefinerNet particularly benefits distant-object detection and fine-grained localization.
- VoxelRPN: Nearly 1 point improvement from replacing lower convolutional layers with Conv3D, while increasing time cost by only 5ms.The change improves information preservation, especially along the Z dimension.
- VoxelRPN: Higher-resolution input significantly improves results without adding much computation.The finer voxel grid has size 800 × 704 × 20, with the first-layer stride changed to 2.
- VoxelRPN: MIXUP improves performance by around 0.5 point and reaches comparable performance with only half the original training epochs.The result is reported for VoxelRPN ablations on the KITTI validation subset.
- VoxelRPN: Using four anchor angles instead of two provides another 0.8 point gain.The four angles are 0°, 45°, 90° and 135°; the authors associate the gain with higher ground-truth matching probability.
- RefinerNet: Attention-based fusion of coordinate and convolution features outperforms simple concatenation by 0.62 point.Coordinate features alone outperform convolution features alone, while combining both provides the strongest performance.
- RefinerNet: RefinerNet improves distant-object AP0.7 from 51.99 to 58.41 for objects 30–50 meters away.The analysis attributes this improvement to using coordinate features to infer structure from sparse point sets.
6. Conclusion
The paper concludes with a fast two-stage 3D detection framework that combines voxel and raw point-cloud representations. It reports performance on par with existing solutions while maintaining higher detection speed.
- 6. Conclusion: The framework uses voxel-based convolutional processing for initial predictions and raw point clouds with convolution features for refinement.This two-stage design is presented as generic, effective, and fast for 3D object detection.
- 6. Conclusion: The method is on par with existing solutions while maintaining higher detection speed.The authors present the design as a way to use different dimensions of point-cloud information.