PaperlayerView exact revision on arXiv

Source-linked AI summary

AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds

Yihan Hu, Zhuangzhuang Ding, Runzhou Ge, Wenxin Shao, Li Huang, Kun Li, Qiang Liu

arXiv:2112.09205v2cs.CV

TL;DR

3D point-cloud detection faces a trade-off between the efficiency of single-stage methods and the accuracy of two-stage methods. AFDetV2 strengthens the first stage with improved features, keypoint supervision, and IoU-aware scoring, and reports state-of-the-art performance while removing the need for second-stage refinement.

  • Problem

    Two-stage detectors usually improve accuracy, but whether their second stage is necessary when the first stage already localizes boxes accurately remains unresolved.

  • Method

    AFDetV2 is an anchor-free single-stage detector using self-calibrated convolution, keypoint auxiliary supervision, and an IoU prediction branch.

  • Results

    AFDetV2 achieves state-of-the-art performance on Waymo Open Dataset and nuScenes Dataset and outperforms prior single-stage and two-stage detectors.

  • Takeaways & Limitations

    Strengthening the first stage and aligning classification with localization can make second-stage refinement unnecessary within the evaluated detection settings.

Abstract

from arXiv · show

There have been two streams in the 3D detection from point clouds: single-stage methods and two-stage methods. While the former is more computationally efficient, the latter usually provides better detection accuracy. By carefully examining the two-stage approaches, we have found that if appropriately designed, the first stage can produce accurate box regression. In this scenario, the second stage mainly rescores the boxes such that the boxes with better localization get selected. From this observation, we have devised a single-stage anchor-free network that can fulfill these requirements. This network, named AFDetV2, extends the previous work by incorporating a self-calibrated convolution block in the backbone, a keypoint auxiliary supervision, and an IoU prediction branch in the multi-task head. As a result, the detection accuracy is drastically boosted in the single-stage. To evaluate our approach, we have conducted extensive experiments on the Waymo Open Dataset and the nuScenes Dataset. We have observed that our AFDetV2 achieves the state-of-the-art results on these two datasets, superior to all the prior arts, including both the single-stage and the two-stage 3D detectors. AFDetV2 won the 1st place in the Real-Time 3D Detection of the Waymo Open Dataset Challenge 2021. In addition, a variant of our model AFDetV2-Base was entitled the "Most Efficient Model" by the Challenge Sponsor, showing a superior computational efficiency. To demonstrate the generality of this single-stage method, we have also applied it to the first stage of the two-stage networks. Without exception, the results show that with the strengthened backbone and the rescoring approach, the second stage refinement is no longer needed.

1 Introduction

3D point-cloud detectors trade computational efficiency against accuracy, motivating whether a second stage is truly necessary. AFDetV2 argues that a strengthened first stage can localize boxes accurately while using IoU-aware rescoring to remove that need.

  • Single-stage detectors are more computationally efficient, whereas two-stage methods usually provide better detection accuracy.
  • The second stage may mainly rescore classification confidence to select better-localized boxes rather than refine their positions.
  • AFDetV2 differs from related approaches through an anchor-free design, self-calibrated convolution, keypoint auxiliary supervision, and an IoU prediction branch.
  • AFDetV2 was evaluated against single-stage and two-stage methods on the Waymo Open Dataset and nuScenes Dataset.
  • 73.12 APH/L2 at 60.06 ms earned AFDetV2 first place in the Real-Time 3D Detection of the WOD Challenge 2021.
  • The proposed components can be added to a two-stage model's first stage so the second stage can be discarded while achieving similar or better detection accuracy.

2 Related Work

Related work spans two-stage and single-stage LiDAR detectors, with designs differing in feature extraction, proposal refinement, and anchor usage. AFDetV2 is presented as a simpler, faster anchor-free alternative for real-time deployment.

  • Two-stage LiDAR detectors generate Region of Interests at a first stage and refine predictions in a second stage.
  • Single-stage LiDAR detectors use representations such as 3D voxels, sparse convolutions, and BEV pillars to reduce computational cost.
  • Anchor-based detectors extend anchors into 3D space, while anchor-free methods formulate detection without those predefined boxes.
  • AFDetV2 is described as a fast, accurate anchor-free single-stage detector with a simpler structure and higher speed than two-stage and anchor-based detectors.

3 Methods

AFDetV2 argues that a strengthened first stage can replace second-stage refinement by combining improved feature extraction, auxiliary keypoint supervision, and IoU-aware confidence scoring. The resulting anchor-free detector targets comparable or better accuracy with lower computational cost.

  • Analysis on Two-stage Methods: The authors identify score refinement, rather than second-stage box regression alone, as the main source of observed AP improvement.Adding box regression alone produced no significant improvement, whereas score refinement significantly improved AP.
  • Network Architecture: AFDetV2 uses an anchor-free single-stage architecture that voxelizes point clouds, extracts sparse 3D features, and applies a multi-scale backbone before detection.The network follows AFDet and sends voxelized inputs through a 3D feature extractor, backbone, and anchor-free detection head.
  • Network Design: AFDetV2 strengthens the first stage with self-calibrated convolutions, keypoint auxiliary supervision, and an IoU-aware confidence prediction branch.The backbone uses SC-Conv to enlarge the receptive field and add channel-wise and spatial attention; the head adds keypoint and IoU-related supervision.
  • IoU-aware Confidence Score Prediction: The network combines classification confidence with predicted localization quality when ranking detections.The post-processing score uses the original classification score and predicted IoU, so ranking reflects both confidence and localization accuracy.

4.1 Datasets

The experiments use the Waymo Open Dataset and nuScenes, with dataset-specific annotations and evaluation metrics. The section also identifies the APH/L2 metric used for the ablation results.

  • nuScenes contains 700 training, 150 validation, and 150 test sequences with annotations for 10 classes.
  • nuScenes uses NDS as its main ranking metric and also reports mean average precision.
  • Waymo contains 798 training, 202 validation, and 150 test sequences for VEHICLE, PEDESTRIAN, and CYCLIST detection.
  • The AFDetV2 ablation study reports results using the 3D APH/L2 metric on the full validation set.

4.2 Experiment Settings

The experiments vary model configurations, training procedures, inference augmentation, and deployment optimizations across Waymo and nuScenes. The setup includes explicit voxel, sweep, precision, and latency choices.

  • nuScenes experiments accumulate 10 LiDAR sweeps, use voxel size [0.075m, 0.075m, 0.2m], and apply no ensembling or test-time augmentation.
  • Waymo models use the top LiDAR, up to 500 objects per frame, a maximum of 5 points per voxel, and 250,000 voxels during training.
  • Waymo evaluates AFDetV2-Lite, AFDetV2-Base, AFDetV2, and AFDetV2 variants sharing the same feature extractor, backbone, and detection head.
  • Training schedules include initial training, trainval fine-tuning, and smaller-grid fine-tuning for AFDetV2-Base and AFDetV2.
  • AFDetV2-Lite, AFDetV2, and AFDetV2-Ens use SWA for test-set submissions, while AFDetV2-Base does not.
  • AFDetV2-Ens uses yaw rotation, global scaling, z-axis translation, and merged LiDAR inputs for test-time augmentation and ensembling.
  • The fastest-model pipeline accelerates inference through GPU preprocessing, half-precision casting, and batch-normalization fusion.

4.3 Comparison with State-of-the-art Methods

AFDetV2 outperforms prior LiDAR-based detectors across nuScenes and Waymo settings, including comparisons emphasizing accuracy, speed, real-time ranking, and ensemble performance.

  • nuScenes test set: AFDetV2 surpasses HotSpotNet by 2.5 NDS or 3.1 mAP on the nuScenes test set.
  • Waymo validation set: AFDetV2-Lite achieves 68.77 APH/L2 on the Waymo validation set, surpassing prior art by 3.9%.
  • Waymo test set: The Waymo test-set comparison separates single-frame, multi-frame, and ensemble methods, with the first two rows targeting onboard real-time use.
  • Onboard real-time scenario: AFDetV2-Lite is 15.4% faster than HIKVISION LiDAR and achieves 2.6 higher ALL APH/L2.
  • Multi-frame comparison: AFDetV2 ranks first among multi-frame LiDAR-only non-ensemble methods under the ALL APH/L2 metric.
  • Offboard scenario: AFDetV2-Ens surpasses published ensemble methods and HorizonLiDAR3D while using only LiDAR and two input frames.

4.4 Waymo Real-Time 3D Detection Challenge

The Waymo real-time challenge rewards high APH/L2 under a 70 ms latency limit, while its efficiency title requires the lowest latency with APH/L2 above 70. AFDetV2 won the real-time challenge, and AFDetV2-Base received the efficiency title.

  • The challenge requires 3D detection within 70 ms per frame and awards the highest APH/L2 satisfying that limit.
  • AFDetV2 won first place in the Real-Time 3D Detection of the Waymo Open Dataset Challenge 2021.
  • AFDetV2-Base was 0.55% lower than AFDetV2 and received the “Most Efficient Model” title for computational efficiency.
  • Figure 3 visualizes test-set boxes for VEHICLE, PEDESTRIAN, and CYCLIST when scores exceed 0.50, using blue, magenta, and olive colors respectively.

4.5 Ablation Studies

Ablations show that keypoint supervision, SC-Conv, and IoU rescoring progressively improve AFDetV2, while comparisons indicate that IoU-aware single-stage detection can match or surpass two-stage alternatives.

  • Component ablations: IoU rescoring improved AFDetV2 by 3.12 ALL APH/L2, exceeding the gains from keypoint auxiliary loss and SC-Conv.Keypoint auxiliary loss added 0.12 ALL APH/L2, while SC-Conv added 0.55 ALL APH/L2.
  • Encoder generality: AFDetV2 surpassed a state-of-the-art PointPillars-based model by 3.17 ALL APH/L2.With the PointPillars encoder, IoU-aware rescoring improved performance by 1.42 ALL APH/L2, compared with 1.1 ALL APH/L2 from CenterPoint second-stage refinement.
  • Two-stage analysis: Second-stage box refinement alone did not improve performance in the CenterPoint-style two-stage analysis.The analysis refined scores and boxes using five RoI features per first-stage proposal.
  • Two-stage analysis: A single-stage detector with IoU-aware rescoring could outperform its corresponding two-stage detector with the same rescoring.Under this comparison, the authors conclude that the second-stage network is unnecessary for a well-designed single-stage network.
  • Two-stage transfer: The study also transferred SC-Conv and the IoU-aware head to Voxel R-CNN for evaluating their use in a two-stage architecture.Voxel R-CNN uses SECOND as its first stage and applies voxel-RoI pooling before second-stage prediction.

5 Conclusion

AFDetV2 is presented as a real-time single-stage anchor-free 3D detector whose strengthened first stage makes second-stage refinement unnecessary. It achieves state-of-the-art performance on Waymo Open Dataset and nuScenes Dataset with high inference speed.

  • AFDetV2 is a real-time single-stage anchor-free 3D object detection model.
  • Experiments show that second-stage refinement is unnecessary when the first stage is strengthened and fully utilized.
  • AFDetV2 achieves state-of-the-art performance on Waymo Open Dataset and nuScenes Dataset with high inference speed.
Loading 2112.09205v2…