Source-linked AI summary
TANet: Robust 3D Object Detection from Point Clouds with Triple Attention
Zhe Liu, Xin Zhao, Tengteng Huang, Ruolan Hu, Yu Zhou, Xiang Bai
TL;DR
3D point-cloud detection remains insufficiently robust for hard objects and degrades under added noise. TANet combines stacked Triple Attention with coarse-to-fine regression, achieving strong noisy-case robustness and state-of-the-art KITTI performance, including the best reported Pedestrian results.
Problem
Existing detectors perform unsatisfactorily on hard objects such as pedestrians and can degrade rapidly when additional noise points are added.
Method
TANet uses stacked channel-wise, point-wise, and voxel-wise attention plus coarse-to-fine regression for robust representations and more accurate localization.
Results
TANet achieves state-of-the-art KITTI performance and greatly outperforms existing approaches in challenging noisy cases, with the best reported result on the Pedestrian class.
Takeaways & Limitations
The results support TANet as a robust and efficient point-cloud-only detector for challenging 3D detection settings.
Abstract
from arXiv · showhide
In this paper, we focus on exploring the robustness of the 3D object detection in point clouds, which has been rarely discussed in existing approaches. We observe two crucial phenomena: 1) the detection accuracy of the hard objects, e.g., Pedestrians, is unsatisfactory, 2) when adding additional noise points, the performance of existing approaches decreases rapidly. To alleviate these problems, a novel TANet is introduced in this paper, which mainly contains a Triple Attention (TA) module, and a Coarse-to-Fine Regression (CFR) module. By considering the channel-wise, point-wise and voxel-wise attention jointly, the TA module enhances the crucial information of the target while suppresses the unstable cloud points. Besides, the novel stacked TA further exploits the multi-level feature attention. In addition, the CFR module boosts the accuracy of localization without excessive computation cost. Experimental results on the validation set of KITTI dataset demonstrate that, in the challenging noisy cases, i.e., adding additional random noisy points around each object,the presented approach goes far beyond state-of-the-art approaches. Furthermore, for the 3D object detection task of the KITTI benchmark, our approach ranks the first place on Pedestrian class, by using the point clouds as the only input. The running speed is around 29 frames per second.
Introduction
TANet addresses difficult pedestrian detection and robustness to noisy point clouds with stacked Triple Attention and coarse-to-fine regression. The approach reports strong noisy-case robustness and state-of-the-art KITTI performance.
- 3D point clouds offer geometric and depth information but remain difficult for detection because they are unordered, sparse, and unevenly distributed.
- Pedestrian detection remains unsatisfactory because pedestrians provide fewer LiDAR points and often appear near confusing background objects.
- TANet evaluates noisy cases by adding random points around objects and reports strong robustness, state-of-the-art KITTI performance, and promising inference speed.
- Triple Attention jointly models channel-wise, point-wise, and voxel-wise attention to enhance discriminative object information and suppress unstable points.
- Coarse-to-fine regression first estimates object boxes coarsely, then refines them using informative fused cross-layer features.
Related Work
Prior 3D detection methods use image, voxel-grid, or raw-point-cloud representations with different accuracy and efficiency trade-offs. TANet combines attention over voxel features with coarse-to-fine regression to seek a better accuracy–efficiency balance.
- Image-based methods are broadly organized into front-view and bird’s-eye-view approaches, but depth loss makes accurate localization difficult.
- Voxel-grid methods divide point-cloud space into regular voxels, encode non-empty voxels, and predict 3D boxes from learned voxel representations.
- TANet applies channel-wise, point-wise, and voxel-wise attention with stacked modules to learn more discriminative voxel representations.
- Voxel-grid detectors can process more than 20 frames per second, whereas raw-point-cloud two-stage methods generally run below 15 frames per second.
- Its coarse-to-fine regression mechanism targets a better trade-off between detection accuracy and efficiency.
3D Object Detection with TANet
TANet is an end-to-end voxel-based detector with stacked Triple Attention and Coarse-to-Fine Regression. Its attention modules weight point, channel, and voxel information before coarse boxes are refined using fused cross-layer features.
- TANet is an end-to-end voxel-based network composed of Stacked Triple Attention and Coarse-to-Fine Regression.
- Stacked Triple Attention: Point-wise attention models spatial responses among points inside each voxel, while channel-wise attention represents feature-channel importance.Point-wise responses arise from channel aggregation; channel-wise responses arise from point aggregation.
- Stacked Triple Attention: Triple Attention combines point-wise and channel-wise attention, then applies voxel-wise weights to enhance crucial features and suppress irrelevant or noisy features.The voxel-wise weight is computed from voxel-center information and the attended voxel representation.
- Stacked Triple Attention: Two TA modules process original and higher-dimensional features, fuse each output with its input, and max-pool point features for CFR.
- Coarse-to-Fine Regression: CFR first generates coarse boxes, then uses PSA cross-layer features with different resolutions and semantic levels for fine regression.The CR module supplies coarse boxes, while PSA enriches features before fusion with CR semantics for FR.
- Coarse-to-Fine Regression: The multi-task loss jointly optimizes coarse and fine regression modules.
Experiments
Experiments evaluate TANet on original and synthetically noised KITTI point clouds across Cars, Pedestrians, and Cyclists. TANet shows strong robustness under noise, competitive overall detection, informative attention behavior, and 34.75 ms average inference time.
- Evaluation Setup: Experiments use KITTI data, mean Average Precision, and official IoU thresholds of 0.7 for Cars and 0.5 for Pedestrians and Cyclists.The evaluation covers three categories and Easy, Moderate, and Hard difficulty levels.
- Noisy Evaluation: 100 noise points are sampled around each object, while models train on official data and test on noised validation point clouds.The added coordinates follow uniform distributions in regions surrounding each object's dimensions.
- Noisy Evaluation: 79.34% 3D mAP with 100 noise points lets TANet outperform PointRCNN by 1.7% for Cars.PointRCNN nevertheless exceeds TANet by 0.43% in 3D mAP for Cars in the reported comparison.
- Noisy Evaluation: 5.8% and 11.9% improvements for Pedestrians over PointPillars and PointRCNN demonstrate stronger robustness for difficult noisy objects.The reported robustness also includes hard Cyclists and hard Cars.
- Original KITTI Evaluation: 62.00% 3D mAP across the three KITTI categories exceeds PointPillars by 1.20% and PointRCNN by 1.22% on original point clouds.For Pedestrians, the improvements are 2.30% over PointPillars and 4.83% over PointRCNN.
- Qualitative Analysis: TANet feature maps focus on salient objects and suppress noisy regions, while producing higher confidence scores than PointPillars.The visualization reports satisfactory confidence for challenging objects that PointPillars fails to detect.
- Efficiency: 34.75 ms average inference time comprises 8.0 ms preprocessing, 12.42 ms voxel feature extraction, and 14.33 ms detection and post-processing.This corresponds to approximately 29 frames per second.
- Ablation Studies: 65.59% baseline 3D mAP rises to 67.38% with parallel point-wise and channel-wise attention, a 1.8% improvement.The ablation compares attention variants without the fine-regression module; PSA further improves performance when combined with TA.
Conclusion
TANet combines Triple Attention with Coarse-to-Fine Regression for point-cloud 3D detection, particularly under added noise. The paper reports state-of-the-art KITTI performance and a large robustness advantage in challenging noisy cases.
- Contributions: TANet uses Triple Attention to enhance crucial object information and suppress interference points.The module jointly models channel-wise, point-wise, and voxel-wise attention.
- Contributions: Coarse-to-Fine Regression provides more accurate detection boxes without excessive computation cost.It is the second core component of TANet.
- Results: State-of-the-art KITTI performance and superior robustness under additional noise are reported as the paper's main outcomes.The conclusion emphasizes that noisy-case performance exceeds existing approaches by a large margin.
Supplemental Material
Supplemental material reports qualitative detections, implementation settings, validation results, and sensitivity to the multi-task loss balance weight.
- Qualitative Results: Figure 6 visualizes projected 3D boxes, LiDAR boxes, and orientations for Cars, Pedestrians, and Cyclists using blue, green, and red boxes.The first and third rows show projections on 2D images; the second and fourth show LiDAR predictions.
- Qualitative Results: TANet accurately predicts object locations and orientations under small-object and heavy-occlusion conditions.These are described as extremely challenging situations.
- Implementation Details: Cars use a 440 × 500 × 1 voxel grid with voxel size 0.16, 0.16, and 4 meters along width, height, and depth.Each voxel uses one anchor box with orientations 0 and π/2 radians.
- Implementation Details: Pedestrian anchors have dimensions wa = 0.6, la = 1.76, ha = 1.73, with positive, negative, and ignored IoU ranges specified.NMS score and IoU threshold are set to 0.1 and 0.6.
- Validation Results: The official KITTI validation set without noise is reported for comparison with future work.Table 5 contains results for Cars, Pedestrians, and Cyclists.
- Hyperparameter Analysis: λ = 2.0 is the best tested balance weight and achieves 71.72% 3D mAP.This experiment evaluates the influence of λ on the multi-task loss.