Source-linked AI summary
Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection
Benjin Zhu, Zhengkai Jiang, Xiangxin Zhou, Zeming Li, Gang Yu
TL;DR
The paper addresses severe class imbalance and the broader multi-category demands of nuScenes 3D detection. It combines sparse 3D feature extraction with class-balanced sampling, augmentation, and a multi-group head, achieving state-of-the-art performance and winning the WAD challenge.
Problem
The nuScenes 3D detection task involves 10 categories, attributes, velocities, and a severe long-tailed class imbalance that disadvantages rare classes.
Method
The method combines sparse 3D feature extraction with class-balanced sampling and augmentation, joint multi-category learning, and a multi-group head for related shapes or sizes.
Results
73.1% improvement over the official PointPillars baseline is reported, with better performance across categories and lower errors across the evaluated metrics.
Takeaways & Limitations
The approach achieves state-of-the-art nuScenes performance and wins the WAD challenge by addressing class imbalance through data, network, and learning-objective strategies.
Abstract
from arXiv · showhide
This report presents our method which wins the nuScenes3D Detection Challenge [17] held in Workshop on Autonomous Driving(WAD, CVPR 2019). Generally, we utilize sparse 3D convolution to extract rich semantic features, which are then fed into a class-balanced multi-head network to perform 3D object detection. To handle the severe class imbalance problem inherent in the autonomous driving scenarios, we design a class-balanced sampling and augmentation strategy to generate a more balanced data distribution. Furthermore, we propose a balanced group-ing head to boost the performance for the categories withsimilar shapes. Based on the Challenge results, our methodoutperforms the PointPillars [14] baseline by a large mar-gin across all metrics, achieving state-of-the-art detection performance on the nuScenes dataset. Code will be released at CBGS.
1. Introduction
The nuScenes challenge extends 3D detection beyond KITTI while introducing severe class imbalance, motivating class-balanced sampling and multi-group detection heads. The method combines these strategies with network and training improvements and reports state-of-the-art performance.
- nuScenes requires simultaneous detection of 10 categories plus object attributes and velocities, compared with KITTI’s 3-category detection task.
- The nuScenes training distribution is long-tailed, with extreme differences in examples between common and rare object classes.
- The report states that joint multi-category training performs better than training each category individually.
- The method proposes class-balanced sampling to address the dataset’s extreme class imbalance.
- A multi-group head lets categories with similar shapes or sizes benefit from each other while reducing interference between dissimilar categories.
- The method achieves state-of-the-art performance on the challenging nuScenes Dataset through network architecture, loss-function, and training-procedure improvements.
2. Methodology
The method combines sparse 3D feature extraction, class-balanced sampling and augmentation, and a multi-group detection head to address nuScenes imbalance and category interference.
- Architecture: The network performs simultaneous 10-category 3D detection, velocity prediction, and attribute prediction with a 3D extractor, proposal network, and multi-group head.The architecture also outperforms training each category separately.
- Class-balanced sampling: DS Sampling duplicates samples from underrepresented categories to produce a smoother training distribution.Fewer category samples receive more duplication.
- Data augmentation: GT-AUG samples annotated ground-truth boxes from an offline database and places them into other point clouds after estimating the ground plane.Ground-plane estimation uses least squares and RANSAC with Ax + By + Cz + D = 0.
- Architecture: Sparse 3D convolutions extract features, which are reshaped for a region proposal network and then processed by the multi-group head.The extractor outputs a 16× downscale representation, while the proposal network generates 8× feature maps.
- Motivation: The approach targets nuScenes’ long-tailed distribution, where common classes dominate annotations and tail classes receive few examples.Car accounts for 43.7% of annotations, 40 times the number of bicycle instances.
- Class-balanced grouping: Classes are grouped by similar shape or size while separating dominant classes to prevent group imbalance.The final configuration contains 6 groups covering all 10 classes.
3. Training Details
Training uses voxelized multi-sweep inputs, geometric and ground-truth augmentation, balanced loss weighting, and specified sparse-convolution settings and optimization procedures.
- Input and augmentation: Training augmentation applies random x-axis flips, scaling, Z-axis rotation, translation, and class-specific GT-AUG box insertion.GT-AUG filters boxes containing fewer than 5 points before sampling and pasting them.
- Ablation: Ablation results identify Database Sampling and Res-Encoder as the largest contributors to mAP.Table 4 evaluates the components on the validation split.
- Optimization: The model uses AdamW with one-cycle learning rate scheduling, batch size 5, and 20 training epochs.The maximum learning rate is 0.04 and weight decay is fixed at 0.01.
- Network implementation: Sparse feature-extractor blocks use 16, 32, 64, and 128 convolution layers, with 1 × 1 convolutions producing final head predictions.Submanifold sparse convolution is used during feature-map downsampling and regular sparse convolution otherwise.
- Loss and targets: Positive area thresholds are 0.6 for sufficiently annotated categories and 0.4 for categories with fewer annotations.Regression weights are 0.2 for velocity and 1.0 for the other listed variables.
4. Results
The method substantially improves over the official PointPillars baseline and reports gains across categories, including long-tail classes and multiple error metrics.
- Overall performance: 73.1%: the method surpasses the official PointPillars baseline in overall performance.The comparison is reported in Table 2.
- Category performance: The method performs better in all categories, with particularly strong improvements for Bicycle, Motorcycle, Bus, and Trailer.The report also describes reliable detection results across all categories.
- Error metrics: The method achieves lower mATE, mASE, mAOE, mAVE, and mAAE errors than the comparison baseline.These metrics cover translation, scale, orientation, velocity, and attribute errors.
5. Conclusion
The report presents a class-imbalance-oriented approach for the challenging nuScenes dataset and reports the best result in the WAD challenge, while noting limited existing results and planned code release.
- Conclusion: nuScenes introduces greater challenges than KITTI, including class imbalance, in large-scale 3D object detection.The conclusion frames the dataset as newly released and large scale.
- Conclusion: The method addresses imbalance and multi-class joint detection through coordinated data, network, and learning-objective strategies.These strategies are described as carefully designed in the conclusion.
- Conclusion: The authors report achieving the best result in the WAD challenge.They note that relatively few methods had reported results on nuScenes and planned to release code.
- Detection examples: Validation examples show ground-truth annotations in green and detections in blue from a model with 51.9% mAP and 62.5% NDS.The examples come from the validation split.