PaperlayerView exact revision on arXiv

Source-linked AI summary

Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection

Benjin Zhu, Zhengkai Jiang, Xiangxin Zhou, Zeming Li, Gang Yu

arXiv:1908.09492v1cs.CV

TL;DR

The paper addresses severe class imbalance and the broader multi-category demands of nuScenes 3D detection. It combines sparse 3D feature extraction with class-balanced sampling, augmentation, and a multi-group head, achieving state-of-the-art performance and winning the WAD challenge.

  • Problem

    The nuScenes 3D detection task involves 10 categories, attributes, velocities, and a severe long-tailed class imbalance that disadvantages rare classes.

    Source: §1 · p. 1, §2.3 · p. 4

  • Method

    The method combines sparse 3D feature extraction with class-balanced sampling and augmentation, joint multi-category learning, and a multi-group head for related shapes or sizes.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    73.1% improvement over the official PointPillars baseline is reported, with better performance across categories and lower errors across the evaluated metrics.

    Source: §4 · p. 6

  • Takeaways & Limitations

    The approach achieves state-of-the-art nuScenes performance and wins the WAD challenge by addressing class imbalance through data, network, and learning-objective strategies.

    Source: §1 · p. 2, §5 · p. 6

Abstract

from arXiv · show

This report presents our method which wins the nuScenes3D Detection Challenge [17] held in Workshop on Autonomous Driving(WAD, CVPR 2019). Generally, we utilize sparse 3D convolution to extract rich semantic features, which are then fed into a class-balanced multi-head network to perform 3D object detection. To handle the severe class imbalance problem inherent in the autonomous driving scenarios, we design a class-balanced sampling and augmentation strategy to generate a more balanced data distribution. Furthermore, we propose a balanced group-ing head to boost the performance for the categories withsimilar shapes. Based on the Challenge results, our methodoutperforms the PointPillars [14] baseline by a large mar-gin across all metrics, achieving state-of-the-art detection performance on the nuScenes dataset. Code will be released at CBGS.

1. Introduction

The nuScenes challenge extends 3D detection beyond KITTI while introducing severe class imbalance, motivating class-balanced sampling and multi-group detection heads. The method combines these strategies with network and training improvements and reports state-of-the-art performance.

  • nuScenes requires simultaneous detection of 10 categories plus object attributes and velocities, compared with KITTI’s 3-category detection task.Source: §1 · p. 1
  • The nuScenes training distribution is long-tailed, with extreme differences in examples between common and rare object classes.Source: §1 · p. 1
  • The report states that joint multi-category training performs better than training each category individually.Source: §1 · p. 2
  • The method proposes class-balanced sampling to address the dataset’s extreme class imbalance.Source: §1 · p. 2
  • A multi-group head lets categories with similar shapes or sizes benefit from each other while reducing interference between dissimilar categories.Source: §1 · p. 2
  • The method achieves state-of-the-art performance on the challenging nuScenes Dataset through network architecture, loss-function, and training-procedure improvements.Source: §1 · p. 2

2. Methodology

The method combines sparse 3D feature extraction, class-balanced sampling and augmentation, and a multi-group detection head to address nuScenes imbalance and category interference.

Architecture

  • The network performs simultaneous 10-category 3D detection, velocity prediction, and attribute prediction with a 3D extractor, proposal network, and multi-group head.The architecture also outperforms training each category separately.Source: §2 · p. 2

Class-balanced sampling

  • DS Sampling duplicates samples from underrepresented categories to produce a smoother training distribution.Fewer category samples receive more duplication.Source: §2.1 · p. 3, Figure 2 · p. 3

Data augmentation

  • GT-AUG samples annotated ground-truth boxes from an offline database and places them into other point clouds after estimating the ground plane.Ground-plane estimation uses least squares and RANSAC with Ax + By + Cz + D = 0.Source: §2.1 · p. 3

Architecture

  • Sparse 3D convolutions extract features, which are reshaped for a region proposal network and then processed by the multi-group head.The extractor outputs a 16× downscale representation, while the proposal network generates 8× feature maps.Source: §2.2 · p. 3, Figure 3 · p. 4

Motivation

  • The approach targets nuScenes’ long-tailed distribution, where common classes dominate annotations and tail classes receive few examples.Car accounts for 43.7% of annotations, 40 times the number of bicycle instances.Source: §2.3 · p. 4

Class-balanced grouping

  • Classes are grouped by similar shape or size while separating dominant classes to prevent group imbalance.The final configuration contains 6 groups covering all 10 classes.Source: §2.3 · p. 4

3. Training Details

Training uses voxelized multi-sweep inputs, geometric and ground-truth augmentation, balanced loss weighting, and specified sparse-convolution settings and optimization procedures.

Input and augmentation

  • Training augmentation applies random x-axis flips, scaling, Z-axis rotation, translation, and class-specific GT-AUG box insertion.GT-AUG filters boxes containing fewer than 5 points before sampling and pasting them.Source: §3 · p. 5

Ablation

  • Ablation results identify Database Sampling and Res-Encoder as the largest contributors to mAP.Table 4 evaluates the components on the validation split.Source: Table 4 · p. 6

Optimization

  • The model uses AdamW with one-cycle learning rate scheduling, batch size 5, and 20 training epochs.The maximum learning rate is 0.04 and weight decay is fixed at 0.01.Source: §3.1 · p. 6

Network implementation

  • Sparse feature-extractor blocks use 16, 32, 64, and 128 convolution layers, with 1 × 1 convolutions producing final head predictions.Submanifold sparse convolution is used during feature-map downsampling and regular sparse convolution otherwise.Source: §3.2 · p. 6

Loss and targets

  • Positive area thresholds are 0.6 for sufficiently annotated categories and 0.4 for categories with fewer annotations.Regression weights are 0.2 for velocity and 1.0 for the other listed variables.Source: §3.2 · p. 6

4. Results

The method substantially improves over the official PointPillars baseline and reports gains across categories, including long-tail classes and multiple error metrics.

Overall performance

  • 73.1%: the method surpasses the official PointPillars baseline in overall performance.The comparison is reported in Table 2.Source: §4 · p. 6

Category performance

  • The method performs better in all categories, with particularly strong improvements for Bicycle, Motorcycle, Bus, and Trailer.The report also describes reliable detection results across all categories.Source: §4 · p. 6

Error metrics

  • The method achieves lower mATE, mASE, mAOE, mAVE, and mAAE errors than the comparison baseline.These metrics cover translation, scale, orientation, velocity, and attribute errors.Source: §4 · p. 6

5. Conclusion

The report presents a class-imbalance-oriented approach for the challenging nuScenes dataset and reports the best result in the WAD challenge, while noting limited existing results and planned code release.

Conclusion

  • nuScenes introduces greater challenges than KITTI, including class imbalance, in large-scale 3D object detection.The conclusion frames the dataset as newly released and large scale.Source: §5 · p. 6
  • The method addresses imbalance and multi-class joint detection through coordinated data, network, and learning-objective strategies.These strategies are described as carefully designed in the conclusion.Source: §5 · p. 6
  • The authors report achieving the best result in the WAD challenge.They note that relatively few methods had reported results on nuScenes and planned to release code.Source: §5 · p. 6

Detection examples

  • Validation examples show ground-truth annotations in green and detections in blue from a model with 51.9% mAP and 62.5% NDS.The examples come from the validation split.Source: Figure 4 · p. 7

Related papers

Loading 1908.09492v1…