Source-linked AI summary
SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud
Wu Zheng, Weiliang Tang, Li Jiang, Chi-Wing Fu
TL;DR
Outdoor 3D detection needs high precision and efficiency, but single-stage designs trade some precision for speed. SE-SSD jointly trains teacher and student SSDs with soft-target consistency, shape-aware augmentation, and ODIoU hard-target supervision, achieving state-of-the-art KITTI car detection with ultra-high inference speed.
Problem
Practical outdoor 3D object detection requires high precision and high efficiency, motivating a single-stage detector that pursues both.
Method
SE-SSD jointly exploits teacher-generated soft targets and annotated hard targets through consistency constraints, shape-aware augmentation, and ODIoU supervision.
Results
SE-SSD outperforms prior state-of-the-art methods on KITTI 3D and BEV car detection while attaining ultra-high inference speed.
Takeaways & Limitations
The proposed training modules improve detector performance and efficiency without introducing extra computation during inference.
Abstract
from arXiv · showhide
We present Self-Ensembling Single-Stage object Detector (SE-SSD) for accurate and efficient 3D object detection in outdoor point clouds. Our key focus is on exploiting both soft and hard targets with our formulated constraints to jointly optimize the model, without introducing extra computation in the inference. Specifically, SE-SSD contains a pair of teacher and student SSDs, in which we design an effective IoU-based matching strategy to filter soft targets from the teacher and formulate a consistency loss to align student predictions with them. Also, to maximize the distilled knowledge for ensembling the teacher, we design a new augmentation scheme to produce shape-aware augmented samples to train the student, aiming to encourage it to infer complete object shapes. Lastly, to better exploit hard targets, we design an ODIoU loss to supervise the student with constraints on the predicted box centers and orientations. Our SE-SSD attains top performance compared with all prior published works. Also, it attains top precisions for car detection in the KITTI benchmark (ranked 1st and 2nd on the BEV and 3D leaderboards, respectively) with an ultra-high inference speed. The code is available at https://github.com/Vegeta2020/SE-SSD.
1. Introduction
Outdoor LiDAR point clouds support robust autonomous-driving detection, but practical 3D detectors must combine high precision with high efficiency. SE-SSD addresses this need by jointly exploiting soft and hard targets through a teacher–student framework, shape-aware augmentation, and ODIoU supervision.
- LiDAR point clouds are robust for autonomous-driving object detection across weather and time-of-day conditions.
- High precision and high efficiency are both required for practical 3D object detection, motivating SE-SSD's single-stage design.
- SE-SSD uses teacher-generated soft targets alongside manually annotated hard targets to jointly optimize the detector.Teacher predictions provide higher-entropy information, while hard targets remain important for supervised convergence.
- Shape-aware augmentation produces object samples intended to encourage inference of complete shapes from incomplete information.The module is designed as a plug-and-play component for 3D detectors.
- SE-SSD combines a consistency constraint, shape-aware ground-truth augmentation, and an Orientation-aware Distance-IoU loss.All designed modules are used during training, without extra inference computation.
- SE-SSD attains state-of-the-art 3D and BEV car-detection performance on KITTI while reaching 32 FPS on commodity CPU-GPU hardware.
2. Related Work
Single-stage detectors generally offer faster inference, whereas two-stage detectors often achieve higher precision through an additional refinement stage. SE-SSD follows the single-stage path while using soft and hard targets, novel constraints, and augmentation to pursue both objectives.
- Single-stage detectors directly regress boxes and confidence, while two-stage detectors refine first-stage predictions using region-proposal-aligned features.
- Two-stage detectors often attain higher precision, whereas single-stage detectors usually run faster because of simpler network structures.
- Recent single-stage detectors have gradually approached the precision of two-stage detectors, motivating efforts to improve both precision and speed.
- SE-SSD differs from prior work by exploiting both soft and hard targets through novel constraints and an augmentation scheme in a fully supervised manner.
- SE-SSD reports the highest average precisions among prior single- and two-stage detectors for KITTI 3D and BEV car detection with very high efficiency.
3. Self-Ensembling Single Stage Detector
SE-SSD jointly trains teacher and student single-stage detectors using filtered soft targets, hard targets, shape-aware augmentation, and specialized consistency and ODIoU losses. The teacher is updated by EMA, while the student learns from augmented inputs without adding inference computation.
- Overall Framework: SE-SSD simultaneously trains same-architecture teacher and student SSDs, using teacher predictions as soft targets for the student.The teacher is initialized with the student and later updated from student parameters using exponential moving average.
- Consistency Loss: IoU-based matching filters low-confidence boxes and weakly overlapping pairs before pairing each student box with its highest-IoU teacher box.The strategy aims to avoid unrelated soft targets and retain relatively precise supervision from highly overlapping pairs.
- Consistency Loss: The consistency loss uses Smooth-L1 terms for box dimensions and normalized classification-confidence differences to align student predictions with teacher predictions.Box coordinates, sizes, and orientation are compared using dimension-specific deviations, while sigmoid scores keep confidence differences in a small range.
- Orientation-Aware Distance-IoU Loss: ODIoU supervises hard targets by constraining predicted box-center distance and BEV orientation difference, complementing conventional box regression.Its orientation term encourages predicted boxes to rotate toward the nearest direction parallel to the ground-truth orientation.
- Shape-Aware Data Augmentation: Shape-aware augmentation divides each object point cloud into six pyramidal subsets and independently applies randomized dropout, swap, and sparsification operations.The scheme is designed to mimic occlusion, distance, and shape-related point-pattern variation and encourage inference of complete object shapes.
4. Experiments
Experiments evaluate SE-SSD on KITTI car detection using LiDAR-only inputs, reporting benchmark comparisons, ablations, and runtime. The method achieves strong accuracy while retaining high inference efficiency.
- Evaluation Setup: SE-SSD is evaluated on KITTI 3D and BEV car detection, using average precision at an IoU threshold of 0.7 across easy, moderate, and hard levels.The dataset contains 7,481 training samples and 7,518 test samples; moderate AP is the official ranking metric.
- State-of-the-Art Comparison: SE-SSD ranks first among state-of-the-art methods for both 3D and BEV detection across all three difficulty levels.The comparison uses 40 sampling recall points on the KITTI test set.
- State-of-the-Art Comparison: 2.6 times faster than Deformable PV-RCNN, SE-SSD ranks second in inference speed among compared methods.The reported speed advantage is attributed to refining features within SSD without extra inference computation.
- Ablation Study: ODIoU improves moderate AP by about 0.6 points over Smooth-L1, while consistency loss improves moderate AP by about 0.9 point.The ablations evaluate 3D AP with 40 recall points on the KITTI validation split.
- Ablation Study: The student-based IoU filter yields the largest AP gain among tested soft-target filtering strategies.It retains soft targets most related to and informative for student predictions.
- Efficiency: SE-SSD requires 30.56 ms overall inference time: 2.84 ms preprocessing, 24.33 ms network forwarding, and 3.39 ms post-processing.Measurements use an Intel Xeon Silver CPU and one TITAN Xp GPU.
5. Conclusion
The paper presents SE-SSD as a self-ensembling single-stage detector for outdoor 3D point clouds. Experiments support its proposed constraints and augmentation scheme, showing strong KITTI accuracy with high efficiency.
- Conclusion: SE-SSD combines consistency-constrained soft-target optimization, ODIoU hard-target supervision, and shape-aware augmentation.The framework uses a teacher-student design and enlarged training-sample diversity.
- Conclusion: Experiments show effectiveness for SE-SSD and each proposed module, alongside high efficiency on KITTI 3D and BEV car detection.The conclusion reports outperformance of prior state-of-the-art methods and ultra-high inference speed.
A. KITTI Car Detection Leaderboards
SE-SSD achieves leading KITTI car-detection leaderboard positions against published and unpublished submissions. It ranks first in BEV and second in 3D detection.
- KITTI Car Detection Leaderboards: SE-SSD ranks first on the KITTI BEV car-detection leaderboard.Figure 7 reports the BEV leaderboard position.
- KITTI Car Detection Leaderboards: SE-SSD ranks second on the KITTI 3D car-detection leaderboard.Figure 8 notes that HRI-ADLab-HZ is unpublished.
B. More Ablation Studies
Additional ablations examine shape-aware augmentation operators, ODIoU orientation weighting, and performance beyond car detection. These studies further support the proposed design choices.
- Other Object Categories: SE-SSD is additionally compared with baseline SSD for cyclist detection using 3D and BEV APs on KITTI validation.The cyclist comparison is presented in Table 8.
- Other Object Categories: SE-SSD is additionally compared with baseline SSD for pedestrian detection using 3D and BEV APs on KITTI validation.The pedestrian comparison is presented in Table 9.
- Shape-aware Data Augmentation: Random dropout, swap, and sparsifying each improve 3D moderate AP in shape-aware augmentation experiments.The operators are evaluated on the KITTI validation split for car detection.
- ODIoU Loss: The ODIoU orientation constraint is important for further boosting precision, with γ finally set to 1.25.Different γ values are evaluated using 3D moderate AP on KITTI validation.
C. Experiments on KITTI Cyclist&Pedestrian
On KITTI cyclist and pedestrian validation data, SE-SSD substantially outperforms the baseline SSD across both 3D and BEV detection. Cyclist gains are especially large on the 3D moderate and hard subsets.
- Cyclist Results: SE-SSD outperforms baseline SSD by a large margin for both 3D and BEV cyclist detection.The comparison uses the KITTI validation split.
- Cyclist Results: About 15 points of improvement are reported for cyclist detection on the 3D moderate and hard subsets.
- Pedestrian Results: SE-SSD also outperforms baseline SSD by a large margin for both 3D and BEV pedestrian detection.