Source-linked AI summary
Transformation-Equivariant 3D Object Detection for Autonomous Driving
Hai Wu, Chenglu Wen, Wei Li, Xin Li, Ruigang Yang, Cheng Wang
TL;DR
Ordinary 3D detectors do not explicitly model rotation and reflection variations, while existing equivariant approaches are computationally expensive for autonomous driving. TED addresses this gap with equivariant sparse voxel features and compact aggregation, achieving first place on KITTI with competitive efficiency.
Problem
Conventional detectors lack explicit rotation and reflection equivariance, while existing equivariant methods require substantial computation for autonomous-driving detection.
Method
TED extracts multi-channel transformation-equivariant voxel features with sparse convolutions and aligns them into lightweight scene- and instance-level representations.
Results
TED ranks 1st among KITTI 3D detection submissions and achieves 85.28% AP for the moderate Car class with competitive speed.
Takeaways & Limitations
TED demonstrates that transformation-equivariant features can support high-performance 3D detection while maintaining competitive efficiency.
Takeaways & Limitations
TED is not strictly transformation-equivariant because transformation discretization and input voxelization introduce approximation; finer discretization increases computational cost.
Abstract
from arXiv · showhide
3D object detection received increasing attention in autonomous driving recently. Objects in 3D scenes are distributed with diverse orientations. Ordinary detectors do not explicitly model the variations of rotation and reflection transformations. Consequently, large networks and extensive data augmentation are required for robust detection. Recent equivariant networks explicitly model the transformation variations by applying shared networks on multiple transformed point clouds, showing great potential in object geometry modeling. However, it is difficult to apply such networks to 3D object detection in autonomous driving due to its large computation cost and slow reasoning speed. In this work, we present TED, an efficient Transformation-Equivariant 3D Detector to overcome the computation cost and speed issues. TED first applies a sparse convolution backbone to extract multi-channel transformation-equivariant voxel features; and then aligns and aggregates these equivariant features into lightweight and compact representations for high-performance 3D object detection. On the highly competitive KITTI 3D car detection leaderboard, TED ranked 1st among all submissions with competitive efficiency.
Introduction
TED addresses the efficiency challenge of transformation-equivariant 3D detection by combining equivariant voxel processing with compact feature aggregation. It achieves strong KITTI and Waymo results, including first place on the KITTI leaderboard with competitive speed.
- Most conventional voxel- and point-based detectors do not explicitly model rotation and reflection equivariance.
- Data augmentation and test-time augmentation improve transformation robustness but require comprehensive samples, larger networks, or repeated inference.
- TED uses a sparse convolution backbone to extract multi-channel equivariant voxel features, then aligns and aggregates them into lightweight representations.
- TED combines TeSpConv, TeBEV pooling, TiVoxel pooling, and distance-aware augmentation for efficient detection and improved distant-object training.
- 85.28% AP in the moderate Car class on KITTI earned TED 1st place among submissions, while experiments covered KITTI and Waymo.
Related Work
Prior work includes voxel-, point-, and image-based 3D detectors, alongside increasingly explicit transformation-equivariant models. TED differs by targeting efficient, real-time outdoor-scene detection and sparse augmentation for distant objects.
- Voxel-based methods use sparse convolution for single- or two-stage 3D detection, while point-based methods use set abstraction operations.
- Transformation-equivariant networks model translation, rotation, or reflection for 3D data, detection, and pose estimation.
- EON is the closest related work, but it does not consider efficiency; TED targets real-time outdoor object detection.
- Copy-and-paste, occlusion-aware, and part-swapping augmentations diversify training objects, whereas TED creates sparse samples from nearby dense objects for distant detection.
Preliminaries
The paper defines equivariance as transforming an operation's output consistently with its transformed input, and applies this principle to 2D bird's-eye-view transformations. Its detector predicts 3D boxes from point clouds under a discrete rotation-reflection group.
- Equivariance requires an operation's output to transform consistently when its input is transformed.
- For autonomous-driving scenes, the analysis focuses on transformations occurring on the 2D BEV plane.
- The transformation group combines 2D translations with discrete rotations and reflections, yielding a subgroup of order 2N.
- The detector maps an input point cloud P to 3D bounding boxes B encoded by coordinates, size, and orientation.
Our Method
TED combines transformation-equivariant sparse convolution with alignment and aggregation modules that compress multi-channel features for efficient 3D detection. Its pipeline includes scene-level proposal generation, instance-level proposal refinement, and distance-aware augmentation for sparse distant objects.
- TeSpConv: TED stacks shared sparse convolutions over transformed point clouds to encode multi-channel rotation- and reflection-equivariant voxel features.The TeSpConv backbone extends sparse convolution by adding transformation channels.
- TeBEV pooling: TeBEV pooling aligns scene-level voxel features and aggregates them by bilinear interpolation and max pooling into a compact BEV map for proposal generation.The resulting lightweight representation is passed to the region proposal network.
- TiVoxel pooling: TiVoxel pooling aligns proposal-centered grid points across transformation channels and aggregates instance-level features into compact transformation-invariant representations.It uses multi-grid pooling followed by cross-grid attention before proposal refinement.
- TiVoxel pooling: Cross-grid attention concatenates grid-wise features across transformation channels, applies learned projections, and averages them into a single feature vector for proposal refinement.The flattened vector is used similarly to recent two-stage detectors.
- Distance-Aware Data Augmentation: Distance-aware augmentation creates sparse distant-object samples by simulating LiDAR scanning and occlusion rather than randomly sampling points.Points are voxelized in spherical coordinates using LiDAR angular resolution, retaining points near voxel centers before occlusion is simulated.
- Results: On the KITTI validation set, TED-S outperforms previous LiDAR-only methods and TED-M outperforms previous multi-modal methods.The table caption states these comparisons for the respective detector settings.
Experiments
TED is evaluated on KITTI and Waymo across benchmark comparisons, ablations, transformation robustness, orientation accuracy, and efficiency. It achieves strong detection results while maintaining competitive inference speed.
- Benchmark results: 3.66% improvement over Voxel-RCNN on moderate Car was achieved by TED-M on the KITTI test set.TED-M also outperformed previous methods in all moderate Car, Pedestrian, and Cyclist classes.
- Benchmark results: 1.83%, 7.63%, and 4.97% improvements over Voxel-RCNN were obtained on Waymo Vehicle(L2), Pedestrian(L2), and Cyclist(L2), respectively.TED-S also outperformed all previous methods on all reported metrics.
- Efficiency and ablations: 11 FPS was achieved with N = 3 rotation transformations on a single 3090 GPU, balancing detection accuracy and computational efficiency.The detection performance for N = 3 was close to N = 4, motivating the selected configuration.
- Efficiency and ablations: 0.43%, 0.61%, and 1.58% improvements came from DA-Aug, TeBEV pooling, and TiVoxel pooling in the LiDAR-only baseline.The corresponding multi-modal improvements were 0.49%, 0.62%, and 1.7%.
- Robustness and orientation: More stable predictions under transformed inputs and significant AOS improvements were observed for TED-S and TED-M relative to the baseline.The transformation robustness is evaluated with performance error bars, while orientation accuracy is measured using official AOS.
Conclusion
TED encodes transformation-equivariant voxel features into compact scene-level and instance-level representations for efficient 3D object detection. It ranks first on the KITTI benchmark, but remains discretely rather than strictly equivariant and requires more GPU memory than VoxelRCNN.
- TED encodes transformation-equivariant voxel features into compact scene-level and instance-level representations for proposal generation and refinement.
- TED ranks 1st among all submissions on the KITTI 3D object detection benchmark.
- TED is not strictly transformation-equivariant because transformations are discretized and inputs are voxelized.
- Using more transformations and smaller voxels would move TED closer to full equivariance but increase computational cost.
- TED excludes scaling transformations because adding transformations increases computation and scaling is observed less often in practical scenes.
- TED requires about 2× the overall GPU memory of the VoxelRCNN baseline.