Source-linked AI summary
HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detection
Maosheng Ye, Shuangjie Xu, Tongyi Cao
TL;DR
HVNet addresses the voxel-size trade-off between fine-grained accuracy and inference efficiency in LiDAR-based 3D detection. It fuses multi-scale voxel features point-wise, uses attentive encoding and feature-map fusion, and achieves state-of-the-art KITTI performance at 31Hz.
Problem
Voxel size requires trading fine geometric detail and small-object localization against feature-map size, computation, memory, and inference speed.
Method
HVNet decouples feature-extraction and pseudo-image projection scales, aggregates hybrid voxel features point-wise, and uses attentive voxel encoding with feature fusion.
Results
HVNet achieves state-of-the-art mAP on KITTI among LiDAR one-stage methods while running at 31Hz.
Takeaways & Limitations
The unified detector combines multi-scale voxel information with high-accuracy detection and real-time inference across KITTI evaluations.
Takeaways & Limitations
Sparse or ambiguous LiDAR observations can split a pickup truck into two cars or misclassify a waste bin and grass as pedestrians or cyclists.
Abstract
from arXiv · showhide
We present Hybrid Voxel Network (HVNet), a novel one-stage unified network for point cloud based 3D object detection for autonomous driving. Recent studies show that 2D voxelization with per voxel PointNet style feature extractor leads to accurate and efficient detector for large 3D scenes. Since the size of the feature map determines the computation and memory cost, the size of the voxel becomes a parameter that is hard to balance. A smaller voxel size gives a better performance, especially for small objects, but a longer inference time. A larger voxel can cover the same area with a smaller feature map, but fails to capture intricate features and accurate location for smaller objects. We present a Hybrid Voxel network that solves this problem by fusing voxel feature encoder (VFE) of different scales at point-wise level and project into multiple pseudo-image feature maps. We further propose an attentive voxel feature encoding that outperforms plain VFE and a feature fusion pyramid network to aggregate multi-scale information at feature map level. Experiments on the KITTI benchmark show that a single HVNet achieves the best mAP among all existing methods with a real time inference speed of 31Hz.
1. Introduction
HVNet addresses the accuracy–speed trade-off in voxel-based LiDAR detection by combining multiple voxel scales point-wise while projecting compact pseudo-image features. It reports state-of-the-art KITTI performance with real-time inference.
- 3D object detection recognizes and localizes objects in 3D scenes for applications including robotics and self-driving cars.
- Voxel feature encoding methods voxelize point clouds, encode point features with PointNet-style networks, and project aggregated features into pseudo-image maps.
- Smaller voxels capture finer geometry and improve localization but increase inference time, whereas coarser voxels are faster but perform worse, especially for small objects.
- HVNet fuses features from multiple voxel scales at the point level, decoupling feature-extraction scales from pseudo-image projection scales.
- 31Hz real-time inference accompanies state-of-the-art KITTI performance, including advantages in mAP and Cyclist detection.
2. Related Work
Prior point-cloud detectors use voxelized compact representations, multi-sensor fusion, or two-stage refinement, but often face efficiency, information-loss, or scale-flexibility constraints. HVNet targets these limitations with a hybrid voxel representation.
- Multi-sensor detectors add information but generally have low inference efficiency and require sensor synchronization, complicating deployment.
- Voxel-based point-cloud methods project sparse data into compact representations, with PointPillars using XY-only voxelization for pseudo-image features.
- PointPillars limits the number of points per voxel, which can cause information loss; HVNet proposes an encoder for lossless 3D representation.
- Two-stage methods improve accuracy through proposal refinement but have inference speeds lower than 20Hz.
- Existing multi-scale voxel methods tie feature maps to voxel sizes, limiting flexibility; HVNet instead separates voxel feature extraction from pseudo-image projection.
3. Approach
HVNet separates voxel feature-extraction scales from pseudo-image projection scales, combining point-wise multi-scale encoding, attentive voxel features, and multi-scale backbone fusion. Its design retains points and voxel indices while using Gather/Scatter operations for efficient sparse processing and predicts boxes with corner-offset localization.
- Overall architecture: HVNet has three stages: hybrid voxel feature extraction, 2D convolutional multi-scale feature fusion, and a detection head for class-specific 3D boxes.The hybrid extractor projects raw point clouds into pseudo-image features before convolutional processing.
- Hybrid voxel feature extraction: Hybrid voxelization records each point’s voxel index across multiple scales while retaining all points instead of using fixed-capacity voxel buffers.Only point-to-voxel mapping tensors are allocated during voxelization.
- Hybrid voxel feature extraction: Compared with standard VFE, HVNet avoids the quadratic feature-map growth caused by finer voxels: reducing voxel scale n times increases NL × NW n2 times in the conventional design.The hybrid strategy uses fine-grained extraction while keeping projection maps compact.
- Hybrid voxel feature extraction: The HVFE module encodes features from several voxel scales into unified point-wise representations, then dynamically projects them into pseudo-image maps at potentially different scales.Feature-extraction scales ST and projection scales SR are decoupled, allowing projection scales outside the input set.
- Attentive voxel feature encoding: Attentive VFE combines point features with voxel-level attention derived from within-voxel feature aggregation and relative coordinates, then aggregates transformed features by max pooling.The attention feature serves as bridge information in both encoder and decoder operations.
- Backbone and detection head: The backbone fuses multi-scale features shallowly through the main stream and deeply through FFPN, while GPU Gather and Scatter operators support index-based propagation.HVNet’s detection head uses BEV anchor-relative corner offsets as its localization objective.
4. Experiments
Experiments on KITTI evaluate HVNet’s accuracy, speed, qualitative detections, and component contributions. Results show strong performance across benchmark comparisons, efficient hybrid-scale processing, and gains from attention, feature fusion, and multi-scale extraction.
- Quantitative Analysis: HVNet outperforms other approaches in mAP and Cyclists while achieving attractive Car and Pedestrian performance at real-time speed.Among one-stage methods, it leads HRI-VoxelFPN by over 1.61% and PointPillars by 8.44% in Moderate.
- Quantitative Analysis: On the KITTI validation set, HVNet achieves the best reported Car performance in both BEV and 3D tasks among methods exposing results.Validation comparisons for Pedestrian and Cyclist are reported across 2D, BEV, and 3D tasks.
- Voxel Scale Study: At a 0.4m feature-projection scale, HVNet matches PointPillar at 0.24m while reducing runtime cost.The comparison uses different hybrid feature and projection voxel scales against PointPillar grid sizes.
- Qualitative Analysis: HVNet produces high-quality 3D detections across classes, including scenes with point-cloud occlusion or densely packed objects.The reported inference time is 32ms, with module timing varying according to the number of input points.
- Multiple Components Study: Adding attention improves BEV Moderate mAP by 2.06, while FPN and FFPN provide gains of 0.58 and 1.42, respectively.Increasing voxel and feature-projection scale numbers to two gives a 2.17 mAP boost, and three scales add another 0.71.
- Hybrid Voxel Feature Extractor: Using three extraction and projection scales provides a tradeoff between speed and performance.The ablation varies the numbers of feature-extraction and projection scales, with the comparison shown in Figure 8.
5. Conclusion
The conclusion presents HVNet as a one-stage detector that aggregates hybrid-scale voxel features into multi-scale pseudo-image representations. Its experiments report state-of-the-art mAP with real-time speed.
- 5. Conclusion: HVNet aggregates hybrid-scale voxel grids into unified point-wise features and projects them into multi-scale pseudo-image features.Attention guides the projection, and a feature fusion pyramid network combines the resulting representations for different categories.
- 5. Conclusion: Experimental studies show state-of-the-art mAP with real-time speed.
1. More About the Attention Mechanism
The attention mechanism is described as a kernel-like transformation of input features. The resulting feature map suppresses background and enhances object shape features.
- Attention Mechanism: Attention transforms input features through learned linear weights and an attention feature, functioning as a second-order kernel for X.The paper writes the transformation as X′=WG · W′X and identifies G as the attention feature.
- Attention Mechanism: With attention, the output feature map suppresses background areas and enhances object shape features.
2. Ablation Studys
The ablations compare localization and loss choices in HVNet. Corner loss performs better than pose loss, while IoU-based and uncertainty-based alternatives do not consistently improve results.
- Corner Loss: Corner loss performs better than the original pose loss on the KITTI validation set.The paper attributes difficulty in directly learning theta partly to angular discontinuity around 0 and 2π.
- Loss Variants: The IoU branch performs well on validation but fails on the test set, while IoU loss fails on both validation and test sets.
- Loss Variants: Uncertainty loss does not improve the result when learning weights for corner, elevation, and classification losses.
3. Quantitative Analysis
The evaluation reports 3D and BEV results on the KITTI validation set, while the test set reports only BEV results in the presented analysis. Overall test results are provided in Table 2.
- KITTI validation results include both 3D and BEV evaluations.
- The KITTI test-set analysis reports only BEV results because of space limitations.
- Overall quantitative test results are provided in Table 2.
4. Qualitative Analysis
Qualitative KITTI results are visualized for validation and test sets, alongside feature visualizations and failure cases. The analysis distinguishes annotation omissions from actual HVNet errors.
- Qualitative results: KITTI validation and test visualizations show HVNet detections using 3D boxes on point clouds and projected 2D boxes on images.Ground-truth boxes are also shown in the validation-set visualization.
- Qualitative results: The paper states that the qualitative results demonstrate the effectiveness of the approach.
- Failure cases: KITTI ground-truth omissions include missing pedestrians and cyclists that HVNet detects.These cases appear in the second examples of the validation and test visualizations.
- Failure cases: Actual HVNet failures include splitting one pickup truck into two cars when points are missing between vehicle parts.LiDAR-only ambiguity also leads to a waste bin being regarded as a pedestrian and grass as a cyclist.
- Feature visualization: The feature visualization presents three output scales, with each point cloud assigned counterpart features at those scales.The first channel of each feature is visualized.