Source-linked AI summary
Bi-Level Routing and Sparse Spatial Attention based Multi-View BEV 3D Object Detection for Autonomous Driving
Jing Zhang, Jiaqi Liu, Zibo Wang
TL;DR
Multi-view BEV 3D detection faces computational cost, insufficient multi-scale feature extraction, and inefficient dense 2D-to-BEV transformation. Sparse-BEVNet combines BRA, CGA, and SSCA to address these issues, achieving 45.2% mAP and 54.5% NDS on nuScenes, surpassing the BEVFormer-S baseline by 3.6% and 2.8% respectively.
Problem
Multi-view BEV 3D detection is limited by computational cost, insufficient multi-scale feature extraction, and inefficient dense 2D-to-BEV view transformation.
Method
Sparse-BEVNet combines BRA for backbone routing, CGA for multi-scale feature fusion, and SSCA for sparse geometry-guided view transformation.
Results
45.2% mAP and 54.5% NDS surpass the BEVFormer-S baseline by 3.6% and 2.8% absolute improvements, respectively, on nuScenes.
Takeaways & Limitations
Sparse-BEVNet outperforms the baseline in both mAP and NDS while addressing dense projection and feature-processing inefficiencies.
Abstract
from arXiv · showhide
Bird's Eye View (BEV)-based multi-view 3D object detection suffers from challenges of computational complexity, multi-scale feature extraction, and efficiency of dense 2D-to-BEV view transformation. To address these problems, this paper proposes an improved BEV 3D object detection algorithm Sparse-BEVNet. Firstly, a Bi-Level Routing Attention (BRA) mechanism is introduced into the image feature extraction network to reduce the computational burden of the backbone. Second, Cascaded Group Attention (CGA) is employed in the feature fusion module, which enhances deep interaction across features of different hierarchical levels without introducing additional computational overhead. Furthermore, a Sparse Spatial Cross-Attention mechanism is adopted to replace the conventional dense view projection pipeline. Experimental results on the public nuScenes dataset demonstrate that the proposed method achieves a mean Average Precision (mAP) of 45.2% and a nuScenes Detection Score (NDS) of 54.5%, corresponding to 3.6% and 2.8% improvements relative to the baseline model, respectively.
I. INTRODUCTION
Sparse-BEVNet addresses the computational and interaction inefficiencies of multi-view BEV 3D detection with three targeted mechanisms for feature extraction, fusion, and view transformation.
- Multi-view BEV perception transforms image features from multiple cameras into an ego-vehicle-centric 3D space.
- Dense 2D-to-BEV projection wastes computation on meaningless background regions and constrains real-time inference.
- Sparse-BEVNet introduces BRA, CGA, and SSCA to reduce backbone computation, improve multi-scale feature interaction, and replace dense view projection.The framework targets computational burden, feature extraction, and dense 2D-to-BEV transformation limitations.
A. Overall Network Architecture
Sparse-BEVNet is organized as a modular multi-view 3D detection pipeline that extracts, fuses, sparsely projects, and decodes image-derived features.
- A. Overall Network Architecture: The framework is designed through modular reconstruction based on Sparse4D and BEVFormer.
- A. Overall Network Architecture: The architecture contains a BiFormer-based backbone, CGA feature-fusion neck, SSCA mechanism, and detection head.
- A. Overall Network Architecture: Figure 1 presents the overall architecture of Sparse-BEVNet.
B. Backbone Feature Extraction Based on Bi-Level Routing
The backbone replaces conventional fixed or local attention with BRA, which routes computation toward semantically relevant regions before fine-grained interaction.
- B. Backbone Feature Extraction Based on Bi-Level Routing: BRA addresses background-weighted computation and limited context by reconstructing the backbone with a lightweight BiFormer-based feature extractor.
- B. Backbone Feature Extraction Based on Bi-Level Routing: The method computes coarse region affinities and selects the top-k relevant regions to form a directed sparse dependency graph.
- B. Backbone Feature Extraction Based on Bi-Level Routing: For each region, BRA gathers key and value tokens only from the k routed relevant regions instead of using global or fixed-window interactions.
- B. Backbone Feature Extraction Based on Bi-Level Routing: Fine-grained attention is then computed between current-region query tokens and the dynamically collected context tensors.
C. Feature Fusion Based on Cascaded Group Attention
CGA replaces simple FPN concatenation with grouped, cascaded attention that deepens cross-scale interaction without additional parameter overhead.
- C. Feature Fusion Based on Cascaded Group Attention: CGA addresses redundant attention and weak cross-scale interaction in standard FPN fusion for objects with large scale variation.
- C. Feature Fusion Based on Cascaded Group Attention: The module divides multi-level features into channel groups and assigns each group a separate attention head.
- C. Feature Fusion Based on Cascaded Group Attention: Each group receives the preceding group’s attention output as contextual input, preventing information isolation between groups.
- C. Feature Fusion Based on Cascaded Group Attention: The grouped outputs are re-concatenated and linearly projected into an enhanced multi-scale 2D feature set.
D. Sparse Spatial Cross-Attention
Sparse Spatial Cross-Attention replaces dense grid-based view transformation with geometry-guided sparse sampling from projected 3D anchors. It aggregates local multi-scale image features only from visible camera views.
- D. Sparse Spatial Cross-Attention: SSCA replaces dense 3D grid projection with deterministic geometric mapping from sparse instance anchors to multi-view image features.The method initializes sparse query features and 3D reference points, then projects them using camera calibration parameters.
- D. Sparse Spatial Cross-Attention: Projected anchors undergo local deformable sampling and feature aggregation instead of time-consuming global feature matching.Sampling adapts to object-scale variation and projection errors through learned offsets and attention weights.
- D. Sparse Spatial Cross-Attention: SSCA aggregates features across visible camera views and multiple feature levels before sending enriched sparse queries to the detection head.Visible views exclude projections outside image boundaries or occluded by the ego-vehicle; the head decodes each sparse instance into 3D box parameters.
A. Experimental Settings
Experiments use a uniformly sampled nuScenes subset with six synchronized surround-view cameras, an 8:2 train-validation split, and fixed training settings. Images are resized and cropped to 704×256 pixels for 100-epoch AdamW training.
- A. Experimental Settings: nuScenes provides synchronized images from six surround-view cameras across sunny, rainy, and nighttime environments.The experiments target single-frame multi-view perception using the six-camera surround-view setup.
- A. Experimental Settings: The experiments use 200 uniformly sampled nuScenes road scenes split into training and validation sets at an 8:2 ratio.The subset was selected because the complete dataset was not used under experimental hardware and validation-efficiency constraints.
- A. Experimental Settings: Training resizes and crops camera images to 704×256 pixels and uses AdamW with an initial learning rate of 2 × 10^-4.The schedule uses cosine annealing, weight decay of 1 × 10^-2, 100 epochs, and global batch size 8.
B. Comparative Analysis of Main Experiments
On the nuScenes validation subset, Sparse-BEVNet improves both detection accuracy and inference efficiency over BEVFormer-S. Its sparse geometric projection avoids the computational burden of dense view transformation.
- B. Comparative Analysis of Main Experiments: 45.2% mAP and 54.5% NDS exceed the BEVFormer-S baseline by 3.6% and 2.8% absolute improvements, respectively.These results are reported in Table I for the nuScenes validation set.
- B. Comparative Analysis of Main Experiments: The accuracy gain is attributed to BRA filtering invalid backgrounds and CGA integrating cross-view truncated features for hard samples.The cited examples include distant pedestrians.
- B. Comparative Analysis of Main Experiments: Sparse-BEVNet reaches 22 FPS by directly mapping sparse 3D anchors to the 2D plane for local sampling through camera calibration.This avoids computational waste from dense interactions over sky and static road regions.
C. Ablation Experiments and Analysis
Ablations show complementary gains from BRA, CGA, and SSCA, while multi-scale fusion performs best when it includes the full hierarchy from C2 through C5. These modules address computation, cross-scale interaction, and fine-detail capture.
- C. Ablation Experiments and Analysis: Sequentially adding BRA, CGA, and SSCA produces complementary gains over a ResNet-101, standard-FPN, dense-projection baseline.The ablation uses the nuScenes validation subset under consistent settings.
- C. Ablation Experiments and Analysis: BRA alone improves mAP by 1.2% and reduces the backbone to roughly 26.0M parameters and 4.5 GFLOPs while reaching 17 FPS.The ResNet-101 baseline has approximately 44.5M parameters and 7.8 GFLOPs.
- C. Ablation Experiments and Analysis: Adding CGA contributes an additional 0.9% mAP by improving cross-scale interaction for overlapping and distant small-object features.The paper describes CGA as compensating for feature truncation and feature loss.
- C. Ablation Experiments and Analysis: Replacing dense grid queries with SSCA achieves the optimal accuracy and raises inference speed to 22 FPS.The result is linked to geometry-guided sparse projection reducing redundant computation in empty regions.
- C. Ablation Experiments and Analysis: The full C2–C5 CGA cascade achieves the highest mAP and NDS, outperforming C4–C5 and C3–C5 alternatives.Including high-resolution shallow features is identified as important for truncated and small targets.
IV. CONCLUSION
Sparse-BEVNet addresses computational redundancy and insufficient multi-scale feature extraction in multi-view 3D object detection by combining BRA, CGA, and SSCA. On nuScenes, it outperforms the baseline in both mAP and NDS.
- IV. CONCLUSION: Sparse-BEVNet combines BRA, CGA, and SSCA to reduce computational redundancy and improve multi-scale feature representation in multi-view 3D detection.BRA reconstructs the backbone, CGA serves as the feature neck, and SSCA replaces dense view projection in geometric mapping.
- IV. CONCLUSION: BRA and CGA filter background noise and enhance representation for cross-scale and truncated objects.
- IV. CONCLUSION: Sparse spatial cross-attention breaks the computational bottleneck of conventional dense 2D-to-3D view projection.
- IV. CONCLUSION: Sparse-BEVNet outperforms the baseline in both mAP and NDS on the nuScenes dataset.