PaperlayerView exact revision on arXiv

Source-linked AI summary

Sparse Recovery Bird's-Eye View Model Soup for Robust Camera-Based 3D Object Detection

Jinwoo Chung, Sangho An, Hyunsung Park, Jangho Kim

arXiv:2610.34976v1cs.CVcs.AIcs.LG

TL;DR

Camera-based BEV models such as Fast-BEV have promising speed and accuracy, but their robustness to real-world corruptions is insufficiently explored. SR-BEVS diversifies sparse-recovered candidates while preserving mode connectivity, then averages them into a dense model. It improves corruption robustness and clean accuracy over Fast-BEV without inference overhead.

  • Problem

    The robustness of camera-based BEV methods to real-world corruptions remains insufficiently explored, although such robustness matters for autonomous-driving perception.

    Source: §I · p. 1

  • Method

    SR-BEVS uses random pruning of low-magnitude weights and recovery fine-tuning to diversify candidates while preserving mode connectivity, then averages them into one dense model.

    Source: §I · p. 1, B. Sparse Recovery Bird’s-Eye View… · p. 4

  • Results

    SR-BEVS improves corruption robustness by up to 33.8% and clean accuracy by 1.8% over Fast-BEV.

    Source: §I · p. 2

  • Takeaways & Limitations

    The recovered candidates average into a model with the original Fast-BEV architecture and zero inference overhead.

    Source: B. Sparse Recovery Bird’s-Eye View… · p. 4

Abstract

from arXiv · show

Camera-based bird's-eye view (BEV) perception methods such as Fast-BEV have demonstrated promising results with fast inference speed and competitive accuracy. However, their robustness to real-world corruptions remains insufficiently explored, posing risks for autonomous driving deployment where environmental variations are inevitable. In this paper, we propose Sparse Recovery Bird's-Eye View Model Soup (SR-BEVS), a sparse recovery learning approach that simultaneously improves clean accuracy and corruption robustness without requiring additional data or architectural modifications. Our method identifies pruning candidates based on layer-wise weight magnitudes, applies random pruning to create diverse sparse models, and retrains each model with a randomly sampled learning rate. By averaging these diverse recovered models into a single dense model, we achieve enhanced robustness against various corruptions while introducing zero inference overhead. We further provide a mode connectivity analysis demonstrating that our pruning-based diversification preserves the connected low-loss basin necessary for stable weight averaging. Experiments on the nuScenes dataset demonstrate up to 33.8% improvement in corruption robustness and a 1.8% gain in clean accuracy over the Fast-BEV baseline, with consistent improvements across sensor noise, weather distortions, and illumination changes. Our simple yet effective approach may be applicable to other BEV models.

I. INTRODUCTION

Fast-BEV offers efficient camera-based BEV detection, but its robustness to corruptions is insufficiently evaluated and degrades sharply under adverse inputs. SR-BEVS creates diverse candidates through controlled pruning and recovery, then averages them while preserving mode connectivity.

  • SR-BEVS consistently beats Fast-BEV and standard Model Soup across corruption types, with up to 33.8% robustness improvement and 1.8% clean-accuracy gain.Source: Figure 1 · p. 1, §I · p. 2
  • Fast-BEV’s corruption robustness has not been systematically evaluated, and severe Gaussian noise can reduce its mAP from 0.277 to 0.017.Failures under adverse weather, illumination changes, or sensor degradation can compromise safety-critical decisions.Source: §I · p. 1
  • Standard Model Soup yields only marginal gains because similarly trained Fast-BEV checkpoints converge to similar solutions, leaving little diversity for averaging.Forcing stronger diversity risks breaking mode connectivity and making weight averaging cross high-loss regions.Source: §I · p. 1
  • SR-BEVS seeks diversity without leaving the connected low-loss basin, enabling stable averaging of candidates with complementary representations.The paper presents this combination of pruning-induced diversity and mode connectivity as key to gains beyond standard Model Soup.Source: §I · pp. 1–2
  • SR-BEVS applies one-time random pruning of low-magnitude weights, briefly recovers candidates, and averages them into a single dense model.Pruning creates distinct sparsity patterns while keeping candidates near the original solution; the final model adds no inference overhead.Source: §I · p. 2

II. RELATED WORK

Related work establishes Fast-BEV as an efficient camera-based BEV baseline and describes corruption robustness, weight averaging, and pruning. SR-BEVS uses random unstructured pruning to diversify candidates while preserving connectivity for averaging.

  • Fast-BEV uses pre-computed Look-Up-Tables to reach 52.6 FPS without depth prediction or transformers, balancing speed and accuracy.The paper selects Fast-BEV as its baseline.Source: §II · pp. 2–3
  • Camera-based 3D detection lacks a corruption benchmark comparable to ImageNet-C, while existing approaches use augmentation or adversarial training that may not anticipate every corruption.The paper tests Fast-BEV using clean training data across seven corruption types and 17 severity levels.Source: §II · p. 3
  • Model Soup averages fine-tuned model weights, but candidates differing only in learning rate or schedule tend to be similar, limiting gains.Safe interpolation depends on candidates remaining in the same loss basin; prior work links diversity alongside connectivity to out-of-distribution robustness.Source: §II · p. 3
  • Unlike compression-oriented pruning, SR-BEVS uses random unstructured removal of low-magnitude weights to diversify candidates while limiting perturbations.Aggressive pruning or removing influential parameters can distort the optimization landscape and break mode connectivity; unstructured pruning typically needs specialized support for latency benefits.Source: §II · p. 3
  • On M0, mAP and NDS remain stable throughout linear interpolation between SR-BEVS candidate pairs, indicating no significant loss barrier.The figure plots mAP and NDS against interpolation coefficient α.Source: Figure 4 · p. 3

A. Backbone Architecture: Fast-BEV

Fast-BEV uses fixed-geometry camera-to-BEV transformation for efficient inference. SR-BEVS diversifies pretrained models through layer-wise candidate selection, random pruning, recovery fine-tuning, and averaging into a dense model.

A. Backbone Architecture: Fast-BEV

  • Fast-BEV uses fixed camera geometry for view transformation, with LUT projection, multi-view-to-one-voxel mapping, and space-to-channel reshaping.These components avoid runtime projection calculations, separate per-camera voxel aggregation, and expensive 3D convolutions.Source: A. Backbone Architecture: Fast-BEV · p. 3

B. Sparse Recovery Bird’s-Eye View Model Soup

  • SR-BEVS creates a layer-wise pool of low-magnitude weights as pruning candidates, with the threshold selecting a specified fraction in each layer.For example, p = 0.2 selects the lowest 20% of weights per layer, limiting excessive pruning in any single layer.Source: B. Sparse Recovery Bird’s-Eye View… · p. 4
  • Each candidate starts from the shared pretrained checkpoint, receives a random pruning ratio and subset of candidate weights, then is fine-tuned at a randomly sampled learning rate.The distinct pruning selections create different sparsity patterns.Source: B. Sparse Recovery Bird’s-Eye View… · p. 4
  • Pruning occurs once before fine-tuning, and the removed weights can recover nonzero values, leaving each candidate dense after recovery.Candidates recover from different damage patterns while remaining close to the pretrained solution.Source: B. Sparse Recovery Bird’s-Eye View… · p. 4
  • Uniformly averaging the recovered candidates produces a model with the original Fast-BEV architecture and parameter count, adding zero inference overhead.The candidates are projected onto two principal weight-space directions in Fig. 5; SR-BEVS candidates are more spread out while remaining in the same low-loss basin.Source: B. Sparse Recovery Bird’s-Eye View… · p. 4, Figure 5 · p. 4

C. Mode Connectivity Analysis

The analysis tests whether SR-BEVS candidates remain connected within a low-loss region, a condition for safe weight averaging. Interpolation and loss-landscape results indicate stable performance and increased candidate diversity within that basin.

C. Mode Connectivity Analysis

  • Candidate models should occupy the same low-loss region or connect through paths that avoid high-loss areas for weight averaging to improve performance.The analysis tests this condition with loss barrier measurements.Source: C. Mode Connectivity Analysis · p. 4
  • The protocol linearly interpolates between two candidates and evaluates validation performance along the path; a middle-path collapse signals different basins and unsafe averaging.Stable performance across the path indicates same-basin connectivity.Source: C. Mode Connectivity Analysis · pp. 4–5
  • On M0, mAP and NDS remain stable across candidate interpolation paths, with no significant loss barrier, and the candidates lie within the same low-loss basin.The loss-landscape view shows SR-BEVS candidates more spread out than standard Model Soup candidates while remaining in that basin.Source: C. Mode Connectivity Analysis · p. 5
  • Connectivity supports safe averaging, while pruning-induced diversity supplies complementary representations; the paper attributes SR-BEVS’s consistent gains over baseline and standard Model Soup to meeting both conditions.The passage identifies low loss barriers as necessary but says connectivity alone is insufficient when candidates are too similar.Source: C. Mode Connectivity Analysis · p. 5

A. Experimental Setup

Experiments use camera-only nuScenes data and evaluate detection under multiple driving-related corruptions. The setup compares Fast-BEV, additional fine-tuning, Model Soup, and SR-BEVS, while the final model adds no inference overhead.

A. Experimental Setup

  • The camera-only nuScenes setup uses 28,130 training samples and 6,019 validation samples, with four temporal frames from six cameras per sample.The dataset includes 3D boxes for 10 categories within a 100 m × 100 m BEV range.Source: A. Experimental Setup · p. 5
  • The corruption benchmark covers seven types across 17 corrupted validation sets, including noise, blur, fog, contrast, brightness, and occlusion at varying severities.Each corruption is applied simultaneously to all six camera views, and each configuration uses the full 6,019-sample validation set.Source: A. Experimental Setup · p. 5
  • Detection is evaluated with mAP for accuracy and NDS, a composite score that also reflects translation, scale, and orientation errors..Source: A. Experimental Setup · p. 5
  • The comparison includes unmodified Fast-BEV, Fast-BEV fine-tuned with the recovery training budget, three-model Model Soup, and SR-BEVS averaging three recovered dense models.The matched fine-tuning budget isolates extra training from pruning-induced diversity.Source: A. Experimental Setup · p. 5
  • SR-BEVS has the same inference architecture and parameter count as one Fast-BEV model, with extra cost limited to recovery training that scales linearly with candidate count.The paper reports diminishing performance returns beyond N=3 in its ablation.Source: A. Experimental Setup · p. 5

B. Quantitative Results

SR-BEVS outperforms the baselines on clean data and across corruption types and severity levels in M0 and M2.

Clean Data Performance

  • SR-BEVS improves clean-data results in both configurations: M0 reaches 0.282 mAP and 0.417 NDS, while M2 reaches 0.332 mAP and 0.460 NDS.These exceed Fast-BEV and standard model soup in each configuration.Source: B. Quantitative Results · p. 6

Corruption Robustness

  • Across M0 and M2, SR-BEVS improves performance across all tested corruption types and severity levels; M0 Gaussian-noise NDS rises 0.077→0.103 (+33.8%), and M2 blur NDS rises 0.171→0.203 (+18.7%).For M0 under heavy fog, mAP improves 0.173→0.192 (+11.0%).Source: B. Quantitative Results · p. 6
  • These gains across diverse corruptions enhance resilience to real-world visual degradations relevant to safe autonomous-driving deployment.Source: B. Quantitative Results · p. 6

C. Qualitative Results

On M0, qualitative examples under severe Gaussian noise and heavy fog show SR-BEVS avoiding false positives and missed detections seen with Fast-BEV and Model Soup.

Severe Gaussian noise: false positives

  • Under strong Gaussian noise, SR-BEVS reduces false positives by avoiding detections of noise patterns as objects, unlike the baseline.Averaging suppresses extreme candidate confidence scores and box predictions, producing more conservative, stable predictions.Source: C. Qualitative Results · p. 6

Heavy fog: missed detections

  • Under heavy fog, SR-BEVS detects objects missed by the baseline, with complementary candidate predictions recovering missed detections.The examples compare two frequently encountered driving corruptions: heavy fog and severe Gaussian noise.Source: C. Qualitative Results · p. 6

Qualitative comparison

  • Fig. 6 contrasts Fast-BEV and Model Soup errors with SR-BEVS detections: noise-pattern false positives above and fog-related misses below.Source: Figure 6 · p. 7

D. Ablation Study

The ablations use M0 on clean nuScenes validation data unless otherwise stated, examining pruning-candidate ratio, candidate count, and diversity.

Optimal Pruning Candidate Ratio

  • A pruning-candidate ratio of p = 0.2 yields 0.282 mAP and 0.417 NDS, outperforming p = 0.1 and p = 0.3.The smaller pool produces insufficiently distinct sparsity patterns; the larger pool includes higher-magnitude weights that are harder to recover during short fine-tuning.Source: D. Ablation Study · p. 7

Number of Candidate Models

  • Using N = 3 reaches 0.282 mAP and 0.417 NDS; N = 5 yields a marginal increase to 0.284 mAP and 0.419 NDS.The study selects N = 3 as a training-cost/performance trade-off because each additional candidate requires a separate fine-tuning run.Source: D. Ablation Study · p. 7, Table IV · p. 7

Model Diversity Analysis

  • SR-BEVS increases candidate-pair L2 distance approximately 3× for M0 and 4× for M2 versus standard model soup; M2 has more parameters for distinct sparsity patterns.For M0, L2 distance rises from 1.706 to 5.064; for M2, from 5.795 to 24.389.Source: D. Ablation Study · pp. 7–8, Table V · p. 7
  • For M0, corrupted average NDS improves by 0.009 (0.324→0.333), versus a clean NDS improvement of 0.005 (0.412→0.417); M2 gains are 0.008 and 0.003, respectively.The reported gains are larger on corrupted than clean data for both configurations.Source: Table V · p. 7, D. Ablation Study · p. 8

V. CONCLUSION

The conclusion presents SR-BEVS as a training-time strategy for improving Fast-BEV robustness without increasing inference cost, while noting the current validation scope.

V. CONCLUSION

  • SR-BEVS uses pruning-induced diversity while preserving mode connectivity, then averages recovered candidates into one dense network with the same architecture and zero inference overhead.Source: §V · p. 8
  • On nuScenes, SR-BEVS consistently outperforms Fast-BEV and standard model soup, improving clean accuracy and reducing corruption-related degradation, with up to 33.8% robustness gain under severe noise.Source: §V · p. 8
  • Validation is currently limited to Fast-BEV and synthetic corruptions.The conclusion describes the approach as a practical training-time recipe for deployment-constrained camera-based BEV perception.Source: §V · p. 8

Related papers

Loading 2610.34976v1…