BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereo

• Camera-based 3D detection still trails LiDAR-based approaches because depth perception is ill-posed, although temporal views offer a way to construct multi-view stereo.Source: Introduction · p. 1
• Multi-view detectors predict discrete depth distributions for image points and project features into a unified BEV map, using overlap between views to support detection.Source: Introduction · p. 1
• Conventional temporal stereo is costly and struggles with static ego vehicles and moving objects; over 10% of nuScenes frames have static ego vehicles and approximately 25% of objects are moving.Source: Introduction · p. 1
• BEVStereo builds a sparse cost volume by dynamically sampling a small number of reference candidate features along the depth axis, using predicted depth center µ and range σ.Source: Introduction · pp. 1–2
• An EM-inspired evolution updates µ and σ to refine candidate features for moving objects and stationary ego vehicles; size-aware Circle NMS also uses box size to remove duplicate boxes.Source: Introduction · p. 2
• The proposed methods are instantiated in BEVDepth as BEVStereo and evaluated on the nuScenes benchmark.Source: Introduction · p. 2
• BEVStereo improves nuScenes mAP and NDS by 1.7% each and achieves new state-of-the-art performance on the camera-only track.Source: Introduction · p. 2

Single-view 3D Object Detection

• Single-view methods estimate 3D objects from image cues or monocular representations, while DFM integrates temporal stereo to improve depth estimation and minimize effects from difficult situations.Source: Single-view 3D Object Detection · p. 2

Multi-view 3D Object Detection

• Multi-view detectors include LSS-based and transformer-based schemas; BEVDet combines LSS feature extraction with a LiDAR detection head, while later approaches add temporal inputs, depth supervision, or transformer attention.Source: Multi-view 3D Object Detection · p. 2

Depth Estimation

• Depth estimation spans single-view CNN approaches and multi-view cost-volume methods, including recurrent, CRF, point-based, cascade, sparse-volume, and adaptive patchmatch variants.Source: Depth Estimation · p. 2

Method

• BEVStereo is a stereo-based multi-view 3D detector that also proposes size-aware circle NMS for proposal suppression.Source: Method · pp. 2–3

Preliminary Knowledge

• LSS-based multi-view detectors extract image features, generate depth and context, transform features to BEV, and use a 3D detection head.Traditional MVS constructs a cost volume by sampling depth candidates, homography-warping source features, and regularizing the volume with 3D convolution.Source: Preliminary Knowledge · p. 3

Dynamic Temporal Stereo

• BEVStereo combines mono and weighted temporal-stereo depth, then uses the resulting BEV features for 3D detection.A Weight Net compares warped source-frame mono depth with reference-frame mono depth to create the stereo-depth weight map.Source: Dynamic Temporal Stereo · p. 3, Figure 1 · p. 4
• The depth module estimates µ and σ to dynamically select depth candidates, warp source features, and predict confidence scores for those candidates.µ and σ represent the cost volume’s depth center and range; the warped source and reference features are used to construct the cost volume.Source: Dynamic Temporal Stereo · p. 3, Figure 2 · p. 4
• An EM-inspired update moves µ toward the confidence-weighted expectation of candidate depths and adjusts σ to confidence, without learnable parameters.When all candidates have low probability, the weight-sum update can maintain µ; depth is split into ranges to reduce the risk of µ starting far from ground truth.Source: Dynamic Temporal Stereo · p. 3

Size-aware Circle NMS

• Size-aware circle NMS uses box-size-adjusted x- and y-axis center-distance thresholds to suppress boxes without computing rotated IoU.The thresholds account for box dimensions and orientation; suppression occurs when both axis distances fall below their respective thresholds.Source: Size-aware Circle NMS · pp. 4–5, Figure 3 · p. 4

Experimental Settings

• Experiments use nuScenes, with LiDAR and image data for training but image data alone for inference; evaluation includes detection, depth, and memory metrics.The image input comprises the key frame and furthest connected sweep, while LiDAR input uses only the key frame.Source: Experimental Settings · p. 5

Analysis

• Compared with BEVDepth using MVS, BEVStereo reduces memory use while improving detection metrics; it also improves BEVDepth's mAP, mATE, and NDS and depth-prediction accuracy.The comparisons are reported in Tables 6, 1, and 2, respectively.Source: Analysis · p. 5, Table 6 · p. 6, Table 1 · p. 6
• Under moving-object and low-ego-velocity conditions, BEVStereo improves performance when MVS fails; it also produces similar results in these conditions without inference-time µ updates.Moving-object recall analysis retains boxes with velocity higher than 1 m/s; Table 4 retains boxes with velocity lower than 1 m/s.Source: Analysis · p. 5, Table 3 · p. 6, Table 4 · p. 6

Ablation Study

• Detection results improve with more µ and σ iterations and with Weight Net; size-aware circle NMS improves class-aware mAP, mATE, and NDS and continues to function under class-agnostic NMS.The iteration, Weight Net, and NMS findings are reported in Tables 7, 1, and 8, respectively.Source: Ablation Study · p. 6, Table 7 · p. 6, Table 8 · p. 6
• Efficient Voxel Pooling v2 reduces latency by up to 40% compared with v1 across the tested resolutions.Its thread mapping processes point features by warp and reduces performance-limiting memory transactions.Source: Ablation Study · pp. 6, 8

Visualization

• Visualizations show improved depth estimation on both moving and static objects, alongside detection examples with more accurate localization and objects found by BEVStereo but missed by the baseline.Figure 5 depicts depth predictions against point-cloud ground truth; Figure 6 marks the detection examples.Source: Figure 5 · p. 7, Figure 6 · p. 7, Visualization · p. 8

Benchmark Result

• BEVStereo achieves the highest mAP and NDS among camera-based methods on both nuScenes test and validation sets.The test-set comparison is in Table 9; the validation-set comparison is in Table 10.Source: Table 9 · p. 7, Benchmark Result · p. 8, Table 10 · p. 8

Conclusion

• BEVStereo applies dynamic temporal stereo to improve multi-view 3D detection without significantly increasing memory usage.The paper also states that its method can resolve some complex scenarios other stereo-based approaches cannot handle.Source: Conclusion · p. 8
• Size-aware circle NMS accounts for box size while avoiding the laborious computation of rotated IoU.The paper reports satisfactory performance under both class-aware and class-agnostic circumstances.Source: Conclusion · p. 8
• Efficient Voxel Pooling v2 speeds up voxel pooling by improving memory-access efficiency.Source: Conclusion · p. 8