PaperlayerView exact revision on arXiv

Source-linked AI summary

Stereo 4D Radar for 3D Object Detection: Integrating Geometric Alignment and Absolute Velocity Estimation

Seung-Hyun Song, Dong-Hee Paek, Woong-Chan Byun, Seung-Hyun Kong

arXiv:2609.02560v1cs.CV

TL;DR

4D Radar offers weather-robust sensing and Doppler information, but clutter-induced sparsity and radial-only velocity limit perception. The paper introduces stereo absolute-velocity estimation and geometry-aware feature fusion, reporting improved 3D detection on an in-house stereo dataset.

  • Problem

    Clutter, ghost reflections, and noise make processed 4D Radar data excessively sparse, while radial-only Doppler measurements do not fully represent object motion.

  • Method

    The framework uses geometric disparity between two Radars to estimate absolute velocity and integrates SR-PFE and SR-BFM to align and fuse complementary stereo features.

  • Results

    8.82% in AP3D and 9.0% in APBEV improvements are reported over state-of-the-art mono 4D Radar baselines.

  • Takeaways & Limitations

    Stereo absolute velocity estimation and feature fusion improve geometric consistency and representation quality beyond mono 4D Radar methods.

Abstract

from arXiv · show

Four-dimensional (4D) Radar is a powerful sensing modality capable of detecting surrounding three-dimensional (3D) objects under diverse weather conditions and providing Doppler-based motion information. However, raw 4D Radar signals contain significant clutter from road surfaces, guardrails, and surrounding vehicles, along with multipath-induced ghost reflections and the receiver's inherent noise floor. Consequently, preprocessing algorithms designed to remove such invalid measurements often make the Radar data excessively sparse. Moreover, the Doppler measurements provided by 4D Radar describe only the radial component of an object's velocity, limiting their ability to recover the full motion state. In this paper, we introduce a stereo 4D Radar-based 3D object detection framework that exploits the geometric disparity between left and right Radars to estimate the absolute velocity of objects and achieve more robust perception through the fusion of their complementary features. The effectiveness of the proposed framework is validated on our in-house stereo 4D Radar dataset, demonstrating performance gains of 8.82 points in AP 3D and 9.0 points in AP BEV over state-of-the-art mono 4D Radar baselines. These results demonstrate that absolute velocity estimation combined with stereo geometry-aware feature fusion leads to substantial improvements in 3D object detection.

I. INTRODUCTION

The paper addresses incomplete Doppler motion information and excessive sparsity in mono 4D Radar by introducing a stereo framework that estimates absolute velocity and fuses complementary Radar features for 3D detection.

  • 4D Radar supports perception in adverse weather and directly measures reflecting-target velocity through the Doppler effect.
  • Doppler velocity captures only the line-of-sight radial component, limiting recovery of an object’s true motion.
  • Clutter, multipath ghosts, and receiver noise make preprocessing sparse and can remove valid Radar returns.
  • The proposed stereo setup uses two Radars with geometric disparity and azimuth measurements to recover objects’ absolute velocity.
  • SR-PFE and SR-BFM align, refine, and fuse left- and right-Radar features to enrich sparse representations.
  • 8.82% in AP3D and 9.0% in APBEV improvements are reported over state-of-the-art mono 4D Radar baselines.

II. RELATED WORK

Prior 4D Radar detection work emphasizes sparse-point encoding, contextual modeling, and multi-representation fusion, while this paper develops stereo processing and feature fusion to address mono-Radar limitations.

  • Encoding-based methods preserve sparse Radar structure through pillarwise or multidimensional tensor representations.
  • Context-based methods model local interactions or density cues to improve representations in sparse regions and diverse environments.
  • Multi-representation methods combine spatial, temporal, scale, or coordinate-system features to improve consistency and object detection.
  • Mono 4D Radar remains constrained by limited field of view and sparse point density despite advances in these architectures.
  • Existing studies use incomplete Doppler for dynamic likelihood estimation, semantic segmentation, camera-Radar fusion, or temporally aggregated detection features.
  • The proposed framework combines stereo preprocessing, SR-PFE, SR-BFM, a 2D backbone, and an anchor-based detection head.

B. Stereo 4D Radar-based Velocity Estimation

The framework estimates ego-motion, separates dynamic points from static background, and clusters moving-object returns before absolute velocity estimation. RANSAC and residual-based masking improve robustness to noise and outliers.

  • Dynamic Point Segmentation: RANSAC estimates horizontal ego-velocity from Doppler measurements and separates dynamically moving points from static background returns.The method uses a 2D translation velocity model and forms a dynamic mask from residuals.
  • Dynamic Point Segmentation: Each Radar point contains spatial coordinates, reflection power, and measured Doppler velocity, while its line-of-sight unit vector supports motion modeling.The line-of-sight vector is defined from the point coordinates.
  • Dynamic Point Segmentation: The velocity model restricts motion estimation to the horizontal plane and assumes static-background Doppler arises solely from ego-vehicle translation.This ground-plane prior models ego-velocity as a 2D vector.
  • Dynamic Point Segmentation: 15 m and ±40° empirically selected regions prioritize distant points in the main viewing sector for more consistent inlier estimation.Near or peripheral points can show larger Doppler variation because of observation-angle differences.
  • Dynamic Point Segmentation: Fig. 3-(b) separates dynamic vehicle points from static structures such as guardrails and trees while suppressing Doppler noise and outlier errors.

2) Dynamic Object Clustering:

Dynamic points are clustered first by 3D spatial proximity and then by Doppler velocity. The two-stage process separates adjacent objects with different motion patterns.

  • Dynamic Object Clustering: DBSCAN first groups dynamic points from both Radars into candidate clusters using 3D spatial proximity in (x, y, z).The clustering operates on points selected by residual-based dynamic masking.
  • Dynamic Object Clustering: A second clustering stage groups points within each spatial cluster by Doppler velocity to separate spatially adjacent objects with different motion patterns.This can distinguish vehicles driving side by side.
  • Dynamic Object Clustering: Fig. 3-(c) and 3-(d) show that Doppler-based clustering separates objects that spatial information alone cannot distinguish.

3) Absolute Velocity Estimation:

Absolute velocity is estimated by combining Doppler observations with stereo geometric constraints in a regularized least-squares model. The resulting estimate is incorporated into stereo feature processing for detection.

  • Absolute Velocity Estimation: The absolute velocity problem is formulated as a regularized RLS model using Doppler measurements from two sensors and their geometric relationship to the target.
  • Absolute Velocity Estimation: For Np Radar points, hi is each point’s LoS unit vector and di is its ego-corrected Doppler measurement.
  • Absolute Velocity Estimation: The pointwise observation constraints are stacked into a linear system whose Doppler vector and LoS matrix support velocity estimation.The system uses d as the measurement vector and H as the matrix of LoS vectors.
  • Absolute Velocity Estimation: The regularization parameter suppresses numerical instability when LoS vectors are nearly parallel, while the optimized estimate is the predicted object velocity.
  • Absolute Velocity Estimation: Stereo velocity estimation recovers absolute velocity components unavailable from a mono sensor and remains robust under noise or insufficient geometric constraints.The regularization term supports this robustness.
  • Stereo Feature Processing: SR-PFE generates BEV representations from Radar attributes, and SR-BFM aligns, refines, and fuses left and right features using stereo geometry.

1) Stereo Radar Pillar Feature Encoding (SR-PFE):

SR-PFE converts left, right, and aggregated Radar point sets into sensor-specific BEV feature maps, while SR-BFM aligns and fuses complementary stereo features for detection.

  • Stereo Radar Pillar Feature Encoding (SR-PFE): SR-PFE partitions each Radar point cloud into spatial pillars and encodes pillar points into BEV feature maps.The encoding process extends Radar PillarNet.
  • Stereo Radar Pillar Feature Encoding (SR-PFE): Three inputs—left, right, and aggregated Radar point sets—are encoded separately to produce FL, FR, and FA.These feature maps feed the subsequent SR-BFM module.
  • Stereo Radar Pillar Feature Encoding (SR-PFE): Each pillar point contributes spatial, reflection-power, and velocity features processed by dedicated encoders before concatenation, max pooling, and BEV scattering.The spatial features use x, y, z; reflection features use PW; velocity features use the velocity components described in the passage.
  • Stereo Radar Bilateral Fusion Module (SR-BFM): SR-BFM sequentially performs sensor identification and positional encoding, attention refinement, and gated correlation-aware fusion.Its AR block aligns stereo features, while GCF combines confidence-based gating with correlation-aware refinement.
  • Stereo Radar Bilateral Fusion Module (SR-BFM): The AR block uses localized cross-attention with aggregated features to align left-right observations and reweights salient channel-spatial structures.The aligned features are passed to GCF for final stereo fusion.
  • Stereo Radar Bilateral Fusion Module (SR-BFM): Pairwise gating assigns BEV-wise confidence weights to left and right features, allowing higher reliance on the more reliable viewpoint.The weights are softmax-normalized so wR + wL = 1 at each BEV location, supporting fusion under occlusion or degraded reflections.
  • Stereo Radar Bilateral Fusion Module (SR-BFM): Correlation-aware refinement uses dilated convolutions to reduce geometric discrepancies before concatenating refined and gated features into the final detection representation.The fused feature map is fed to the backbone and detection head.

IV. EXPERIMENTS

The experiments evaluate absolute velocity estimation and stereo 4D Radar detection using a newly constructed dataset containing the same scenes from two spatially separated sensors.

  • IV. EXPERIMENTS: The evaluation uses a new stereo 4D Radar dataset that captures identical scenes from two spatially separated sensors.The dataset supports assessment of both absolute velocity estimation and 3D object detection.

A. Dataset

The multimodal dataset centers on stereo 4D Radar, with LiDAR annotations and forward-facing cameras for visualization, and contains 37 sequences recorded in clear daytime driving.

  • Dataset: The dataset combines stereo 4D Radars with LiDAR for annotation and multiple forward-facing cameras for visualization.
  • Dataset: 37 sequences contain 22K frames and approximately 66K Sedan 3D bounding boxes.The recordings use clear daytime driving scenarios.
  • Dataset: Training uses PyTorch 1.12 for 30 epochs with learning rate 0.001 and batch size 8 on an RTX 4070 Ti GPU.Radar points are restricted to the stated region of interest.
  • Dataset: Dynamic object clustering uses spatial and Doppler thresholds of 2.5 m and 0.5 m/s, with minPtsxyz = 6 and minPtsdop = 2.
  • Dataset: The detection evaluation follows the K-Radar protocol and reports AP3D and APBEV for Sedan at IoU 0.3.

C. Quantitative Analysis of Mono vs. Stereo 4D Radar Configurations

Stereo Radar input consistently improves detection over mono input, and adding estimated absolute velocity provides further gains; the proposed network achieves the strongest reported performance.

  • Quantitative Analysis of Mono vs. Stereo 4D Radar Configurations: Stereo input consistently improves AP3D and APBEV across evaluated networks compared with mono-Radar input.RadarPillarNet rises from AP3D 48.02 to 57.23 and from APBEV 49.52 to 58.25.
  • Quantitative Analysis of Mono vs. Stereo 4D Radar Configurations: Adding estimated absolute velocities to stereo Doppler input further improves AP3D and APBEV in most networks.The recovered motion information supports more accurate localization and motion interpretation of dynamic objects.
  • Quantitative Analysis of Mono vs. Stereo 4D Radar Configurations: AP3D 58.92 and APBEV 59.73 are achieved by the proposed network, which outperforms existing 4D Radar-based detectors.
  • Absolute Velocity Estimation: Stereo velocity estimation recovers lateral and complex object motions that mono-Radar methods distort or misinterpret.The qualitative scenes include lateral movement, lane changing, and oncoming traffic.

E. Ablation Study

The ablation study evaluates SR-BFM components and shows that the complete model performs best, while removing individual alignment, refinement, or encoding components reduces detection accuracy.

  • SR-BFM components: APBEV = 58.02 and AP3D = 56.72 without the GCF Block, the largest observed degradation.This indicates that correlation-aware refinement and gating are central to effective stereo fusion.
  • SR-BFM components: APBEV = 58.87 and AP3D = 57.91 without the SI-PE Block, showing that sensor identity embeddings support stereo alignment.The SI-PE ablation reduces performance relative to the full model.
  • SR-BFM components: APBEV = 58.76 and AP3D = 57.83 without the AR Block, indicating that cross-attention-based alignment mitigates left-right geometric inconsistencies.Removing the alignment block lowers accuracy compared with the complete model.
  • SR-PFE: APBEV = 59.10 and AP3D = 58.01 without SR-PFE, demonstrating its role in producing discriminative BEV features before stereo fusion.The SR-PFE ablation also produces a noticeable performance drop.
  • Overall ablation: APBEV = 59.73 and AP3D = 58.92 for the full model, which achieves the highest performance.The result verifies that SR-PFE and all SR-BFM components jointly enhance stereo 4D Radar detection.
  • GCF Block: The study further analyzes the Coarse Branch and Dilated Convolution as detailed components of the GCF Block.Table III compares variants without the Coarse Branch and without Dilated Convolution.

APPENDIX

The in-house stereo 4D Radar dataset was collected in Daejeon under clear daytime conditions across urban, highway, and suburban driving sequences, with training and testing splits and sedan-only annotations.

  • Sensor platform: Two 4D Radars were mounted on the front bumper with a fixed baseline of 0.6 m.Each Radar operates in the 76-79 GHz band and uses 12 transmit and 16 receive antennas.
  • Data collection: The dataset contains 10 urban, 11 highway, and 16 suburban sequences recorded under clear daytime conditions.Data were collected in Daejeon, South Korea, using the described sensor platform.
  • Dataset split: The dataset is split into 18K training frames and 4K testing frames.The split supports training and evaluation across the collected driving environments.
  • Annotations: A total of 66K 3D bounding boxes were annotated for the Sedan class.Annotations were initialized automatically from LiDAR data and then manually refined.
Loading 2609.02560v1…