PaperlayerView exact revision on arXiv

Source-linked AI summary

RADDet: Range-Azimuth-Doppler based Radar Object Detection for Dynamic Road Users

Ao Zhang, Farzan Erlik Nowruzi, Robert Laganiere

arXiv:2105.00363v1cs.CVeess.SP

TL;DR

Radar object detection research faces fragmented data representations and a shortage of public datasets. The paper introduces an annotated RAD dataset and a dual-head detector for dynamic road users, reporting state-of-the-art performance with radar data.

  • Problem

    Radar object-detection research is split across input and output representations, and public datasets suitable for benchmarking are scarce.

    Source: §I · p. 1, §II · p. 2

  • Method

    The paper introduces a dataset of annotated RAD data, an automatic method for labeling RAD and Cartesian representations, and a ResNet-based detector with 3D RAD and 2D Cartesian heads.

    Source: §I · p. 2

  • Results

    The proposed model is reported to achieve state-of-the-art performance in object detection with radar data.

    Source: §I · p. 2

  • Takeaways & Limitations

    The publicly available RAD dataset provides annotated data for future research in the field.

    Source: §I · p. 2

    The authors report that limited dataset size constrains model performance, alongside label imbalance and class ambiguity.

    Source: C. Discussion · p. 7

Abstract

from arXiv · show

Object detection using automotive radars has not been explored with deep learning models in comparison to the camera based approaches. This can be attributed to the lack of public radar datasets. In this paper, we collect a novel radar dataset that contains radar data in the form of Range-Azimuth-Doppler tensors along with the bounding boxes on the tensor for dynamic road users, category labels, and 2D bounding boxes on the Cartesian Bird-Eye-View range map. To build the dataset, we propose an instance-wise auto-annotation method. Furthermore, a novel Range-Azimuth-Doppler based multi-class object detection deep learning model is proposed. The algorithm is a one-stage anchor-based detector that generates both 3D bounding boxes and 2D bounding boxes on Range-Azimuth-Doppler and Cartesian domains, respectively. Our proposed algorithm achieves 56.3% AP with IOU of 0.3 on 3D bounding box predictions, and 51.6% with IOU of 0.5 on 2D bounding box prediction. Our dataset and the code can be found at https://github.com/ZhangAoCanada/RADDet.git.

I. INTRODUCTION

The paper addresses fragmented radar-detection research and a shortage of public datasets by introducing an annotated RAD dataset and a dual-head detector for dynamic road users. It reports state-of-the-art object-detection performance with radar data.

  • FMCW radar is widely used in ADAS and is robust to adverse weather and lighting, but has low output resolution and high noise levels.Source: §I · p. 1
  • RAD represents range, angle, and velocity in a 3D tensor, and this study uses spectra obtained by applying FFT to radar signals.Source: §I · p. 1
  • Radar-detection research is split across input and output representations, while public datasets suitable for benchmarking are scarce; many studies focus on dynamic objects.Source: §I · pp. 1–2
  • The introduced public dataset provides RAD representations with corresponding annotations for multiple object classes.Source: §I · p. 2
  • An automatic annotation method generates ground-truth labels across RAD dimensions and in Cartesian form.Source: §I · p. 2
  • The detector uses a ResNet-based backbone and dual heads for 3D detection on RAD data and 2D detection in Cartesian coordinates.Source: §I · p. 2
  • The proposed model is reported to achieve state-of-the-art object-detection performance with radar data, making radar a viable competitor to camera- and lidar-based methods.Source: §I · p. 2

II. RELATED WORK

The section describes a shortage of public radar datasets amid hardware and configuration variation, and reviews differences in sensing inputs and detection approaches. Prior datasets often combine radar with cameras or LiDAR.

II. RELATED WORK

  • Public radar datasets are scarce partly because hardware and configurable radar settings change data resolution and detection range.Different transmitter and receiver counts affect data resolution; sweep magnitudes and sampling rates affect maximum range and range resolution.Source: §II · p. 2
  • Earlier radar datasets often use cameras or LiDAR for sensor fusion, with some using multiple high-precision sensors for annotation.The cited examples include Lidar, Synthetic Aperture Radar, and cameras.Source: §II · p. 2
  • Radar inputs are broadly cluster-based or spectrum-based: clusters can merge or separate under noise and limited range resolution, while RAD spectra retain input consistency without discretization.Palffy et al. add cropped spectrum data to clusters to address cluster-based issues; spectrum inputs can increase algorithm complexity and potentially affect runtime.Source: §II · p. 2
  • Radar research has adapted vision architectures and detection heads, including U-Net and FCN, while one-stage anchor-based heads remain common and two-stage heads are considered slower.The section also notes growing interest in anchor-free heads and self-attention,, [30].Source: §II · p. 2

III. DATASET PREPARATION

The dataset preparation section outlines the collection setup, an auto-annotation approach, manual validation, and statistical analysis.

III. DATASET PREPARATION

  • Dataset preparation covers radar and stereo-camera hardware and introduces an auto-annotation method for the challenging radar annotation task.Extracted annotations are manually validated before dataset statistical analysis.Source: §III · p. 3
  • The annotation workflow includes manual validation to ensure dataset quality.Source: §III · p. 3
  • The section concludes with statistical analysis of the dataset.Source: §III · p. 3

A. Hardware Setup

The dataset uses a calibrated radar-and-stereo setup and an annotation pipeline that combines radar instance extraction with stereo-derived categories. Automated annotations cover about 75% of objects and are manually corrected.

A. Hardware Setup

  • Data collection used a Texas Instruments AWR1843-BOOST2 radar and two DFK 33UX2733 stereo cameras, calibrated with a self-made trihedral corner reflector at randomly selected distances of 5–50 m.The 3D-stereo-to-bird-eye-view radar projection matrix was computed with least-squares optimization; raw radar ADC data had shape (256, 8, 64).Source: A. Hardware Setup · p. 3

B. Annotation Method

  • Dynamic road users are the annotation target because their Doppler information is richer and their detection rate is higher than for static users.The proposed annotation method is intended to address OS-CFAR false detections, including missed points on large objects and noise mislabeled as detections.Source: B. Annotation Method · p. 3
  • The method connects linear patterns in Range-Doppler outputs to enrich detections, then uses DBSCAN to separate instances that overlap at similar ranges and speeds.Rigid-body motion coherence on the Doppler axis supports vehicle detection; pedestrians are treated as rigid bodies when range resolution limits observable body-motion complexity.Source: B. Annotation Method · pp. 3–4
  • Stereo disparity maps and Mask-RCNN segmentation masks are triangulated into categorized point clouds, then transformed into the radar frame.The segmentation masks are projected onto disparity maps before triangulation.Source: B. Annotation Method · p. 4
  • Auto-annotation retrieves labels for around 75% of objects; radar extraction, stereo depth, Mask-RCNN errors, and narrower camera FOV leave errors or unlabeled objects, so annotations are manually corrected.Source: B. Annotation Method · p. 4

C. Dataset Analysis

The dataset analysis reports collection and split procedures, and the following model section describes a ResNet-based backbone and a YOLO-inspired dual detection head. The model omits multiresolution neck layers because radar reveals objects' true scale.

C. Dataset Analysis

  • 10,158 frames were collected during sunny-weather sessions at random locations from September to October, after corrupted frames were removed.Source: C. Dataset Analysis · p. 4
  • The dataset is split 80%/20% into training and test sets by class-wise random sampling, with 90% of training data used for training and 10% for validation.The split is intended to preserve the overall distributions shown in Figure 3.Source: C. Dataset Analysis · p. 4
  • Figure 3 presents object counts across categories and the range distribution across objects.Source: Figure 3 · p. 4

IV. RADDET

  • The detector uses a ResNet backbone and a YOLO-based dual detection head, omitting multiresolution neck layers because radar reveals true object scale.The text contrasts this with images, where perspective geometry changes apparent object size with distance.Source: §IV · p. 4

A. Global Normalization of the Input

The model normalizes each input tensor using dataset-wide statistics before passing it to a RadarResNet backbone. RadarResNet adapts residual and downsampling blocks to produce a (16, 16, 256) output.

A. Global Normalization of the Input

  • Each complex-valued (256, 256, 64) Range-Azimuth-Doppler input tensor is normalized using mean and variance values computed over the entire dataset.The normalized tensor is the input to the backbone.Source: A. Global Normalization of the Input · p. 4

B. RadarResNet

  • RadarResNet adapts ResNet to radar by treating Range-Azimuth as input dimensions and Doppler as the channel dimension.Its residual blocks follow ResNet, while downsampling blocks use max-pooling; the final output size is (16, 16, 256).Source: B. RadarResNet · pp. 4–5
  • Figure 4 presents the dataflow of the proposed network architecture.Source: Figure 4 · p. 5

C. Dual Detection Head

The dual detection head has anchor-based 3D and 2D branches. The 3D branch predicts boxes in RAD, while the 2D branch transforms polar features to Cartesian form before box prediction.

C. Dual Detection Head

  • Both anchor-based branches use structures similar to YOLO [9], with six anchors defined for each branch by K-means clustering on ground-truth boxes.The error-rate threshold is 10%.Source: C. Dual Detection Head · p. 5
  • The 3D RAD YOLO Head reshapes predictions to (16, 16, 4, num_of_anchors, 7+num_of_classes), encoding objectness and 3D box center and size.Its Doppler output size is 4, and the branch uses six anchors defined by K-means clustering with a 10% error-rate threshold.Source: 2) 2D Detection Head · p. 5, C. Dual Detection Head · p. 5

2) 2D Detection Head:

  • The 2D head transforms polar feature maps to Cartesian form before feeding them to a classic YOLO Head [9] for 2D box predictions.It applies two fully connected layers to each (16, 16) feature map, reshapes each output to (32, 16), concatenates the maps, and postprocesses them with a residual block.Source: 2) 2D Detection Head · p. 5
  • The traditional range-and-azimuth to Cartesian transformation can double the input width in Cartesian form.Source: 2) 2D Detection Head · p. 5

D. Loss Functions

The total loss combines box regression, objectness, and classification losses. The authors weight the box loss and adjust negative-sample focal loss to address box-loss dominance and non-object predictions.

D. Loss Functions

  • The total loss is Ltotal = β · Lbox + Lobj + Lclass, combining box regression, objectness, and classification losses.Source: D. Loss Functions · pp. 5–6
  • The box regression loss is chosen from, focal loss is used for objectness, and cross entropy is used for classification.Source: D. Loss Functions · p. 6
  • The negative-sample coefficient α is set to 0.01 to address non-object prediction dominance, and β is set to 0.1 because box loss easily dominated total loss during training.Source: D. Loss Functions · p. 6

V. EXPERIMENTS

Experiments explore backbones and additional layers, training the shared backbone with the more complex RAD YOLO head before fine-tuning the 2D head. The paper reports the main hyperparameters and evaluation setup.

V. EXPERIMENTS

  • The exploration compares VGG and ResNet backbones, and tests self-attention and channel-wise MLP layers before bounding-box extraction.The selected self-attention proposals are SAGAN and SAUNet [30].Source: §V · p. 6
  • The shared backbone is trained with the RAD YOLO head, then fine-tuned on the 3D head before its layers are frozen for 2D-head training.Source: §V · p. 6
  • Experiments use batch size 3, learning rate 0.0001, Adam [31], and mean Average Precision, with objectness threshold 0.5 and NMS thresholds 0.1 for 3D and 0.3 for 2D.Learning-rate decay is 0.96 every 10K steps after 60K warm-up steps; experiments run on an RTX 2080Ti GPU with TensorFlow.Source: §V · p. 6

A. Backbone Model Search

The backbone search favored RadarResNet with max-pooling, and the detection-head comparisons found different best-performing backbones for the RAD and 2D heads. The selected RadarResNet uses repeat counts of 2, 4, 8, and 16 before its max-pooling layers.

A. Backbone Model Search

  • A (16, 16, 256) backbone output was selected after (16, 16) gave the highest 3D-detection performance among the tested output sizes.The search also set the number of channel-expansion and downsampling blocks to two and four, respectively.Source: A. Backbone Model Search · p. 6
  • The selected RadarResNet uses residual-block repeat counts of 2, 4, 8, and 16 before each max-pooling layer.The Table II caption identifies its samples as backbone-size searches with bracketed repeat counts before downsampling.Source: A. Backbone Model Search · p. 6, Table II · p. 7
  • ResNet-based backbones generally performed better than VGG, and max-pooling downsampling improved performance over residual-block downsampling.These findings informed the RadarResNet design.Source: A. Backbone Model Search · p. 6

B. Detection Head Comparisons

  • For the RAD YOLO Head, RadarResNet outperforms all other backbones; for the 2D YOLO Head, it has better overall precision and inference speed than RadarResNet+SA (SAUNet).RadarResNet+SA (SAUNet) indicates slightly better performance on the 2D head, while RadarResNet has better overall precision.Source: B. Detection Head Comparisons · p. 6, Table I · p. 7

C. Discussion

The discussion links false detections to dataset limitations, including label imbalance, limited size, and ambiguity between classes with similar radar traces. The conclusion also notes that the dataset is smaller than image-based counterparts and that embedded deployment would slow inference drastically.

C. Discussion

  • Label imbalance, limited dataset size, and ambiguous classes are identified as factors contributing to false detections and constrained performance.Mini-vans assigned to “truck” and large SUVs assigned to “car” can confuse the model because their radar signal traces look similar.Source: C. Discussion · p. 7

VI. CONCLUSION

  • The dataset remains much smaller than image-based counterparts, and the authors say more targeted data could balance object categories and improve model performance.Source: §VI · p. 7
  • Although inference is fast on an RTX 2080Ti, deploying the model on embedded platforms would cause a drastic slowdown, motivating hardware-aware model exploration.Source: §VI · p. 7
  • The authors describe their work as the first model to take raw radar inputs and predict dynamic road users’ position, speed, and category.Source: §VI · p. 7
  • The authors suggest that attention layers designed for this specific radar data could improve performance.Source: §VI · p. 7
Loading 2105.00363v1…