PaperlayerView exact revision on arXiv

Source-linked AI summary

RADIATE: A Radar Dataset for Automotive Perception in Bad Weather

Marcel Sheeny, Emanuele De Pellegrin, Saptarshi Mukherjee, Alireza Ahrabian, Sen Wang, Andrew Wallace

arXiv:2010.09076v3cs.CVcs.RO

TL;DR

Existing automotive perception datasets are dominated by camera and LiDAR data collected in good weather, while public radar datasets with labelled actors in adverse conditions are scarce. RADIATE addresses this gap with a large, multimodal radar dataset and baseline detection experiments, reporting promising results for radar-based perception in adverse weather.

  • Problem

    Public automotive radar datasets with object annotations are scarce, especially for bad weather, limiting available data for adverse-condition perception research.

  • Method

    RADIATE provides a high-resolution radar dataset with labelled road actors, multimodal sensors, and data collected across challenging weather and driving conditions.

  • Results

    RADIATE's baseline experiments report promising radar-based object-detection results in adverse conditions, including settings where LiDAR recognition fails at short range in fog.

  • Takeaways & Limitations

    RADIATE supports research on radar-based object detection, tracking, and scene understanding for automotive perception in adverse weather.

Abstract

from arXiv · show

Datasets for autonomous cars are essential for the development and benchmarking of perception systems. However, most existing datasets are captured with camera and LiDAR sensors in good weather conditions. In this paper, we present the RAdar Dataset In Adverse weaThEr (RADIATE), aiming to facilitate research on object detection, tracking and scene understanding using radar sensing for safe autonomous driving. RADIATE includes 3 hours of annotated radar images with more than 200K labelled road actors in total, on average about 4.6 instances per radar image. It covers 8 different categories of actors in a variety of weather conditions (e.g., sun, night, rain, fog and snow) and driving scenarios (e.g., parked, urban, motorway and suburban), representing different levels of challenge. To the best of our knowledge, this is the first public radar dataset which provides high-resolution radar images on public roads with a large amount of road actors labelled. The data collected in adverse weather, e.g., fog and snowfall, is unique. Some baseline results of radar based object detection and recognition are given to show that the use of radar data is promising for automotive applications in bad weather, where vision and LiDAR can fail. RADIATE also has stereo images, 32-channel LiDAR and GPS data, directed at other applications such as sensor fusion, localisation and mapping. The public dataset can be accessed at http://pro.hw.ac.uk/radiate/.

I. INTRODUCTION

RADIATE addresses the limited availability of public automotive radar datasets with object annotations, particularly for adverse weather. It provides multimodal, high-resolution radar data covering varied weather and driving conditions for perception research.

  • Visible-spectrum camera and LiDAR data are affected by attenuation, multiple scattering, and turbulence in bad weather.
  • Few public automotive radar datasets provide object annotations, especially in bad weather.
  • 3 hours of radar images are fully annotated, containing more than 200K labelled object instances across 8 road-actor categories.
  • The dataset combines radar with stereo camera, LiDAR, and GPS data collected across challenging weather conditions and driving scenarios.
  • A demonstrated use case shows that RADIATE supports robust vehicle detection in adverse weather when optical sensors fail.

II. RELATED WORK

Existing automotive perception datasets are dominated by optical sensing and generally lack broad radar annotations across adverse weather. RADIATE addresses this gap with high-resolution scanning-radar data and extensive actor labels.

  • Most established automotive datasets use cameras and LiDAR, with adverse weather often secondary or absent.
  • Radar is more resilient to fog, rain and snow, but conventional automotive radar provides low-resolution imagery that challenges recognition and scene mapping.
  • Scanning radar improves azimuth resolution by measuring each azimuth with a moving antenna.
  • Existing radar datasets commonly provide sparse point clouds, lack object annotations, or contain only about 500 annotated frames.
  • RADIATE is presented as the first public radar dataset with a large set of publicly available actor annotations.
  • The authors hope RADIATE will boost autonomous-driving research in the community.

III. THE RADIATE DATASET

RADIATE was collected over one year using a ROS-based multi-sensor system. The release includes structured sequence data and an SDK for calibration, visualization and preprocessing.

  • RADIATE data was collected between February 2019 and February 2020 using a system built with ROS.
  • ROS rosbags were used to extract sensor information with its respective timestamp.
  • The dataset release provides a folder structure for each sequence and an SDK for calibration, visualization and preprocessing.

A. Perception Sensors

RADIATE combines scanning radar, stereo cameras and 32-channel LiDAR for perception and sensor-fusion research. Radar supplies high-resolution 360° range-azimuth imagery, while optical and LiDAR sensing can degrade in severe weather.

  • RADIATE includes radar, LiDAR and stereo cameras in its perception sensor setup.
  • The stereo camera records 672 × 376 images at 15 frames per second per camera and is protected by waterproof housing.
  • Rain, dense fog and heavy snow can seriously blur, haze or fully block the camera images, while fog or snow can severely attenuate and reflect LiDAR signals.
  • The Navtech CTS350-X scanning radar provides 360° high-resolution range-azimuth images.
  • The radar has 100 meters maximum operating range, 0.175m range resolution, 1.8° azimuth resolution and 1.8° elevation resolution.
  • The radar currently does not provide Doppler information.

B. Sensor Calibration

RADIATE calibrates its heterogeneous sensors for correspondence and fusion by defining the radar as the local-frame origin and estimating six-degree-of-freedom transformations.

  • Calibration is required for multi-sensor fusion, feature correspondence and actor correspondence.
  • Stereo-camera intrinsic parameters and distortion coefficients are calibrated with the Matlab camera calibration toolbox before rectification and depth calculation.
  • The radar is selected as the origin of the local coordinate frame because it is the main sensor.
  • Radar, camera and LiDAR extrinsics are represented as six-degree-of-freedom transformations comprising translation and rotation.
  • Extrinsic parameters are initialized by measuring sensor distances and fine-tuned by aligning measurements between sensor pairs.
  • Calibration parameters are provided in a YAML file, while sensors operating at different frame rates are synchronized using the time of arrival as the timestamp.

C. Data Collection Scenarios

RADIATE spans seven driving and weather scenarios, from sunny parked and urban settings to motorway, night, rain, fog, and snow. These conditions represent differing levels of perception challenge, including clutter, low illumination, precipitation, and limited fog availability.

  • Seven scenarios cover sunny parked, sunny/overcast urban, overcast motorway, night motorway, rain suburban, fog suburban, and snow suburban data.
  • Sunny (parked): The parked scenario is intended as the easiest setting for detection, tracking, and trajectory prediction because the vehicle senses passing actors while stationary.
  • Sunny/Overcast (Urban): Urban driving is challenging because busy traffic, dense buildings, numerous road actors, and reflections from non-road objects clutter radar observations.
  • Overcast (motorway): The motorway scenario is relatively easy because surrounding dynamic actors are mostly vehicles and the background is similar; it was collected in overcast weather.
  • Night (motorway): Night motorway data tests perception under missing illumination, while radar and LiDAR are expected to remain effective as active sensors.
  • Rain and fog: Rain data comprises 18 minutes in a suburban setting, whereas fog was collected opportunistically because its occurrence and density are difficult to predict.
  • Snow: Snow data contains 34 minutes, with 3 minutes labelled; heavy snowfall can block sensors within 3 minutes, and snowflakes can affect LiDAR, camera, and radar data.

D. Labelling

RADIATE uses a multisensor annotation workflow because radar objects are difficult to recognise directly and single-sensor annotation tools are inadequate. It labels eight road-actor categories with rotated 2D bounding boxes.

  • Radar-image labelling is difficult because human annotators cannot easily recognise objects even after checking corresponding camera images.
  • Existing annotation tools designed for a single sensor are inadequate, so the authors developed a tool that correlates and visualises calibrated multisensor data.
  • Figure 7 demonstrates radar annotations alongside projected LiDAR data, including false LiDAR points caused by snowflakes.
  • Eight labelled road-actor categories are cars, vans, trucks, buses, motorbikes, bicycles, pedestrians, and groups of pedestrians.
  • Each bounding box is represented by x, y, width, height, and a counter-clockwise rotation angle.
  • The annotation process uses the CamShift tracker to improve annotation efficiency.
  • More than 200K bounding boxes span 44K radar images, averaging 4.6 bounding boxes per image, with distributions reported by driving scenario.

IV. EXAMPLE OF USE: RADAR BASED VEHICLE DETECTION

The paper uses RADIATE to establish a radar-based vehicle-detection baseline and examine performance across weather conditions. It defines weather-diverse training and test sets for evaluation and benchmarking.

  • The paper presents RADIATE’s first baseline results for radar-based object detection in extreme weather, where existing autonomous-driving work is lacking.
  • Three evaluation datasets provide good-weather training, mixed good-and-bad-weather training, and a test set spanning both weather types.
  • The test set includes good and bad weather and is used for evaluation and benchmarking.
  • The baseline evaluates single-image vehicle detection over cars, vans, trucks, buses, motorbikes, and bicycles.

A. Radar based Vehicle Detection in the Wild

A Faster R-CNN detector adapted to radar evaluates vehicle detection across weather and driving scenarios. Performance varies mainly with scenario data type: radar remains effective in fog and at night, while rain and snow are more difficult.

  • Faster R-CNN with a ResNet-101 backbone is trained under good-weather-only or mixed-weather conditions and tested across all weather and driving scenarios.
  • The radar detector uses predefined vehicle-sized anchors and rotated Region Proposal Network outputs suited to metric-scale radar images.
  • The detector is evaluated using Average Precision at IoU 0.5, matching the PASCAL VOC and DOTA evaluation metrics.
  • The AP difference between good-weather-only and mixed-weather training is marginal, suggesting weather has no or only subtle impact on radar-based detection.
  • Similar AP distributions for both training regimes further support radar-based object detection across weather conditions.
  • Scenario type primarily biases results: parked data is easiest at close to 80% AP, while rain and snow perform more poorly.
  • Fog achieves considerably better radar results, making radar a good solution for perception in dense fog, while nighttime motorway results approach daytime performance.

V. CONCLUSIONS

RADIATE is presented as a labelled, large-scale radar dataset for perception research, with documented collection processes and examples across weather and road scenarios. Baseline vehicle-detection results are promising in adverse conditions, while motion-based recognition and tracking remain limited by the available data.

  • RADIATE is a new, labelled, large-scale dataset for research into radar-based perception.
  • The paper describes the sensor setup, calibration, labelling processes, and examples collected under different weather and road scenarios.
  • The dataset is demonstrated for radar-based vehicle detection in varied driving scenarios and weather conditions.
  • 45.77 Average Precision is achieved by ResNet-50 trained on good and bad weather, compared with 45.31 when trained on good weather only.
  • 46.55 Average Precision is achieved by ResNet-101 trained on good and bad weather, compared with 45.84 when trained on good weather only.
  • Radar-based object detection is reported as less affected by adverse weather, including foggy conditions where LiDAR recognition fails at short range.
  • The preliminary recognition study does not use motion, and future tracking work would require long-term motion memory because Doppler data is unavailable.
  • RADIATE is intended to facilitate research on robust perception in adverse weather.
Loading 2010.09076v3…