Source-linked AI summary
TJ4DRadSet: A 4D Radar Dataset for Autonomous Driving
Lianqing Zheng, Zhixiong Ma, Xichan Zhu, Bin Tan, Sen Li, Kai Long, Weiqi Sun, Sihan Chen, Lu Zhang, Mengyue Wan, Libo Huang, Jie Bai
TL;DR
Autonomous-driving research lacks sufficiently diverse, accurately annotated 4D-radar data for 3D perception. TJ4DRadSet addresses this gap with synchronized multimodal sequences and a 4D-radar 3D-detection baseline, which shows radar's potential while retaining a substantial gap to lidar.
Problem
Few autonomous-driving datasets contain 4D radar, limiting deep-learning research on 4D-radar point clouds.
Method
The paper constructs TJ4DRadSet with synchronized 4D radar, lidar, camera, and GNSS data, 3D annotations, track ids, diverse scenarios, and a 3D-detection baseline.
Results
The 4D-radar baseline achieves over 30% average accuracy across all classes at the 0.25 IoU threshold in BEV, although it remains behind lidar under the same algorithm.
Takeaways & Limitations
TJ4DRadSet supports research on 4D-radar-based 3D perception and indicates promising potential for high-level autonomous driving.
Abstract
from arXiv · showhide
The next-generation high-resolution automotive radar (4D radar) can provide additional elevation measurement and denser point clouds, which has great potential for 3D sensing in autonomous driving. In this paper, we introduce a dataset named TJ4DRadSet with 4D radar points for autonomous driving research. The dataset was collected in various driving scenarios, with a total of 7757 synchronized frames in 44 consecutive sequences, which are well annotated with 3D bounding boxes and track ids. We provide a 4D radar-based 3D object detection baseline for our dataset to demonstrate the effectiveness of deep learning methods for 4D radar point clouds. The dataset can be accessed via the following link: https://github.com/TJRadarLab/TJ4DRadSet.
I. INTRODUCTION
TJ4DRadSet addresses the limited availability of autonomous-driving datasets with high-resolution 4D radar by providing synchronized multimodal data, 3D annotations, diverse scenarios, and a 3D detection baseline.
- I. INTRODUCTION: The dataset responds to the scarcity of 4D-radar datasets needed for large-scale, accurately annotated deep-learning research.Training 3D detection networks requires diverse data and accurate ground truth.
- I. INTRODUCTION: TJ4DRadSet provides 4D radar point clouds with synchronized lidar, camera, and GNSS data for autonomous-driving perception research.The dataset is intended to support 4D-radar-based perception algorithms.
- I. INTRODUCTION: The dataset includes 40K synchronized frames, including 7757 frames across 44 sequences with annotated 3D bounding boxes and track ids.Annotations use joint multi-sensor annotation and multiple rounds of manual checks.
- I. INTRODUCTION: TJ4DRadSet covers elevated roads, complex intersections, one-way roads, urban roads, intense light, and darkness.These conditions provide varied settings for developing 4D-radar-based 3D perception algorithms.
- I. INTRODUCTION: The paper provides a 4D-radar-based 3D object-detection baseline and reports promising potential for high-level autonomous driving.The introduction identifies the baseline as evidence supporting 4D radar for 3D perception.
II. RELATED WORK
Prior driving datasets often lack radar, elevation information, Doppler velocity, tracking, or sufficient scale, motivating datasets with high-resolution 4D radar.
- II. RELATED WORK: Existing radar datasets differ in sensing capability, with some lacking elevation information and others lacking Doppler velocity or using radar as image data.The related work contrasts low-resolution FMCW and scanning-radar datasets with 4D point-cloud formats.
- II. RELATED WORK: Astyx provides only 545 frames and lacks tracking information, illustrating limitations in existing high-resolution 4D-radar data.The passage identifies dataset scale and tracking as specific shortcomings.
- II. RELATED WORK: VoD contains 8600 synchronized, calibrated frames with 4D radar, lidar, camera, 3D annotations, and track ids, while TJ4DRadSet claims richer and more challenging scenario clips.The comparison concerns scenario diversity rather than annotation or sensor completeness.
III. THE TJ4DRADSET DATASET
The dataset section presents TJ4DRadSet through sensor parameters, calibration, data collection, annotation, statistical analysis, and visualization.
- III. THE TJ4DRADSET DATASET: TJ4DRadSet is documented through sensor parameters, calibration, data collection, annotation, statistical analysis, and visualization.These components define the dataset description provided in the paper.
A. Sensors
TJ4DRadSet combines roof-mounted camera and lidar with a forward-facing 4D radar, supplemented by high-precision GNSS positioning.
- A. Sensors: The camera and lidar are mounted on the roof bracket, while the 4D radar is installed at the middle of the front ventilation grille.The lidar scans 360 degrees, whereas the camera and radar cover the forward driving view.
- A. Sensors: GNSS data are corrected using real-time kinematic positioning to provide high-precision localization.The passage identifies GNSS as an additional included sensor modality.
B. Sensor Calibration
The dataset calibrates its camera, lidar, and 4D radar sensors through intrinsic, extrinsic, and temporal alignment procedures.
- B. Sensor Calibration: Intrinsic calibration corrects camera distortion, while factory procedures provide the intrinsic parameters of the 4D radar and lidar.Camera calibration uses MATLAB Toolkit and a checkerboard; rectified images are then obtained.
- B. Sensor Calibration: Extrinsic calibration aligns camera-lidar data in 2D-3D space and 4D radar-lidar data as 3D-3D point clouds.Camera and radar extrinsics are derived from the remaining calibrated sensor transformations.
- B. Sensor Calibration: Sensor data are temporally aligned using arrival timestamps because the sensors operate at different frame rates.The aligned radar and lidar point clouds are projected into the image.
C. Data Collection and Annotation
TJ4DRadSet was collected across varied lighting and road environments, with multimodal sensor recordings and object-level 3D annotations.
- C. Data Collection and Annotation: Data collection in Suzhou covered normal, bright, and dark lighting across urban roads, elevated roads, industrial zones, intersections, and one-way streets.The scenarios range from object-dense intersections to one-way streets containing few objects.
- C. Data Collection and Annotation: The acquisition system used ROS, and all sensor data were recorded completely in rosbag files.
- C. Data Collection and Annotation: Each object is annotated with a 3D bounding box, class, and track id, plus occlusion and truncation indicators.Bounding boxes encode center, dimensions, and yaw; the dataset defines eight object classes before class mapping.
- C. Data Collection and Annotation: Annotations primarily use lidar and images, while radar-visible objects with few lidar points from occlusion are also labeled.This accounts for objects appearing in the radar field of view because of multipath effects.
D. Dataset Statistics
Dataset statistics characterize class frequencies, ego-vehicle speeds, and the differing point-cloud densities and features of lidar and 4D radar.
- D. Dataset Statistics: Car is the most numerous class, followed by Cyclist, while Truck and Pedestrian occur in approximately equal amounts.
- D. Dataset Statistics: 4D radar point clouds are sparser than lidar point clouds but contain additional features such as Doppler velocity.
- D. Dataset Statistics: The statistics include the ego vehicle’s speed distribution and the point-cloud density distributions for lidar and 4D radar.
IV. BASELINE EXPERIMENTS
The baseline evaluates PointPillars for 4D-radar and lidar 3D detection using fixed sequence-based splits and adapted radar features. 4D radar shows potential, but lidar performs better under the same algorithm.
- IV. BASELINE EXPERIMENTS: The fixed sequence-based split contains 5717 training samples and 2040 test samples with broad test-set coverage.
- IV. BASELINE EXPERIMENTS: 4D radar achieves over 30% average precision across all classes in BEV at the 0.25 IoU threshold, while lidar completely outperforms it under PointPillars.Results are reported for 50 m and 70 m detection ranges and BEV and 3D views.
- IV. BASELINE EXPERIMENTS: Labels are transferred from the lidar coordinate system to the 4D radar system using the lidar-radar extrinsic matrix.
- IV. BASELINE EXPERIMENTS: PointPillars is adapted for sparse radar data and trained with five-dimensional radar features: x, y, z, Doppler velocity, and signal-to-noise ratio.The model uses a 69.12 m x-axis detection range, 0.16 m × 0.16 m pillars, augmentation, and 80 Adam-training epochs.
- IV. BASELINE EXPERIMENTS: Average precision is evaluated at IoU thresholds of 0.5 and 0.25 for Car, Truck, and Cyclist, and only 0.25 for Pedestrian.
- IV. BASELINE EXPERIMENTS: The radar-lidar performance gap may reflect radar’s lower point density and the influence of data augmentation choices.
V. CONCLUSION AND FUTURE WORK
TJ4DRadSet is a multi-modal autonomous driving dataset with 4D radar point clouds for studying 4D radar-based 3D perception. The paper provides dataset details and baseline experiments, with future work targeting expansion and improved radar processing and fusion.
- The authors establish baseline experiments and plan to expand the dataset while investigating fusion algorithms, point-cloud enhancement, and 4D radar feature representation.