PaperlayerView exact revision on arXiv

Source-linked AI summary

2D Car Detection in Radar Data with PointNets

Andreas Danzer, Thomas Griebel, Martin Bach, Klaus Dietmayer

arXiv:1904.08414v3cs.CVcs.LGstat.ML

TL;DR

Sparse radar makes 2D object recognition difficult, and prior radar approaches did not jointly provide classification and bounding-box estimation. The paper adapts PointNets to classify and segment radar targets and regress amodal 2D boxes from a single radar sensor, with promising results on real-world radar data.

  • Problem

    Sparse radar data makes recognizing different objects challenging, while prior radar methods did not jointly perform classification and bounding-box estimation.

  • Method

    The method adapts PointNets to radar point clouds for object classification, target segmentation, and amodal 2D bounding-box regression.

  • Results

    The proposed detector shows promising classification, segmentation, and bounding-box regression results on real-world radar data, with 2.9 ms inference time per patch.

  • Takeaways & Limitations

    PointNets show potential for detecting 2D object hypotheses directly from sparse high-resolution radar data.

  • Takeaways & Limitations

    The dataset contains only one object per measurement cycle, uses one ground-truth object size, and includes only one object class.

Abstract

from arXiv · show

For many automated driving functions, a highly accurate perception of the vehicle environment is a crucial prerequisite. Modern high-resolution radar sensors generate multiple radar targets per object, which makes these sensors particularly suitable for the 2D object detection task. This work presents an approach to detect 2D objects solely depending on sparse radar data using PointNets. In literature, only methods are presented so far which perform either object classification or bounding box estimation for objects. In contrast, this method facilitates a classification together with a bounding box estimation of objects using a single radar sensor. To this end, PointNets are adjusted for radar data performing 2D object classification with segmentation, and 2D bounding box regression in order to estimate an amodal 2D bounding box. The algorithm is evaluated using an automatically created dataset which consist of various realistic driving maneuvers. The results show the great potential of object detection in high-resolution radar data using PointNets.

I. INTRODUCTION

The paper addresses sparse-radar 2D object detection, where prior radar methods did not jointly perform classification and bounding-box estimation. It adapts PointNets to process radar point clouds directly for this task.

  • I. INTRODUCTION: Sparse radar data is challenging for recognizing different objects, despite high-resolution sensors generating multiple detections per object.Radar remains much sparser than lidar point clouds or camera images.
  • I. INTRODUCTION: PointNets are used because they directly process radar point clouds without requiring transformation into a regular grid-map representation.Each radar target is represented using spatial coordinates, ego-motion-compensated Doppler velocity, and radar cross section values.
  • I. INTRODUCTION: The proposed method jointly classifies objects, segments radar targets, and estimates amodal 2D bounding boxes from a single radar sensor.An amodal box estimates the entire object even when the radar captures only part of it.
  • I. INTRODUCTION: The study introduces an automatically generated radar dataset and evaluates the proposed detector on real-world radar data.The paper describes the dataset and training process before reporting evaluation results.

II. RELATED WORK

Prior radar research separately addressed object classification or bounding-box estimation, while PointNet research established direct point-cloud processing and Frustum PointNets extended it to detection.

  • II. RELATED WORK: Existing radar classification methods use extracted or stochastic features, accumulated grid maps, or neural networks to recognize pedestrians, vehicles, and static objects.The cited approaches include random forests, long short-term memory networks, and deep neural networks.
  • II. RELATED WORK: Radar bounding-box studies estimate vehicle orientation, dimensions, or position using high-resolution radar measurements and specialized geometric algorithms.One approach combines measurements from two radars with oriented bounding-box and L-fit algorithms.
  • II. RELATED WORK: PointNet supports point clouds as input, avoiding the need to transform them into regular structures such as image grids or grid maps.Its described applications include 3D classification and semantic segmentation of lidar point clouds.
  • II. RELATED WORK: The presented problem uses a four-dimensional radar point set from one measurement cycle and seeks 2D object classification and localization.Each point contains x and y coordinates, compensated Doppler velocity, and radar cross section; measurements are not accumulated over time.

IV. 2D OBJECT DETECTION WITH POINTNETS

The detector processes a radar target list through patch proposal, PointNet-based classification and segmentation, and amodal 2D box regression.

  • IV. 2D OBJECT DETECTION WITH POINTNETS: Patch dimensions and target counts are selected to support distinguishing car patches from clutter patches and car targets from clutter targets.The design requires patches to include enough radar targets for both classification and segmentation.
  • IV. 2D OBJECT DETECTION WITH POINTNETS: The system first proposes multiple regions of interest, or patches, from the complete radar target list.A patch is centered around each radar target and sized to contain the entire vehicle of interest.
  • IV. 2D OBJECT DETECTION WITH POINTNETS: The proposed pipeline classifies patches, segments their radar targets, and regresses an amodal 2D bounding box from segmented car targets.The final regression stage uses the m segmented car radar targets.
  • IV. 2D OBJECT DETECTION WITH POINTNETS: Patches are normalized to a center view before entering the classification and segmentation network, providing rotation invariance.All radar targets within each normalized patch are forwarded to the network.

B. Classification and Segmentation

The classification and segmentation module identifies car-containing patches and assigns car-membership probabilities to their radar targets, after which segmented points support box estimation.

  • B. Classification and Segmentation: The network classifies each patch as car or clutter and predicts a car-membership probability for every radar target in car patches.The masking step uses these predictions to retain radar targets classified as belonging to the car.
  • B. Classification and Segmentation: The classification and segmentation design can be extended to multiple classes by using patch-class predictions in the segmentation step.The current described classification distinguishes car and clutter.
  • B. Classification and Segmentation: A Transformer PointNet estimates the amodal box center and transforms segmented radar targets into coordinates relative to that center.The transformation is motivated by the relationship between viewing angle and the centroid of segmented object targets.
  • B. Classification and Segmentation: The box-regression PointNet predicts center, heading angle, length, and width for the 2D bounding box.Heading and size use classification with predefined size templates and associated residual predictions.
  • B. Classification and Segmentation: For multiple classes, box regression also uses class information and requires size templates extended to categories such as pedestrians or cyclists.This class-conditioned extension is described as a mechanism rather than evaluated here.

D. Network Architectures

The detector adapts PointNet and Frustum PointNets to four-dimensional radar target lists for joint classification, segmentation, and amodal 2D box estimation.

  • D. Network Architectures: The architecture processes radar target lists as four-dimensional points containing spatial coordinates, compensated Doppler velocity, and RCS.The classification and box-regression networks receive radar data, while segmented object points are passed to box estimation.
  • D. Network Architectures: Classification and segmentation operate on an n-point patch, producing an m-point object list for the 2D bounding box estimation network.

E. Dataset

The dataset uses real-world radar recordings from a test track, with automatically generated labels and maneuver-divided training, validation, and testing data.

  • E. Dataset: The dataset records a target vehicle using one front-corner radar sensor, with ADMA and DGPS supporting bounding-box ground truth.The target vehicle is a Mercedes E-Class station wagon, and ground truth includes position, orientation, and dimension.
  • E. Dataset: Eleven driving maneuvers, including circles, figure eights, approach, departure, and passing scenarios, are represented under varying weather conditions.
  • E. Dataset: Radar targets are automatically labeled as car or clutter using the target vehicle’s ground-truth bounding box, extended by 0.35 m in length and width.
  • E. Dataset: Car patches require at least 2 car targets, clutter patches require at least 16 clutter targets, and only 4.77% of patches are car patches.
  • E. Dataset: Training, validation, and testing data are separated by repeated maneuver runs, with training and validation sharing maneuver types.

F. Training

Training jointly optimizes classification, segmentation, and bounding-box objectives, using sampling, augmentation, weighted losses, and Adam optimization.

  • F. Training: The multi-task objective jointly optimizes patch classification, segmentation, T-Net, and amodal 2D bounding-box estimation losses.The formulation includes center, heading, size, and corner terms for box regression.
  • F. Training: For clutter patches, the box-loss weight is set to zero, so bounding-box estimation is not performed.
  • F. Training: Training samples up to 48 radar targets for classification and segmentation and up to 32 points for amodal box estimation.
  • F. Training: Augmentation perturbs target positions, Doppler velocity, and RCS values to reduce overfitting.
  • F. Training: Adam training uses learning rate 0.0001, batch size 32, and 11 epochs on one NVIDIA GeForce GTX 1070 GPU.
  • F. Training: Loss weights are set to wcls = 2 and wseg = 2 for car patches, wcls = 1 and wseg = 1 for clutter patches, wbox = 1, and wcorner = 10.

V. EXPERIMENTS

The experiments evaluate the proposed radar detector on classification, segmentation, and 2D bounding-box estimation, while also discussing results and restrictions.

  • V. EXPERIMENTS: The evaluation covers object classification, radar-target segmentation, and 2D bounding-box estimation.

A. Evaluation

The evaluation measures classification, segmentation, and amodal 2D bounding-box estimation on full-dataset and individual-maneuver tests. Prediction takes 2.9 ms per patch, with best bounding-box results when following a target vehicle with or without overtaking.

  • A. Evaluation: Full-dataset testing uses patches containing at least 2 car targets and 16 clutter targets, with 10 m patch dimensions covering an entire vehicle.
  • A. Evaluation: Evaluation uses accuracy and F1 for classification and segmentation, and mean IoU plus IoU-threshold ratios for bounding-box regression.The test set includes the full dataset and individual driving maneuvers.
  • A. Evaluation: 2.9 ms is required per patch for classification, segmentation, and amodal 2D bounding-box prediction.The reported inference time covers the complete object-detector prediction.
  • A. Evaluation: The detector’s best 2D bounding-box results occur while following a target vehicle, both with and without overtaking.

B. Discussion

The discussion reports promising radar-based detection but emphasizes dataset restrictions and the detector’s intended role before multi-radar tracking. Current data contain one object, one class, and fixed object size, while future work targets broader coverage.

  • B. Discussion: The dataset contains one object per radar measurement cycle, a single class, and fixed ground-truth object size.
  • B. Discussion: Future extensions will evaluate multiple objects, multiple classes, and varying object sizes.
  • B. Discussion: The detector is intended as a preprocessing module for multi-object tracking that fuses measurements from multiple radar sensors.The tracker is expected to handle sparse-data misdetections and clutter measurements.

VI. CONCLUSION

The work proposes PointNet variants for classification, 2D segmentation, and amodal 2D bounding-box estimation from sparse radar data. Its promising results remain limited by evaluation with one object class and motivate testing multiple classes and object sizes.

  • VI. CONCLUSION: The proposed detector combines object classification, 2D segmentation, and amodal 2D bounding-box estimation for sparse radar data.
  • VI. CONCLUSION: The reported results are promising, but the evaluation considers only one object class.
  • VI. CONCLUSION: Future evaluation will add several classes and same-class objects with different sizes to examine object-dimension effects.
Loading 1904.08414v3…