PaperlayerView exact revision on arXiv

Source-linked AI summary

High Resolution Point Clouds from mmWave Radar

Akarsh Prabhakara, Tao Jin, Arnav Das, Gantavya Bhatt, Lilly Kumari, Elahe Soltanaghaei, Jeff Bilmes, Swarun Kumar, Anthony Rowe

arXiv:2206.09273v2cs.ROcs.NIeess.SP

TL;DR

Single-chip mmWave radar offers sensing through occlusions, but its low angular resolution limits point-cloud detail. RadarHD learns lidar-like point clouds from radar input, reporting low error against lidar ground truth and downstream use in odometry and mapping.

  • Problem

    Single-chip radar has much lower angular resolution than lidar, while radar artifacts complicate recovering dense, accurate point clouds.

    Source: §I · p. 1

  • Method

    RadarHD uses supervised radar-lidar pairs to train a customized end-to-end neural network that generates lidar-like point clouds from low-resolution radar input.

    Source: §I · pp. 1–2

  • Results

    RadarHD reports 24 cm point-cloud error versus lidar ground truth and 3.5× better performance than traditional radar point clouds.

    Source: §I · p. 2

  • Takeaways & Limitations

    RadarHD’s point clouds are demonstrated in odometry and mapping using existing lidar workflows, including Google Cartographer.

    Source: §I · p. 2, B. Odometry and Mapping Comparison · p. 6

Abstract

from arXiv · show

This paper explores a machine learning approach for generating high resolution point clouds from a single-chip mmWave radar. Unlike lidar and vision-based systems, mmWave radar can operate in harsh environments and see through occlusions like smoke, fog, and dust. Unfortunately, current mmWave processing techniques offer poor spatial resolution compared to lidar point clouds. This paper presents RadarHD, an end-to-end neural network that constructs lidar-like point clouds from low resolution radar input. Enhancing radar images is challenging due to the presence of specular and spurious reflections. Radar data also doesn't map well to traditional image processing techniques due to the signal's sinc-like spreading pattern. We overcome these challenges by training RadarHD on a large volume of raw I/Q radar data paired with lidar point clouds across diverse indoor settings. Our experiments show the ability to generate rich point clouds even in scenes unobserved during training and in the presence of heavy smoke occlusion. Further, RadarHD's point clouds are high-quality enough to work with existing lidar odometry and mapping workflows.

I. INTRODUCTION

Single-chip mmWave radar can sense through occlusions, but its angular resolution is far below lidar’s, limiting point-cloud detail. RadarHD learns to generate lidar-like point clouds from radar, with evaluations showing low error and use in odometry and mapping.

I. INTRODUCTION

  • Single-chip radar has angular resolution two orders of magnitude below lidar, restricting it to coarse collision-avoidance applications; higher-resolution mechanical radars add bulk and cost.Lidar’s dense, low-noise point clouds support mapping and localization, but lidar and visible-light cameras fail in fog, smoke, or dust.Source: §I · p. 1
  • RadarHD is a customized end-to-end neural network that generates lidar-like point clouds from low-resolution radar input, learning features traditional signal-processing pipelines may discard.The approach uses supervised radar-lidar pairs from identical scenes and customizes the representation, architecture, and loss functions.Source: §I · pp. 1–2
  • Radar measurements contain sidelobes, specular reflections, and other artifacts, while coarse radar data struggles to resolve sharp environmental features.The artifacts must be distinguished from true objects to construct dense, accurate point clouds.Source: §I · p. 1
  • 24 cm point-cloud error versus lidar ground truth, and 3.5× better performance than traditional radar point clouds, are reported for RadarHD.The system also demonstrates its point clouds in odometry and mapping using Google Cartographer.Source: §I · p. 2
  • RadarHD is evaluated in unseen environments and severe occlusions such as smoke, and the paper reports a repository of raw radar I/Q and lidar pairs with source code.The authors collected 200,000 radar-lidar pairs across indoor and outdoor environments.Source: §I · p. 2

II. RELATED WORK

Prior radar super-resolution work uses synthetic aperture, sensor fusion, or learning, but often assumes precise motion or static single-object scenes. RadarHD instead targets general-purpose high-resolution point clouds from single-chip radar in complex environments and contributes a larger radar-lidar dataset.

Radar Super Resolution

  • SAR and sensor-fusion approaches can require precise motion or auxiliary sensors, limiting their fit for portable, arbitrarily moving, or occluded settings.In portable applications, obtaining mm-accurate motion information for SAR can be expensive; camera and lidar fusion also faces occlusion limits.Source: §II · p. 2
  • RadarHD targets high-resolution point clouds from single-chip radar in static or arbitrarily moving, complicated real environments, unlike prior learning setups focused on static platforms and single objects.Earlier approaches include SAR-based methods and deep learning; HawkEye uses a mechanically scanning radar on a large-aperture slider.Source: §II · p. 2

Radar Odometry and Mapping

  • RadarHD addresses a different problem from radar odometry and mapping systems by generating general-purpose point clouds that can feed multiple lidar-processing pipelines.Prior systems mainly use scan matching or fuse radar with velocity estimates, IMU, or RGB; RadarHD replaces CFAR points with higher-resolution point clouds.Source: §II · p. 2

Radar-Lidar Datasets

  • Because existing radar-lidar datasets provide limited indoor pairs, the authors collected their own dataset for larger-scale training and testing.The closest dataset had about 15k indoor radar-lidar pairs, only a fraction of its total.Source: §II · pp. 2–3

III. RadarHD SYSTEM DESIGN

RadarHD addresses radar artifacts and coarse spatial resolution with a learning-based design that combines global scene understanding and upsampling. Its pipeline is customized across representation, architecture, and loss functions.

III. RadarHD SYSTEM DESIGN

  • RadarHD must distinguish true objects from sidelobes, specular reflections, and other artifacts while recovering sharp environmental features from coarse radar images.Sidelobes from strong reflectors create sinc-like patterns across azimuth because of poor azimuth resolution.Source: §III · p. 3
  • Rather than relying on local pixel-neighborhood interpolation, RadarHD uses machine learning for global image understanding before upscaling radar data.Radar neighborhoods may contain spreading sidelobes and ghost points, so local similarity is unreliable.Source: §III · p. 3
  • RadarHD adapts a U-net encoder-decoder for radar super resolution, with the encoder representing scene objects and the decoder performing upsampling.The section organizes design choices around radar I/Q representation, artifact removal while preserving true objects, and losses that retain sharp lines.Source: §III · p. 3

A. Radar Data Representation

RadarHD feeds the network a lightly thresholded radar heatmap rather than raw I/Q or a heavily filtered point cloud. Its polar layout matches radar’s radial measurements and azimuthal artifacts to the spatial operations of convolutional filters.

A. Radar Data Representation

  • RadarHD’s hybrid representation preserves weaker returns that high-threshold CFAR would discard while avoiding direct learning from raw I/Q data.Raw I/Q requires learning phase information and initial processing steps, while highly thresholded input can remove feeble objects masked by sinc lobes.Source: A. Radar Data Representation · p. 3, Figure 3 · p. 4
  • RadarHD applies a low heatmap threshold to retain dominant and weak reflectors along with artifacts, leaving the model to filter spurious returns.The thresholded input has 1,606 nonzero pixels, compared with 110 after CFAR thresholding.Source: A. Radar Data Representation · p. 3
  • The input uses polar coordinates because radar measures radially and sidelobes spread azimuthally, allowing convolution filters to traverse range and azimuth.The image places range along rows and azimuth along columns; the radar input is 64 × 256, narrower than the 512 × 256 lidar representation.Source: A. Radar Data Representation · pp. 3–4

B. Neural network architecture

RadarHD uses a multiscale U-net to denoise and interpret radar scenes, then extends its decoder for super resolution. It also uses past frames to address specularity and balances output sharpness against feature retention.

B. Neural network architecture

  • The U-net encoder builds global scene understanding across resolutions, while the decoder performs super resolution and is extended to produce the wider output.The adapted architecture is asymmetric because the super-resolution output image is wider than the input.Source: B. Neural network architecture · p. 4
  • Dice loss sharpens lines but can remove important features, so RadarHD balances sharpness and accuracy.The qualitative ablation contrasts Dice’s sharper output with features it misses, as indicated by colored circles.Source: Figure 4 · p. 4
  • RadarHD stacks 40 past radar frames, covering two seconds, as input channels to model persistence and reduce jitter, including in static settings.Past views help address specular objects that appear and disappear as viewing orientation changes.Source: B. Neural network architecture · p. 4

C. Neural network training methodology

RadarHD combines pixel-wise BCE and Dice loss to match lidar-derived images while encouraging sharper boundaries. The losses trade off pixel accuracy against preserving important features.

C. Neural network training methodology

IV. IMPLEMENTATION

RadarHD uses a TI AWR1843 single-chip radar and a time-synchronized lidar testbed, with raw radar I/Q paired to lidar ground truth. Training uses office data, while testing spans unseen trajectories and environments.

IV. IMPLEMENTATION

  • The system uses a TI AWR1843 radar with theoretical 3.75 cm range and 15° azimuth resolution, collecting raw I/Q samples through a DCA1000EVM.Source: §IV · p. 5
  • The time-synchronized testbed pairs radar with an Ouster OS-0 64-beam lidar; ground truth uses forward-facing points within ±30 cm elevation.The lidar ground-truth configuration has 0.35° azimuth resolution.Source: §IV · p. 5
  • The baseline is Cell-Averaging CFAR thresholding, a non-learning method that relies only on radar signals and works with static or mobile radar.Source: §IV · p. 5
  • Training uses 28 trajectories from an 839 m² office, while 39 test trajectories cover the same setting, other offices, and new lobby and outdoor environments.The repository contains about 200k radar I/Q–lidar pairs; tests use unseen data.Source: §IV · p. 5

A. Point cloud Comparison

RadarHD produces point clouds closer to lidar ground truth than CFAR and retains meaningful structure across new environments. In the tested smoke densities, its performance does not degrade.

A. Point cloud Comparison

  • Point-cloud errors are evaluated against lidar using Chamfer and modified-Hausdorff nearest-neighbor distances.The smoke test captures the same static scene without smoke and at varying smoke levels.Source: A. Point cloud Comparison · pp. 5–6
  • 0.24 m modified-Hausdorff median error and 0.36 m Chamfer median error are reported for RadarHD, which improves on CFAR by 3.5× and 2.7×, respectively.The comparison uses 18k radar–lidar pairs over 19 trajectories; CFAR’s density varies substantially with threshold.Source: A. Point cloud Comparison · p. 5, Figure 10 · p. 6
  • With increasing CFAR threshold from 1 to 8 dB, point-cloud error CDFs shift left, but higher thresholds produce sparse clouds that lack ground-truth structure.Beyond 8 dB, CFAR yields fewer than 1% as many points as lidar.Source: Figure 10 · p. 6
  • In similar and different environments, RadarHD’s median errors are 0.75 m and 0.94 m modified-Hausdorff, and 0.8 m and 1.03 m Chamfer, respectively.The CDFs begin at 0.08 m in both settings, versus 0.4 m for CFAR; Fig. 8 shows meaningful generated points.Source: A. Point cloud Comparison · p. 6
  • Even at the densest tested smoke level, RadarHD performance does not degrade, while lidar receives no points with one smoke pellet.The radar signals with and without smoke were almost identical, including with four pellets.Source: A. Point cloud Comparison · p. 6

B. Odometry and Mapping Comparison

RadarHD point clouds support odometry and mapping through an existing lidar SLAM workflow without additional sensors. They outperform CFAR in odometry and yield maps structurally similar to lidar maps.

B. Odometry and Mapping Comparison

  • RadarHD odometry outperforms CFAR regardless of threshold across tested environments and reaches performance comparable to the 0.8 m reported for radar-plus-IMU odometry pipelines.The evaluation uses Absolute Trajectory Error in Google Cartographer without adding another sensor such as an IMU.Source: B. Odometry and Mapping Comparison · p. 6
  • RadarHD produces a structurally similar map to lidar, whereas CFAR provides no meaningful features for extracting corresponding keypoints.Mapping error is the Euclidean distance between keypoints corresponding to the same physical feature.Source: B. Odometry and Mapping Comparison · p. 6
  • The authors attribute successful odometry and mapping to RadarHD’s artifact-free, meaningful point clouds, which also allow visual debugging when downstream performance is poor.Source: B. Odometry and Mapping Comparison · p. 6

VI. CONCLUSION AND FUTURE WORK

RadarHD generates lidar-like high-resolution point clouds from single-chip mmWave radar for robotic settings where lidar fails. The authors report rich point clouds in new environments and smoke occlusion, while identifying 3D output and dynamic scenes as future work.

VI. CONCLUSION AND FUTURE WORK

  • RadarHD generates high-resolution point clouds from low-resolution single-chip mmWave radar for robotic applications where lidar fails.The pipeline selects design parameters to address radar artifacts.Source: §VI · p. 6
  • RadarHD produces rich point clouds in completely new environments and under occlusions such as smoke.Source: §VI · p. 6
  • The authors identify 3D point clouds, highly dynamic scenes, and 3D mobile platforms as future challenges.The collected repository contains radar I/Q–lidar pairs and is described as useful for other perception tasks.Source: §VI · pp. 6–7

Related papers

Loading 2206.09273v2…