PaperlayerView exact revision on arXiv

Source-linked AI summary

Survey on LiDAR Perception in Adverse Weather Conditions

Mariella Dreissig, Dominik Scheuble, Florian Piewak, Joschka Boedecker

arXiv:2304.06312v2cs.ROcs.CV

TL;DR

Adverse weather can degrade LiDAR measurements used in safety-critical autonomous-vehicle perception, while labeling noisy point clouds is difficult and costly. This survey analyzes data, point-cloud processing, and robust-perception approaches, and identifies research gaps and promising directions.

  • Problem

    Weather-induced noise can impair LiDAR scene understanding, while labeling sparse, noisy point clouds is difficult, expensive, and error-prone.

    Source: §I · p. 1

  • Method

    The survey analyzes LiDAR weather datasets, point-cloud processing and denoising, and robust perception approaches, including sensor fusion and training adjustments.

    Source: §I · p. 2

  • Results

    The survey reviews current LiDAR weather-perception research and identifies remaining gaps and promising research directions.

    Source: §V · p. 7

  • Takeaways & Limitations

    The survey highlights weather augmentation as more effective than denoising for robustness, while identifying future research directions for LiDAR perception.

    Source: B. Enhancing Training with Data… · p. 6, §V · p. 7

Abstract

from arXiv · show

Autonomous vehicles rely on a variety of sensors to gather information about their surrounding. The vehicle's behavior is planned based on the environment perception, making its reliability crucial for safety reasons. The active LiDAR sensor is able to create an accurate 3D representation of a scene, making it a valuable addition for environment perception for autonomous vehicles. Due to light scattering and occlusion, the LiDAR's performance change under adverse weather conditions like fog, snow or rain. This limitation recently fostered a large body of research on approaches to alleviate the decrease in perception performance. In this survey, we gathered, analyzed, and discussed different aspects on dealing with adverse weather conditions in LiDAR-based environment perception. We address topics such as the availability of appropriate data, raw point cloud processing and denoising, robust perception algorithms and sensor fusion to mitigate adverse weather induced shortcomings. We furthermore identify the most pressing gaps in the current literature and pinpoint promising research directions.

I. INTRODUCTION

LiDAR provides sparse but accurate depth information for autonomous driving, yet adverse weather can distort its measurements and impair safety-critical scene understanding. This survey reviews LiDAR-specific coping strategies across data, point-cloud processing, and robust perception.

I. INTRODUCTION

  • LiDAR supplies sparse but accurate depth information and complements camera and radar for autonomous-driving perception.It emits light pulses and estimates distance from the time taken for reflected light to return.Source: §I · p. 1
  • Backscattering or loss of emitted light in airborne particles produces noisy distance and reflectance measurements that can harm scene-understanding algorithms in safety-critical driving.Weather can also affect intensity values, point counts, and other point-cloud characteristics; sensor technology, model, and mounting position influence deterioration.Source: §I · p. 1
  • Accurately labeling sparse, weather-corrupted LiDAR point clouds is difficult, expensive, and error-prone, motivating methods to simulate or augment weather-specific noise.Most recent deep-learning approaches require labels such as object bounding boxes or point-wise segmentation classes.Source: §I · p. 1
  • The survey addresses a gap in comprehensive coverage of algorithmic strategies for improving LiDAR perception in adverse weather, which earlier weather surveys did not cover in LiDAR-specific detail.It organizes the topic around data availability, point-cloud manipulation, and robust perception.Source: §I · p. 2

II. ADVERSE WEATHER DATA

Weather-corrupted LiDAR data are costly and difficult to label, while real-world recordings, augmentation, and simulation offer different ways to obtain weather-specific point clouds. Real-world datasets provide realism but may have incomplete labels or limited applicability when recorded in weather chambers.

II. ADVERSE WEATHER DATA

  • Weather-corrupted sparse LiDAR data are expensive and cumbersome to label, making high-quality labeled datasets a central need.The difficulty is greater when raw point clouds contain weather-induced noise.Source: §II · p. 2
  • Adverse-weather point clouds can come from real-world recordings, augmented clear-weather clouds, or physical-model- or DL-based simulation.Augmentation can retain existing annotations, while real-world recordings use test cars with appropriate sensor setups.Source: §II · p. 2

A. Real-world Datasets

  • Most LiDAR perception benchmark datasets use favorable-weather recordings, although some extensive datasets also include rain, snow, and fog.Source: A. Real-world Datasets · p. 2
  • Real-world datasets offer high realism, but may have only partial point-wise labels or weather-chamber recordings with limited applicability to complex real-world scenarios.Manual point-wise labeling is particularly difficult because clutter or noise may be impractical to distinguish from actual reflections.Source: A. Real-world Datasets · p. 2

B. Weather Augmentation

Weather augmentation creates adverse-weather point clouds from clear-weather data, offering a way to expand training data while retaining existing scenes and annotations. The survey reviews physical, empirical, and DL-based approaches, while noting that realism is difficult to assess.

B. Weather Augmentation

  • Augmentation maps clear-weather points to adverse-weather conditions and can generate large datasets while preserving existing scenes and annotations.Physics-based or empirical models are often applied to real-world or simulated clear-weather clouds.Source: B. Weather Augmentation · p. 2

WEATHER LABEL

  • Fog and rain models use visibility, intensity, or noise thresholds to add scatter points or remove points.Some rain models also simulate beam divergence; related approaches extend the modeling to snow and fog.Source: WEATHER LABEL · p. 3
  • Power-domain models compare attenuated returns with distance-dependent noise thresholds to add scatter points or lose points.Other models compute intensity profiles to represent occlusions; one also models wet-ground point loss.Source: WEATHER LABEL · p. 3
  • Empirical spray augmentation models use experimental observations or dataset scenes, while a GAN-based approach transforms sunny clouds into foggy or rainy ones.The GAN results were compared qualitatively with real foggy and rainy point clouds from a weather chamber.Source: WEATHER LABEL · pp. 3–4
  • Assessing augmentation realism is challenging; some studies compare against controlled weather effects or assess whether augmentation aids perception in real adverse weather.Source: WEATHER LABEL · p. 4

III. POINT CLOUD PROCESSING & DENOISING

The section surveys weather-coping approaches at the sensor or point-cloud level, independent of the perception task. It covers robustness across sensors, degradation estimation, and classical or DL-based point-cloud denoising.

III. POINT CLOUD PROCESSING & DENOISING

  • Sensor-technique- and point-cloud-based approaches address adverse weather independently of the perception task.Source: §III · p. 4
  • The surveyed approaches include estimating performance degradation from weather conditions and removing weather-induced point-cloud noise with classical methods or DL.Source: §III · p. 4
  • The section also examines general sensor-dependent weather robustness.Source: §III · p. 4

A. Sensor-related Weather Robustness

Weather effects vary with LiDAR technology, characteristics, and configuration. The section also discusses full waveform LiDAR, which captures weaker returns and can help analyze the surrounding medium despite high computational demands.

A. Sensor-related Weather Robustness

  • LiDAR weather sensitivity varies with technology, sensor characteristics, and configuration; 1550 nm models appear to offer improved visibility in heavy fog compared with 905 nm models.The text attributes the apparent visibility improvement to higher emitted power.Source: A. Sensor-related Weather Robustness · p. 4
  • Full Waveform LiDAR measures weaker returns as well as stronger ones, providing more information about the surrounding and noise at high computational cost.It has been useful for analyzing the surrounding medium, which can support understanding changing conditions and dynamic adjustment.Source: A. Sensor-related Weather Robustness · p. 4

B. Sensor Degradation Estimation and Weather Classification

Because LiDAR sensors degrade differently across weather conditions, estimating degradation and classifying weather are presented as ways to identify sensing limits and address corrupted point clouds. The survey covers physical and learned estimation approaches, while noting that most methods need real-world evaluation.

B. Sensor Degradation Estimation and Weather Classification

C. Point Cloud Denoising

Denoising methods remove weather-specific noise to reconstruct clearer LiDAR measurements, using classical filters or learned models. The survey describes their evaluation needs, computational trade-offs, and limits in validating unsupervised outputs.

C. Point Cloud Denoising

  • Classical denoising is evaluated by precision for preserving environmental features and recall for filtering weather noise, requiring point-wise weather labels.Source: C. Point Cloud Denoising · p. 4
  • Distance-adjusted and statistical outlier filters address natural sparsity at range, but their high runtimes limit autonomous-driving use; FCSOR and DSOR reduce computational load,,,.Source: C. Point Cloud Denoising · p. 5
  • Weather errors associate with dense, low-intensity, close-range, fast-decaying points; LIOR and DDIOR use intensity, with DDIOR reporting compelling snowy-scan results at low computational cost,.Source: C. Point Cloud Denoising · p. 5
  • Roadside denoisers use historical background models and intensity or density cues, but are not readily applicable to vehicle-mounted LiDAR,.Source: C. Point Cloud Denoising · p. 5
  • CNN-based, especially voxel-based, denoisers outperform classical methods in noise filtering and have lower inference time through faster GPU computations.Temporal data can further help distinguish weather noise because it changes faster than scene backgrounds and objects.Source: C. Point Cloud Denoising · p. 5
  • Unsupervised CycleGANs convert noisy inputs into clear scans, but outputs remain noisy and are difficult to validate for realism,.Source: C. Point Cloud Denoising · p. 5

IV. ROBUST LIDAR PERCEPTION

Robust LiDAR perception can use sensor fusion, weather-specific augmentation, or general methods for domain shifts. The survey presents fusion as a way to offset complementary sensor weaknesses, while noting task-coverage and runtime considerations.

IV. ROBUST LIDAR PERCEPTION

  • Robustness methods span sensor fusion, weather-noise augmentation, and general approaches to domain shift, independent of data quality and noise level.Source: §IV · p. 5

A. Combating Adverse Weather with Sensor Fusion

  • Sensor-fusion approaches are the only ones described as tackling multiple perception tasks beyond object detection; semantic-segmentation work is absent.Source: §IV · p. 5
  • Sensor fusion combines complementary strengths because LiDAR, cameras, and radar have different weather sensitivities and perception capabilities.LiDAR is affected by airborne particles, cameras by strong light and blooming, and radar lacks static-object and fine-structure detection.Source: A. Combating Adverse Weather with… · p. 5
  • Early fusion can filter false detections in radar-LiDAR depth maps, whereas late fusion can rely on an intact sensor when streams degrade unequally,,,.The preferred approach depends on sensor choice, data representation, and expected failure rates.Source: A. Combating Adverse Weather with… · p. 5
  • Fusion methods also combine early and late processing with temporal or region-based fusion, attention maps, or adaptive entropy-steered fusion,,.Source: A. Combating Adverse Weather with… · p. 6
  • A metric combining drivable-space segmentation performance with inference runtime ranked the LiDAR-only model best.Source: A. Combating Adverse Weather with… · p. 6

B. Enhancing Training with Data Augmentation

Weather-specific augmentation trains models to accommodate weather noise rather than removing it, and the survey reports greater robustness than denoising. Its effectiveness also varies with model architecture and object representation.

B. Enhancing Training with Data Augmentation

  • Weather augmentation is reported as more effective than denoising for robustness, suggesting emphasis on training with weather-specific noise.Augmentation trains models to accommodate the noise rather than remove it from point clouds.Source: B. Enhancing Training with Data… · p. 6
  • Point-based and voxelizing detectors appear less affected by augmented weather effects, but architecture interacts with corruption type and wet-ground extension benefits only some models,,.Source: B. Enhancing Training with Data… · p. 6
  • Smaller, underrepresented classes suffer more from weather augmentation than well-represented classes, and clear-training-set annotation counts indicate detection performance,,.Weather augmentation also appears to aid performance under clear conditions and vice versa.Source: B. Enhancing Training with Data… · p. 6

C. Robust Perception Algorithms

Weather-to-weather domain adaptation is a potential route to robust LiDAR perception, but existing dataset-to-dataset evaluations confound weather shifts with other differences. Reported approaches include teacher-student training, robust detection, and multimodal domain adaptation.

C. Robust Perception Algorithms

  • Dataset-to-dataset evaluations leave the weather-specific contribution unclear because sensor resolution and label strategy can overshadow the weather-induced gap.The domain shift is not limited to weather, and the overall dataset-to-dataset shift is very strong.Source: C. Robust Perception Algorithms · p. 6
  • Teacher-student training from sunny Waymo Open to rainy Kirkland yielded students that appeared to generalize better to the target domain [78].The teacher generated labels for part of Waymo Open and part of Kirkland; the student was trained on all labels.Source: C. Robust Perception Algorithms · p. 6
  • A maximum-discrepancy-loss joint training approach showed high performance on both sunny KITTI and rainy CADC after the proposed robust pipeline failed to perform well on both simultaneously.The pipeline used attention mechanisms and global context-aware feature extraction.Source: C. Robust Perception Algorithms · p. 6
  • Multimodal, multi-target domain adaptation using domain discriminators and self-supervised pretraining generalized well to scenarios such as fog [80].The work combined sensor fusion and data augmentation for RGB cameras and LiDAR.Source: C. Robust Perception Algorithms · p. 6
  • Existing domain-adaptation overviews mainly address differences in sensor resolution or available data and labels, rather than weather shifts specifically [77].Source: C. Robust Perception Algorithms · p. 6

V. DISCUSSION AND CONCLUSION

The survey reviews training data, point-cloud processing and denoising, and robust LiDAR perception under adverse weather. It identifies gaps in real-world data and evaluation, systematic perception-impact analysis, and LiDAR-only robustness, while outlining potential research directions.

V. DISCUSSION AND CONCLUSION

  • The survey analyzes training-data availability, perception-independent point-cloud processing and denoising, and state-of-the-art robust LiDAR perception approaches.It also identifies remaining gaps and promising research directions.Source: §V · p. 7
  • Real-world adverse-weather datasets need more suitable data, especially point-wise class labels, and augmentation methods still lack a way to evaluate realism.Most available datasets provide object labels, but only one has point-wise class labels.Source: §V · p. 7
  • A systematic analysis of how adverse weather affects perception algorithms is missing, and sensor degradation estimation may help address this gap.LiDAR technologies react differently to adverse weather; existing statistical denoising methods have been less efficient than weather augmentation during training.Source: §V · p. 7
  • Despite compelling sensor-fusion results, LiDAR-only robustness remains a research direction, with domain adaptation and weather-noise perspectives identified as possible avenues.The survey suggests anomaly detection, uncertainty modeling, and investigation of weather-induced domain-gap quality.Source: §V · p. 7

Related papers

Loading 2304.06312v2…