PaperlayerView exact revision on arXiv

Source-linked AI summary

CNN-based Lidar Point Cloud De-Noising in Adverse Weather

Robin Heinzler, Florian Piewak, Philipp Schindler, Wilhelm Stork

arXiv:1912.03874v2cs.CVcs.RO

TL;DR

Adverse weather creates erroneous lidar measurements that can impair scene understanding. The paper introduces CNN-based weather segmentation and de-noising, trained and evaluated using controlled-weather data and realistic augmentation, and reports significant improvement over geometric filtering methods.

  • Problem

    Adverse weather produces erroneous lidar points that can cause missed detections and false positives in environment perception.

  • Method

    The paper uses a CNN-based WeatherNet for point-wise weather segmentation, supported by controlled-weather labels and realistic weather-effect augmentation.

  • Results

    The authors report significant performance improvement over previous state-of-the-art, with qualitative results preserving pedestrians and cyclists amid dense-weather scatter.

  • Takeaways & Limitations

    CNN-based weather segmentation can distinguish adverse-weather scatter from valid solid-object points as a lidar perception preprocessing step.

Abstract

from arXiv · show

Lidar sensors are frequently used in environment perception for autonomous vehicles and mobile robotics to complement camera, radar, and ultrasonic sensors. Adverse weather conditions are significantly impacting the performance of lidar-based scene understanding by causing undesired measurement points that in turn effect missing detections and false positives. In heavy rain or dense fog, water drops could be misinterpreted as objects in front of the vehicle which brings a mobile robot to a full stop. In this paper, we present the first CNN-based approach to understand and filter out such adverse weather effects in point cloud data. Using a large data set obtained in controlled weather environments, we demonstrate a significant performance improvement of our method over state-of-the-art involving geometric filtering. Data is available at https://github.com/rheinzler/PointCloudDeNoising.

I. INTRODUCTION

Adverse weather introduces erroneous lidar measurements that impair environment perception, motivating dedicated point-cloud de-noising. The paper proposes CNN-based weather filtering, supported by controlled-weather evaluation, augmentation, and point-level assessment.

  • Fog, rain, dust, snow, pollution, and smog impair lidar range measurements through back-scattered light from water drops or airborne particles.
  • Erroneous weather points can degrade scene understanding, especially for algorithms that directly use low-level point-cloud geometry.The introduction contrasts this need with CNN-based perception algorithms that may cope better through learning capacity.
  • The evaluation uses controlled environments with point-level weather ground truth across clear, rain, and fog conditions.
  • The paper presents the first CNN-based approach to lidar point-cloud weather segmentation and de-noising.
  • The authors report a significant performance boost over previous state-of-the-art while remaining very efficient.
  • The method combines realistic adverse-weather data augmentation with quantitative and qualitative evaluation of de-noising algorithms.

II. RELATED WORK

Prior lidar de-noising work includes image-inspired, spatial, statistical, and segmentation-based filters, but sparse adverse-weather point clouds remain difficult to process. Their limitations reflect lidar sparsity and declining point density with distance.

  • Only a few sparse-point-cloud de-noising algorithms are publicly available, while most state-of-the-art datasets use favorable weather.
  • Camera-image de-noising methods are organized into spatial, statistical, and segmentation-based categories.
  • Spatial smoothing can produce successful snow de-noising but generally fails to preserve edges when it assumes small neighborhood variations.
  • Statistical filters remove minor sensor errors by estimating whether points lie on surfaces.
  • Segment-based filters smooth local regions with identical labels, helping preserve corners and finer structures.
  • Sparse lidar point clouds become substantially less dense at medium and far ranges because conventional sensors have limited angular resolution.

B. Sparse Point Cloud De-Noising

Sparse lidar filters commonly use spatial neighborhoods to identify outliers, but this strategy can remove isolated valid measurements and weather scatter under denser conditions.

  • SOR, ROR, and related filters decide whether points are noise using neighbor distances, neighbor counts, or spatial vicinity.
  • Spatial filters increasingly discard valuable distant information because isolated reflections lack nearby points.This can additionally limit the sensor’s effective range, particularly for autonomous-vehicle use at higher speeds.
  • Sparsity cannot reliably distinguish fog or drizzle scatter once water-drop distributions become sufficiently dense.

C. Semantic Segmentation for Sparse Point Clouds

The paper adapts CNN semantic-segmentation ideas to sparse lidar data, using global scene structure and intensity alongside projected two-dimensional representations for efficient weather filtering.

  • The proposed CNN filter learns underlying point-cloud structure and incorporates intensity information to generalize across distances and clutter distributions.
  • Semantic-segmentation methods can generalize to recognizing objects at different distances and orientations.
  • Because preprocessing requires high computation speed, the approach focuses on two-dimensional input representations such as bird’s-eye-view or image-projection views.
  • The paper proposes a two-dimensional CNN approach inspired by PointPillars for point-wise semantic segmentation rather than object detection.
  • A rotating lidar scan is merged into a cylindrical depth image represented as a two-dimensional matrix of range and related measurements.

B. Autolabeling for Noise Caused by Rain or Fog

The paper derives point-wise rain and fog labels by comparing adverse-weather lidar distances with accumulated reference measurements. This raw-data method is fast and avoids requiring 3D information.

  • Manual weather segmentation labels are difficult, so time-synchronized camera images support labeling decisions for sparse lidar point clouds.The task is deciding whether each point was caused by a water droplet.
  • Reference lidar images from multiple frames are stacked into a ground-truth distance representation for comparison with rain or fog recordings.Accumulation over reference frames accounts for minor sensor measurement inaccuracies.
  • A point is labeled clutter when its adverse-weather distance differs sufficiently from all corresponding ground-truth distances.The comparison uses a search threshold of ΔR = ±35cm to reduce false negatives.
  • Comparing matching transmitter–receiver distances is fast, uses raw sensor data directly, and does not require 3D information.A kd-tree point-cloud comparison produced no significantly different results.
  • Applying the labeling procedure between split reference recordings quantifies its false-label rate under unchanged weather conditions.The evaluation expects all points to be valid because the reference recording contains no weather changes.

C. Data Augmentation

The augmentation pipeline extends favorable-weather lidar data with modeled fog and rain effects, including weather-induced scattering and altered sensing attributes. Fog parameters cover multiple visibility ranges, while rain augmentation uses calibrated scatter probabilities.

  • The augmentation approach adapts a fog model to generate rain effects, enabling large training data without manual annotation and with error-free ground truth.It is applied to lidar images so individual point attributes can be manipulated.
  • The augmentation changes more than point presence: adverse weather also reduces viewing range and lowers intensity and echo-pulse-width contrast.
  • 1) Fog Model:: Fog augmentation derives maximum sensing range from visual range, atmospheric extinction coefficient β, and detector threshold CT.Because lidar is active, its maximum sensor range is half the maximum viewing distance.
  • 1) Fog Model:: Fog scattering-point intensities follow a logarithmic normal distribution whose parameters are estimated from clutter intensities in climate-chamber data.The model covers meteorological visibility of 10−100m and augmented visibility of 30−3000m.
  • Rain augmentation sets β = 0.01 and uses point scatter rate p to control the per-point probability of random scatter points.The parameters are adapted so augmented scatter points correspond to natural rainfall.

D. Network Architecture

WeatherNet adapts the LiLaNet sparse-point-cloud CNN for three-class lidar de-noising. The design reduces depth and expands spatial context while adding dropout for generalization.

  • WeatherNet adopts CNN architectures designed for semantic segmentation of sparse lidar point clouds.It is presented as an efficient variant of LiLaNet.
  • The network depth is reduced because de-noising uses three classes rather than the thirteen classes of full multi-class semantic segmentation.
  • A dilated convolution enlarges the receptive field to provide more information about spatial vicinity.The modified inception layer incorporates this convolution.
  • A dropout layer is inserted to increase the network’s generalization capability.Training uses Adam with learning-rate decay after every epoch.

A. Road Data Set

The study combines controlled climate-chamber recordings with augmented road data to build reproducible weather-segmentation benchmarks. Reference conditions enable point-wise labels for clear, rain, and fog, while analysis links clutter to weather degradation.

  • A. Road Data Set: Augmentation reuses favorable-weather datasets, adds road recordings, and helps maintain balanced class distributions while reducing overfitting to chamber scenes.
  • A. Road Data Set: A climate-chamber dataset records four realistic road scenarios under constant, reproducible fog, rain, and reference conditions.Fog visibility ranges from 10−100m, and rainfall intensities are 15, 33, and 55mm/h.
  • B. Climate Chamber Scenarios and Ground Truth Labels: Reference recordings enable automatic point-wise labels for clear points, rain clutter, and fog clutter without manual weather annotation.
  • B. Climate Chamber Scenarios and Ground Truth Labels: Lidar clutter counts correlate with visibility or rainfall rate, while valid and weather-induced points together preserve the reference point count.
  • B. Climate Chamber Scenarios and Ground Truth Labels: Rainfall rate cannot be estimated directly from scatter-point increases, but lidar sensor degradation can be estimated from the weather-induced measurements.This degradation information can support autonomous-vehicle adaptation to environmental conditions and sensor performance.
  • A. Road Data Set: The dataset contains about 175,941 training, validation, and test samples split approximately 60%−15%−25%.It includes 72,800 chamber samples and 103,141 road samples.

V. EXPERIMENTS

Experiments evaluate WeatherNet using controlled-weather and road data, with IoU-based segmentation results, confusion matrices, and comparisons against geometric and CNN baselines. Adding road data and weather augmentation improves overall and per-class performance, while WeatherNet is efficient and outperforms DROR by an order of magnitude.

  • Experimental Setup: Evaluations use chamber-only, chamber-plus-road, and chamber-plus-road-with-augmentation training settings, assessed with Intersection-over-Union.All evaluations use the test set from experiment 2.
  • Segmentation Performance: WeatherNet performance significantly increases when road data and proposed weather augmentation are added, improving overall and per-class IoU.The result indicates weather influences are identified across both domains.
  • Comparison with Geometric Filtering: The baseline DROR filter shows that local vicinity alone is inadequate for filtering scatter points caused by dense water drops.DROR parameters follow prior work, with horizontal resolution adapted to the Velodyne VLP32C.
  • Comparison with Geometric Filtering: An order of magnitude: the CNN approach outperforms DROR on the reported de-noising comparison.The comparison concerns filtering scatter points in adverse weather.
  • Comparison with CNN Baselines: WeatherNet mostly outperforms RangeNet21, RangeNet53, and LiLaNet, especially in experiment 3, while using fewer trainable parameters and less inference time.These efficiency gains support using the network as a preprocessing step.
  • Error Analysis: Rain and fog are mostly confused because both consist of water droplets, differing in distribution, density, and size; this confusion does not hinder point-cloud filtering.The distinction matters more for weather-condition classification than for separating clutter from valid points.

A. Qualitative Results on Dynamic Chamber Data

Dynamic chamber scenes test de-noising in dense fog at 30 m visibility, where WeatherNet removes fog clutter while retaining small dynamic objects. Compared with DROR and RangeNet53, it better preserves or distinguishes the cyclist and pedestrian in the qualitative examples.

  • A. Qualitative Results on Dynamic Chamber Data: At 30 m visibility in dense fog, the evaluated scene contains a cyclist and pedestrian that are barely recognizable before filtering.The objects are highlighted with black boxes in the figure.
  • A. Qualitative Results on Dynamic Chamber Data: WeatherNet retains the pedestrian and cyclist while discarding fog clutter, whereas DROR cannot distinguish rain from fog in this example.DROR colors all filtered points as rain points because it lacks a rain–fog distinction.
  • A. Qualitative Results on Dynamic Chamber Data: WeatherNet distinguishes the pedestrian and cyclist from scatter points, while RangeNet53 filters most parts of both objects despite comparable evaluated performance.RangeNet53 preserves fine structures and edges but removes substantial portions of the small objects.
  • A. Qualitative Results on Dynamic Chamber Data: Spatial filtering fails when fog points are distributed as densely as real-object points, although it can filter some isolated scattering points.DROR also removes distant single points that are not weather-caused.
  • A. Qualitative Results on Dynamic Chamber Data: Fog scatter-point counts correlate with meteorological visibility, enabling visibility estimation from weather segmentation and degradation estimation from the scatter-to-valid-point ratio.These estimates use lidar point-cloud information rather than direct rainfall-rate estimation.

B. Qualitative Results on Dynamic Road Data

Road experiments show that the approach works on lidar point clouds corrupted by natural rainfall, including dynamic scenes with pedestrians and passing cars. The conclusion frames CNN weather segmentation as an efficient preprocessing step for distinguishing weather scatter from valid object points.

  • B. Qualitative Results on Dynamic Road Data: The approach operates on point clouds corrupted by natural rainfall recorded on roads.This extends evaluation beyond controlled chamber weather.
  • B. Qualitative Results on Dynamic Road Data: In a light-rain road scene, the method handles a crossing pedestrian and multiple passing cars despite training based on chamber data complemented by augmented real-world data.The authors describe this as generalization to a different real-world scenario.
  • VI. CONCLUSION: The CNN approach is presented as an essential preprocessing step that distinguishes adverse-weather scatter points from valid points belonging to solid objects.The conclusion contrasts its global traffic-scene understanding with local-spatial-statistics filters.
  • VI. CONCLUSION: Quantitative and qualitative results demonstrate superior performance over state-of-the-art while maintaining high efficiency.The paper points readers to additional dynamic-scene results in the dataset and supplementary video.
Loading 1912.03874v2…