PaperlayerView exact revision on arXiv

Source-linked AI summary

CNN-based Lidar Point Cloud De-Noising in Adverse Weather

Robin Heinzler, Florian Piewak, Philipp Schindler, Wilhelm Stork

arXiv:1912.03874v2cs.CVcs.RO

TL;DR

Adverse weather creates erroneous lidar measurements that can impair scene understanding. The paper introduces CNN-based weather segmentation and de-noising, trained and evaluated using controlled-weather data and realistic augmentation, and reports significant improvement over geometric filtering methods.

  • Problem

    Adverse weather produces erroneous lidar points that can cause missed detections and false positives in environment perception.

    Source: §I · p. 1

  • Method

    The paper uses a CNN-based WeatherNet for point-wise weather segmentation, supported by controlled-weather labels and realistic weather-effect augmentation.

    Source: §I · pp. 1–2, D. Network Architecture · p. 4

  • Results

    The authors report significant performance improvement over previous state-of-the-art, with qualitative results preserving pedestrians and cyclists amid dense-weather scatter.

    Source: §I · p. 2, A. Qualitative Results on Dynamic… · pp. 6–7, §VI · p. 7

  • Takeaways & Limitations

    CNN-based weather segmentation can distinguish adverse-weather scatter from valid solid-object points as a lidar perception preprocessing step.

    Source: §VI · p. 7

Abstract

from arXiv · show

Lidar sensors are frequently used in environment perception for autonomous vehicles and mobile robotics to complement camera, radar, and ultrasonic sensors. Adverse weather conditions are significantly impacting the performance of lidar-based scene understanding by causing undesired measurement points that in turn effect missing detections and false positives. In heavy rain or dense fog, water drops could be misinterpreted as objects in front of the vehicle which brings a mobile robot to a full stop. In this paper, we present the first CNN-based approach to understand and filter out such adverse weather effects in point cloud data. Using a large data set obtained in controlled weather environments, we demonstrate a significant performance improvement of our method over state-of-the-art involving geometric filtering. Data is available at https://github.com/rheinzler/PointCloudDeNoising.

I. INTRODUCTION

Adverse weather introduces erroneous lidar measurements that impair environment perception, motivating dedicated point-cloud de-noising. The paper proposes CNN-based weather filtering, supported by controlled-weather evaluation, augmentation, and point-level assessment.

  • Fog, rain, dust, snow, pollution, and smog impair lidar range measurements through back-scattered light from water drops or airborne particles.Source: §I · p. 1
  • Erroneous weather points can degrade scene understanding, especially for algorithms that directly use low-level point-cloud geometry.The introduction contrasts this need with CNN-based perception algorithms that may cope better through learning capacity.Source: §I · p. 1
  • The evaluation uses controlled environments with point-level weather ground truth across clear, rain, and fog conditions.Source: §I · pp. 1–2
  • The paper presents the first CNN-based approach to lidar point-cloud weather segmentation and de-noising.Source: §I · p. 1
  • The authors report a significant performance boost over previous state-of-the-art while remaining very efficient.Source: §I · p. 2
  • The method combines realistic adverse-weather data augmentation with quantitative and qualitative evaluation of de-noising algorithms.Source: §I · p. 2

II. RELATED WORK

Prior lidar de-noising work includes image-inspired, spatial, statistical, and segmentation-based filters, but sparse adverse-weather point clouds remain difficult to process. Their limitations reflect lidar sparsity and declining point density with distance.

B. Sparse Point Cloud De-Noising

Sparse lidar filters commonly use spatial neighborhoods to identify outliers, but this strategy can remove isolated valid measurements and weather scatter under denser conditions.

  • SOR, ROR, and related filters decide whether points are noise using neighbor distances, neighbor counts, or spatial vicinity.Source: B. Sparse Point Cloud De-Noising · p. 2
  • Spatial filters increasingly discard valuable distant information because isolated reflections lack nearby points.This can additionally limit the sensor’s effective range, particularly for autonomous-vehicle use at higher speeds.Source: B. Sparse Point Cloud De-Noising · p. 2
  • Sparsity cannot reliably distinguish fog or drizzle scatter once water-drop distributions become sufficiently dense.Source: B. Sparse Point Cloud De-Noising · p. 2

C. Semantic Segmentation for Sparse Point Clouds

The paper adapts CNN semantic-segmentation ideas to sparse lidar data, using global scene structure and intensity alongside projected two-dimensional representations for efficient weather filtering.

B. Autolabeling for Noise Caused by Rain or Fog

The paper derives point-wise rain and fog labels by comparing adverse-weather lidar distances with accumulated reference measurements. This raw-data method is fast and avoids requiring 3D information.

  • Manual weather segmentation labels are difficult, so time-synchronized camera images support labeling decisions for sparse lidar point clouds.The task is deciding whether each point was caused by a water droplet.Source: B. Autolabeling for Noise Caused by… · p. 3
  • Reference lidar images from multiple frames are stacked into a ground-truth distance representation for comparison with rain or fog recordings.Accumulation over reference frames accounts for minor sensor measurement inaccuracies.Source: B. Autolabeling for Noise Caused by… · p. 3
  • A point is labeled clutter when its adverse-weather distance differs sufficiently from all corresponding ground-truth distances.The comparison uses a search threshold of ΔR = ±35cm to reduce false negatives.Source: B. Autolabeling for Noise Caused by… · p. 3
  • Comparing matching transmitter–receiver distances is fast, uses raw sensor data directly, and does not require 3D information.A kd-tree point-cloud comparison produced no significantly different results.Source: B. Autolabeling for Noise Caused by… · p. 3
  • Applying the labeling procedure between split reference recordings quantifies its false-label rate under unchanged weather conditions.The evaluation expects all points to be valid because the reference recording contains no weather changes.Source: B. Autolabeling for Noise Caused by… · p. 3

C. Data Augmentation

The augmentation pipeline extends favorable-weather lidar data with modeled fog and rain effects, including weather-induced scattering and altered sensing attributes. Fog parameters cover multiple visibility ranges, while rain augmentation uses calibrated scatter probabilities.

  • The augmentation approach adapts a fog model to generate rain effects, enabling large training data without manual annotation and with error-free ground truth.It is applied to lidar images so individual point attributes can be manipulated.Source: C. Data Augmentation · p. 3
  • The augmentation changes more than point presence: adverse weather also reduces viewing range and lowers intensity and echo-pulse-width contrast.Source: C. Data Augmentation · p. 4

1) Fog Model:

  • Fog augmentation derives maximum sensing range from visual range, atmospheric extinction coefficient β, and detector threshold CT.Because lidar is active, its maximum sensor range is half the maximum viewing distance.Source: 1) Fog Model · p. 4
  • Fog scattering-point intensities follow a logarithmic normal distribution whose parameters are estimated from clutter intensities in climate-chamber data.The model covers meteorological visibility of 10−100m and augmented visibility of 30−3000m.Source: 1) Fog Model · p. 4
  • Rain augmentation sets β = 0.01 and uses point scatter rate p to control the per-point probability of random scatter points.The parameters are adapted so augmented scatter points correspond to natural rainfall.Source: 2) Rain Model · p. 4

D. Network Architecture

WeatherNet adapts the LiLaNet sparse-point-cloud CNN for three-class lidar de-noising. The design reduces depth and expands spatial context while adding dropout for generalization.

  • WeatherNet adopts CNN architectures designed for semantic segmentation of sparse lidar point clouds.It is presented as an efficient variant of LiLaNet.Source: D. Network Architecture · p. 4
  • The network depth is reduced because de-noising uses three classes rather than the thirteen classes of full multi-class semantic segmentation.Source: D. Network Architecture · p. 4
  • A dilated convolution enlarges the receptive field to provide more information about spatial vicinity.The modified inception layer incorporates this convolution.Source: D. Network Architecture · p. 4
  • A dropout layer is inserted to increase the network’s generalization capability.Training uses Adam with learning-rate decay after every epoch.Source: D. Network Architecture · p. 4

A. Road Data Set

The study combines controlled climate-chamber recordings with augmented road data to build reproducible weather-segmentation benchmarks. Reference conditions enable point-wise labels for clear, rain, and fog, while analysis links clutter to weather degradation.

A. Road Data Set

B. Climate Chamber Scenarios and Ground Truth Labels

  • Reference recordings enable automatic point-wise labels for clear points, rain clutter, and fog clutter without manual weather annotation.Source: B. Climate Chamber Scenarios and… · p. 5
  • Lidar clutter counts correlate with visibility or rainfall rate, while valid and weather-induced points together preserve the reference point count.Source: B. Climate Chamber Scenarios and… · p. 5
  • Rainfall rate cannot be estimated directly from scatter-point increases, but lidar sensor degradation can be estimated from the weather-induced measurements.This degradation information can support autonomous-vehicle adaptation to environmental conditions and sensor performance.Source: B. Climate Chamber Scenarios and… · p. 5

A. Road Data Set

  • The dataset contains about 175,941 training, validation, and test samples split approximately 60%−15%−25%.It includes 72,800 chamber samples and 103,141 road samples.Source: C. Data Split · p. 5

V. EXPERIMENTS

Experiments evaluate WeatherNet using controlled-weather and road data, with IoU-based segmentation results, confusion matrices, and comparisons against geometric and CNN baselines. Adding road data and weather augmentation improves overall and per-class performance, while WeatherNet is efficient and outperforms DROR by an order of magnitude.

Experimental Setup

  • Evaluations use chamber-only, chamber-plus-road, and chamber-plus-road-with-augmentation training settings, assessed with Intersection-over-Union.All evaluations use the test set from experiment 2.Source: §V · p. 5

Segmentation Performance

  • WeatherNet performance significantly increases when road data and proposed weather augmentation are added, improving overall and per-class IoU.The result indicates weather influences are identified across both domains.Source: §V · pp. 5–6

Comparison with Geometric Filtering

  • The baseline DROR filter shows that local vicinity alone is inadequate for filtering scatter points caused by dense water drops.DROR parameters follow prior work, with horizontal resolution adapted to the Velodyne VLP32C.Source: §V · p. 6
  • An order of magnitude: the CNN approach outperforms DROR on the reported de-noising comparison.The comparison concerns filtering scatter points in adverse weather.Source: §V · p. 6

Comparison with CNN Baselines

  • WeatherNet mostly outperforms RangeNet21, RangeNet53, and LiLaNet, especially in experiment 3, while using fewer trainable parameters and less inference time.These efficiency gains support using the network as a preprocessing step.Source: §V · p. 6

Error Analysis

  • Rain and fog are mostly confused because both consist of water droplets, differing in distribution, density, and size; this confusion does not hinder point-cloud filtering.The distinction matters more for weather-condition classification than for separating clutter from valid points.Source: §V · p. 6

A. Qualitative Results on Dynamic Chamber Data

Dynamic chamber scenes test de-noising in dense fog at 30 m visibility, where WeatherNet removes fog clutter while retaining small dynamic objects. Compared with DROR and RangeNet53, it better preserves or distinguishes the cyclist and pedestrian in the qualitative examples.

A. Qualitative Results on Dynamic Chamber Data

  • At 30 m visibility in dense fog, the evaluated scene contains a cyclist and pedestrian that are barely recognizable before filtering.The objects are highlighted with black boxes in the figure.Source: Figure 7 · p. 7
  • WeatherNet retains the pedestrian and cyclist while discarding fog clutter, whereas DROR cannot distinguish rain from fog in this example.DROR colors all filtered points as rain points because it lacks a rain–fog distinction.Source: Figure 7 · p. 7
  • WeatherNet distinguishes the pedestrian and cyclist from scatter points, while RangeNet53 filters most parts of both objects despite comparable evaluated performance.RangeNet53 preserves fine structures and edges but removes substantial portions of the small objects.Source: A. Qualitative Results on Dynamic… · p. 7
  • Spatial filtering fails when fog points are distributed as densely as real-object points, although it can filter some isolated scattering points.DROR also removes distant single points that are not weather-caused.Source: A. Qualitative Results on Dynamic… · p. 7
  • Fog scatter-point counts correlate with meteorological visibility, enabling visibility estimation from weather segmentation and degradation estimation from the scatter-to-valid-point ratio.These estimates use lidar point-cloud information rather than direct rainfall-rate estimation.Source: A. Qualitative Results on Dynamic… · p. 7

B. Qualitative Results on Dynamic Road Data

Road experiments show that the approach works on lidar point clouds corrupted by natural rainfall, including dynamic scenes with pedestrians and passing cars. The conclusion frames CNN weather segmentation as an efficient preprocessing step for distinguishing weather scatter from valid object points.

B. Qualitative Results on Dynamic Road Data

  • The approach operates on point clouds corrupted by natural rainfall recorded on roads.This extends evaluation beyond controlled chamber weather.Source: B. Qualitative Results on Dynamic Road… · p. 7
  • In a light-rain road scene, the method handles a crossing pedestrian and multiple passing cars despite training based on chamber data complemented by augmented real-world data.The authors describe this as generalization to a different real-world scenario.Source: B. Qualitative Results on Dynamic Road… · p. 7

VI. CONCLUSION

  • The CNN approach is presented as an essential preprocessing step that distinguishes adverse-weather scatter points from valid points belonging to solid objects.The conclusion contrasts its global traffic-scene understanding with local-spatial-statistics filters.Source: §VI · p. 7
  • Quantitative and qualitative results demonstrate superior performance over state-of-the-art while maintaining high efficiency.The paper points readers to additional dynamic-scene results in the dataset and supplementary video.Source: §VI · p. 7

Related papers

Loading 1912.03874v2…