PaperlayerView exact revision on arXiv

Source-linked AI summary

Weather Influence and Classification with Automotive Lidar Sensors

Robin Heinzler, Philipp Schindler, Jürgen Seekircher, Werner Ritter, Wilhelm Stork

arXiv:1906.07675v1cs.CV

TL;DR

Autonomous-driving perception must account for adverse weather because rain and fog can affect lidar performance. The paper analyzes automotive lidar point clouds using controlled and road datasets, and develops lidar-only weather classification. In controlled environments, the VLP16 classifier achieves a mean IoU of 97.14%.

  • Problem

    The paper addresses limited consideration of weather effects in autonomous-driving perception by studying how rain and dense fog influence lidar performance.

  • Method

    The paper records controlled and road datasets with two automotive lidar sensors, analyzes point-cloud and object-perception changes, and classifies weather from lidar output alone.

  • Results

    97.14% mean IoU is achieved for the VLP16 weather classifier with an SVM in controlled environments.

  • Takeaways & Limitations

    Weather classification can provide sensor-specific information for adapting fusion confidence values and system behavior to current perception performance.

Abstract

from arXiv · show

Lidar sensors are often used in mobile robots and autonomous vehicles to complement camera, radar and ultrasonic sensors for environment perception. Typically, perception algorithms are trained to only detect moving and static objects as well as ground estimation, but intentionally ignore weather effects to reduce false detections. In this work, we present an in-depth analysis of automotive lidar performance under harsh weather conditions, i.e. heavy rain and dense fog. An extensive data set has been recorded for various fog and rain conditions, which is the basis for the conducted in-depth analysis of the point cloud under changing environmental conditions. In addition, we introduce a novel approach to detect and classify rain or fog with lidar sensors only and achieve an mean union over intersection of 97.14 % for a data set in controlled environments. The analysis of weather influences on the performance of lidar sensors and the weather detection are important steps towards improving safety levels for autonomous driving in adverse weather conditions by providing reliable information to adapt vehicle behavior.

I. INTRODUCTION

Autonomous-driving systems must account for changing environmental conditions because perception constrains system availability and performance. This paper investigates rain and dense fog as lidar-specific environmental influences.

  • I. INTRODUCTION: Environmental conditions must be recognized and handled to improve autonomous-driving system availability and performance.The introduction links environmental awareness to detecting system boundaries and appropriate system responses.
  • I. INTRODUCTION: Robust perception and sensor fusion require understanding how environmental conditions degrade different sensor types.This supports recognizing system boundaries and adapting operation under environmental impairments.
  • I. INTRODUCTION: The paper investigates how rain and dense fog influence state-of-the-art lidar sensors.The study evaluates recorded data under changing environmental conditions and includes weather classification from lidar output.

II. RELATED WORK

Prior research has examined harsh-weather effects on lidar, including fog, rain, dust, and snow, while many established datasets were recorded in favorable weather.

  • II. RELATED WORK: Many state-of-the-art datasets are recorded under favorable weather conditions, whereas prior literature examines fog, rain, dust, and snow effects on lidar.The related-work discussion frames harsh-weather analysis as an established but distinct research area.

A. Lidar Sensors in Adverse Weather Conditions

Earlier studies report weather-dependent lidar degradation, investigate physical attenuation and reflections, and propose filtering or wavelength-based analyses. These works motivate quantifying weather effects for sensor fusion and trajectory planning.

  • A. Lidar Sensors in Adverse Weather Conditions: Multisensor experiments found significant lidar attenuation in challenging environments, with objects potentially disappearing behind airborne dust.A filtering method used radar data to remove dust reflections from laser measurements.
  • A. Lidar Sensors in Adverse Weather Conditions: Theoretical fog-and-rain models represent layered reflection, transmission, and absorption, but experimental setups were limited in length and target distance.The cited simulator had a maximum length of 4 m and a 10 m sensor-to-target distance.
  • A. Lidar Sensors in Adverse Weather Conditions: Rain studies report smaller changes in detected distance but dramatic decreases in lidar intensity and point counts.These results were obtained for one lidar sensor observing a static scene.
  • A. Lidar Sensors in Adverse Weather Conditions: Prior dust studies describe systematic, predictable lidar effects, while one chamber study lacked artificial fog and therefore did not investigate fog influence.The dust analysis examined a 2D laser scanner, and the cited chamber could not generate artificial fog.
  • A. Lidar Sensors in Adverse Weather Conditions: Wavelength comparisons disagree across cited analyses: 905 nm outperformed 1550 nm under modeled attenuation, while another study found 1550 nm superior in adverse weather.The latter result was attributed to fewer emitted-light restrictions under laser class 1.
  • A. Lidar Sensors in Adverse Weather Conditions: A fog-reflection removal method used beam penetration, intensity, and geometric features, but its single static scene could allow classifier overfitting.The reported scenes had visibility ranges of 2 and 6 m.
  • A. Lidar Sensors in Adverse Weather Conditions: Recognizing and quantifying weather-dependent lidar degradation is fundamental for robust perception, sensor fusion, and modality weighting.The paper connects this evaluation to fusion and trajectory planning for autonomous cars.

B. Main Contribution

The paper contributes a realistic, controllable dataset for analyzing rain and fog effects across lidar scenarios, plus lidar-only weather classification. It separates sensor inputs to preserve redundancy and support weather-aware system adaptation.

  • B. Main Contribution: The paper’s first contribution is a detailed analysis of rain and fog effects on different lidar sensors using a realistic, controllable, large dataset.The dataset is described as covering varied scenarios while emphasizing realism, controllability, and size.
  • B. Main Contribution: The second contribution is lidar-only weather detection with controlled fog visibility from 20–60 m and stabilized rainfall of 55 mm/h.The dataset includes dynamic traffic scenarios and road recordings across sunny, cloudy, and rainy day- and nighttime conditions.
  • B. Main Contribution: Weather classification evaluates lidar point-cloud output to identify adverse environmental conditions rather than classifying individual disturbance points.The approach focuses on weather-related sensor-performance degradation, especially reduced range.
  • B. Main Contribution: Dynamic chamber scenes include stationary lane, reflector, mannequin, and movable pedestrian, cyclist, and car objects.The reference condition is clear, without rain or fog.
  • B. Main Contribution: Road recordings cover varied environmental conditions, traffic situations, road types, and both daytime and nighttime operation.The road dataset includes no-rain, occasional-rain, and almost-permanent-rain scenarios.
  • B. Main Contribution: Sensor-specific weather information is intended to adjust confidence values independently for different sensor types in fusion algorithms.The paper excludes filtering one sensor’s raw input using another sensor to preserve redundancy and avoid cross-dependencies.

III. EXPERIMENTAL SETUP

The study uses controlled chamber recordings and road data to evaluate lidar under clear, foggy, and rainy conditions. Two automotive lidar sensors with different scanning and return-measurement designs provide the sensor data.

  • Controlled weather environment: Defined fog and rain conditions are produced in a climate chamber, including homogeneous fog with controlled visibility and stabilized rainfall.The chamber is described as publicly accessible and supports controlled visual range and rainfall rate.
  • Chamber scenarios: The experimental data include static and dynamic chamber setups designed to reduce time correlation and classifier overfitting.Setup A uses objects with 5%, 50%, and 90% reflectivity.
  • Road scenarios: Road recordings cover dry, occasional-rain, and nearly permanent-rain scenarios across daytime, nighttime, traffic, and road types.The road set includes highways, rural roads, and inner-city roads.
  • Lidar sensors: The sensor setup contains Velodyne VLP16 and Valeo Scala lidar sensors operating at about 905 nm.The sensors differ in scanning mechanics and measured return attributes: intensity for VLP16 versus echo pulse width for Scala.
  • Recorded data: The VLP16 road recording contains 270,000 frames.

IV. METHOD

The method represents each lidar frame as a structured point cloud, extracts spatial, return, and signal statistics, and classifies weather using selected frame-level features. Analysis focuses on a near-range ego-lane region and distinguishes returns by echo number.

  • Point-cloud representation: Each point cloud is represented as a matrix whose rows are points and whose columns are sensor-specific attributes.Attributes include Cartesian and spherical coordinates, echo number, intensity, and echo pulse width.
  • Region of interest: The analysis is spatially restricted to x ≤20m and −1.5m ≤y ≤+1.5m to focus on near-range weather effects in the ego-lane.The region of interest also reduces scenario dependence and computation time.
  • Echo-based features: Echo number distinguishes first, second, and third returns because atmospheric particles can create additional scattering returns.The return groups are represented by M_t, and the number of points for each echo is captured by N_t(k).
  • Feature extraction: Frame-level features include attribute means and variances, mean return distance, and covariance-matrix eigenvalues describing spatial point distribution.The spatial distribution is summarized using eigenvalues for x, y, and z.
  • Visualization: Figure 4 visualizes a representative Scala point cloud by echo number, marking the pedestrian with red boxes and the car with blue stars.Weather conditions are labeled using visibility, rainfall rate, or clear reference conditions.
  • Feature selection and classification: A 16-dimensional feature vector is down-selected using neighboring component analysis, while total point count is excluded because it depends strongly on scenario.The VLP16 feature set uses intensity instead of Scala’s echo pulse width.

V. EXPERIMENTAL RESULTS

The results section evaluates point-cloud and object-perception changes across clear, foggy, and rainy conditions using meteorological visibility and rainfall measurements as ground truth. Boxplots summarize object-related raw point-cloud attributes across recorded frames.

  • Ground truth: The evaluation uses meteorological visibility V in meters and rainfall rate R in mm/h as chamber ground-truth measurements.For road recordings, rain intensity is obtained from a rain-light sensor used to control windshield-wiper speed.
  • Object perception evaluation: Figure 5 summarizes Scala object perception for a car and pedestrian using x-coordinate, y-coordinate, echo, and echo pulse width.The weather conditions are ordered by descending meteorological visibility on the ordinate axes.
  • Object perception evaluation: The boxplot analysis includes 1,200 frames per weather condition, except for rain with 921 frames.

A. Weather Influence on Point Clouds and Object Perception

Adverse weather reduces lidar point returns, detection range, and object point density, with dense fog producing especially severe perception degradation. Multiecho returns and object-density measurements expose these effects across cars and pedestrians.

  • Weather Influence on Point Clouds and Object Perception: 55 mm/h rain produces fewer points at the climate-chamber endpoint, indicating reduced lidar detection range.A car at approximately 19 m remains detected in all tested scenarios, but fog and rain shift its dominant detection to the second echo.
  • Weather Influence on Point Clouds and Object Perception: Dense fog causes primary echoes to cluster below 5 m, severely limiting environment perception and sensor range.At 20–40 m visibility, nearly all primary echoes are attributed to fog; highly reflective tail-light targets can still generate secondary or tertiary echoes.
  • Weather Influence on Point Clouds and Object Perception: Multiecho sensing returns weaker fog and rain reflections while maintaining reasonable object-detection performance compared with single-return sensors.This provides additional weather-sensitive returns without eliminating useful object detections.
  • Weather Influence on Point Clouds and Object Perception: Rain and clear conditions mainly produce first echoes, whereas fog increases second and third returns and can shift object detections toward secondary echoes.The car at approximately 19 m is detected by both sensors in every tested scenario, but fog and rain make the second echo dominant.
  • Weather Influence on Point Clouds and Object Perception: 0.36 median car density for ’VLP16’ and 0.04 for ’Scala’ occur at 20−30 m visibility, while pedestrian density reaches zero below 40 m visibility.The density is normalized against clear-weather object returns, revealing stronger degradation for non-retro-reflective objects in dense fog.

B. Weather Influence on Feature Vector

Rain and fog alter lidar feature statistics in static and dynamic scenes, affecting return counts, distances, intensities, epw values, and covariance eigenvalues. These weather-dependent signals support distinguishing environmental conditions, although dynamic scenes increase variance and outliers.

  • Weather Influence on Feature Vector: Return-count features distinguish weather conditions, with fog and rain increasing second and third returns through multiple reflections.The variance of the second echo differs significantly among foggy, rainy, and clear conditions, while dense fog and rain show no distinct difference in N3.
  • Weather Influence on Feature Vector: Mean echo distance decreases for the first return in fog, while rain produces the greatest distance variance and fog or rain increase r2,3 statistics.These distinct mean and variance patterns make echo-specific distance features useful for estimating weather presence.
  • Weather Influence on Feature Vector: Fog increases Scala epw approximately inversely with fog density, whereas rain reflections produce smaller epw values because raindrops are less dispersed.The measured intensity also changes with weather, making epw a weather-sensitive signal.
  • Weather Influence on Feature Vector: Rain and fog influence eigenvalues of the covariance matrix for x but show no derived dependency for eig(cov(y)).The paper attributes the absence of a y-axis dependency potentially to the setup’s symmetry and does not evaluate eig(z) because of the small vertical field of view.
  • Weather Influence on Feature Vector: Dynamic scenes increase variance and outlier counts while reducing differences between feature means compared with static scenes.The increase is especially pronounced for intensity-related signals.

C. Weather Classification by Means of Lidar Sensors

The paper classifies clear, fog, and rain conditions from lidar point-cloud features using kNN and SVM classifiers. In controlled environments, ’VLP16’ classification is highly effective, while road data and the ’Scala’ sensor perform worse.

  • Weather Classification by Means of Lidar Sensors: The classifiers predict one of three responses: ’clear’, ’fog’, or ’rain’, without distinguishing fog visibility ranges.Fog visibility ranges are omitted because the feature differences were considered small under those conditions.
  • Weather Classification by Means of Lidar Sensors: 97.14 % mean IoU is achieved by the SVM classifier for ’VLP16’ in controlled environments.The corresponding kNN result is 96.40 % mean IoU.
  • Weather Classification by Means of Lidar Sensors: 96.40 % mean IoU is achieved by kNN for ’VLP16’ in controlled environments, compared with 58.89 % for ’Scala’.The lower ’Scala’ result could be caused by its smaller vertical field of view and consequently fewer points per frame.
  • Weather Classification by Means of Lidar Sensors: Training uses setups A and B while setup C is reserved for testing, reducing time correlation in the climate-chamber data.For road data, separate recordings are assigned to training and testing across clear, occasional-rain, and permanent-rain conditions.
  • Weather Classification by Means of Lidar Sensors: 86.88 % mean IoU is achieved by kNN for ’VLP16’ on road data.The rain-class IoU is 77.04%, versus 96.72% for ’clear’; the paper relates the rain decrease to rainfall-rate variety and less accurate ground truth.

VI. CONCLUSION

The paper analyzes how fog and rain influence lidar sensors and introduces point-cloud weather classification for controlled and uncontrolled environments. It reports impaired object perception in adverse weather, satisfactory classification results for most classes, and several directions for extending the approach.

  • VI. CONCLUSION: The study combines an in-depth analysis of fog and rain effects on lidar sensors with a novel point-cloud approach for classifying weather status.The approach is evaluated in both controlled and uncontrolled environments.
  • VI. CONCLUSION: Adverse weather significantly reduces points per object and decreases the variance of measured intensity or epw values.The paper therefore expects reduced detection range and impaired object perception under fog and rain.
  • VI. CONCLUSION: Reduced intensity contrast is expected to increase misclassifications and wrong detections, making weather recognition especially important for low-reflective objects.This conclusion concerns the additional perception degradation caused by adverse weather.
  • VI. CONCLUSION: Compared with prior work, the data set covers a larger meteorological visibility range, a larger spatial measuring range, and controlled and uncontrolled rain recordings.The setup also used closed-loop stabilization and more extensive data.
  • VI. CONCLUSION: Future extensions include advanced classifiers, temporal accumulation, finer class divisions, and dynamically selected regions of interest.Dynamic ROI selection could use map data, while the ROI design also reduces processing time for high-resolution lidar sensors.
Loading 1906.07675v1…