Source-linked AI summary
A Benchmark for Lidar Sensors in Fog: Is Detection Breaking Down?
Mario Bijelic, Tobias Gruber, Werner Ritter
TL;DR
Level-five autonomous driving requires perception in adverse weather, but fog degrades lidar-based environment sensing. This paper benchmarks four state-of-the-art lidar systems in a controlled fog chamber, characterizes disturbance patterns, and tests internal-parameter tuning. All four scanners break down in fog, while parameter adjustments and processing techniques provide only limited gains.
Problem
Adverse weather degrades environmental perception, while most state-of-the-art detection algorithms rely on undisturbed lidar data.
Method
The paper evaluates four state-of-the-art lidar scanners in controlled fog conditions and investigates disturbance patterns and internal-parameter tuning.
Results
All four scanners break down in fog; below 40 m meteorological visibility, perception is limited to less than 25 m.
Takeaways & Limitations
Multiple echoes and adaptive noise levels improve performance by a few meters, but dense fog remains far from reliably perceivable.
Takeaways & Limitations
The authors assume that measured points prevent further laser-power increases, potentially constraining automatic adaptation.
Abstract
from arXiv · showhide
Autonomous driving at level five does not only means self-driving in the sunshine. Adverse weather is especially critical because fog, rain, and snow degrade the perception of the environment. In this work, current state of the art light detection and ranging (lidar) sensors are tested in controlled conditions in a fog chamber. We present current problems and disturbance patterns for four different state of the art lidar systems. Moreover, we investigate how tuning internal parameters can improve their performance in bad weather situations. This is of great importance because most state of the art detection algorithms are based on undisturbed lidar data.
I. INTRODUCTION
Level-five autonomous driving requires perception in adverse weather, motivating lidar evaluation because current lidar technologies show major problems in fog. The paper presents a reproducible benchmark for comparing lidar systems under such conditions.
- Adverse weather such as fog, haze, mist, and rain challenges the continuous environment perception required for level-five autonomous driving.
- Lidar currently has an advantage over gated and time-of-flight cameras among proposed automotive sensor technologies.
- The DENSE project introduces an easy, reproducible benchmark for state-of-the-art lidar sensors in adverse weather.
- The benchmark evaluates disturbance patterns and establishes a baseline for subsequent sensor development.
B. Related Work
Prior work established experimental characterization and benchmarking of laser range sensors, including laboratory and real-environment evaluations. However, earlier adverse-weather studies varied in environmental control and coverage.
- Early experiments quantified range and angular precision and verified lidar measurement models.
- Subsequent benchmarks evaluated multiple commercial range sensors and extended testing from laboratory settings to underground environments.
2) Influence of bad weather on lidar systems:
The paper compares four automotive lidar scanners and examines how their designs affect scan quality and adverse-weather behavior. The systems differ substantially in field of view, resolution, return handling, and hardware configuration despite sharing 905 nm wavelength and mechanical beam steering.
- Four scanners—Velodyne HDL-64S2, HDL-64S3, Ibeo LUX, and Ibeo LUX HD—are compared in the test.
- Longer-exposure scans show increased uniformity for the S3D compared with the S2.
- The Velodyne S3 generation includes dual-return S3D operation, which records strongest and last echoes while reducing horizontal resolution by approximately 30%.
- The Ibeo LUX HD modifies the LUX with a lens and tinted covering for more robust close-distance detection in dusty environments, while lowering maximum viewing distance.
- Velodyne scanners provide 360° FOV and substantially higher resolution, whereas Ibeo scanners provide 85° FOV.
A. Intensity calculation
The paper describes raw-intensity calibration for Velodyne scanners and the proprietary role of intensity measurement in robust perception. It also notes that scattering media worsen beam divergence and may require visibility-aware correction.
- Intensity calibration ranges from raw level 0 measurements to level 3 values intended to be correct and identical across systems.
- Only the Velodyne S2 and S3D provide raw level 0 intensity measurements for analysis under fog.
- The scanners can automatically select among eight laser-power levels when clear object reflections are not obtained.
- Factory calibration corrects raw intensity IR to corrected intensity IC using measured distance and per-laser beam-divergence parameters.
- Beam divergence becomes worse in scattering media, motivating more sophisticated correction methods when visibility is known.
III. EXPERIMENTS
The experiments compare four lidar sensors in controlled radiation and advection fog, using a static scene and varying fog density to assess performance and tunable parameters.
- III. EXPERIMENTS: The CEREMA climate chamber produces stable radiation and advection fog with continuously controlled meteorological visibility V.Fog density is monitored through atmospheric transmission and expressed as visibility.
- III. EXPERIMENTS: Four sensors—Velodyne HDL64-S2, HDL64-S3D, Ibeo LUX, and Ibeo LUX HD—are mounted on a test vehicle in the chamber.Velodyne sensors are mounted on top of the vehicle, while Ibeo sensors use bumper-like mounting.
- III. EXPERIMENTS: A static scene containing vehicles, mannequins, signs, road markings, and reflective targets is recorded across fog densities and droplet distributions.The setup supports evaluation of fog effects on lidar performance in a fixed environment.
A. Qualitative Evaluation
Qualitative bird’s-eye views show that fog heavily degrades lidar scans through near-sensor clutter and attenuation, while sensor designs differ in clutter and detection distance.
- A. Qualitative Evaluation: Fog heavily impairs lidar performance by reflecting signals from droplets near the vehicle and creating a clutter cloud around the sensor.Filtering these clutter points drastically reduces effective resolution.
- A. Qualitative Evaluation: Attenuation limits maximum viewing distance because reflected signals disappear into noise when illumination power is limited.The effect is visible in the Velodyne scans and contributes to reduced perception range.
- A. Qualitative Evaluation: The Velodyne HDL64-S3D delivers better bird’s-eye-view results than the older HDL64-S2.The passage reports this qualitative comparison without specifying a numerical improvement.
- A. Qualitative Evaluation: The Ibeo LUX HD contains less clutter than the Ibeo LUX, while both detect all experiment targets from V = 36 m.At those distances, the Velodyne sensors still have difficulties with larger distances.
- A. Qualitative Evaluation: The Ibeo sensors show no visible difference between advection and radiation fog in the reported bird’s-eye views.The comparison concerns the qualitative scans described for the experiment setup.
C. Strongest Return vs. Last Return
The strongest and last returns behave differently in fog: increasing fog reduces detected points, while the last return retains greater overlap with the clear-weather reference.
- C. Strongest Return vs. Last Return: Increasing fog density decreases the number of points in depth bins averaged over 100 point clouds.The static point cloud is binned into 122 slices of 0.25 m for comparison across echo types.
- C. Strongest Return vs. Last Return: The last return shows greater overlap with the clear-weather reference point cloud than the strongest return as fog density increases.In clear weather, both return modes produce the same point cloud used as ground truth.
- C. Strongest Return vs. Last Return: The higher point count relative to ground truth is attributed to clutter received at very close distance.This clutter contributes points that do not correspond to the clear-weather reference cloud.
- C. Strongest Return vs. Last Return: Strongest and last returns differ in modality, with strongest return generally offering a more stable and accurate measure.The passage also notes that more received pulses in adverse weather can increase the probability of obtaining actual object responses.
E. Intensity Evaluation
Intensity corresponds to measured signal peak amplitude and decreases with attenuation, but ring-to-ring inhomogeneity prevents a confident exponential scattering fit; low-reflectivity targets are weaker.
- E. Intensity Evaluation: Intensity values correspond directly to measured signal peak amplitude and determine when a signal disappears.The evaluation plots raw intensities for 5% and 90% reflective targets against distance.
- E. Intensity Evaluation: A confident exponential scattering-model fit is not possible because intensity values jump when one laser ring disappears and another replaces it.The inhomogeneity between HDL64-S3D laser rings is visible in the scanner measurements.
- E. Intensity Evaluation: Fig. 6 uses red, blue, green, and yellow to encode Ibeo scanning layers 1, 2, 3, and 4, respectively.The bird’s-eye views cover different fog types and fog densities.
- E. Intensity Evaluation: ≈20 % lower intensity is observed for 5 % reflective targets than for 90 % targets on average.The target disappears before reaching the minimum noise level.
V. CONCLUSION AND OUTLOOK
All four tested state-of-the-art laser scanners lose performance in fog, while manual parameter tuning recovers only a few meters and does not provide reliable perception in dense fog. The benchmark identifies parameter trade-offs and motivates continued testing and sensor development.
- All four state-of-the-art laser scanners break down in fog, with most sensors showing similar performance under strict class-1 NIR eye-safety constraints.
- Manual parameter adjustment increases performance, including longer HDL64-S3D range at higher laser output power.
- Below 40 m meteorological visibility, perception is limited to less than 25 m.
- Adaptive noise levels and multiple echoes improve performance by a few meters, but reliable perception remains unresolved in dense fog.
- The benchmark establishes a baseline for future sensor development, while further adverse-weather testing is needed for upcoming systems and lidar-based fusion methods.