Source-linked AI summary
Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather
Martin Hahner, Christos Sakaridis, Dengxin Dai, Luc Van Gool
TL;DR
Fog degrades LiDAR with attenuation and spurious returns, while collecting annotated foggy data is expensive and existing datasets are limited. The paper physically transforms clear-weather point clouds into foggy data, and reports improved detection robustness on real dense-fog data across several state-of-the-art methods.
Problem
Fog creates attenuated and spurious LiDAR returns, while large-scale annotated foggy datasets remain scarce because collection and annotation are costly.
Method
The paper uses a physically based LiDAR transmission model to transform clear-weather points into foggy counterparts by modifying their measured range and intensity.
Results
The authors report improved performance of several state-of-the-art 3D object detection methods on real-world dense-fog point clouds.
Takeaways & Limitations
The simulation provides physically controllable fog generation applicable to real clear-weather LiDAR datasets and supports training for robust perception in real fog.
Takeaways & Limitations
The simulation assumes a homogeneous optical medium, and the expected benefit of 360° annotations cannot be tested because no such public dataset is available.
Abstract
from arXiv · showhide
This work addresses the challenging task of LiDAR-based 3D object detection in foggy weather. Collecting and annotating data in such a scenario is very time, labor and cost intensive. In this paper, we tackle this problem by simulating physically accurate fog into clear-weather scenes, so that the abundant existing real datasets captured in clear weather can be repurposed for our task. Our contributions are twofold: 1) We develop a physically valid fog simulation method that is applicable to any LiDAR dataset. This unleashes the acquisition of large-scale foggy training data at no extra cost. These partially synthetic data can be used to improve the robustness of several perception methods, such as 3D object detection and tracking or simultaneous localization and mapping, on real foggy data. 2) Through extensive experiments with several state-of-the-art detection approaches, we show that our fog simulation can be leveraged to significantly improve the performance for 3D object detection in the presence of fog. Thus, we are the first to provide strong 3D object detection baselines on the Seeing Through Fog dataset. Our code is available at www.trace.ethz.ch/lidar_fog_simulation.
1. Introduction
Fog attenuates LiDAR signals and creates spurious returns, challenging robust outdoor perception. Because real adverse-weather data are scarce and expensive to collect, the paper converts clear-weather LiDAR into foggy counterparts using a physically based transformation.
- Motivation: Fog attenuates returns from solid objects and creates spurious peaks at incorrect ranges, producing noisy LiDAR point clouds.These effects make robust performance under changing weather difficult for outdoor applications.
- Motivation: Existing LiDAR datasets rarely support adverse-weather evaluation, and only CADC and STF specifically address such conditions.Consequently, large quantities of foggy LiDAR data for training remain unavailable.
- Motivation: Collecting and annotating large-scale adverse-weather datasets is time-, labor-, and cost-intensive.
- Approach: The method transforms each clear-weather point’s range and intensity using a LiDAR transmission model and a formal clear-to-fog response connection.The resulting partially synthetic data can train state-of-the-art 3D object detection pipelines for improved robustness on real foggy data.
- Observed fog effects: Fog noise depends on scene geometry: open space beyond a guardrail produces many spurious returns, whereas a nearby solid target suppresses them.The STF example uses a Velodyne HDL-64E capable of strongest and last returns.
2. Related Work
Prior work studied weather effects on LiDAR and developed image-based fog simulation, but LiDAR fog simulation for training remained comparatively limited. The section also reviews major LiDAR-based 3D object detection architectures.
- Weather and LiDAR simulation: Earlier studies modeled fog, haze, and other adverse-weather effects on automotive LiDAR, while LIBRE evaluated multiple LiDAR sensors in weather conditions.
- Weather and LiDAR simulation: Automotive fog simulation was largely image-based, producing foggy versions of Cityscapes and Synscapes from depth information.ACDC additionally provides pixel-level annotations for adverse conditions.
- Weather and LiDAR simulation: Rain research modeled LiDAR degradation for ADAS evaluation, while other methods rendered rain on images to improve or evaluate robustness.
- 3D object detection: PointNet directly processes point clouds, whereas VoxelNet quantizes 3D space into voxels and applies PointNet-like layers within them.VoxelNet uses computationally intensive 3D convolutions and is relatively heavy.
- 3D object detection: PointPillars processes points in a 3D pillar grid, enabling faster 2D convolutions; PointPainting adds image-derived pseudo-class labels to points.
- 3D object detection: PointRCNN proposes and refines 3D boxes in two stages, Part-A2 uses object-part locations, and PV-RCNN variants combine coarse voxels with raw points.
3. Fog Simulation on Real LiDAR Point Clouds
The method models LiDAR returns through a linear optical-system formulation, then relates clear-weather hard-target responses to fog responses containing attenuation and distributed backscattering. It uses these relations to transform real clear-weather point measurements into physically based fog counterparts.
- Optical system model: The received LiDAR power is modeled as the convolution of transmitted pulse power with the environment’s impulse response.The environmental response is decomposed into optical-channel and target impulse responses.
- Optical system model: The optical-channel response combines inverse-square range loss, transmission loss, and a crossover function for the bistatic transmitter–receiver geometry.The crossover function is approximated piecewise linearly because commercial sensors’ detailed optical configurations are generally unavailable.
- Optical system model: The simulation assumes a homogeneous medium, yielding the transmission loss T(R) = exp(-αR), where α is the constant attenuation coefficient.The attenuation coefficient increases as visibility range decreases, so channel response varies with visibility.
- Fog response: Under fog, the same hard target is supplemented by a soft fog target modeled as distributed backscattering, so the target response is the superposition of both contributions.The resulting fog response includes an attenuated hard-target term and a soft-target term whose magnitude depends on the hard-target distance.
- Clear-weather response: In clear weather, the target is represented as a hard target with a Dirac-delta impulse response at range R0 and differential reflectivity β0.The transmitted automotive LiDAR pulse is modeled with a sin^2 time signature, using peak power P0 and halfpower width τH.
- Fog response: The soft fog contribution can exceed the hard-target contribution, causing the measured range to equal the soft-target response’s maximum.This effect explains why fog can shift the measured LiDAR range away from the solid object.
4. Results
The physically grounded fog simulation produces more realistic fog effects than the prior heuristic simulation and improves detection across real foggy test conditions. Training with simulated fog generally outperforms clear-weather baselines while largely preserving clear-weather performance.
- 4.1. Fog Simulation: Our simulation models the soft-target response P soft R,fog physically, unlike the heuristic response used in [2].In a matched scene, only the proposed simulation reproduces a similar half-circle pattern of fog noise.
- 4.2.1 Quantitative Results: Across classes on the STF dense-fog split, fog-simulation training outperforms the clear-weather baseline and the [2] fog-simulation runs in mAP and for Car.Table 1 reports 3D AP for Car, Cyclist and Pedestrian and 3D mAP over all three classes using 40 recall positions.
- 4.2.1 Quantitative Results: The strongest ∩last filter usually improves the clear-weather model but does not surpass most fog-simulation runs.Cyclist performance is generally lower, which the authors attribute to underrepresentation in STF’s dense-fog test split.
- 4.2.1 Quantitative Results: On Car across dense-fog, light-fog and clear splits, fog-simulation training outperforms other runs in dense fog and achieves state-of-the-art mAP in most cases.Mixing simulated fog into training does not hurt clear-weather performance too much.
- 4.2.2 Qualitative Results: Examples show fewer false positives, more true positives and more accurate predictions after training on mixed clear-weather and simulated-fog data.The comparison uses the same confidence threshold for both PV-RCNN models.
5. Conclusion
The paper introduces a physically accurate method for converting clear-weather LiDAR point clouds into foggy data with controllable parameters. It improves several state-of-the-art 3D detectors on real dense-fog data, while broader gains from 360° annotations remain untested.
- 5. Conclusion: The method converts real-world clear-weather point clouds into foggy point clouds with control over physical simulation parameters.It can simulate different fog densities and the influence of fog on basically any current LiDAR sensor.
- 5. Conclusion: Using the simulation improves several state-of-the-art 3D object detection methods on point clouds collected in real-world dense fog.The conclusion reports this result without restricting it to a single detector.
- 5. Conclusion: The authors believe the simulation may apply beyond 3D object detection to other tasks.This is presented as an expectation rather than an experimentally established result.