Source-linked AI summary
LiDAR Snowfall Simulation for Robust 3D Object Detection
Martin Hahner, Christos Sakaridis, Mario Bijelic, Felix Heide, Fisher Yu, Dengxin Dai, Luc Van Gool
TL;DR
LiDAR-based 3D object detection under snowfall is difficult because adverse-weather data are hard to collect and snow degrades measurements. The paper simulates snowfall and wet ground on clear-weather point clouds, then uses the synthetic data to train detectors; across seven methods, this improves heavy-snow performance by up to 2.1% AP without sacrificing clear-weather performance.
Problem
Snowfall degrades LiDAR measurements through attenuation and spurious returns, while real adverse-weather training data are difficult to collect.
Method
The paper uses physically based modeling of LiDAR pulse transmission, explicitly sampled snow particles, and Fresnel-based wet-ground reflection to augment clear-weather point clouds.
Results
Up to 2.1% AP improvement in heavy snowfall is achieved across seven 3D object detection methods, while clear-weather performance is not sacrificed.
Takeaways & Limitations
Partially synthetic snowfall and wet-ground data can train 3D detectors that are more robust on real snowy STF data while retaining clear-weather performance.
Takeaways & Limitations
The LiDAR model assumes snow is the only scatterer in unoccupied medium regions and relies on assumptions about sensor polarization and calibration.
Abstract
from arXiv · showhide
3D object detection is a central task for applications such as autonomous driving, in which the system needs to localize and classify surrounding traffic agents, even in the presence of adverse weather. In this paper, we address the problem of LiDAR-based 3D object detection under snowfall. Due to the difficulty of collecting and annotating training data in this setting, we propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds. Our method samples snow particles in 2D space for each LiDAR line and uses the induced geometry to modify the measurement for each LiDAR beam accordingly. Moreover, as snowfall often causes wetness on the ground, we also simulate ground wetness on LiDAR point clouds. We use our simulation to generate partially synthetic snowy LiDAR data and leverage these data for training 3D object detection models that are robust to snowfall. We conduct an extensive evaluation using several state-of-the-art 3D object detection methods and show that our simulation consistently yields significant performance gains on the real snowy STF dataset compared to clear-weather baselines and competing simulation approaches, while not sacrificing performance in clear weather. Our code is available at www.github.com/SysCV/LiDAR_snow_sim.
1. Introduction
LiDAR measurements degrade in snowfall because particles attenuate target returns and create spurious echoes, producing noisy point clouds and a domain shift from clear weather. The paper introduces physically based snowfall and wet-ground simulation to train detectors that perform better in snowy conditions.
- Motivation: Snow particles absorb, reflect, or refract laser photons, attenuating target returns and creating spurious echoes at incorrect ranges.These effects severely degrade measurement quality and create a large domain shift from clear-weather point clouds.
- Motivation: Adverse-weather LiDAR data are difficult to collect, motivating simulation methods for reducing the clear-to-adverse-weather data gap.Prior simulation work addressed camera data in fog and rain and LiDAR data in fog and rain.
- Approach: The proposed method simulates snowfall on real clear-weather LiDAR point clouds by sampling snow particles and modeling their effect on LiDAR pulse transmission.Snow particles are represented as opaque spheres whose size is controlled by snowfall rate.
- Approach: The method also models wet ground using Fresnel equations and thin-surface reflection to produce adapted reflectance values for wet surfaces.Snowfall-associated ground wetness increases the specular component of ground reflection.
- Results: Models trained with the generated partially synthetic point clouds achieve significant gains on the real snowy STF dataset over clear-weather baselines and competing simulation methods.The synthetic data are used to optimize state-of-the-art 3D object detectors for greater snowfall robustness.
2. Related Work
Related work spans meteorological modeling, phenomenological analysis, and computer-vision applications for adverse weather. The paper positions its simulation as a physically grounded LiDAR approach that models continuous power-domain effects and wet roads.
- Adverse-weather research: Adverse-weather research covers meteorological foundations, phenomenological disturbance analysis, and computer-vision applications across weather conditions.A general theoretical framework has modeled LiDAR effects from rain, fog, and snow.
- Adverse-weather research: Prior algorithmic responses include robust fusion, adverse-weather simulation for cameras and LiDAR, and enhancement methods that remove weather effects.LiDAR simulation has addressed fog, rain, snow, and combinations of weather conditions.
- Simulation rationale: Adverse-weather simulation reduces data-collection difficulty and enables reproducible conditions with clear ground truth for training and analysis.These benefits motivate synthetic data generation for both image and LiDAR applications.
- Positioning: Compared with a prior general weather simulator, the proposed snowfall model uses continuous power-domain formulation to combine particle reflections and reason about particle-target occlusions.The paper also incorporates wet-road effects, extending the modeled phenomena beyond atmospheric particles.
- 3D object detection: LiDAR-based detectors generally occupy the top positions on many 3D detection dataset leaderboards.The paper evaluates multiple detector families differing in point-cloud representations and detection stages.
3. Snowfall Simulation on LiDAR Point Clouds
The simulation models LiDAR snowfall by combining calibrated pulse propagation with explicit snow-particle echoes, occlusion, attenuation, and cluttered returns. It also augments point clouds with a geometric model of wet-ground reflection.
- Snowfall Simulation: The method models received LiDAR power by combining transmitted-pulse timing with channel and target impulse responses.The channel includes transmission, range attenuation, and beam overlap; the target response represents scene reflectivity.
- Snowfall Simulation: Snow particles are sampled as nonintersecting opaque spheres whose number is controlled by snowfall rate and whose positions can occlude targets or other particles.Each particle has a reflectivity, diameter, distance, and beam-opening contribution.
- Snowfall Simulation: The snowfall response is a superposition of target and snow-particle echoes, with each echo shaped by particle angle and inverse-square distance attenuation.The simulator evaluates the maximum received-power peak as the LiDAR return.
- Snowfall Simulation: When a snow-particle peak exceeds the target peak, the simulator replaces the true return with a cluttered point at the snow-particle range; otherwise, target intensity is attenuated by occlusion.The complete procedure samples snowflakes in 2D for each LiDAR layer and processes particles intersecting each beam.
- Wet Ground Model: The wet-ground model represents road wetness through geometric optics and thin-layer reflection, using refractive indices, incident and refracted angles, and polarization-dependent light paths.The model reconstructs modified ground intensity from dry measurements after estimating the ground plane and incident angle.
4. Experiments
Experiments evaluate simulated snowfall and wet-ground augmentation on STF across seven detectors and three weather splits. The full simulation improves heavy-snow performance consistently and preserves or improves clear-weather performance.
- Dataset and evaluation: Table 1 reports 3D average precision for moderate cars across heavy-snowfall, light-snowfall, and clear-weather STF test splits.The reported splits contain 1,404 heavy-snowfall, 2,512 light-snowfall, and 1,816 clear-weather samples.
- Quantitative results: Our snow+wet simulation consistently improves heavy-snowfall performance for all detectors over the clear-weather baseline and competing simulation or denoising methods.The improvement is especially pronounced for PV-RCNN, where the full simulation exceeds the clear-weather baseline by 2.1% in AP.
- Quantitative results: The full simulation achieves the best performance among methods on light snowfall for PV-RCNN, CenterPoint, and Part-A², extending benefits across snowfall intensities.The snowfall-only simulation also improves clear-weather performance for six of seven detection methods compared with the clear-weather baseline.
- Comparison with competing methods: Fog simulation transfers poorly to snowfall, while DROR can remove valid points and achieves among the lowest results.The fog simulation is slightly below the clear-weather baseline on both snowfall splits for most detectors; DROR also removes non-clutter points.
- Qualitative results: Qualitative examples show that the proposed augmentation detects snowfall-obscured pedestrians and lead vehicles while competing methods produce additional false positives.The comparison includes the clear-weather baseline, DROR, LISA, and the proposed snow+wet augmentation on heavy-snowfall STF samples.
5. Conclusion
The paper introduces physically accurate synthesis of snowy and wet-ground LiDAR scenes from clear captures. Across seven detectors, augmentation consistently improves performance, reaching up to 2.1% AP gains in heavy snowfall.
- The method synthesizes winter LiDAR scenes from clear captures by modeling snowfall and wet surfaces in a physically accurate way.
- Testing with seven 3D object detectors yields consistent improvements of up to 2.1% in AP under heavy snowfall.
- Future work will explore temporal cues for robust LiDAR-based 3D object detection.