Source-linked AI summary

Beyond the Snowflakes: Disentangling Weather Effects from Confounding Meta-Factors in LiDAR Object Detection

Yiming Yang, Jeremy P. Bos

arXiv:2610.13891v1cs.CVcs.RO

TL;DR

Snowy LiDAR evaluation lacks systematic analysis across snowfall intensities and can be confounded by object-level shifts. The paper evaluates WADS-3D and CADC with ablation and meta-information analysis, finding detection performance improves with heavier snowfall as object point counts increase.

  • Problem

    Snowfall can degrade LiDAR detection, while systematic quantitative analysis across snowfall intensities remains limited.

    Source: §I · p. 1
  • Method

    The authors evaluate WADS-3D and CADC using ablation and meta-information analyses to examine detection performance across snowfall intensities.

    Source: §I · p. 1
  • Results

    Detection performance improves with heavier snowfall across WADS-3D and CADC, while snow filtering preserves the positive trend.

    Source: 1) Results on WADS-3D · p. 4, 2) Results on CADC · p. 4
  • Takeaways & Limitations

    Weather-severity bins alone provide incomplete robustness profiles when object-level shifts such as increased point counts are unaccounted for.

    Source: A. Conclusion · p. 7

    The proposed reflectance account remains a hypothesis: the authors suspect snow accumulation shifts vehicle surfaces from specular to diffuse-like reflectance.

    Source: E. Discussion and Hypotheses · p. 6

Abstract

from arXiv · show

Light Detection and Ranging (LiDAR) is critical for autonomous perception, yet its performance is known to degrade in adverse weather. While existing literature models snow as atmospheric noise that degrades performance, our granular evaluation on the WADS-3D and CADC datasets reveals a counter-intuitive phenomenon: detection accuracy consistently improves as snow intensity increases. Through ablation study and meta-information analysis, we reconcile this by confirming that while falling snow acts as noise, its negative impact is outweighed by a dominant, concurrent factor: significantly increased object point count. We hypothesize that this object point count boost acts as a geometric enhancement, driven by snow accumulation increasing vehicle surface area and a shift from specular to diffuse reflectance. Our findings demonstrate that evaluating perception systems using broad weather-severity bins without accounting for correlated object-level data can lead to misleading conclusions about detector robustness. We conclude that accurate evaluation requires decoupling atmospheric noise from surface accumulation effects and outline future research to validate these physical models. The WADS-3D dataset is available at: https://digitalcommons.mtu.edu/wads-3d/.

I. INTRODUCTION

LiDAR supports core autonomous-vehicle perception tasks, but snowfall can add noise and occlusion that degrade detection. The paper addresses limited systematic analysis across snowfall intensities with WADS-3D and CADC evaluations.

I. INTRODUCTION

  • Snowflakes can obscure objects and create transient noise points, with errors potentially propagating to safety-critical downstream modules.Adverse weather can also introduce occlusion and false returns.Source: §I · p. 1
  • Snow research remains comparatively limited, with scarce systematic quantitative analysis across snowfall intensities because data collection is difficult and costly.Prior studies often rely on theoretical scattering models or qualitative observations.Source: §I · p. 1
  • The authors introduce WADS-3D and evaluate it alongside CADC, finding detection performance can improve with heavier snowfall as object complexity outweighs environmental complexity.Ablation, error-matrix, and meta-information analyses attribute the trend to confounding variables rather than falling snow directly.Source: §I · p. 1

A. LiDAR-based 3D Object Detection

The related work spans LiDAR detection architectures, snowy-weather effects, complexity measures, and meta-information analyses. The paper situates its study within LiDAR-only perception and prior efforts to quantify factors associated with detector performance.

A. LiDAR-based 3D Object Detection

  • Modern LiDAR detectors use voxel, pillar, point-based, or hybrid representations; this study focuses on LiDAR-only methods to isolate snowy-condition effects.The literature also includes geometry-based clustering and sensor-fusion approaches.Source: A. LiDAR-based 3D Object Detection · p. 2

B. Impact of Snow on LiDAR Perception

  • Prior snow studies report degradation from airborne effects, while heterogeneous sensor datasets can introduce domain shifts that complicate interpretation of weather effects.Simulation work often models airborne snow without surface accumulation; other work reports vehicle-detection drops of up to 50% under heavy snow.Source: B. Impact of Snow on LiDAR Perception · p. 2

C. Object- and Scene-Level Complexity Metrics

  • Object- and scene-level complexity measures vary across benchmarks, including object point counts, visibility, range, annotation effort, geometric measures, and point-cloud entropy.The cited examples include LiDAR-centric object metrics in Waymo and post-hoc error analysis in nuScenes,.Source: C. Object- and Scene-Level Complexity… · p. 2

D. Analyzing the Influence of Meta-Information

  • Meta-information studies have used Random Forest and SHAP for camera detectors and Kendall’s Tau and SHAP for LiDAR pedestrian detection.Prior work also examined false-negative and false-positive errors to account for multivariate dependencies [47].Source: D. Analyzing the Influence of… · p. 2

E. Research Gap

The paper identifies limits in snowy-LiDAR evaluation and contributes WADS-3D, separate WADS-3D and CADC benchmarks, and meta-information analysis. The datasets also differ in observed object attributes.

E. Research Gap

  • Existing snowy-LiDAR studies often use coarse weather categories, omit surface accumulation, or combine different sensors, complicating interpretation of weather effects.The paper proposes independent WADS-3D and CADC benchmarks and meta-information analysis to address these gaps.Source: E. Research Gap · p. 2

A. Dataset

  • WADS-3D contains 4,100 labeled frames and 30,104 cars; CADC contributes 7,000 LiDAR frames captured with a different sensor suite.WADS-3D is derived from WADS and uses 2,050 unique 360-degree scans rotated for forward-view requirements.Source: A. Dataset · p. 2
  • WADS-3D vehicles have higher average points per bounding box despite hardware differences, and are generally observed farther away with lower mean intensity.Car dimensions also vary across the datasets.Source: A. Dataset · p. 2
  • Figure 2 compares car distance, points per car, mean intensity, and dimensions across KITTI, CADC, and WADS-3D; WADS-3D has the highest points-per-car distribution.The caption attributes CADC’s lower point count to its 32-channel lidar.Source: Figure 2 · p. 3

B. Snowfall Intensity Classification

The evaluation bins frames by quantified snowfall intensity, measures detection with Average Precision, and uses five detector architectures with dataset-specific training and evaluation setups.

B. Snowfall Intensity Classification

  • Snowfall bins use snow-point counts from DSOR and LIOR filtering, with quantile binning to keep comparable numbers of ground-truth cars per bin.Snow points range from 764–19,386 (0.69%–28.34%, avg. 8.97%) in WADS-3D and 142–4,447 (0.66%–21.38%, avg. 6.71%) in CADC.Source: B. Snowfall Intensity Classification · p. 3

C. Evaluation Metircs

  • The evaluation uses KITTI’s 3D detection protocol and Average Precision to summarize the precision-recall curve.The same metric is used for the ablation study to enable comparison with the main results.Source: C. Evaluation Metircs · p. 3

D. Implementation Details

  • Five detectors span voxel, pillar, point-based, and hybrid architectures; WADS-3D uses KITTI-pretrained models, while CADC uses a 50:50 fine-tuning and evaluation split.CADC training data is snow-filtered, but evaluation uses original snowy frames.Source: D. Implementation Details · p. 3

E. Analysis Methodology

The analysis combines snow-filter ablations, error analysis, and meta-information methods to separate snowfall effects from other influences. On WADS-3D, the positive AP trend persists after filtering, pointing to a confounding factor rather than falling snow as its driver.

E. Analysis Methodology

  • The framework uses raw-versus-denoised ablations, error matrices, and meta-information rankings to analyze snowfall effects and detector outcomes.It applies Kendall’s Tau, Mutual Information, and Random Forest with SHAP analysis.Source: E. Analysis Methodology · p. 3
  • Meta-information is divided into object-level features, including point count and distance, and scene-level features, including snowfall intensity and spatial complexity.The Random Forest models predict false negatives and true positives; conditional sampling is used for SHAP to limit extrapolation with correlated features.Source: E. Analysis Methodology · p. 4

1) Results on WADS-3D:

  • On WADS-3D, AP rises with snowfall intensity across architectures, and the positive trend remains after snow filtering.Filtering improves absolute scores, but does not remove the trend.Source: 1) Results on WADS-3D · p. 4

2) Results on CADC:

CADC confirms that detection performance trends upward with snowfall intensity, although the pattern is more volatile than on WADS-3D. Filtering preserves the trend, while cross-dataset filtering gains and architectural comparisons distinguish noise sensitivity and model robustness.

2) Results on CADC:

  • CADC confirms a positive AP trend across snowfall intensities, but with greater volatility than WADS-3D.Snow filtering preserves the positive trend without improving CADC performance.Source: 2) Results on CADC · p. 4, Figure 4 · p. 4

3) Impact of Explicit Snow Filtering:

  • Snow filtering improves WADS-3D AP by an average of 6.72%, but yields negligible gains on CADC.The authors associate WADS-3D’s greater snow-noise volume with clutter and object concealment, unlike CADC’s lower number of snow points.Source: 3) Impact of Explicit Snow Filtering · p. 4, Figure 4 · p. 4

4) Architectural Robustness Under Snowfall:

  • Voxel-based SECOND-IoU achieves the highest performance, ahead of pillar-based PointPillars, hybrid PV-RCNN, and point-based Part-A2.The authors suggest voxel aggregation filters sparse noise and that anchor-based models outperform anchor-free Part-A2.Source: 4) Architectural Robustness Under… · p. 4

B. Performance Decomposition & The Recall Paradox

As snowfall increases, both datasets show recall-related improvements in error distributions and precision-recall behavior. Increased true positives and reduced false negatives outweigh increased background clutter, producing the net AP increase.

  • In WADS-3D, increasing snowfall intensity coincides with more true positives, fewer false negatives, and PR curves shifting upward and rightward despite background-clutter fluctuations.Background errors dominate localization errors among false positives.Source: 1) Error Decomposition &… · p. 4, Figure 5 · p. 5
  • In CADC, false negatives decline across snowfall levels, while PR curves shift outward relative to WADS-3D.The outward shift delays precision dropoff and maintains high precision in the high-recall regime.Source: 1) Error Decomposition &… · p. 4, Figure 6 · p. 5
  • Across the reported results, increased true positives and reduced false negatives outweigh increased background clutter, driving the net AP increase.Source: 1) Error Decomposition &… · p. 5

2) The Physical Paradox:

The apparent improvement in heavy snow conflicts with the expected effects of atmospheric noise, so the authors examine object and scene factors associated with recall. Their WADS-3D meta-analysis identifies object point count and distance as the strongest recall determinants, while snowfall intensity has negligible importance.

2) The Physical Paradox:

  • Although snow noise is expected to attenuate signals and occlude objects, objects are easier to detect in heavy-snow frames, suggesting object characteristics may explain the improvement.Source: 2) The Physical Paradox · p. 5

3) Transition to Meta-Analysis:

  • The authors analyze recall through a binary true-positive/false-negative framework to test whether heavy-snow subsets contain easier-to-detect objects.The analysis considers physical object properties such as distance, point count, and size.Source: 3) Transition to Meta-Analysis · p. 5

1) Analysis of Recall Determinants on WADS-3D:

  • Across five architectures, object point count and distance are the strongest recall factors; snowfall intensity has negligible feature importance (<0.03).Point count and distance have Kendall’s τ of approximately 0.3 and −0.3, respectively; together they receive about 40–50% of model decision weight.Source: 1) Analysis of Recall Determinants on… · p. 5, Figure 7 · p. 6

2) Analysis of Recall Determinants on CADC:

On CADC, object point count is the primary determinant of recall, with intensity a secondary cue and snowfall intensity negligible. The analysis relates these patterns to object-level feature changes across snowfall levels and discusses possible physical explanations.

2) Analysis of Recall Determinants on CADC:

  • On CADC, object point count is the primary recall determinant (SHAP ≈0.55); mean point intensity is secondary (Kendall’s τ ≈0.3; SHAP ≈0.18), while snowfall intensity is negligible.Object dimensions also have negligible impact, likely because larger vehicles do not yield more returns at the given sensor resolution.Source: 2) Analysis of Recall Determinants on… · p. 6

D. Identifying Key Meta-Factors Driving Recall

  • Across snowfall bins, object point count correlates positively with severity in WADS-3D (r = 0.820, p < 0.01) and CADC (r = 0.733); CADC mean object intensity also correlates positively (r = 0.694).The analysis uses Pearson correlations between snowfall intensity bins and dataset-specific recall drivers; WADS-3D ego-distance shows no significant trend (r = −0.282, p = 0.43).Source: D. Identifying Key Meta-Factors… · p. 6, Table III · p. 6

E. Discussion and Hypotheses

  • The discussion proposes scene smoothing, snow returns entering car boxes, and a specular-to-diffuse reflectance shift as possible explanations for snow-related detection gains.The authors state that the physical mechanisms behind the statistical role of object point count remain uncharacterized.Source: E. Discussion and Hypotheses · p. 6
  • Controlled findings support two physical changes from snow accumulation: geometric inflation that raises point count and a radiometric shift that enhances return intensity.The reported intensity enhancement is especially apparent at high incidence angles and long ranges.Source: E. Discussion and Hypotheses · p. 7

A. Conclusion

The conclusion attributes improved detection under heavier snowfall to increased object point count outweighing falling-snow noise, and argues that weather-bin evaluations should account for object-level shifts. Future work targets other object types, mitigation strategies, and simulation of surface accumulation.

A. Conclusion

  • Across WADS-3D and CADC, heavier snowfall coincides with better 3D detection because increased object point count outweighs falling-snow noise, making weather-bin-only robustness evaluation incomplete.The conclusion calls for safety validation to disentangle precipitation noise from correlated geometric changes.Source: A. Conclusion · p. 7

B. Future Work

  • Future work will test whether accumulation similarly enhances geometry for smaller, non-rigid targets or whether atmospheric attenuation dominates their smaller cross-sections.The authors note that this may require targeted data collection.Source: B. Future Work · p. 7
  • Planned follow-ups include dynamic point-density normalization, camera-LiDAR fusion, and simulations that model physical snow accumulation on scene geometry.These mitigation strategies target geometric distortions introduced by snow accumulation; the simulation direction moves beyond airborne-noise-only models.Source: B. Future Work · p. 7
Loading 2610.13891v1…