PaperlayerView exact revision on arXiv

Source-linked AI summary

Weather-Aware Domain Adaptation for Street-View Weather Recognition

Hossein Maghsoumi, George Atia, Yaser P. Fallah

arXiv:2610.02000v1cs.CVcs.LG

TL;DR

Street-view weather recognition is underexplored, and non-street-view training data differ from real driving imagery. The paper proposes weather-conditioned domain adaptation and a multi-dataset benchmark; across backbones, WA-ADDA outperforms source-only training and classical ADDA with more balanced adverse-weather performance.

  • Problem

    Multi-class weather recognition from street-view images is underexplored, while non-street-view datasets and domain shift limit generalization to real driving imagery.

    Source: A. Weather Classification in… · p. 2

  • Method

    WA-ADDA conditions the domain discriminator on predicted weather, and the paper benchmarks non-street-view source datasets against street-view targets using macro accuracy as the primary metric.

    Source: §I · p. 1

  • Results

    Across multiple backbones, WA-ADDA outperforms source-only training and classical ADDA, with higher macro and overall accuracy and more balanced performance, particularly for fog, rain, and sand.

    Source: §V · p. 6

  • Takeaways & Limitations

    The authors present WA-ADDA as a backbone-agnostic framework that requires no target annotations and can be integrated into vision-based perception systems.

    Source: §V · p. 6

Abstract

from arXiv · show

Adverse conditions such as rain, snow, fog, and dust remain challenging for camera-based perception in autonomous driving. We study multi-class weather recognition from street-view images under domain shift, where most available training data come from non-street-view sources that differ markedly from real driving scenes. We propose Weather-Aware Adversarial Discriminative Domain Adaptation (WA-ADDA), which conditions the domain discriminator on predicted weather to promote features that are both domain-invariant and weather-sensitive. We also assemble a multi-dataset benchmark by unifying diverse non-street-view weather collections as sources and real street-view images as targets, and define a standardized evaluation protocol with macro accuracy as the primary metric. Across backbones (ResNet-50, EfficientNet, VGG, DenseNet), WA-ADDA consistently improves street-view performance and yields strong per-class recalls in challenging conditions while preserving clear-weather accuracy. These findings highlight the feasibility of domain-adapted weather recognition and the value of our benchmark for advancing robust, on-board perception.

I. INTRODUCTION

Street-view weather recognition remains difficult because camera perception degrades in adverse conditions and most weather data differ from real driving imagery. WA-ADDA and a multi-dataset benchmark address this domain shift, with reported gains across backbones.

I. INTRODUCTION

  • Camera images provide immediate, localized evidence of on-scene weather, supporting timely adaptation of detection and control.External weather services may be coarse in space and time and can miss local phenomena.Source: §I · p. 1
  • Direct multi-class weather recognition for street-view images remains underexplored, and differences in viewpoint, optics, and scene composition impair generalization from non-street-view data.Source: §I · p. 1
  • Generic ADDA aligns source and target features but does not use weather-specific structure that could stabilize alignment and preserve class semantics.Source: §I · p. 1
  • WA-ADDA conditions the domain discriminator on predicted weather probabilities to encourage features that are domain-invariant yet weather-sensitive.Source: §I · p. 1
  • The benchmark unifies non-street-view weather datasets as sources and street-view images as targets, using macro accuracy as its primary metric to account for class imbalance.Source: §I · p. 1
  • WA-ADDA improves target-domain performance under challenging weather while maintaining accuracy on clear-weather scenes across evaluated backbones.The evaluation includes EfficientNet, ResNet-50, DenseNet, and VGG.Source: §I · p. 1

II. RELATED WORK

Prior work and available datasets leave multi-class street-view weather recognition under domain shift comparatively underdeveloped. The paper positions weather-conditioned adaptation and a unified benchmark as responses to that gap.

A. Weather Classification in Autonomous Driving

  • Weather classification from street-view images is underexplored, while many existing datasets are non-street-view or synthetically augmented and generalize poorly to driving scenes.Source: A. Weather Classification in… · p. 2

B. Domain Adaptation for Visual Perception

  • Adversarial and discrepancy-based domain adaptation have been applied to adverse-weather driving tasks, but the cited advances predominantly address detection and segmentation rather than multi-class weather classification.Source: B. Domain Adaptation for Visual… · p. 2

C. Weather-Aware DA and Benchmarks

D. Our Position

  • The paper proposes weather-conditioned ADDA and a multi-dataset benchmark with standardized evaluation for street-view weather classification.Source: D. Our Position · p. 2

III. METHOD

  • The task is to train a weather classifier using labeled source data to predict labels on an unlabeled street-view target dataset.WA-ADDA conditions the discriminator on the current weather posterior and adds unsupervised target-side regularizers for confident, stable predictions.Source: §III · p. 2

A. Source Pretraining

The method starts from source-supervised ADDA training, then uses a weather-aware discriminator to align target features without target weather labels.

A. Source Pretraining

  • The source encoder and classifier are first trained on labeled source data, after which the encoder is frozen and initializes the target encoder.Source: A. Source Pretraining · p. 2

B. Weather-Aware Discriminator

  • WA-ADDA conditions the domain discriminator on concatenated features and predicted class probabilities, making domain discrimination sensitive to weather predictions.Source: B. Weather-Aware Discriminator · p. 2
  • The discriminator learns to distinguish source from target features while the target encoder is trained to fool it, encouraging alignment within weather modes without target labels.Source: B. Weather-Aware Discriminator · p. 2

C. Target-Side Objectives

Target-side objectives combine adversarial alignment with regularizers for confident and stable predictions. The optional consistency term uses mildly perturbed target images to reduce sensitivity to nuisance weather artifacts.

C. Target-Side Objectives

  • With the source encoder frozen, the target encoder is updated to fool the discriminator during adversarial alignment.Source: C. Target-Side Objectives · p. 3
  • Minimizing Shannon entropy encourages confident target predictions and avoids degenerate uniform posteriors.Source: C. Target-Side Objectives · p. 3
  • The optional label-free consistency regularizer compares predictions on target images and mildly adverse-style augmented views, improving robustness to small visibility changes and nuisance weather artifacts.Source: C. Target-Side Objectives · p. 3

D. Overall Objectives and Training

Training proceeds from supervised source pretraining to adversarial target adaptation, with optional target-side regularizers and inference using only the target encoder and classifier. The benchmark pairs diverse non-street-view sources with unlabeled street-view targets and evaluates performance with class-balanced metrics.

D. Overall Objectives and Training

  • WA-ADDA first trains the source encoder and classifier on labeled data, then adversarially adapts an initialized target encoder while keeping the source encoder fixed.The discriminator and target encoder are updated in alternation during adaptation.Source: D. Overall Objectives and Training · p. 3
  • The discriminator uses weather probabilities alongside features to align domains within weather modes, while optional entropy minimization and adverse-style consistency stabilize target predictions.Only the target encoder and classifier are retained for street-view weather classification at inference.Source: Figure 1 · p. 3, D. Overall Objectives and Training · p. 3

A. Datasets and Benchmark Protocol

  • The benchmark combines five publicly available non-street-view weather datasets as sources with the Street-View dataset as target, using target labels only for evaluation.The datasets vary in resolution, collection environments, and weather conditions.Source: A. Datasets and Benchmark Protocol · p. 3
  • Source and target samples are re-weighted so each weather category contributes equally to the loss, and training uses random flips and mild color jitter.Adverse-weather classes in the Street-View dataset are relatively scarce compared with Clear images.Source: A. Datasets and Benchmark Protocol · pp. 3–4
  • Evaluation reports macro accuracy, the average of class-wise accuracies, alongside overall accuracy, and averages results over three random-seed runs.The benchmark describes macro accuracy as its primary metric to account for inter-class imbalance.Source: A. Datasets and Benchmark Protocol · p. 4

B. Implementation Details

The implementation evaluates four ImageNet-initialized backbones with a shared classifier design and a two-phase source-pretraining and adversarial-adaptation procedure. Adaptation updates the target encoder and discriminator, while the source encoder remains fixed.

B. Implementation Details

  • Experiments use ResNet-50, EfficientNet-B0, VGG16-BN, and DenseNet-121 as ImageNet-initialized feature extractors.The classifier uses two fully connected layers with batch normalization and ReLU, followed by a five-category softmax.Source: B. Implementation Details · p. 4
  • Algorithm 1 first trains the source encoder and classifier with cross-entropy, copies source weights to initialize the target encoder, then alternates discriminator and target-encoder updates.The target encoder is optimized with the weighted objective; the source encoder is kept fixed during adaptation.Source: B. Implementation Details · p. 4
  • Source pretraining uses SGD for 30 epochs with batch size 64, learning rate 10^-3, momentum 0.9, and weight decay 10^-4.The adaptation hyperparameter table summarizes configurations for each backbone.Source: B. Implementation Details · p. 4, Table II · p. 4
  • During adaptation, Adam optimizes the target feature extractor and discriminator; batch normalization is fixed in evaluation mode and only final high-level convolutional stages are unfrozen.Each experiment is repeated with three random seeds and mean performance is reported.Source: B. Implementation Details · p. 4

C. Baselines

The comparison evaluates Source-Only, standard ADDA, and WA-ADDA under shared preprocessing, initialization, and optimization settings. WA-ADDA adds classifier-aware discrimination and target-side regularization to standard ADDA.

C. Baselines

  • Source-Only trains on the aggregated multi-source data without domain adaptation and serves as a lower-bound reference for the source-target domain gap.All methods share data preprocessing, network initialization, and optimization settings for comparison.Source: C. Baselines · p. 4
  • ADDA pretrains a source encoder and classifier, then adversarially trains a target encoder against a discriminator that distinguishes source and target features.It is the primary baseline for assessing the classifier-aware extension.Source: C. Baselines · pp. 4–5
  • WA-ADDA conditions the discriminator on feature embeddings and classifier logits, and applies entropy minimization and consistency regularization to stabilize adaptation and reduce pseudo-label noise.The controlled comparison is intended to isolate feature-logit coupling and regularization relative to ADDA.Source: C. Baselines · p. 5

D. Quantitative Results

On the Street-View target set, adversarial methods outperform Source-Only across backbones, while WA-ADDA shows strong, balanced weather performance and preserves Clear-category accuracy. Results also report backbone-wide consistency and stability across random seeds.

D. Quantitative Results

  • WA-ADDA's more uniform class-wise generalization contrasts with ADDA's occasional high accuracy in individual classes alongside degradation in others and less stable macro performance.WA-ADDA also shows only small macro-accuracy variance across random seeds on the Street-View target set.Source: D. Quantitative Results · p. 5
  • Source-dataset results are reported as a high-performing upper-bound reference for the backbones and contextualize target-domain differences against domain shift.Table III reports macro, overall, and per-class accuracy on the source dataset.Source: Table III · p. 5, D. Quantitative Results · p. 5
  • Both ADDA and WA-ADDA significantly outperform Source-Only across all backbones on the Street-View target dataset.WA-ADDA consistently achieves strong macro and overall accuracy, often matching or surpassing ADDA, with stronger performance in Fog, Rain, and Sand while preserving Clear accuracy.Source: Table IV · p. 5, D. Quantitative Results · p. 5
  • Consistent gains across ResNet-50, DenseNet-121, EfficientNet-B0, and VGG16-BN indicate performance across the evaluated architectures.Source: D. Quantitative Results · p. 5

E. Qualitative Results

The feature visualizations show progressively closer source–target alignment from source-only training to ADDA and WA-ADDA, with WA-ADDA producing the most coherent distributions. Fig. 3 organizes these t-SNE views by backbone and training setting.

E. Qualitative Results

  • Before adaptation, source and target samples form clearly separated clusters across backbones, especially for EfficientNet-B0 and VGG16-BN.The separation reflects domain differences associated with weather conditions, illumination, and visibility.Source: E. Qualitative Results · p. 5
  • Fig. 3 presents t-SNE source–target feature distributions by backbone in columns and by training setting in rows.The settings are source-only, ADDA, and WA-ADDA.Source: E. Qualitative Results · p. 5, Figure 3 · p. 6
  • ADDA partially aligns source and target features, but embeddings often remain fragmented or elongated, consistent with class-wise imbalance.The quantitative results also show ADDA improving some classes while degrading others.Source: E. Qualitative Results · pp. 5–6
  • WA-ADDA produces more coherent source–target feature distributions, with reduced fragmentation and improved overlap across multiple clusters.The authors describe the alignment as smoother and more structured, while noting that perfect overlap is neither expected nor required.Source: E. Qualitative Results · p. 6

V. CONCLUSION

The paper proposes WA-ADDA for multi-class weather recognition under severe domain shift and evaluates it on a multi-source-to-street-view benchmark. Across backbones, WA-ADDA outperforms source-only training and classical ADDA, with more balanced performance under adverse weather.

V. CONCLUSION

  • WA-ADDA extends ADDA by conditioning the domain discriminator on classifier predictions for semantically informed feature alignment.The paper addresses multi-class weather recognition where labeled street-view data are scarce and existing weather datasets differ substantially from real driving imagery.Source: §V · p. 6
  • WA-ADDA consistently outperforms source-only training and classical ADDA across backbone architectures on the multi-source-to-street-view benchmark.It achieves higher macro and overall accuracy with more balanced per-class performance, particularly for fog, rain, and sand.Source: §V · p. 6
  • The authors characterize WA-ADDA as a backbone-agnostic framework that requires no target annotations and can be integrated into vision-based perception systems.They describe it as a practical solution for robust weather recognition in real-world driving scenarios.Source: §V · p. 6
Loading 2610.02000v1…