PaperlayerView exact revision on arXiv

Source-linked AI summary

ICM: Intra-class Mixing for Domain Adaptation in Adverse Weather

Boying Li, Chang Liu, Britta Ayano Wilde, György Kovács, Tosin Adewumi, Björn Backe, Hamam Mokayed

arXiv:2609.27533v1cs.CV

TL;DR

Adverse-weather appearance shifts challenge semantic-segmentation UDA and reduce target pseudo-label reliability. ICM enforces prediction consistency between clean images and same-image, same-class mixes. On Cityscapes→ACDC it achieves 75.7% mIoU, +1.9 pp over the previous best, while the authors identify weaker effectiveness in severe night conditions as a limitation.

  • Problem

    Adverse-weather changes widen the domain gap and make target-domain pseudo-labels less reliable for semantic-segmentation UDA.

    Source: §I · p. 1

  • Method

    ICM mixes confident regions into unconfident regions of the same class within an image and enforces consistency between mixed and clean predictions.

    Source: §I · p. 1, A. Intra-class mixing · p. 3

  • Results

    75.7% mIoU on Cityscapes→ACDC, improving on the previous best result by +1.9 pp.

    Source: §VII · p. 8

  • Takeaways & Limitations

    ICM establishes a new state of the art on Cityscapes→ACDC and is compatible with existing UDA frameworks.

    Source: §VII · p. 8

    The authors suspect ICM is less effective in severe night conditions, and its robustness to different λsup and λKL weighting scales remains untested.

    Source: §VI · pp. 7–8

Abstract

from arXiv · show

Unsupervised domain adaptation (UDA) for semantic segmentation remains challenging under adverse weather conditions because severe appearance changes enlarge the domain gap and degrade the reliability of pseudo labels in the target domain. To address this problem, we propose an Intra-Class Mixing Consistency (ICM) framework that enforces prediction consistency between an intra-class mixed image and its original counterpart. Unlike previous mixing-based consistency methods that combine regions across different images or domains and may introduce unrealistic semantic inconsistencies, ICM performs mixing within the same image and semantic class, preserving realistic semantic layout for consistency regularization. With ICM, we establish a new state-of-the-art performance for clear-to-adverse-weather unsupervised domain adaptation (UDA) in semantic segmentation. On the Cityscapes $\rightarrow$ ACDC benchmark, our method achieves 75.7\% mIoU, outperforming the previous state of the art by +1.9 pp, demonstrating its effectiveness in mitigating class confusion under challenging environmental conditions. The code is provided in the supplementary material.

I. INTRODUCTION

UDA offers a way to adapt from labeled clear-weather data to unlabeled adverse-weather data, but large appearance shifts and visually similar classes make reliable segmentation difficult. ICM addresses this by enforcing consistency between intra-class-mixed images and clean images, and improves Cityscapes-to-ACDC results over the previous state of the art.

I. INTRODUCTION

  • Adverse weather changes illumination, texture, and image quality, increasing domain gaps and confusion between visually similar classes such as road and sidewalk.The introduction also notes persistent failures on snow-covered sidewalks and on sky, vegetation, and buildings in dark nights, even with a DINOv3 backbone.Source: §I · p. 1
  • ICM enforces consistency between predictions on intra-class-perturbed images and clean counterparts; its plug-in module uses EMA-teacher pseudo-labels to mix confident pixels into unconfident regions of the same class.The method is validated within MIC and DAFormer.Source: §I · p. 1
  • The method sets a new state of the art on Cityscapes→ACDC, exceeding the previous state of the art by +1.9 pp mIoU.It also outperforms the proposed baseline by 1.3–2 pp mean IoU on this UDA task.Source: §I · p. 1

A. Unsupervised Domain Adaptation

  • UDA transfers knowledge from labeled source data to an unlabeled target domain, while prior approaches use distribution alignment, self-training, and teacher–student methods to address domain shift.For adverse-weather adaptation, existing work also explores masked consistency, weather-invariant feature alignment, image translation, and restoration-guided adaptation.Source: A. Unsupervised Domain Adaptation · p. 2

B. Mixing and Consistency Regularization

Prior mixing methods combine regions across images or domains and can disrupt scene semantics, while ICM mixes patches within the same image and semantic class. It focuses on stuff classes and uses teacher confidence to pair hard and easy patches while preserving class boundaries.

B. Mixing and Consistency Regularization

  • Cross-image and cross-domain mixing can introduce contextual inconsistencies, while ICM restricts uncertainty-aware patch mixing to semantically consistent regions within one target image.The cited examples include vehicles appearing in sky regions, and confusion among road, sidewalk, and sky under adverse weather.Source: B. Mixing and Consistency Regularization · p. 2

A. Intra-class mixing

  • ICM restricts mixing to the same image and semantic class, perturbing local appearance while preserving the scene’s semantic layout.The method applies mixing to stuff classes, where adverse-weather conflation is especially severe and disrupting geometric structure is considered less detrimental than for thing classes.Source: A. Intra-class mixing · p. 3
  • Source-image class labels come from ground truth, whereas target-image class labels come from EMA-teacher pseudo-labels.These labels provide the semantic information used for intra-class mixing.Source: A. Intra-class mixing · p. 3
  • The method divides images into grid patches and retains patches whose occupancy by a given stuff class exceeds a threshold.Only pixels labeled as that class in both paired patches are overwritten.Source: A. Intra-class mixing · p. 3
  • Teacher confidence guides mixing by pairing high-confusion, hard patches with low-confusion, easy patches of the same class.Low-confusion patches provide pixels that overwrite matching-class pixels in high-confusion patches.Source: A. Intra-class mixing · p. 3

B. Mixing and Consistency Regularization

  • The intra-class overwrite changes texture, illumination, or style within a class while preserving its semantic boundary.Earlier mixing approaches include class-agnostic geometric replacement and class-aware mixing across images or domains; intra-image permutation methods also often do not account for structured pseudo-label noise in UDA.Source: B. Mixing and Consistency Regularization · p. 2, A. Intra-class mixing · p. 3

B. Intra-Class Mix (ICM) Loss

ICM trains on an intra-class-mixed image and its clean counterpart using supervised segmentation and KL-based consistency losses. Semantic and mixing-region masks allow the loss region to be selected rather than applied indiscriminately.

B. Intra-Class Mix (ICM) Loss

  • ICM combines pixel-wise supervised segmentation loss with KL-based consistency regularization between clean and mixed predictions.The overview identifies supervision and consistency losses as the objectives for the mixed image and its clean counterpart.Source: B. Intra-Class Mix (ICM) Loss · p. 3, Figure 1 · p. 4
  • The loss region can be selectively chosen using semantic and mixing-region masks instead of including every image region.The semantic mask identifies stuff-class pixels.Source: B. Intra-Class Mix (ICM) Loss · p. 3
  • ICM mixing preserves the clean image’s label because it operates within the same image and semantic class.The figure comparison states that the intra-class mixed image retains the clean image’s label.Source: Figure 2 · p. 4

1) Supervised Segmentation Loss:

The supervised segmentation loss trains on the intra-mixed image using source labels or target pseudo-labels, with pixel weights that account for pseudo-label confidence.

1) Supervised Segmentation Loss:

  • The loss applies effective labels from source ground truth or target teacher-generated pseudo-labels to the intra-mixed image.The pixel weights are 1 for valid source pixels, w for valid target pseudo-labeled pixels, and 0 for ignored pixels.Source: 1) Supervised Segmentation Loss · p. 4
  • The target-pixel weight w is the proportion of target pixels whose maximum teacher probability exceeds confidence threshold τ.Teacher probabilities are defined per class and pixel.Source: 1) Supervised Segmentation Loss · p. 4
  • The supervised loss is normalized weighted cross-entropy over the mixed image, using the effective labels and pixel weights.The normalization is by the sum of weights over the region.Source: 1) Supervised Segmentation Loss · p. 4

2) Intra-Class Mix Consistency:

ICM regularizes prediction consistency between clean and intra-class-mixed images. It uses pixel-wise KL divergence, treating clean-image predictions as fixed targets while updating predictions on the mixed image.

2) Intra-Class Mix Consistency:

  • ICM enforces prediction consistency between the clean image and its intra-class-swapped version.The design seeks stable semantic predictions under intra-class spatial perturbations.Source: 2) Intra-Class Mix Consistency · pp. 4–5
  • The consistency loss is a pixel-wise KL divergence between temperature-scaled predictions on the clean and mixed images.The clean-image distribution is the fixed target, with gradients propagated only through the mixed-image prediction.Source: 2) Intra-Class Mix Consistency · pp. 4–5

3) Optimization Objective:

The optimization combines the ICM objective with a baseline loss. The ICM objective weights supervised segmentation and KL consistency terms, while the baseline includes source segmentation, DACS mixing, and MIC masking consistency losses.

3) Optimization Objective:

  • The total objective adds the ICM objective to the baseline loss.The ICM objective weights supervised segmentation and KL consistency losses by λsup and λKL.Source: 3) Optimization Objective · p. 5
  • The baseline loss sums source segmentation, DACS class-mixing, and MIC masking-consistency losses.The baseline follows DACS, DAFormer, HRDA, and MIC.Source: 3) Optimization Objective · p. 5

A. Implementation Details

ICM improves Cityscapes-to-ACDC results over prior UDA methods, and the paper reports implementation settings, datasets, and an ablation setup for examining the method.

A. Implementation Details

  • The implementation builds on DAFormer with HRDA and MIC, using a ConvNeXt-Base-DINOv3-LVD1689M encoder and freezing its first two stages.The EMA teacher momentum is 0.999; intra-class mixing divides images into an 8×8 grid, and the ICM loss scales are 0.5.Source: A. Implementation Details · p. 5

B. Datasets

  • Experiments use Cityscapes as labeled source data and ACDC training images as unlabeled adverse-weather target data.ACDC covers night, fog, rain, and snow; its validation split is used for ablations and hyperparameter tuning.Source: B. Datasets · p. 5

C. Comparison to the State of the Art in UDA

  • ICM exceeds previous UDA approaches by 1.9 pp in mean IoU and achieves best or second-best IoU in 14 of 19 classes on Cityscapes→ACDC.The strong MIC-DINO baseline combines MIC with a DINOv3 backbone; ICM further improves adaptation performance.Source: C. Comparison to the State of the Art… · p. 5

D. Analysis and Ablative Study

  • The ablation uses DAFormer without HRDA or MIC on Cityscapes→ACDC, with each experiment run for 20k iterations across three runs.The authors report that ICM provides consistent, robust improvements independently of MIC.Source: D. Analysis and Ablative Study · p. 5

1) Apply ICM to source or target:

Across the ablations, applying ICM to both domains performs best; component and region studies favor preserving semantic identity and applying consistency within stuff regions.

2) Ablation of components:

  • Inter-class mixing drops performance by 2.1 pp versus baseline; KL consistency is inapplicable when cross-class mixing changes semantics.Inter-class mixing occurs within one image but across classes, with overwritten destination pixels assigned the source class ID.Source: 2) Ablation of components · pp. 5–6, Table III · p. 6

1) Apply ICM to source or target:

2) Ablation of components:

  • Confusion-guided mixing adds 0.6 pp over random mixing; supervised-only gives +1.7 pp versus +0.8 pp for KL-only, while full ICM gives +1.9 pp.Source: 2) Ablation of components · p. 6

3) Loss region:

  • Performance stays similar when ICM losses cover stuff regions, but thing-only losses fall below baseline, possibly from unstable object mixing and pseudo-label noise.Source: 3) Loss region · p. 6

4) Patch size:

An 8×8 grid balances structural coherence with appearance variation, while ICM improves the MIC-DINO and re-trained MIC baselines on Cityscapes→ACDC.

E. Effectiveness of ICM

  • ICM surpasses MIC-DINO by 1.3–2.0 pp; with MiT-B5, ICM reaches 66.5 mIoU versus MIC-Reproduce’s 65.0 mIoU (+1.5 pp).Source: E. Effectiveness of ICM · pp. 6–7
  • Across snow, fog, night, and rain, ICM improves road, sidewalk, and traffic-light segmentation and mitigates foggy sky/building and snowy sky/road conflation.Source: Figure 3 · p. 7

4) Patch size:

  • The 8×8 grid balances structural coherence and effective intra-class invariance; performance remains stable across 4–16 grids.Large patches disturb global layout, while very small patches provide weak appearance variation.Source: 4) Patch size · p. 6, Table V · p. 7

V. DISCUSSION

ICM sets a new Cityscapes→ACDC state of the art, with its largest class-wise gains among stuff classes. The discussion also reports a severe-night weakness and open questions about class-group behavior and weighting-scale robustness.

V. DISCUSSION

  • Truck, Train, and Bus also improve despite mixing being applied only to stuff regions, possibly benefiting from surrounding stuff context.The authors offer contextual information as a possible explanation for the gains in nearby vehicles.Source: §V · p. 7

VI. LIMITATION

  • The interpretation of stuff-versus-thing IoU behavior in the loss-region analysis remains preliminary; representation-level analysis is a future direction.Source: §VI · p. 7
  • ICM may be less effective in severe night conditions; degradation is observed on the ACDC night subset and Cityscapes→DarkZurich.The authors contrast this with ACDC improvements arising mainly from fog and snow conditions, where intra-class consistency is more reliable.Source: §VI · pp. 7–8

V. DISCUSSION

  • Road, Sky, and Sidewalk show the largest gains among stuff classes; Vegetation drops in the DINO Test setting.The figure compares per-class IoU improvement for three experimental settings.Source: §V · p. 7, Figure 4 · p. 8

VI. LIMITATION

  • Robustness to different λsup and λKL weighting scales remains untested.Source: §VI · p. 8

VII. CONCLUSION

  • 75.7% mIoU on Cityscapes→ACDC sets the new state of the art, +1.9 pp over the previous best.ICM uses confusion-guided intra-class patch mixing within the same image while preserving semantic identity.Source: §VII · p. 8
Loading 2609.27533v1…