PaperlayerView exact revision on arXiv

Source-linked AI summary

Guiding Pseudo-labels with Uncertainty Estimation for Source-free Unsupervised Domain Adaptation

Mattia Litrico, Alessio Del Bue, Pietro Morerio

arXiv:2303.03770v3cs.CVcs.AIcs.LG

TL;DR

In source-free UDA, adaptation must proceed without source data, and noisy pseudo-labels remain a challenge. The paper refines labels using neighbour information, estimates their uncertainty to reweight classification loss, and reports improved benchmark results across three datasets.

  • Problem

    Source-free UDA adapts a model without source data, while noisy pseudo-labels challenge existing self-training and refinement approaches.

    Source: §1 · p. 1

  • Method

    The method refines target pseudo-labels through neighbour aggregation, weights classification loss by their estimated uncertainty, and uses contrastive learning with noisy-label-robust negative-pair exclusion.

    Source: §3.2 · p. 4, §6 · p. 9

  • Results

    The method reports 90.0% accuracy on VisDA-C and 69.6% accuracy on DomainNet, each with a +1.8% gain, plus PACS gains of +12.3% single-source and +6.6% multi-target.

    Source: §1 · p. 2, §4.2 · pp. 6–7

  • Takeaways & Limitations

    Analyses report robustness to noise and progressively more accurate pseudo-labels that guide adaptation.

    Source: §6 · p. 9

    The approach adds the overhead of maintaining and updating a queue during training.

    Source: §5 · p. 8

Abstract

from arXiv · show

Standard Unsupervised Domain Adaptation (UDA) methods assume the availability of both source and target data during the adaptation. In this work, we investigate Source-free Unsupervised Domain Adaptation (SF-UDA), a specific case of UDA where a model is adapted to a target domain without access to source data. We propose a novel approach for the SF-UDA setting based on a loss reweighting strategy that brings robustness against the noise that inevitably affects the pseudo-labels. The classification loss is reweighted based on the reliability of the pseudo-labels that is measured by estimating their uncertainty. Guided by such reweighting strategy, the pseudo-labels are progressively refined by aggregating knowledge from neighbouring samples. Furthermore, a self-supervised contrastive framework is leveraged as a target space regulariser to enhance such knowledge aggregation. A novel negative pairs exclusion strategy is proposed to identify and exclude negative pairs made of samples sharing the same class, even in presence of some noise in the pseudo-labels. Our method outperforms previous methods on three major benchmarks by a large margin. We set the new SF-UDA state-of-the-art on VisDA-C and DomainNet with a performance gain of +1.8% on both benchmarks and on PACS with +12.3% in the single-source setting and +6.6% in multi-target adaptation. Additional analyses demonstrate that the proposed approach is robust to the noise, which results in significantly more accurate pseudo-labels compared to state-of-the-art approaches.

1. Introduction

Source-free adaptation addresses the lack of source data during target adaptation, where noisy pseudo-labels challenge existing approaches. The paper estimates uncertainty to reweight refined labels, adds contrastive negative-pair exclusion, and reports gains across three benchmarks.

1. Introduction

  • Source-free UDA adapts using unlabelled target data without source data, which common UDA methods require during adaptation.The paper notes that source data may be unavailable when privacy or transmission bandwidth is critical.Source: §1 · p. 1
  • Prior self-training approaches can be harmed by noisy pseudo-labels, while equally weighting refined labels fails to account explicitly for their uncertainty.The paper links this to noise overfitting and poor adaptation.Source: §1 · p. 1
  • The method reweights classification loss using uncertainty estimated from neighbours’ predictions after pseudo-label refinement, penalising labels with high uncertainty.The approach builds on initial target pseudo-labels assigned by the pretrained source model.Source: §1 · pp. 1–2
  • A self-supervised contrastive framework regularises the target feature space to support knowledge aggregation from neighbouring samples.The paper says refinement requires neighbourhoods to contain semantically similar samples.Source: §1 · p. 2
  • A temporal-queue strategy identifies and excludes same-class negative pairs even when pseudo-labels are noisy.The paper also reports that its approach progressively reduces pseudo-label noise better than the state of the art.Source: §1 · p. 2
  • 90.0% accuracy on VisDA-C and 69.6% accuracy on DomainNet, each with a +1.8% gain; PACS improves by +12.3% in single-source and +6.6% in multi-target settings.The paper reports these as state-of-the-art results on the named benchmarks and settings.Source: §1 · p. 2

2. Related Work

Prior domain-adaptation methods commonly require source and target data, while source-free approaches use a source model and unlabelled target data. The paper positions its approach against methods that do not address pseudo-label noise or require a noise-free validation set.

2. Related Work

  • Earlier domain-adaptation methods aligned distributions or used generative models but required access to both source and target data during adaptation.More recent source-free methods adapt using a source model and unlabelled target data.Source: §2 · p. 3
  • Among source-free adaptation methods using only a source model and unlabelled target data, [5] used self-supervision to refine pseudo-labels, but weighted them equally and applied no other noise countermeasure.Source: §2 · p. 3
  • Meta-learned loss-reweighting methods [24] [51] require a noise-free validation set and do not refine noisy labels during training.The related-work discussion also surveys noise-robust losses, noise-transition estimation, and clean-sample selection.Source: §2 · p. 3
  • The proposed reweighting estimates uncertainty from neighbour consensus without target labels or a noise-free validation set; unlike NEL, refinement needs no network ensemble.NEL combines Negative Learning with ensemble-based pseudo-label refinement, which the paper describes as computationally costly.Source: §2 · p. 3

3. Proposed Method

The method adapts a source model using only unlabeled target data, refining noisy pseudo-labels through neighbors and weighting them by estimated uncertainty. A temporal queue excludes likely same-class negative pairs, while self-supervised training supports target-space learning.

3. Proposed Method

  • Source-free adaptation uses the trained source model to adapt on unlabeled target images without access to source data.The source model first assigns target pseudo-labels, which can be incorrect under domain shift.Source: §3 · p. 3

3.1. Pseudo-label Refinement via Nearest neighbours Knowledge Aggregation

  • Neighboring target samples refine pseudo-labels by averaging their class probabilities and assigning the argmax of the resulting soft-vote vector.The method selects K nearest samples by cosine distance from a feature bank of weakly augmented targets.Source: §3.1 · pp. 3–4

3.2. Loss Reweighting with Pseudo-labels Uncertainty Estimation

  • The classification loss gives greater weight to refined pseudo-labels whose neighbors' averaged predictions have low entropy and less weight to high-entropy predictions.Uncertainty is estimated from neighbors rather than the sample's own pseudo-label; the exponential weighting is intended not to over-penalize samples near decision boundaries.Source: §3.2 · pp. 4–5

3.3. Temporal Queue for Negative Pairs Exclusion

  • A temporal queue excludes a sample pair from contrastive negatives if the pair shared a pseudo-label at least once during the past T epochs.This history-based rule is designed to identify same-class pairs despite noisy current pseudo-labels, unlike exclusion based only on current labels.Source: §3.3 · p. 5

3.4. Joint training with self-learning

  • Refined weak-view labels supervise strongly augmented targets through uncertainty-weighted negative learning, alongside self-supervised contrastive training.Refinement is iterative and progressively improves pseudo-label accuracy; the method uses negative learning without a positive classification loss and includes regularization to prevent posterior collapse.Source: §3.4 · pp. 5–6, §3.3 · p. 5

4. Experiments and Results

The experiments compare the method on PACS, VisDA-C, and DomainNet-126, reporting gains over prior methods in both single-source and multi-target settings. Ablations and further analyses examine the contributions of the pipeline components and its robustness to noisy pseudo-labels.

4.1. Experimental setup

  • Experiments cover PACS, VisDA-C, and seven DomainNet-126 domain shifts, using ResNet-18, ResNet-101, and ResNet-50 backbones, respectively.The models are initialized for adaptation from source-model parameters.Source: §4.1 · p. 6

4.2. Results

  • +1.8% average over prior SF-UDA methods on both VisDA-C and DomainNet-126, alongside PACS gains of +12.3% in single-source and +6.6% in multi-target adaptation.The PACS single-source experiments surpass the SOTA across six standard domain shifts; the multi-target settings merge domains into ACP or PCS targets.Source: §4.2 · pp. 6–7, Table 3 · p. 7, Table 4 · p. 7

4.3. Analysis

  • Adding contrastive regularisation to pseudo-label refinement raises VisDA-C accuracy by +26.6%, while negative learning, queue-based exclusion, and uncertainty reweighting add +3.2%, +3.7%, and +4.2%, respectively.Pseudo-label refinement alone reaches 52.3% accuracy.Source: Table 5 · p. 7, §4.3 · p. 8
  • A temporal queue length of T = 5 gives the best accuracy, while T = 1, using only current predictions for exclusion, gives the worst.The authors report negligible memory overhead for storing the five past predictions.Source: §4.3 · p. 8, Figure 4 · p. 8
  • Exponential loss reweighting outperforms linear reweighting by +4.9%, and using only negative learning yields a +4.8% gain over the positive-loss comparison.The smoother reweighting also performs better than hard sample selection based on an entropy threshold.Source: §4.3 · p. 8
  • Unlike AdaContrast [5], which progressively loses pseudo-label accuracy, the proposed reweighting and exclusion strategies accompany progressively improving pseudo-label accuracy.The authors associate this trend with greater robustness to noise and learning from more accurate labels.Source: Figure 5 · p. 8, §4.3 · p. 8

5. Limitations and Impact

Compared with other source-free UDA methods, the approach requires maintaining and updating a queue during training. The paper also notes that UDA systems can be put to inappropriate use or lead to harmful applications.

5. Limitations and Impact

  • Training requires maintaining and updating a queue, a stated overhead relative to other source-free UDA methods.The paper also notes possible inappropriate or harmful applications of UDA systems.Source: §5 · p. 8

6. Conclusion

The approach refines pseudo-labels through neighbor aggregation and uncertainty estimation, and adds negative-pair exclusion within a contrastive framework. It surpasses state of the art on major domain-adaptation benchmarks, with analyses showing robustness to noise and improved pseudo-label accuracy.

  • The method aggregates knowledge from neighboring samples to refine pseudo-labels and estimates their uncertainty to mitigate wrong assignments.Source: §6 · p. 8
  • The method surpasses state of the art on major domain-adaptation benchmarks, and analyses show robustness to noise and improved pseudo-label accuracy.Source: §6 · p. 9
  • A negative-pair exclusion strategy within a contrastive framework identifies and excludes pairs made of same-class samples.Source: §6 · p. 9
Loading 2303.03770v3…