PaperlayerView exact revision on arXiv

Source-linked AI summary

C-SFDA: A Curriculum Learning Aided Self-Training Framework for Efficient Source Free Domain Adaptation

Nazmul Karim, Niluthpol Chowdhury Mithun, Abhinav Rajvanshi, Han-pang Chiu, Supun Samarasekera, Nazanin Rahnavard

arXiv:2303.17132v1cs.CV

TL;DR

SFDA addresses domain adaptation when labeled source data cannot be accessed during adaptation, but existing self-training can be affected by early noisy pseudo-labels and memory-bank costs. C-SFDA uses curriculum-based selective self-training to prioritize reliable labels, and reports state-of-the-art results across image recognition, segmentation, and online test-time adaptation.

  • Problem

    UDA commonly requires labeled source data during adaptation, while SFDA self-training faces early pseudo-label noise and memory-bank costs.

    Source: §1 · p. 1

  • Method

    C-SFDA uses curriculum learning to train first on reliable pseudo-labels, then propagate label information to less reliable samples, without memory-bank-based refinement.

    Source: §1 · p. 2, §5 · p. 8

  • Results

    C-SFDA reports state-of-the-art results on image-recognition and segmentation benchmarks and online test-time adaptation; on GTA5→Cityscapes it achieves 48.3% 19-way averaged mIoU, outperforming HCL.

    Source: §1 · p. 2, §4.2 · p. 6

  • Takeaways & Limitations

    The memory-efficient framework is applicable to online test-time domain adaptation.

    Source: §1 · p. 2, §5 · p. 8

    The setting assumes source and target share a label distribution and common label set, while source data is unavailable during adaptation.

    Source: §3 · p. 3

Abstract

from arXiv · show

Unsupervised domain adaptation (UDA) approaches focus on adapting models trained on a labeled source domain to an unlabeled target domain. UDA methods have a strong assumption that the source data is accessible during adaptation, which may not be feasible in many real-world scenarios due to privacy concerns and resource constraints of devices. In this regard, source-free domain adaptation (SFDA) excels as access to source data is no longer required during adaptation. Recent state-of-the-art (SOTA) methods on SFDA mostly focus on pseudo-label refinement based self-training which generally suffers from two issues: i) inevitable occurrence of noisy pseudo-labels that could lead to early training time memorization, ii) refinement process requires maintaining a memory bank which creates a significant burden in resource constraint scenarios. To address these concerns, we propose C-SFDA, a curriculum learning aided self-training framework for SFDA that adapts efficiently and reliably to changes across domains based on selective pseudo-labeling. Specifically, we employ a curriculum learning scheme to promote learning from a restricted amount of pseudo labels selected based on their reliabilities. This simple yet effective step successfully prevents label noise propagation during different stages of adaptation and eliminates the need for costly memory-bank based label refinement. Our extensive experimental evaluations on both image recognition and semantic segmentation tasks confirm the effectiveness of our method. C-SFDA is readily applicable to online test-time domain adaptation and also outperforms previous SOTA methods in this task.

1. Introduction

Source-free domain adaptation (SFDA) adapts a source-trained model to unlabeled target data without access to source data. C-SFDA uses curriculum learning and reliable pseudo-label selection to address noisy self-training while avoiding memory-bank-based refinement.

1. Introduction

  • UDA typically requires labeled source data during adaptation, whereas SFDA uses only a source-pretrained model and unlabeled target data.Source-data access may be infeasible because of privacy concerns or limited device resources.Source: §1 · p. 1
  • Early noisy pseudo-labels can mislead cluster-based refinement, propagate noise through later training, and require burdensome memory banks.The passage also notes that most memory-bank-dependent methods are unsuitable for online test-time adaptation.Source: §1 · p. 1
  • C-SFDA prioritizes reliable pseudo-labels through curriculum learning to reduce early memorization and improve subsequent self-training.The approach aims to propagate accurate label information from reliable to less reliable samples.Source: §1 · p. 2
  • C-SFDA achieves state-of-the-art performance on image-recognition and semantic-segmentation benchmarks and online test-time adaptation.The framework is presented as memory-efficient and applicable to online test-time adaptation.Source: §1 · p. 2

2. Related Work

Prior work applies selective pseudo-labeling and curriculum learning in UDA, while SFDA methods often refine labels using feature-space structure. C-SFDA adapts selective, curriculum-based learning to SFDA without expensive refinement.

2. Related Work

  • Prior UDA work uses selective pseudo-labeling and curriculum learning, but applying them to SFDA requires a framework suited to that more challenging setting.The passage says the effectiveness of this curriculum depends on the proposed selective pseudo-labeling technique.Source: §2 · p. 2
  • SFDA methods such as SHOT, G-SFDA, and NRC refine labels using centroids or local-neighbor consistency, while AdaContrast adds contrastive feature learning.These approaches commonly depend on large memory queues, and early pseudo-label noise remains an issue.Source: §2 · p. 2
  • C-SFDA reduces label-noise memorization without rigorous, expensive pseudo-label refinement.Its overview uses reliable pseudo-labels for early CE and contrastive losses, then label propagation as the model becomes more confident.Source: §2 · p. 3, Figure 2 · p. 3

3. Proposed Method

C-SFDA adapts a source-trained model using target-only self-training, selecting reliable pseudo-labels and gradually extending learning to harder samples. It adapts this selection strategy to per-category pixel thresholds for semantic segmentation.

3. Proposed Method

  • SFDA adaptation assumes shared label classes but no access to labeled source data while adapting the source-trained model on unlabeled target data.Target labels are reserved for evaluation.Source: §3 · p. 3

3.1. Curriculum SFDA

  • A teacher generates target pseudo-labels for a student initialized with the same source-model weights, while augmentation averaging and EMA stabilize teacher predictions.The method uses 12 augmented copies per input and updates the teacher with an exponential moving average of student weights.Source: §3.1 · p. 3

3.1.1 Selective Pseudo-Labelling

  • Binary reliability selection uses prediction confidence and augmentation-based uncertainty, with batch-mean thresholds that avoid per-dataset threshold tuning.The uncertainty estimate uses variation across augmented predictions to approximate domain shift using target data only.Source: §3.1.1 · pp. 3–4

3.1.2 Loss Functions

  • Reliable samples receive class-balanced cross-entropy, less reliable samples use label propagation, and label-independent contrastive learning limits reliance on noisy labels.The total loss combines these terms with coefficients that control their contributions.Source: §3.1.2 · p. 4

3.1.3 Curriculum Learning

  • Curriculum weights initially restrict learning from less reliable samples, then gradually decrease as reliability improves, while high batch difficulty limits the weight change.The contrastive-loss coefficient also decays during training.Source: §3.1.3 · p. 4

3.1.4 Semantic Segmentation

  • For segmentation, per-category confidence and uncertainty percentiles select reliable pixels, which receive cross-entropy alongside prediction-entropy loss.The method uses the 55th percentile and updates batch-normalization layers while freezing other parameters.Source: §3.1.4 · p. 5

4. Experiments

Across image-classification and semantic-segmentation benchmarks, C-SFDA improves on prior SFDA results. The experiments also evaluate online adaptation and ablate selection, loss, and curriculum choices.

4.1. Experimental Settings

  • Experiments evaluate classification adaptation on Office-31, Office-Home, VisDA, and DomainNet, and segmentation adaptation from GTA5 and SYNTHIA to Cityscapes, plus Cityscapes to Dark-Zurich.VisDA results use per-class accuracy and its average, while DomainNet reports top-1 accuracy. Cityscapes validation uses 19-way performance; SYNTHIA→Cityscapes uses 16-way and 13-way performance.Source: §4.1 · p. 5

4.2. Experimental Results

  • 0.4% on Office-31, 0.6% on Office-Home, 1% over AdaCon on VisDA, and 1.2% over prior SOTA on DomainNet are reported.The Office-31 and Office-Home improvements are over previous SOTA results.Source: Table 1 · p. 5, §4.2 · p. 6, Table 2 · p. 6
  • 48.3% 19-way averaged mIoU on GTA5→Cityscapes beats HCL, while SYNTHIA→Cityscapes improves mIoU by 1.1% over prior SOTA.C-SFDA also achieves SOTA on both mIoU and mIoU* for SYNTHIA→Cityscapes.Source: §4.2 · pp. 6–7, Table 5 · p. 7, Table 6 · p. 7
  • 3.5% accuracy gain on VisDA, 0.8% mIoU gain on Cityscapes→Dark-Zurich, and 4% gain on SYNTHIA→Cityscapes are reported for online test-time adaptation.The online experiments train for one epoch.Source: §4.2 · p. 7, Table 7 · p. 7

4.2.1 Ablation Study

  • Confidence, uncertainty, and DoC each affect accuracy; prediction uncertainty is especially influential, and using enough augmentations is crucial to SOTA average accuracy.The augmentation-count study varies L.Source: §4.2.1 · pp. 7–8, Figure 4 · p. 8
  • On VisDA, confidence and selected reliable samples rise while uncertainty falls; training shifts CE emphasis from DR to DU while minimizing contrastive loss throughout.Average accuracy improves significantly during training.Source: Figure 3 · p. 8
  • CE alone gives satisfactory performance, while propagation loss without contrastive loss may degrade results; contrastive learning helps prevent label-noise memorization.The component ablation is reported in Table 8.Source: §4.2.1 · p. 8, Table 8 · p. 8

5. Conclusion

C-SFDA uses curriculum-based selective self-training to adapt without source data or memory-bank label refinement, and the authors report superior performance across image-classification and semantic-segmentation benchmarks. They identify extreme domain shifts as a boundary because initial pseudo-label reliability can be severely misleading.

  • Extensive evaluations show superior performance across image-classification and semantic-segmentation benchmarks.Source: §5 · p. 8
  • C-SFDA prioritizes highly reliable pseudo-labels, then propagates label information to less reliable samples, replacing label refinement with hyperparameter-independent selection.Source: §5 · p. 8
  • Contrastive loss-based representation learning supports consistent feature representations, and C-SFDA’s memory efficiency suits online test-time adaptation.Source: §5 · p. 8
  • Under extreme domain shifts, initial pseudo-label reliability may be severely misleading, making selection over-restrictive; the authors suggest additional robust self-supervision or strongly augmented source training.Source: §5 · p. 8

Related papers

Loading 2303.17132v1…