PaperlayerView exact revision on arXiv

Source-linked AI summary

Domain Adaptation without Source Data

Youngeun Kim, Donghyeon Cho, Kyeongtak Han, Priyadarshini Panda, Sungeun Hong

arXiv:2007.01524v4cs.CVcs.LGeess.IV

TL;DR

The paper addresses domain adaptation when source samples cannot be accessed because of real-world availability and privacy constraints. SFDA uses a pre-trained source model, reliable target prototypes, pseudo-label filtering, and progressive self-learning, and is reported to outperform conventional source-data-based models without direct source-sample access.

  • Problem

    Domain adaptation commonly assumes access to source and target samples during training, but source-data access may be infeasible and privacy-sensitive.

  • Method

    SFDA freezes a pre-trained source model and progressively trains an initialized target model using entropy-selected class prototypes, similarity-based pseudo labels, set-to-set filtering, and source-oriented regularization.

  • Results

    SFDA achieves higher performance than conventional models trained with labeled source data without accessing source samples during training.

  • Takeaways & Limitations

    SFDA proposes a source-data-free domain-adaptation protocol that uses source-model parameters rather than source samples and is presented as relevant to data privacy and security.

  • Takeaways & Limitations

    SFDA may limit flexibility for modifying or adding source data, including annotation or preprocessing, when more flexible adaptation is required.

Abstract

from arXiv · show

Domain adaptation assumes that samples from source and target domains are freely accessible during a training phase. However, such an assumption is rarely plausible in the real-world and possibly causes data-privacy issues, especially when the label of the source domain can be a sensitive attribute as an identifier. To avoid accessing source data that may contain sensitive information, we introduce Source data-Free Domain Adaptation (SFDA). Our key idea is to leverage a pre-trained model from the source domain and progressively update the target model in a self-learning manner. We observe that target samples with lower self-entropy measured by the pre-trained source model are more likely to be classified correctly. From this, we select the reliable samples with the self-entropy criterion and define these as class prototypes. We then assign pseudo labels for every target sample based on the similarity score with class prototypes. Furthermore, to reduce the uncertainty from the pseudo labeling process, we propose set-to-set distance-based filtering which does not require any tunable hyperparameters. Finally, we train the target model with the filtered pseudo labels with regularization from the pre-trained source model. Surprisingly, without direct usage of labeled source samples, our PrDA outperforms conventional domain adaptation methods on benchmark datasets. Our code is publicly available at https://github.com/youngryan1993/SFDA-SourceFreeDA

1 Introduction

Unsupervised domain adaptation typically requires source and target data during training, creating practical and privacy concerns. SFDA instead uses a pre-trained source model and reliable target samples to progressively adapt without direct source-data access.

  • Motivation: Unsupervised domain adaptation reduces train–test discrepancy by transferring knowledge from labeled source data to unlabeled target data.Collecting labeled target-like samples is costly and time-consuming.
  • Motivation: UDA assumes that source and target samples are freely available during training, an assumption that can create privacy risks when source labels contain biometric or confidential information.Sensitive labels can identify samples or individuals, and improper disclosure may harm data providers and organizations.
  • SFDA: SFDA decouples adaptation from direct source-data usage by leveraging a pre-trained source model and progressively updating a target model with reliable target samples.The approach uses self-entropy to identify target samples whose predictions are more likely to be correct.
  • SFDA: Reliable target samples become class prototypes, whose feature similarities provide pseudo labels for target samples; set-to-set distance filtering reduces pseudo-label uncertainty.The framework combines source-oriented pseudo labels, target-oriented pseudo labels, and progressively increasing self-learning influence.
  • Results: Without accessing source samples during training, SFDA is reported to achieve higher performance than conventional models trained with labeled source data.The contribution statement identifies this as a central empirical result.

2 Source data-free domain adaptation

SFDA performs domain adaptation without source samples by using a frozen pre-trained source model and unlabeled target data. It progressively adapts a target model through prototypes, pseudo-labeling, confidence filtering, and source-domain regularization.

  • Problem setup: SFDA differs from UDA by forbidding access to source samples during training and using a pre-trained source model instead.The protocol is motivated by source-data privacy constraints.
  • SFDA framework: The framework uses a frozen source model and a trainable target model initialized from the source model.The target model evolves with two losses, while the source model remains fixed.
  • Adaptive prototype memory: Adaptive Prototype Memory selects multiple class prototypes from target samples using class-wise self-entropy and an adaptive threshold without hyperparameters.The threshold is the largest class-specific minimum entropy, allowing different numbers of prototypes across classes.
  • Pseudo labeling: Target samples receive pseudo labels by comparing their embedded features with class prototypes and selecting the most similar class.Each class may contain a different number of prototypes in memory.
  • Confidence-based filtering: Set-to-set distance compares the nearest and second-nearest prototype sets to filter unreliable pseudo labels without tunable filtering hyperparameters.A sample is reliable when the prototypes of its most similar class are closer than those of the second most similar class.
  • Optimization: Training combines source-oriented pseudo-label supervision with self-learning loss, computes loss only for confident samples, and gradually increases their balance from 0 to 1.The source-oriented loss preserves source-domain information and regularizes the target model.

3 Experiments

SFDA is evaluated against source-data-using domain adaptation methods across three benchmark datasets, with analyses of filtering, feature alignment, reliable samples, update periods, and loss balancing.

  • Experimental setup: SFDA is evaluated on Office-31, Office-Home, and VisDA-C under the official UDA protocol without directly using source samples during training.Office-31 includes six transfers, Office-Home includes twelve, and VisDA-C evaluates a synthetic-to-real scenario.
  • Experimental results: SFDA achieves comparable performance to state-of-the-art UDA methods across datasets and gains 12.2% over the source-only baseline on VisDA-C.Office-31 results are reported as mean and standard deviation over five runs.
  • Empirical analysis: Confidence filtering initially uses a small portion of target samples, but the number of valid training samples gradually increases during training.The filtering scheme is reported to address uncertainty in imperfect pseudo labels.
  • Empirical analysis: SFDA produces better source-target feature alignment than ResNet-50 in t-SNE visualization on the A ⇒W Office-31 transfer.The comparison visualizes features from the baseline ResNet-50 and the proposed SFDA.
  • Empirical analysis: Reliable samples are defined using entropy below 0.2, initially covering about 30% of samples and eventually exceeding 50% as training progresses.The accuracy of pseudo labels for reliable samples also increases during training.
  • Empirical analysis: Shorter APM update periods improve accuracy but increase computation, while dynamic α outperforms fixed α settings.Update periods above 1000 degrade performance; the authors use an update period of 100 across datasets and gradually increase α from 0 to 1.

4 Related work

The paper situates SFDA among unsupervised domain adaptation, fine-tuning, and knowledge distillation. These approaches differ in how they address domain discrepancy, transfer pretrained knowledge, and use data across domains.

  • Unsupervised domain adaptation: UDA methods reduce source–target discrepancy through distribution matching or domain-adversarial training.Examples include MMD-based domain confusion, distribution-matching variants, and domain-adversarial methods.
  • Fine-tuning: Fine-tuning adapts pretrained models to specific target tasks and has been widely applied across vision and video benchmarks.The passage cites ImageNet-pretrained VGG or ResNet models and Kinetics-trained 3D CNNs as examples.
  • Knowledge distillation: SFDA resembles knowledge distillation because two models are involved, but KD generally transfers between teacher and student on the same dataset.Unlike SFDA, knowledge distillation generally does not involve covariate shift between source and target domains.

5 Conclusion

The conclusion presents SFDA as a source-data-free domain-adaptation paradigm that adapts a target model from a pretrained source model. It reports higher performance than conventional source-data methods while identifying closed-set adaptation as the addressed scenario.

  • 5 Conclusion: SFDA uses a pretrained source model instead of directly using source data to adapt a target model.The framework uses target-oriented pseudo labels for self-learning and source-oriented pseudo labels to prevent self-bias.
  • 5 Conclusion: Despite not directly accessing source data, SFDA achieves higher performance than conventional models trained with source data.The conclusion attributes this claim to extensive empirical analysis in SFDA scenarios.
  • 5 Conclusion: The study addresses closed-set domain adaptation, where source and target domains have identical class sets.The authors note that differing class sets could affect class-prototype selection in real-world scenarios.

Broader Impact

The broader-impact discussion frames SFDA as a way to reduce privacy concerns and widen participation in adaptation research when source data are sensitive or large-scale. It also notes reduced flexibility for modifying source data.

  • Broader Impact: SFDA is presented as a way for enterprises and government organizations to address privacy concerns involving labeled source datasets.The discussion focuses especially on source data that may contain sensitive attributes.
  • Broader Impact: Using pretrained models may broaden participation in domain-adaptation research when source datasets are large-scale and difficult to access directly.The passage connects this benefit to the growing practice of sharing pretrained models.
  • Broader Impact: SFDA potentially limits flexibility for modifying or adding source data, including annotation and preprocessing.The limitation is most relevant when more flexible domain adaptation is required or expected.

Appendix (A): Reimplementation

The appendix explains that early UDA results require reimplementation for fair comparison because they used Caffe while newer methods used PyTorch. It reports higher reimplemented results than earlier reports.

  • Appendix (A): Reimplementation: Early UDA methods used Caffe whereas recent works, including SFDA, used PyTorch, making direct reported-performance comparisons potentially unfair.The appendix therefore reimplements ResNet-50, DAN, DANN, and MSTN using PyTorch.
  • Appendix (A): Reimplementation: The experimental setup uses dataset-specific iteration limits, batch size 32, SGD optimization, and learning-rate scheduling.The stated schedule is lrp = lr0(1 + α · p)^−β.
  • Appendix (A): Reimplementation: Reimplemented PyTorch results are higher than previously reported Caffe results, especially for the ResNet baseline.Tables 4 and 5 compare reported and reimplemented accuracy on Office-31 and Office-Home.

Appendix (B): Additional ablation on VisDA-C

The VisDA-C ablation shows that combining source-regularization and self-learning is preferable to relying on either loss alone, while dynamic α performs best overall. The analysis is reported for ResNet-101 classification accuracy.

  • Trade-off parameter α: Using only source-regularization loss (α = 0) or self-learning loss (α = 1) degrades performance because each relies on one-sided information.The comparison covers static α values from 0.0 to 1.0.
  • Trade-off parameter α: Combining source and target information produces promising results with static α settings.
  • Trade-off parameter α: Dynamic α outperforms all static α settings on VisDA-C.
  • Evaluation setting: Table 6 reports classification accuracy (%) on VisDA-C using ResNet-101.

Appendix (C): Confidence-base filtering

Confidence-base filtering estimates pseudo-label confidence by comparing a target sample's embedded feature with prototype sets for the two most similar classes. Hausdorff-distance simplifications exploit the sets' discrete structure and the singleton target set.

  • Filtering objective: Confidence-base filtering estimates pseudo-label confidence from corner cases between prototype sets.
  • Set definitions: Q is the singleton set containing the embedded feature f_t of target sample x_t.
  • Set definitions: M_t1 contains multiple prototypes from the class most similar to x_t according to s_c(x_t).
  • Set definitions: M_t2 contains multiple prototypes from the second most similar class.
  • Hausdorff-distance formulation: Because Q, M_t1, and M_t2 are closed discretized sets, suprema and infima become maxima and minima, respectively.Because Q is a singleton containing f_t, the expressions can be simplified by substituting f_t for q.
  • Hausdorff-distance formulation: For the second most similar class, Hausdorff distance is modified to select the opposite corner case.
Loading 2007.01524v4…