PaperlayerView exact revision on arXiv

Source-linked AI summary

Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation

Jian Liang, Dapeng Hu, Jiashi Feng

arXiv:2002.08546v6cs.CVcs.LG

TL;DR

Source-data access makes unsupervised domain adaptation difficult for private, decentralized data. SHOT freezes a trained source classifier and learns target features using information maximization and self-supervised pseudo-labeling; it achieves competitive or state-of-the-art performance across several adaptation settings, including reported Office-Home gains.

  • Problem

    Existing unsupervised domain adaptation methods require source-data access during learning, motivating adaptation when only a trained source model is available for private, decentralized data.

  • Method

    SHOT freezes the source classifier hypothesis and learns a target-specific feature module using information maximization and self-supervised pseudo-labeling.

  • Results

    71.8% average accuracy versus 67.6% in Office-Home closed-set adaptation, 79.3% versus 71.8% for partial transfer, and 72.8% versus 69.5% for open-set adaptation.

  • Takeaways & Limitations

    SHOT provides a source-data-free unsupervised domain adaptation framework evaluated across closed-set, partial-set, and open-set tasks, with competitive and state-of-the-art results reported for digit and object recognition.

  • Takeaways & Limitations

    The method assumes a K-way classification setting, and digit experiments select the source hypothesis using the source test set as validation.

Abstract

from arXiv · show

Unsupervised domain adaptation (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain. Prior UDA methods typically require to access the source data when learning to adapt the model, making them risky and inefficient for decentralized private data. This work tackles a practical setting where only a trained source model is available and investigates how we can effectively utilize such a model without source data to solve UDA problems. We propose a simple yet generic representation learning framework, named \emph{Source HypOthesis Transfer} (SHOT). SHOT freezes the classifier module (hypothesis) of the source model and learns the target-specific feature extraction module by exploiting both information maximization and self-supervised pseudo-labeling to implicitly align representations from the target domains to the source hypothesis. To verify its versatility, we evaluate SHOT in a variety of adaptation cases including closed-set, partial-set, and open-set domain adaptation. Experiments indicate that SHOT yields state-of-the-art results among multiple domain adaptation benchmarks.

1. Introduction

The paper addresses unsupervised domain adaptation when source data are unavailable, proposing SHOT to adapt a trained source model through target representation learning. SHOT is evaluated across closed-set, partial-set, and open-set settings, with strong benchmark results.

  • Motivation: Existing domain adaptation methods typically access source data during adaptation, creating transmission inefficiency and potential privacy-policy violations for decentralized private data.The proposed setting transfers only a trained source model to the unlabeled target domain.
  • Method: SHOT freezes the source classifier hypothesis and learns a target-specific feature encoder to align target representations with the source hypothesis without source data.The framework assumes a shared decomposition into feature encoding and classifier modules across domains.
  • Method: SHOT augments information maximization with self-supervised pseudo-labeling to improve target representation learning when source-classifier pseudo-labels may be noisy.The pseudo-labeling strategy uses intermediate target-domain class-wise prototypes.
  • Evaluation: SHOT is applied to closed-set, partial-set, and open-set unsupervised domain adaptation tasks and achieves state-of-the-art results across multiple benchmarks.The reported evaluation covers several adaptation cases rather than a single domain-adaptation setting.
  • Evaluation: 71.8% average accuracy improves on 67.6% in Office-Home closed-set adaptation, while 79.3% improves on 71.8% for partial transfer and 72.8% improves on 69.5% for open-set adaptation.These comparisons are reported as SHOT versus the cited best prior methods on Office-Home.

2. Related Work

The related work spans unsupervised domain adaptation, hypothesis transfer learning, pseudo-labeling, and federated learning. SHOT differs by addressing unsupervised adaptation with only a source model rather than labeled target data, multiple hypotheses, or source gradients.

  • Unsupervised Domain Adaptation: Unsupervised domain adaptation methods include moment matching, adversarial alignment, and sample-based approaches for transferring knowledge across domains without target labels.The cited applications include object recognition, object detection, semantic segmentation, and sentiment classification.
  • Hypothesis Transfer Learning: Hypothesis Transfer Learning generally assumes the target hypothesis relates to a source hypothesis but commonly requires labeled target examples, limiting its use in semi-supervised adaptation.This requirement distinguishes conventional HTL from SHOT's unlabeled-target setting.
  • Pseudo Labeling: Pseudo-labeling assigns labels to unlabeled data using predicted probabilities and supports fine-tuning or discriminative distribution alignment in domain adaptation.Prior DA methods use pseudo-labels either as regularization or within adaptation modules.
  • Federated Learning: Federated learning transfers knowledge across decentralized devices without exchanging data samples, while federated domain adaptation can aggregate source gradients but may not address the single-source setting.SHOT instead operates with a pre-trained source model and no source-data access.

3. Method

SHOT addresses UDA when only a trained source model is available by freezing its classifier and adapting a target-specific feature encoder without source data. It combines information maximization with self-supervised pseudo-labeling to align target representations with the source hypothesis and supports several UDA settings.

  • SHOT formulates source-model transfer for K-way UDA using a trained source function and unlabeled target data, without retaining source data during adaptation.
  • The source network decomposes into a feature encoder and classifier, while prior alignment methods require source access and shared feature encoders during adaptation.The source model is fs(x) = hs(gs(x)), with gs producing d-dimensional features and hs producing K-dimensional outputs.
  • SHOT keeps the source classifier fixed and learns a target encoder so target features are classifiable by the source hypothesis and aligned with the unseen source representation distribution.The target model uses ht = hs and learns gt: Xt → Rd.
  • SHOT with Information Maximization: Information maximization makes target predictions individually certain and globally diverse, avoiding the trivial solution in which all unlabeled samples receive the same one-hot label.The objective combines conditional entropy minimization with diversity promotion through the mean target output.
  • Self-supervised Pseudo-labeling: Because information maximization can match targets to the wrong source hypothesis, SHOT derives cleaner pseudo-labels from target class-wise centroids and trains the encoder with them.The centroids are computed from an intermediate target model, and nearest-centroid pseudo-labels are updated in multiple rounds, with one update reported as sufficient.
  • The full SHOT objective combines information maximization and pseudo-label supervision while freezing the source hypothesis; the framework also extends to partial-set and open-set domain adaptation.The pseudo-label term is balanced by β > 0.

4. Experiments

SHOT is evaluated across digit, object-recognition, closed-set, partial-set, open-set, multi-source, and multi-target adaptation settings. Results generally show competitive or state-of-the-art performance without direct access to source data, with gains on several benchmarks and identifiable sensitivity to target-domain size.

  • Experimental settings: SHOT is evaluated on Digits, Office, Office-Home, VisDA-C, partial-set, open-set, multi-source, and multi-target domain adaptation benchmarks.The experiments cover vanilla closed-set adaptation and asymmetric or multi-domain settings.
  • Digit recognition: SHOT obtains the best or second-best mean accuracy on each of three digit-recognition tasks and exceeds prior work in average accuracy.SHOT also performs slightly better than SHOT-IM, while approaching target-supervised results on MNIST→USPS.
  • Object recognition: SHOT performs best on the D→A and W→A Office tasks but worse than the prior state-of-the-art method on other reported object-recognition tasks.The authors attribute this pattern to the small size of D and W when they serve as target domains.
  • Ablation studies: The self-supervised pseudo-labeling strategy consistently outperforms naive pseudo-labeling, while combining diversity promotion with entropy minimization improves over the pseudo-labeling baseline.The ablation identifies the diversity-promoting objective Ldiv as important for the combined configuration.
  • Ablation studies: Batch normalization boosts source-model-only results, whereas weight normalization and label smoothing can reduce them; combining components improves source-model-only and SHOT-IM performance.SHOT-IM achieves its best performance when all three components are used together.
  • Beyond vanilla closed-set: SHOT achieves the best results in both multi-source and multi-target settings and slightly exceeds ETN on ImageNet→Caltech without accessing source data.The multi-source comparison also reports that FADA performs much worse than SHOT.

5. Conclusion

SHOT enables unsupervised domain adaptation using only a trained source model, without accessing private source data. It learns target-specific features through information maximization and self-supervised pseudo-labeling, achieving competitive or state-of-the-art performance.

  • 5. Conclusion: SHOT performs unsupervised domain adaptation without accessing source data, using only a well-trained source model.This setting addresses decentralized data and privacy constraints.
  • 5. Conclusion: SHOT freezes the source classifier and learns a target-specific feature module using information maximization and self-supervised pseudo-labeling.The algorithm initializes the target feature extractor from the source model and updates it using target pseudo-labels.
  • 5. Conclusion: Experiments on digit and object recognition show that SHOT achieves competitive and even state-of-the-art performance.The conclusion summarizes results across the evaluated recognition settings.

A. UDA scenarios and Network Architecture

UDA includes closed-set, open-set, and partial-set scenarios with different relationships between source and target label spaces. Digit experiments evaluate SHOT on MNIST, USPS, and SVHN using the architectures shown in Figure 4.

  • UDA scenarios: Closed-set DA assumes that source and target domains share the same label space.Open-set DA adds unseen target classes, whereas partial-set DA has a source label space that subsumes the target label space.
  • UDA scenarios: Open-set DA includes unseen target classes, while partial-set DA considers a target label space contained within the source label space.These settings extend beyond the shared-label-space assumption of closed-set DA.
  • Network Architecture: Digit experiments use architectures for MNIST↔USPS and SVHN→MNIST adaptation tasks.Figure 4 distinguishes the architecture used for MNIST↔USPS from that used for SVHN→MNIST.

B. Details of SHOT for Partial-set DA (PDA)

For partial-set DA, SHOT adapts its objectives and pseudo-labeling to the target domain’s sparse label distribution. It removes diversity promotion and discards tiny class centroids before evaluating centroid-size effects on Office-Home.

  • Objective design: The diversity-promoting objective Ldiv is removed for PDA because the target contains only some source classes and therefore has a sparse label distribution.Uniform target-label distributions are suitable for closed-set DA but not for partial-set DA.
  • Pseudo-labeling: SHOT discards PDA centroids smaller than threshold Tc because tiny centroids should be treated as empty.The effect of Tc is evaluated using average accuracy on Office-Home.

C. Details of SHOT for Open-set DA (ODA)

For open-set DA, SHOT must identify unknown target samples while adapting features with a fixed classifier. It uses entropy-based uncertainty and confidence thresholding, with Figure 6 tracking scores across training epochs on Ar→Cl.

  • Unknown-sample rejection: SHOT uses confidence thresholding to reject unknown target samples because its final classifier is difficult to change during open-set adaptation.Uncertainty is computed from network-output entropy and normalized by log K.
  • Open-set metrics: The open-set formulation treats the K source-known classes and the (K+1)-th class as unknown.The reported OS* score measures per-class accuracy on the known classes.
  • Evaluation: Figure 6 tracks different SHOT scores against the number of training epochs for the Ar→Cl open-set DA task on Office-Home.The experiment uses ResNet-50.
Loading 2002.08546v6…