PaperlayerView exact revision on arXiv

Source-linked AI summary

Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer

Jian Liang, Dapeng Hu, Yunbo Wang, Ran He, Jiashi Feng

arXiv:2012.07297v3cs.CVcs.LG

TL;DR

The paper addresses UDA when source data are unavailable because they cannot be accessed or shared. It proposes SHOT and SHOT++, which adapt target representations and transfer confident labels, achieving comparable or better state-of-the-art results across multiple UDA settings.

  • Problem

    Existing UDA methods require source-data access, creating a privacy-relevant gap for confidential or non-shareable source datasets.

  • Method

    SHOT freezes the source classification hypothesis while learning a target-specific feature encoder with information maximization and self-supervision; SHOT++ adds confidence-based labeling transfer with semi-supervised learning.

  • Results

    SHOT and SHOT++ achieve results comparable to or better than the state-of-the-art across three UDA scenarios and semi-supervised domain adaptation on digit and object-recognition tasks.

  • Takeaways & Limitations

    Source-data-absent UDA can be pursued by reusing a trained source model and exploiting unlabeled target-domain information through representation and labeling transfer.

  • Takeaways & Limitations

    The formulation assumes a K-way visual classification task with aligned source and target task mappings.

Abstract

from arXiv · show

Unsupervised domain adaptation (UDA) aims to transfer knowledge from a related but different well-labeled source domain to a new unlabeled target domain. Most existing UDA methods require access to the source data, and thus are not applicable when the data are confidential and not shareable due to privacy concerns. This paper aims to tackle a realistic setting with only a classification model available trained over, instead of accessing to, the source data. To effectively utilize the source model for adaptation, we propose a novel approach called Source HypOthesis Transfer (SHOT), which learns the feature extraction module for the target domain by fitting the target data features to the frozen source classification module (representing classification hypothesis). Specifically, SHOT exploits both information maximization and self-supervised learning for the feature extraction module learning to ensure the target features are implicitly aligned with the features of unseen source data via the same hypothesis. Furthermore, we propose a new labeling transfer strategy, which separates the target data into two splits based on the confidence of predictions (labeling information), and then employ semi-supervised learning to improve the accuracy of less-confident predictions in the target domain. We denote labeling transfer as SHOT++ if the predictions are obtained by SHOT. Extensive experiments on both digit classification and object recognition tasks show that SHOT and SHOT++ achieve results surpassing or comparable to the state-of-the-arts, demonstrating the effectiveness of our approaches for various visual domain adaptation problems. Code is available at \url{https://github.com/tim-learn/SHOT-plus}.

1 INTRODUCTION

The paper addresses source-data-absent UDA by adapting a target feature encoder using a fixed source classifier, then transfers confident labels to less-confident target samples. SHOT and SHOT++ achieve competitive or superior results across several domain-adaptation settings.

  • Motivation: Existing UDA methods typically require source-data access, which can conflict with privacy policies when data contain private information.The paper motivates hypothesis-based transfer as an alternative when source data cannot be shared.
  • Results: Experiments show competitive or state-of-the-art results across closed-set, partial-set, and multi-source UDA, with further evidence in semi-supervised domain adaptation.The reported evaluation covers several benchmark scenarios and supports the methods’ versatility.
  • SHOT: SHOT adapts to the target domain with only a provided source model, retaining the source classifier while learning a target-specific feature encoding module.The fixed classifier represents the source hypothesis used to guide target representation learning.
  • SHOT: SHOT uses information maximization and two self-supervised auxiliary tasks to exploit unlabeled target-domain knowledge and improve target representations.These objectives address potentially incorrect matching between target features and the source hypothesis.
  • SHOT++: SHOT++ divides target samples into high-confidence ‘easy’ and low-confidence ‘hard’ splits, then uses semi-supervised learning to transfer labeling information from easy to hard samples.The split is determined using an adaptive confidence threshold after SHOT predictions.
  • Extensions: The paper extends earlier work with relative-rotation self-supervision and a labeling-transfer strategy that can operate with a black-box source model.Labeling transfer can also be combined with hypothesis transfer to produce SHOT++.

2 RELATED WORK

Related work covers domain adaptation, hypothesis transfer, self-supervised learning, and semi-supervised learning. The paper positions SHOT as an end-to-end source-data-absent approach that uses flexible feature learning and confidence-based intra-domain supervision.

  • Domain Adaptation: UDA research includes shallow distribution-alignment methods, deep transferable representations, and scenarios distinguished by relationships between source and target label spaces.The cited scenarios include closed-set, partial-set, open-set, and universal UDA.
  • Hypothesis Transfer Learning: Hypothesis transfer learning transfers prior knowledge through source hypotheses rather than source data but traditionally requires some labeled target samples.Recent source-data-absent methods use encoded source information, while the paper emphasizes end-to-end feature learning.
  • Self-supervised Learning: Self-supervised learning generates labels from unlabeled data through pretext tasks, with image examples including colorization, relative position, rotation, and jigsaw prediction.The paper uses self-supervision in UDA to support domain alignment.
  • Semi-supervised Learning: Semi-supervised learning methods include pseudo-labeling, entropy minimization, tri-training, virtual adversarial training, and MixMatch.The paper instead treats confident target samples as labeled and the remaining samples as unlabeled for intra-domain learning.

3 METHOD

The method assumes a K-way visual classification task with a pretrained source model but no source-data access during adaptation. It transfers the source model and learns target predictions under this constrained setting.

  • Problem Setting: The task is K-way visual classification with labeled source data, unlabeled target data, and the goal of predicting target labels.The source and target tasks are assumed to share the same mapping structure.
  • Problem Setting: The source model consists of a feature encoding module and a hypothesis module trained from source data before transfer to the target domain.After this initial training, adaptation proceeds without accessing the source data.

3.1 Source Model Generation

The source model is trained with cross-entropy and label smoothing, producing a feature encoder and classifier hypothesis for later transfer.

  • 3.1 Source Model Generation: The source classification model is learned by minimizing cross-entropy over labeled source examples.The loss uses the source model’s soft-max outputs and one-hot source labels.
  • 3.1 Source Model Generation: Label smoothing replaces each one-hot target with k_k = (1 − α)q_k + α/K, using α = 0.1.The smoothed labels are intended to encourage tight and evenly separated feature clusters.
  • 3.1 Source Model Generation: The source model consists of a feature encoding module and a classifier module called the hypothesis.The classifier maps encoded features to class outputs and is later transferred without source-data access.

3.2 Hypothesis Transfer with Information Maximization

SHOT adapts without source data by learning a target-specific encoder under the frozen source classifier, aligning target outputs through information maximization.

  • 3.2 Hypothesis Transfer with Information Maximization: SHOT learns a domain-specific target encoder while fixing the source classifier hypothesis.The shared classifier is used to transfer information from the unseen source distribution to target features.
  • 3.2 Hypothesis Transfer with Information Maximization: Figure 2 compares t-SNE features for unseen source and target data across five classes, showing improved alignment with SHOT-IM but some wrong hypothesis matching.The comparison is against the source-model-only method.
  • 3.2 Hypothesis Transfer with Information Maximization: Output alignment requires target features passed through the shared classifier to produce outputs similar to one-hot class encodings.This output constraint is presented as a necessary condition for feature alignment when source features cannot be observed.
  • 3.2 Hypothesis Transfer with Information Maximization: Information maximization makes target predictions individually certain and globally diverse.Its diversity term avoids the trivial solution in which all unlabeled target samples receive the same class encoding.

3.3 Hypothesis Transfer with Self-supervised Learning

SHOT supplements information maximization with self-supervised semantic and structure-aware learning, then uses labeling transfer to refine uncertain target predictions with semi-supervised learning.

  • 3.3 Hypothesis Transfer with Self-supervised Learning: The complete hypothesis-transfer framework freezes the source hypothesis and learns the target encoder using pseudo labels and rotation labels.The rotation classifier is added after the target feature encoder as an auxiliary self-supervised component.
  • 3.3 Hypothesis Transfer with Self-supervised Learning: Self-supervised learning is motivated by errors from semantically wrong matches between target representations and the source hypothesis.The paper uses it to focus representation learning on semantically meaningful features.
  • 3.3 Hypothesis Transfer with Self-supervised Learning: SHOT uses target-domain class prototypes to generate more reliable pseudo labels through nearest-centroid assignment.The centroids are obtained in an unsupervised manner and characterize target category distributions.
  • 3.3 Hypothesis Transfer with Self-supervised Learning: Relative-rotation prediction supplies image-specific self-supervision while avoiding ambiguity that makes absolute rotation unsuitable for some tasks.The rotation classifier predicts one of four relative rotation degrees from concatenated image-pair features.
  • 3.3 Hypothesis Transfer with Self-supervised Learning: SHOT++ divides target predictions into high-confidence easy and low-confidence hard splits, then applies labeling transfer to improve the less-confident predictions.The strategy uses only prediction information from SHOT for the split and follows with semi-supervised learning.

3.4 Labeling Transfer with Semi-supervised Learning

Labeling transfer uses SHOT predictions to divide target samples by confidence, treating reliable samples as labeled and refining less-confident predictions with semi-supervised learning.

  • Confidence-based split: SHOT prediction entropies divide target samples into high-confidence easy and low-confidence hard splits.The easy split is treated as reliable labeling information, while the hard split contains potentially inaccurate predictions.
  • Confidence-based split: The method treats the high-confidence split as labeled data and the low-confidence split as unlabeled data.
  • Semi-supervised refinement: MixMatch refines predictions for the unlabeled split using semi-supervised learning.The approach combines consistency regularization, entropy minimization, and MixUp regularization.
  • Confidence-based split: An adaptive threshold determines the labeled-split proportion from the entropy distribution.For each class, samples with the smallest entropy values are selected into the labeled split.
  • Two-stage framework: The resulting SHOT++ framework first performs hypothesis transfer, then applies labeling transfer using only SHOT predictions for the split.The feature encoder is initialized from SHOT, while the classification module is newly initialized and no longer frozen.

3.5 Extension to Multi-source Domain Adaptation

For multi-source domain adaptation, SHOT is run separately on each source-target pair, and their probabilistic predictions are combined to classify target samples.

  • Multi-source extension: SHOT and SHOT-IM are applied independently to every source-target pair in the multi-source setting.
  • Multi-source extension: Probabilistic scores from the source-target pairs are summed, and argmax produces the final target predictions.

3.6 Extension to Partial-set Domain Adaptation

For partial-set adaptation, SHOT modifies its label-distribution and prototype handling because target classes form a subset of the source classes.

  • Problem setting: Partial-set domain adaptation has a target label space that is a subset of the source label space.
  • Objective adjustment: SHOT removes the uniform-label-distribution objective because it is unsuitable when the target contains only some source classes.The affected term is the diversity-promoting loss L_div.
  • Prototype handling: SHOT discards target centroids smaller than T_c because tiny centroids may represent empty classes in partial-set adaptation.

3.7 Extension to Semi-supervised Domain Adaptation

For semi-supervised domain adaptation, SHOT combines supervised learning on labeled target data with its complete adaptation loss on unlabeled target data.

  • Problem setting: SSDA includes some labeled target data, unlike unsupervised domain adaptation.
  • Training objective: The method uses supervised training loss for labeled target samples and the complete SHOT loss for unlabeled target samples.
  • Target centroids: Labeled target data are also included when computing target-specific centroids.
  • Network architecture: Weight normalization keeps classifier weight-vector norms equal, while batch normalization reduces internal dataset shift across domains.The architecture rationale links classifier weight normalization to feature-to-class weight distances and batch normalization to shared first- and second-order moments.

4 EXPERIMENTS

Experiments evaluate SHOT and SHOT++ across digit, object-recognition, multi-source, partial-set, semi-supervised, and special-case adaptation settings, with ablations and analyses examining their mechanisms and stability.

  • Digit recognition: SHOT obtains the best mean accuracy on each evaluated digit adaptation task and outperforms source-model-only and SHOT-IM.Self-supervised learning improves SHOT over information maximization alone.
  • Object recognition: 88.8% average accuracy makes SHOT competitive with MDD and BNM on Office, while SHOT++ matches TransNorm and GVB-GD in mean accuracy.SHOT is best on D→A and W→A; labeling transfer benefits cross-domain object recognition.
  • Object recognition: 73.0% average accuracy raises Office-Home performance from GVB-GD’s 70.4%, with SHOT++ best on 11 of 12 tasks.For Re→Ar, SHOT++ reaches 73.7% versus GVB-GD’s 74.6%; labeling transfer adds nearly one point to average accuracy.
  • Object recognition: 85.5% per-class accuracy lets SHOT exceed STAR’s 82.7% on VisDA-C with ResNet-101, while labeling transfer improves overall results but decreases truck accuracy.SHOT++ wins 8 of 12 tasks; the authors attribute the truck decrease to error in the labeled split.
  • Additional settings: SHOT outperforms competing methods in multi-source, partial-set, semi-supervised, and off-the-shelf-model settings, while SHOT++ further improves partial-set averages.On PDA, SHOT++ raises averages from 79.5% to 79.9% and from 73.6% to 77.0%; on VisDA-C with ResNet-50, SHOT++ reaches 77.1% versus SHOT’s 76.7%.
  • Model analysis: Ablations indicate that diversity and self-supervised pseudo-labeling terms contribute to performance, while labeling transfer improves the complete framework.BN and WN benefit adaptation, BN helps more than WN, and label smoothing improves source and target accuracies.

5 CONCLUSION

The conclusion presents SHOT as a source-data-absent UDA framework requiring only a trained source model, and SHOT++ extends it with labeling transfer. Experiments report comparable or better results than state-of-the-art methods across several adaptation settings.

  • SHOT learns target-specific features using information maximization and self-supervised learning to fit a source hypothesis without source data.
  • SHOT++ adds a semi-supervised labeling transfer strategy that exploits intra-domain information after hypothesis transfer.
  • Experiments find SHOT and SHOT++ comparable to or better than state-of-the-art methods across unsupervised and semi-supervised domain adaptation scenarios.
Loading 2012.07297v3…