Cycle Self-Training for Domain Adaptation

Hong Liu; Jianmin Wang; Mingsheng Long

Source-linked AI summary

Cycle Self-Training for Domain Adaptation

Hong Liu, Jianmin Wang, Mingsheng Long

arXiv:2103.03571v3Mar 5, 2021cs.LGcs.CV

TL;DR

UDA must transfer knowledge from labeled source data to an unlabeled target domain despite distributional shift, which makes standard pseudo-labels unreliable. The paper proposes Cycle Self-Training, which alternates forward pseudo-label generation with reverse target-classifier and representation updates, and reports stronger benchmark performance alongside theoretical guarantees and hard-case recovery.

Problem
Distributional shift makes target pseudo-labels unreliable, while standard selection strategies can be brittle and require task-specific threshold or weight tuning.
Method
Cycle Self-Training alternates source-model pseudo-label generation, target-classifier training, and shared-representation updates that improve target-classifier performance on source data, using Tsallis entropy regularization.
Results
CST outperformed previous state-of-the-art methods in 21 out of 25 object-recognition and sentiment-classification tasks, with theoretical guarantees and hard cases where it recovered target ground truth.
Takeaways & Limitations
CST explicitly addresses distribution shift by learning pseudo-labels that generalize across domains rather than relying on hard-to-tweak selection protocols.
Takeaways & Limitations
The evaluation focuses on standard UDA benchmarks, while future work is left to study other semi-supervised techniques under distribution shift.

Abstract

from arXiv · show

Mainstream approaches for unsupervised domain adaptation (UDA) learn domain-invariant representations to narrow the domain shift. Recently, self-training has been gaining momentum in UDA, which exploits unlabeled target data by training with target pseudo-labels. However, as corroborated in this work, under distributional shift in UDA, the pseudo-labels can be unreliable in terms of their large discrepancy from target ground truth. Thereby, we propose Cycle Self-Training (CST), a principled self-training algorithm that explicitly enforces pseudo-labels to generalize across domains. CST cycles between a forward step and a reverse step until convergence. In the forward step, CST generates target pseudo-labels with a source-trained classifier. In the reverse step, CST trains a target classifier using target pseudo-labels, and then updates the shared representations to make the target classifier perform well on the source data. We introduce the Tsallis entropy as a confidence-friendly regularization to improve the quality of target pseudo-labels. We analyze CST theoretically under realistic assumptions, and provide hard cases where CST recovers target ground truth, while both invariant feature learning and vanilla self-training fail. Empirical results indicate that CST significantly improves over the state-of-the-arts on visual recognition and sentiment analysis benchmarks.

1 Introduction

UDA transfers supervision from a labeled source domain to an unlabeled target domain despite distributional shift. The paper shows why standard self-training struggles and introduces Cycle Self-Training to make pseudo-labels generalize across domains.

UDA adapts models trained on labeled source data to unlabeled target data under distributional shift.
Feature adaptation narrows source–target feature-distribution distance, but invariant representations have intrinsic limitations under label shift and support shift.
Standard self-training jointly trains on source ground-truths and target pseudo-labels, but selected pseudo-labels can be brittle under domain shift.
Domain shift makes pseudo-label distributions diverge sharply from target ground truth and reduces the accuracy of entropy- and confidence-based selection.
CST cycles between training a target classifier with target pseudo-labels and updating shared representations so that classifier performs well on source data.
21 out of 25 tasks improved over previous state-of-the-art methods, while theory and hard cases support CST’s target-performance guarantees.

2 Preliminaries

The paper formalizes UDA with source and target distributions, feature extractors, classifiers, and expected risks, then reviews feature adaptation and self-training. VisDA-2017 analyses show that domain shift biases pseudo-labels, limits their denoising, and weakens selection criteria.

UDA considers source distribution P and target distribution Q over input-label pairs, with a model combining feature extractor h_φ and classifier g_θ.
Feature adaptation trains on source data while matching source and target representation distributions using a distribution distance.
Standard self-training generates target pseudo-labels with a source-trained model and retrains using source labels together with those pseudo-labels.
2.1 Limitations of Standard Self-Training: On VisDA-2017, domain shift biased pseudo-labels toward several classes, making self-training risky because many target examples could be misclassified.
2.1 Limitations of Standard Self-Training: dTV remained 0.26 throughout training, upper-bounding pseudo-label accuracy at 0.74 despite a decreasing pseudo-label error rate.
2.1 Limitations of Standard Self-Training: Under domain shift, pseudo-label selection quality fell from AUC=0.89 without shift to AUC=0.78, weakening entropy- and confidence-based filtering.

3 Approach

CST alternates forward pseudo-label generation with a reverse update that tests target-trained predictions on source data, refining labels without hand-tuned selection. Tsallis entropy regularization further controls target uncertainty during iterative optimization.

3.1 Cycle Self-Training: CST iterates between generating target pseudo-labels with a source classifier and updating shared representations so a target classifier generalizes to the source domain.The method formulates this cycle as a bi-level optimization procedure with inner target-head training and outer representation updates.
3.1 Cycle Self-Training: Traditional confidence selection is brittle under domain shift because deep-network confidence can be miscalibrated and threshold tuning is expensive.CST instead aims to refine pseudo-labels through the model’s training cycle without cumbersome selection or thresholding.
3.1 Cycle Self-Training: The reverse step trains a target head on target pseudo-labels, then jointly enforces source performance for both the source and target heads through the cycle loss.The objective combines source loss for the source classifier with source loss for the target classifier trained from pseudo-labels.
3.2 Tsallis Entropy Minimization: Tsallis entropy generalizes Gibbs entropy through an entropic index α, recovering Gibbs entropy as α approaches 1 and Gini impurity when α = 2.CST minimizes a selected α-Tsallis entropy term on target predictions.
3.2 Tsallis Entropy Minimization: Smaller α penalizes uncertain predictions more strongly, whereas larger α allows similar class scores and can help correct initially incorrect pseudo-label dimensions.CST searches α over [2] by evaluating a target head trained with the resulting pseudo-labels on source data, updating α each epoch.
3.3 Algorithm: Algorithm 1 combines forward labeling, reverse target-head training, feature-extractor updates, source-head updates, and α-Tsallis entropy minimization across epochs.The feature extractor uses source losses for both heads plus target Tsallis entropy, while the source head uses source loss and target entropy.

4 Theoretical Analysis

The theoretical analysis links CST’s cycle objective to low target error under expansion, agreement, and robustness assumptions. A constructed hard case further shows that CST can recover target ground truth where feature adaptation and standard self-training select spurious solutions.

4.1 Theoretical Guarantee: CST’s theory assumes equal class priors, disjoint class-conditional supports, and a metric-based neighborhood structure for analyzing target error.These assumptions define the setting in which source and target distributions can be related through local expansion.
4.1 Theoretical Guarantee: The expansion assumption requires neighborhoods of sufficiently large class-conditional sets to grow by at least the smaller of a fixed amount and the set’s mass.This condition is imposed on the mixture of corresponding source and target class-conditional distributions.
4.1 Theoretical Guarantee: When source and target classifiers agree on target predictions and the target classifier is locally robust, minimizing cycle source error bounds the source classifier’s target error.The robustness-constrained formulation measures local prediction stability across source and target distributions.
4.1 Theoretical Guarantee: The finite-sample theorem bounds target error using the CST loss, an intrinsic distribution parameter, and function-class complexity measured through empirical Rademacher complexity.The analysis connects the training objective to population target error under the stated expansion setting.
4.2 Hard Case for Feature Adaptation and Standard Self-Training: In the quadratic-network hard case, x[2] is the target-working feature, while spurious coordinates can solve the source problem but fail on the target domain.The construction uses different coordinate relationships across source and target distributions, with realizability assumed.
4.2 Hard Case for Feature Adaptation and Standard Self-Training: Feature adaptation learns a spurious feature because the ideal target feature is farther away in feature space, while standard self-training chooses randomly among source-valid solutions.CST distinguishes the spurious solution because its pseudo-label-trained classifier cannot perform well on the source domain.
4.2 Hard Case for Feature Adaptation and Standard Self-Training: CST recovers the target ground truth in the constructed hard case with high probability once the target sample size exceeds a logarithmic threshold in the failure rate.The stated result is given for target dataset size nt > Θ(log 1/ξ) and probability at least 1 − ξ.

5 Experiments

Experiments evaluate CST across visual recognition and sentiment classification against feature adaptation, self-training, and combined baselines. Results show consistent gains, while analyses attribute improvements to more reliable pseudo-labels and Tsallis entropy regularization.

Results: CST significantly outperforms feature adaptation and self-training methods on 9 of 12 Office-Home tasks.Standard self-training methods suffer when large domain shifts degrade pseudo-label quality.
Method comparison: CST does not require manually setting confidence thresholds or reweighting.This contrasts with prior UDA self-training approaches that use hard-to-tweak selection protocols.
Results: CST surpasses state-of-the-art methods on VisDA-2017 with both ResNet-50 and ResNet-101 backbones.Combining feature adaptation with self-training performs worse than CST.
Results: CST outperforms all baselines on most sentiment-classification tasks while directly working with BERT after replacing the head and training objective.Most feature adaptation baselines improve over source-only training only marginally.
Ablation study: Replacing Tsallis entropy with standard entropy makes CST 3.7% worse, while CST is 5.4% better than FixMatch.FixMatch+Tsallis remains 3.6% behind CST and has a much larger pseudo-label total variation distance from ground truth.
Analysis: During training, pseudo-label error and total variation distance decay while target-classifier error on source data decreases, indicating progressive refinement.The analysis contrasts this behavior with standard self-training.
Analysis: Gibbs entropy produces largest softmax outputs close to 1, whereas Tsallis entropy avoids over-confidence.Over-confident incorrect predictions can be difficult to correct through self-training.

6 Related Work

Related work spans self-training for semi-supervised learning and feature-alignment methods for domain adaptation. The paper distinguishes CST by explicitly addressing domain shift rather than relying only on generic self-training or invariant representations.

Self-Training: Self-training uses unlabeled data by training on pseudo-labels generated by a source model.Related methods also study consistency regularization and data augmentation, but these approaches do not explicitly account for domain shift.
Domain Adaptation: Domain-adaptation methods commonly minimize source-target distribution distances to learn domain-invariant features.DANN approximates domain distance through adversarial learning, followed by methods that refine this alignment strategy.

7 Conclusion

The paper proposes cycle self-training to address distribution shift explicitly and provides theoretical and empirical support for the approach. It also identifies consistency regularization and data augmentation under shift as future directions.

Conclusion: Cycle self-training explicitly addresses distribution shift in domain adaptation and is shown to work provably under the expansion assumption.The paper also presents hard cases where feature adaptation and standard self-training fail.
Conclusion: Future work may study consistency regularization and data augmentation under distribution shift for domain adaptation.These techniques are identified as other semi-supervised learning directions beyond pseudo-labeling.

A.1 Proof of Theorem 1

Theorem 1 establishes a target-error bound for the source classifier when the target classifier agrees with source pseudo-labels and is locally robust, under structural assumptions on source and target distributions.

Assumptions: The analysis assumes matched class priors, disjoint supports across classes, and a constant-expansion property connecting neighborhoods within each class.The expansion assumption formalizes continuity of class sub-populations across domains.
Assumptions: CST measures target-classifier robustness as the probability that neighboring inputs receive different predictions.The robustness term R(ft) is evaluated over the average of source and target distributions.
Theorem 1: Theorem 1 bounds target error by source error of the target classifier, classifier disagreement, and robustness-dependent terms: ErrQ(fs) ≤ ErrP(ft) + c + 2q + ρ / min{ϵ, q}.The bound applies when target-source disagreement is at most c and robustness is at most ρ.
Proof: Lemma 1 shows robustness is guaranteed for a large proportion of sub-populations under the theorem’s conditions.The proof partitions classes according to whether their local inconsistency is below min{ϵ, q}.
Proof: Lemma 2 uses expansion to show that robust sub-populations have source and target errors that are simultaneously low or simultaneously high.The resulting alternatives are ErrPi(ft), ErrQi(ft) ∈ [0, 2q] or both in [1 − 2q, 1].
Theorem 2: Theorem 2 states that minimizing an objective consistent with CST yields small target loss for the source classifier with high probability.The result applies to solutions of the specified objective under Theorem 1 and Assumption 3.

A.4 Proof of Theorem 3

Theorem 3 constructs a quadratic-network domain-adaptation setting where feature adaptation and standard self-training select source solutions that generally fail on the target domain.

Feature adaptation: Feature adaptation prefers the non-transferable solutions because they produce a smaller source-target feature-distribution distance.The comparison is expressed through total variation between feature distributions.
Standard self-training: Standard self-training leaves the learned coordinate unchanged because it trains on source labels and target pseudo-labels generated by the same source solution.Since only l = 2 transfers, the probability of recovering the target ground truth is 1 / (d − 1).

A.5 Proof of Theorem 4

Theorem 4 shows that CST selects the transferable coordinate in the constructed quadratic-network setting and recovers the target ground truth with high probability.

CST objective: CST’s objective is zero for l = 2 but nonzero for l ∈ {3, · · ·, d}, distinguishing the target-working solution from the alternatives.This follows because l = 2 makes source and target feature spans identical.
Proof: The proof establishes recovery by showing the source consistency term is zero for the transferable solution and nonzero for the alternatives.This separates the target-compatible coordinate through the cycle objective.
CST objective: When the non-transferable coordinate is used, the target classifier becomes zero and the source consistency loss equals 0.18.The calculation evaluates the squared source-label discrepancy against the zero target classifier.
Theorem 4: For target sample sizes exceeding Θ(log 1/ξ), CST recovers the target ground truth with probability at least 1 − ξ over target-data sampling.The theorem states this recovery for the learned feature extractor and head.

B Implementation Details

The implementation uses standard deep-learning infrastructure, bilevel CST optimization with label sharpening and Tsallis entropy, and evaluations across visual and sentiment-analysis domain-adaptation datasets.

Experimental infrastructure: Experiments use PyTorch, 2080Ti GPUs, pretrained torchvision ResNet models, and the official BERT checkpoint and Transformers code.Several baselines use original-paper results or adaptations from official implementations.
Datasets: The evaluation covers OfficeHome, VisDA-2017, DomainNet, and Amazon Review datasets spanning visual recognition and sentiment analysis.VisDA experiments separately resample i.i.d., covariate-shift, and label-shift settings.
Optimization: CST’s inner loop trains a target classifier on shared representations using pseudo-labels sharpened from the source classifier’s outputs.The target classifier depends on the source classifier and representations through the pseudo-labels.
Optimization: Because label sharpening is nondifferentiable, gradients through pseudo-labels are set to zero, yielding the algorithm’s update rules together with Tsallis entropy loss.This follows standard self-training practice in the implementation.
Optimization: MSE loss enables an analytical least-squares target-classifier solution and avoids second-order derivatives in the outer loop.For multiclass classification, the implementation uses one-hot outputs and a multivariate regressor.
Uncertainty regularization: The uncertainty parameter α is selected from {1.0, 1.1, · · ·, 2.0} and tends toward smaller values during training.Smaller α increases uncertainty penalization, while its decrease is slower for heuristically distant domains.
Diagnostics: Pseudo-label analyses compare distributions and confidence-based selection under domain shift on Art→Clipart, Product→Art, and Clipart→Real World.The broader analysis also includes Art→Real World and Real World→Product tasks.

C.2 Results on digit datasets

On digit datasets, CST is evaluated without pre-training using a DTN architecture and achieves performance comparable to state-of-the-art methods.

Experimental setup: CST is evaluated on digit datasets without using pre-training.The experiments use the DTN architecture following Long et al.
Results: Results in Table 8 indicate that CST achieves comparable performance to the state-of-the-art.The table reports accuracy (%) on digit datasets with DTN.
Results: Table 8 reports digit-dataset accuracy (%) for the DTN-based evaluation.

D Limitations of CST and Future Directions

The paper identifies unresolved extensions of CST beyond pseudo-labeling and highlights risks from biased unlabeled data in real-world deployment.

Future directions: Extending CST to consistency regularization and self-ensembling under distribution shift remains an open question.The paper also suggests studying adversarial unlabeled-data poisoning and consistency regularization as a potential defense.
Limitations: Real-world unlabeled datasets may contain bias that is difficult to detect or mitigate during preprocessing.The paper contrasts this with the comparatively monitorable quality of labeled datasets.
Limitations: Self-training on biased unlabeled data may amplify biased model predictions.