PaperlayerView exact revision on arXiv

Source-linked AI summary

Understanding Self-Training for Gradual Domain Adaptation

Ananya Kumar, Tengyu Ma, Percy Liang

arXiv:2002.11361v1cs.LGstat.ML

TL;DR

Models deployed in evolving environments need to adapt using unlabeled data, but it is unclear whether gradual shifts improve adaptation over direct transfer. This paper analyzes gradual self-training, proving non-vacuous and tight error bounds and deriving practical insights about regularization and label sharpening.

  • Problem

    Models face evolving data distributions, while repeatedly collecting labeled examples for retraining can be impractical; most adaptation methods also ignore gradual shift structure.

  • Method

    The paper theoretically analyzes gradual self-training, including hard-label pseudolabeling and ramp-loss-based regularization, across general and isotropic-Gaussian settings.

  • Results

    The analysis provides the first non-vacuous theoretical bound for gradual domain adaptation, shows the bound can be tight, and recovers a Bayes-optimal target classifier in an isotropic-Gaussian setting with infinite unlabeled data.

  • Takeaways & Limitations

    Regularization and label sharpening remain essential with infinite data, while gradual structure can improve adaptation over direct domain adaptation.

  • Takeaways & Limitations

    The paper does not show that self-training converges to the constrained minimum of U, leaving optimization analysis for future work.

Abstract

from arXiv · show

Machine learning systems must adapt to data distributions that evolve over time, in applications ranging from sensor networks and self-driving car perception modules to brain-machine interfaces. We consider gradual domain adaptation, where the goal is to adapt an initial classifier trained on a source domain given only unlabeled data that shifts gradually in distribution towards a target domain. We prove the first non-vacuous upper bound on the error of self-training with gradual shifts, under settings where directly adapting to the target domain can result in unbounded error. The theoretical analysis leads to algorithmic insights, highlighting that regularization and label sharpening are essential even when we have infinite data, and suggesting that self-training works particularly well for shifts with small Wasserstein-infinity distance. Leveraging the gradual shift structure leads to higher accuracies on a rotating MNIST dataset and a realistic Portraits dataset.

1 Introduction

Gradual domain adaptation uses unlabeled intermediate domains to adapt a source classifier as data distributions evolve toward a target. The paper analyzes self-training theoretically and shows that leveraging gradual shifts can improve target-domain accuracy, with regularization, label sharpening, and small Wasserstein-infinity shifts playing important roles.

  • Motivation: Evolving deployment distributions make repeated collection of labeled data impractical, motivating adaptation from unlabeled examples.The motivation includes sensor aging, changing road conditions, and changing neural signals.
  • Contribution: The paper provides the first theoretical analysis showing improvements from gradual domain adaptation over direct adaptation to the target.It analyzes self-training, also called pseudolabeling, using unlabeled intermediate domains.
  • Theory: In the margin setting, gradual self-training can have a non-vacuous error bound after T steps even when direct target adaptation can reach 0% accuracy.The bound is errT ≤ ecT (α0 + O(1/√n)) and is tight for gradual self-training.
  • Theory: Under isotropic Gaussian class-conditionals, a nearly Bayes-optimal source classifier can recover a Bayes-optimal target classifier with infinite unlabeled data.This stronger-assumption setting identifies conditions under which self-training can outperform the exponential bound.
  • Algorithmic insights: Regularization and label sharpening are essential for gradual self-training, even with infinite data.The method pseudolabels examples with hard labels, while the ramp loss supplies robustness toward outliers.
  • Experiments: On Portraits, gradual self-training improves accuracy from 77% to 84%, while removing regularization reduces accuracy from 84% to 77% on Portraits and from 88% to 46% on rotating MNIST.The paper also reports that gradual self-training does not help when interpolation creates a large Wasserstein-infinity shift.

2 Setup

The setup models domains that shift gradually from a labeled source to an unlabeled target and evaluates classifiers by target-domain error. Gradual self-training successively pseudolabels each intermediate domain and retrains on those labels.

  • Problem setup: The domains are joint distributions P0, P1, . . . , PT, with P0 as source, PT as target, and intermediate domains between them.The shift is assumed gradual when consecutive domains are close under a distributional distance.
  • Models and objectives: A model outputs a confidence score, predicts sign(Mθ(x)), and is evaluated using 0-1 loss.The prediction is 1 for nonnegative scores and −1 for negative scores.
  • Problem setup: The objective is to find a classifier with low error on the target distribution PT, while online evaluation may also track error at each current Pt.The paper denotes target performance by Err(θ, PT).
  • Baseline methods: Self-training pseudolabels unlabeled examples with the current model and selects a new model minimizing loss on the pseudolabeled dataset.The paper calls these hard labels label sharpening rather than probabilistic labels.
  • Gradual self-training: Gradual self-training applies self-training successively to finite unlabeled samples from each domain, producing θT for evaluation on PT.The direct-adaptation baseline instead self-trains the source model on target data, while repeated target self-training is denoted STk(θ0, ST).

3 Theory for the margin setting

The margin-setting theory assumes small Wasserstein-infinity shifts, bounded regularized models, and per-domain low-loss classifiers, then bounds gradual self-training despite cases where direct target adaptation fails completely. The analysis identifies regularization, label sharpening, and ramp loss as essential, while showing that the exponential dependence can be tight.

  • Main result: The paper’s main theorem shows gradual self-training improves over direct target adaptation under small shifts and per-step margin classifiers.The classifier may differ at each time step, making the assumption more general than covariate shift.
  • Assumptions: Regularized linear models bound weight norms and encourage a geometric margin, helping preserve correct classifications after mild shifts.The regularization strength is tied to the margin enforced by the model class.
  • Assumptions: The ramp loss is bounded above, making it robust to outliers and central to the theoretical guarantees for gradual self-training.Unlike hinge loss, it prevents a single point from contributing without bound to the loss.
  • Assumptions: Wasserstein-infinity distance W∞ limits how far points move between consecutive distributions, supporting the gradual-shift assumption.The analysis uses small consecutive shifts rather than requiring source and target supports to overlap.
  • Domain shift: baselines fail: Direct adaptation can fail completely: a source-perfect classifier may have target loss 1, and self-training directly on the target may still have loss 1.Example 3.1 establishes this even when every domain is separable.
  • Gradual self-training improves error: With source loss α0, gradual self-training has an error bound proportional to exp(cT)α0, and this exponential growth is tight even with infinite unlabeled data.The result provides control absent from direct adaptation, but does not generally yield sub-exponential guarantees.
  • Essential ingredients for gradual self-training: Label sharpening is necessary to make self-training update parameters, while without sharpening every model can remain a minimizer of the soft-label objective.This observation also applies to finite data by treating the empirical distribution as P.
  • Essential ingredients for gradual self-training: Hinge-loss self-training cannot control gradual-adaptation error in the theory, even with infinite examples, whereas ramp loss supports the theorem’s guarantees.An example reaches target error 1 under hinge-loss self-training.

4 Theory for the Gaussian setting

In an idealized Gaussian setting, the paper studies when gradual self-training maintains a good classifier under small distribution shifts and infinite unlabeled data. Starting near the source-domain optimum, the analysis shows that sequential updates can recover the target-domain optimum despite non-convex unlabeled loss.

  • Setting: The Gaussian analysis assumes isotropic class-conditionals, gradually changing means, separated class means, and infinite unlabeled data.The class-conditional distributions are isotropic Gaussians in d dimensions, with ∥µ_t+1 − µ_t∥_2 ≤ B and ∥µ_t∥_2 ≥ B.
  • Design implications: Regularization and label sharpening are essential to the gradual self-training analysis, including in the infinite-data setting.Without regularization, self-training has no incentive to change predictions; hard labels are used in the theoretical analysis.
  • Algorithm: The algorithm initializes with a regularized source classifier and sequentially self-trains on unlabeled data from each successive domain.Each update generates pseudolabels for P_t and trains a regularized supervised classifier on them.
  • Proof intuition: The proof tracks a nearby unlabeled-loss minimum because the loss is non-convex and may contain other bad local minima.The local-neighborhood argument connects gradual movement of the labeled optimum to recovery at the next domain.

5 Experiments

Experiments on synthetic, semi-synthetic, and real datasets evaluate whether sequentially exploiting gradual shifts improves adaptation. Gradual self-training outperforms direct and pooled baselines across the three datasets, while regularization and hard labels remain important and the benefit depends on the shift metric.

  • Experimental setup: Experiments use Gaussian, rotating MNIST, and Portraits datasets to compare gradual self-training with direct, pooled, and source-only baselines.The datasets span a synthetic Gaussian setting, rotated handwritten digits, and historical high-school portraits.
  • Main results: Gradual self-training improves over direct target adaptation and pooled unlabeled-data self-training on all three datasets.Table 1 reports classification accuracies with 90% confidence intervals over 5 runs.
  • Important ingredients: Explicit regularization and hard labels produce the strongest gradual self-training performance, whereas removing regularization or using soft labels reduces accuracy.The comparison controls the initial source model when testing regularization during self-training.
  • When gradual shift helps: 33.5 ± 1.5% versus 33.0 ± 2.2% is the target accuracy for gradual versus direct self-training under a small total-variation but large Wasserstein shift.In this modified rotating MNIST setting, gradual self-training does not outperform direct adaptation.

6 Related work

The paper situates gradual domain adaptation among self-training, unsupervised adaptation, and related sequential-learning settings. It emphasizes that existing theory often assumes source-target overlap or lacks guarantees, whereas gradual shifts offer structure for analyzing larger domain differences.

  • Self-training: Self-training is a semi-supervised and domain-adaptation method related to entropy minimization, co-training, and robust self-training.Prior semi-supervised theory studies when unlabeled data can help but does not provide bounds for particular algorithms.
  • Unsupervised adaptation: Unsupervised domain adaptation directly transfers from a labeled source domain to an unlabeled target domain, with support mismatch posing a central theoretical challenge.Importance-weighting methods assume overlap, while density ratios can scale exponentially with dimension.
  • Existing approaches: Representation-learning approaches seek domain-invariant features but lack theoretical guarantees and depend on task-specific heuristics.The cited work reports that these methods work well on some tasks but not others.
  • Positioning: Gradual domain adaptation differs from online, lifelong, and concept-drift learning because only unlabeled data from shifted distributions are available.The paper presents its theory as the first analysis of when and why gradual structure helps.

A Proofs for Section 3

The appendix proves the gradual self-training results through loss comparisons, shift bounds, and examples showing both guarantees and failure modes. The proofs establish that gradual updates can control loss growth, while direct target self-training can fail completely or accumulate error exponentially.

  • Lower-bound examples: A source classifier can have zero source loss but target loss 1, and direct self-training on the target cannot correct it.The failure persists even when every domain is separable.
  • Shift control: A small shift relative to the geometric margin preserves most source classifications, allowing self-training to learn a low-margin-loss classifier on the shifted distribution.The margin acts as a buffer against misclassification after a small distribution shift.
  • Tightness: The gradual bound is tight in a constructed example where self-training loss can grow exponentially despite infinite unlabeled data.At each step, the loss can increase by a constant factor under the specified construction.
  • Gradual guarantee: If the initial loss is α_0, gradual self-training yields final loss at most α_0(T + 1) under the proposition’s assumptions.The argument applies the unlabeled-loss bound and loss-growth lemma at each of T steps.
  • Proof tools: The proof framework uses unlabeled-loss lower bounds, monotonicity under self-training, a triangle inequality, and an upper bound on per-step loss growth.These lemmas combine inductively to control the final labeled loss.

B Proofs for Section 4

The proofs establish key properties of the self-training loss and use them inductively to show recovery of the optimal classifier across gradual shifts in the Gaussian setting.

  • Proof strategy: If ||µ||2 and ||µ′||2 are bounded below by B > 0, the proof controls how much w∗(µ) changes between nearby distributions.This Lipschitz-style control supports the one-step induction.
  • Proof strategy: The key lemma shows that w∗(µ) is a strict local minimizer of U(w, Pµ,σ).Nearby feasible parameters have strictly higher loss.
  • Proof strategy: Scaling parameters strictly decreases the loss, allowing the proof to assume ||w||2 = 1 without loss of generality.This follows from the monotonicity of φ and positive probability under the Gaussian mixture.
  • Proof strategy: The symmetry argument pairs reflected points and shows the Gaussian mixture assigns greater density to one point, yielding a strict expected-loss inequality.A positive-volume set has strictly lower contribution under w∗ than under w.
  • Theorem 4.1: Under the theorem’s initialization condition, each one-step update recovers w∗(µt), and induction gives wT = w∗(µT).The constrained optimization argument makes w∗(µt) the unique minimizer at each step.

C Experimental details for Section 5

Additional experiments report that gradual self-training remains better than direct target self-training across several alternative training choices.

  • Robustness: Gradual self-training has a very small number of hyperparameters, and its findings are robust to different parameter choices.The reported variations include removing confidence thresholding, training for more iterations, and using a smaller window size.
  • Robustness: Gradual self-training does better than self-training directly to the target when confidence thresholding is omitted.The passage reports this comparison among the ablation settings.
  • Robustness: Gradual self-training does better than direct target self-training under longer training and smaller-window ablations.These are among the alternative choices tested in the additional experiments.

C.1 Datasets

The experiments use synthetic Gaussian and rotating-MNIST shifts alongside the realistic Portraits dataset, whose changing class proportions violate the theory’s constant-label assumption.

  • Gaussian: The Gaussian dataset uses 100-dimensional two-class distributions with independently varied means and non-diagonal covariances, while keeping class probabilities at 0.5.Initial and final class parameters can differ across the shift.
  • Rotating MNIST: Rotating MNIST divides 50,000 images into labeled source, unlabeled intermediate, and unseen target sets across rotations from 0 to 60 degrees.The source covers 0–5 degrees, intermediate examples span 5–60 degrees, and targets span 55–60 degrees.
  • Portraits: Portraits contains high-school-senior photos collected across years, with 2,000 source images, 14,000 unlabeled intermediate examples, and 2,000 unseen targets.Images are downsampled to 32x32 without other preprocessing.
  • Portraits: Portraits includes label shift because the proportions of males and females change over time, unlike the theory’s assumption that label probabilities remain constant.Figure 3 visualizes the changing female-image fraction with rolling averages and 90% confidence intervals.

C.2 Algorithm and baselines

Gradual self-training processes unlabeled intermediate data in windows, filters low-confidence pseudolabels, and compares against target self-training under matched step or data-access conditions.

  • Gradual self-training: The algorithm trains on labeled source examples, then repeatedly pseudolabels each intermediate window and retrains on its confident examples.Each window contributes filtered inputs and pseudolabels to the next training stage.
  • Gradual self-training: Confidence filtering removes the α fraction of examples on which the classifier is least confident, using maximum softmax output as confidence.The passage describes this filtering as standard in many self-training methods.
  • Baselines: The target self-train baseline self-trains iteratively on target data for |I|/W rounds, matching gradual self-training’s total number of self-training steps.This controls the number of update stages in the comparison.
  • Baselines: Synthetic experiments give target self-train the same total number of unlabeled examples as gradual self-training.For Portraits, additional target examples cannot be generated, so the comparison uses another matching strategy.

C.3 Models and parameter settings

This section describes the models, training parameters, confidence filtering, and robustness checks used in the experiments.

  • Models: The Gaussian experiments use logistic regression with l2 regularization 0.02, while rotating MNIST and Portraits use a three-layer convolutional network.The convolutional network uses 5x5 filters, 2x2 strides, 32 output channels, and ReLU activations.
  • Models: The convolutional architecture applies dropout(0.5), batch normalization, a flattened final layer, and a class-sized linear logits layer followed by softmax and cross-entropy.The number of logits is 10 for rotating MNIST and 2 for Portraits.
  • Robustness checks: The tested architecture modifications, including an extra layer, different channel counts, and an additional fully connected layer, have little impact on the results.The architecture was not tuned for the experiments.
  • Parameters: The experiments use window sizes W = 500 for Gaussian and W = 2000 for rotating MNIST and Portraits, with 100, 10, and 20 training epochs per round respectively.The window and epoch settings were chosen using source-domain validation data without examining intermediate or target data.
  • Confidence thresholding: Confidence thresholding filters the 10% least confident examples using α = 0.1; removing this filtering makes all methods slightly worse while preserving their relative ordering.Gradual self-training remains significantly better than the other methods without filtering.

C.4 Ablations

The ablations show that gradual self-training remains superior when confidence filtering or larger windows are removed, but performance declines for a farther Portraits extrapolation.

  • Confidence thresholding: Without confidence thresholding, all methods perform worse, while gradual self-training remains significantly better than the other methods.This result is reported for rotating MNIST and Portraits.
  • Window sizes: With smaller window sizes reduced from 2000 to 1000, gradual self-training still performs significantly better than the other methods on rotating MNIST and Portraits.The comparison uses the smaller-window ablation reported in Table 5.
  • Additional Portraits ablations: With 50% more training epochs on Portraits, gradual self-training reaches 83.9 ± 0.4% accuracy over 5 trials, versus 80.7 ± 1.1% for target self-training.Self-training on all unlabeled examples reaches 79.6 ± 2.2%, while the non-adaptive baseline reaches 77.3 ± 1.0%.
  • Additional Portraits ablations: For a farther Portraits extrapolation, gradual self-training achieves 60.6 ± 1.4%, compared with 56.5 ± 1.4% for direct target self-training and 57.4 ± 0.3% using all unlabeled data.All methods perform quite poorly in this setting, which the paper identifies as a direction for future work.
Loading 2002.11361v1…