PaperlayerView exact revision on arXiv

Source-linked AI summary

A Prototype-Oriented Framework for Unsupervised Domain Adaptation

Korawat Tanwisuth, Xinjie Fan, Huangjie Zheng, Shujian Zhang, Hao Zhang, Bo Chen, Mingyuan Zhou

arXiv:2110.12024v1cs.LGcs.CVstat.ML

TL;DR

Unsupervised domain adaptation methods often align source and target samples directly, despite sensitivity to sampling variability and class mismatch and the need to access source data. This paper uses class prototypes and probabilistic bidirectional transport, achieving competitive performance across varied settings without adding model parameters.

  • Problem

    Direct source–target sample alignment is sensitive to sampling variability and mini-batch class mismatch, and can require source-data access during adaptation.

    Source: §1 · pp. 1–2

  • Method

    The method uses class prototypes and probabilistic bidirectional transport to align target features, with linear-classifier weights serving as prototypes.

    Source: §1 · p. 2

  • Results

    Across several widely used benchmarks, the prototype-oriented method works across multiple datasets, is robust against class imbalance, and performs adaptation without direct source-data access.

    Source: §5 · p. 10

  • Takeaways & Limitations

    The method achieves competitive performance with no additional model parameters and only a moderate increase in computation over the source model alone.

    Source: §1 · p. 2, §5 · p. 10

    The method addresses closed-category adaptation and assumes source and target domains share the same label space.

    Source: §2 · p. 3

Abstract

from arXiv · show

Existing methods for unsupervised domain adaptation often rely on minimizing some statistical distance between the source and target samples in the latent space. To avoid the sampling variability, class imbalance, and data-privacy concerns that often plague these methods, we instead provide a memory and computation-efficient probabilistic framework to extract class prototypes and align the target features with them. We demonstrate the general applicability of our method on a wide range of scenarios, including single-source, multi-source, class-imbalance, and source-private domain adaptation. Requiring no additional model parameters and having a moderate increase in computation over the source model alone, the proposed method achieves competitive performance with state-of-the-art methods.

1 Introduction

The introduction frames direct source–target feature alignment as vulnerable to sampling variability and mini-batch class mismatch, and sometimes impractical when source data are unavailable. It proposes class prototypes and bidirectional transport to address these concerns across varied adaptation settings.

1 Introduction

  • Direct sample alignment can be sensitive to mini-batch outliers and class coverage, may pull features from different classes together, and requires source-data access during adaptation.These concerns are especially relevant when label proportions shift or source data are unavailable, as in personal healthcare.Source: §1 · pp. 1–2
  • Class prototypes represent source samples in latent space, avoiding source sampling variability, mini-batch class mismatch, and source-data privacy concerns while bidirectional transport addresses label-proportion shifts.The approach transports target features to prototypes and vice versa.Source: §1 · p. 2
  • The method avoids adversarial training and uses learnable prototypes to bypass direct source-data access, while processing mini-batches end-to-end rather than requiring clustering-based repeated forward passes.The authors say this makes the algorithm robust and faster to converge.Source: §1 · p. 2
  • Across single-source, multi-source, class-imbalance, and source-private adaptation, the prototype-oriented method achieves highly competitive performance with no additional parameters and moderate extra computation.The paper introduces probabilistic alignment using linear-classifier weights as class prototypes.Source: §1 · p. 2

2 Prototype-oriented conditional transport

Prototype-oriented Conditional Transport (PCT) aligns target features with learned class prototypes through probabilistic bidirectional transport and estimates target class proportions. Its components address prototype construction, target alignment, and class imbalance.

2 Prototype-oriented conditional transport

  • PCT aligns target features with learned class prototypes through probabilistic bidirectional transport and estimates target class proportions without adding alignment parameters.The method is learned end-to-end.Source: §2 · pp. 2–3, §2.3 · p. 5

2.1 Learning class prototypes

  • Learnable classifier weights serve as class prototypes, replacing sample averages and enabling source-private adaptation while representing every class at each training update.This approach adds no parameters and avoids the computation needed to average latent features.Source: §2.1 · p. 3

2.2 Bi-directional prototype-oriented conditional transport

  • Under the cluster assumption, minimizing expected pairwise costs under two joint distributions encourages target features toward prototypes and away from decision boundaries.The cluster assumption holds that same-class instances form separated clusters and decision boundaries should not cross high-density regions.Source: §2.2 · p. 4

2.2.1 Moving from target domain to class prototypes

  • Target-to-prototype cost assigns samples by prototype similarity and target-class prior, but alone can concentrate assignments on only a few prototypes.The passage connects this mode-seeking behavior to the risk of dropping class prototypes.Source: §2.2.1 · p. 4

2.2.2 Moving from class prototypes to target domain

  • The reverse cost normalizes each prototype’s assignments across target samples, encouraging every prototype to receive nearby target features and countering dropped classes.The method treats prototypes as fixed during both transport costs, enabling adaptation with only the source model available and yielding more stable training empirically.Source: §2.2.2 · p. 5

2.3 Learning class proportions in the target domain

  • Target class proportions are inferred by maximizing unlabeled-data likelihood with EM, using average target predictions to update the estimates iteratively.The prior starts uniform; the update can use a decaying rate, with its initial value set to zero for a uniform prior or a small positive value to infer proportions.Source: §2.3 · p. 5

3 Related work

Prior deep-learning domain adaptation aligns marginal or joint feature distributions, while related approaches use prototypes for pseudo-labeling, address shifts in class proportions, or adapt without source data. This paper instead aligns target features with prototypes.

Feature distribution alignment

  • Deep-learning domain adaptation commonly aligns marginal or joint feature distributions, but Jensen–Shannon divergence provides no useful gradients when source and target supports do not overlap.Source: §3 · p. 5
  • The method computes transport probabilities in O(dfMK), matching predictive-probability computation, and aligns target features with prototypes rather than source features.Source: §3 · p. 6

Prototype-based alignment

  • Unlike prior prototype methods that use prototypes for pseudo-label assignment or average latent features, this method uses parametric prototypes as representative source samples to define the loss.Source: §3 · p. 6

Learning under shifted class proportions

  • For shifted class proportions, prior approaches include kernel embeddings, EM updates, meta-priors, and causal or anti-causal formulations; the proposed method combines prototypes with proportion estimation.Source: §3 · p. 6

Source-private adaptation

  • In source-private adaptation, the method uses the source model and target data without raw source data, with prototypes serving as class centers suited to mini-batch stochastic-gradient training.Source: §3 · p. 6

4 Experiments

Experiments evaluate PCT across single-source, multi-source, class-imbalance, and source-private adaptation, using Office-31, Office-Home, and DomainNet among other datasets. PCT reports improvements over baselines across these settings, while analyses examine ablations, convergence, and feature alignment.

4.1 Main results

  • PCT exceeds MDD by 1.1% in average accuracy on Office-31 and by 3.7% on Office-Home, outperforming baselines on 10 of 12 Office-Home transfers.On Office-31, PCT has the highest average accuracy while using fewer parameters than most methods.Source: §4.1 · p. 7, Table 1 · p. 7, Table 2 · p. 7
  • PCT outperforms multi-source algorithms on both Office-Home and DomainNet, despite using a single classifier without domain labels.PCT is categorized as source-combined; the authors relate the performance gain to mitigating sampling variability as source-domain variation increases.Source: §4.1 · p. 7, Table 3 · p. 7, Table 4 · p. 8
  • On sub-sampled source data, uniform-prior PCT beats IWCDAN by 4% on Office-31 and 6.6% on Office-Home; on sub-sampled target data, learnable priors add 1.0% and 0.4%, respectively.On target data, uniform-prior PCT already exceeds IWCDAN by 1.9% on Office-31 and 5.2% on Office-Home; learning target proportions does not improve results for the sub-sampled source setting.Source: §4.1 · p. 8, Table 5 · p. 8, Figure 3 · p. 9
  • In source-private adaptation, PCT's bi-directional loss beats SHOT's IM and pseudo-label losses on Office-31, while Office-Home comparisons favor PCT over each loss but SHOT's combined loss scores higher.On Office-31, SHOT's combined loss is 0.2% higher, a difference reported as not statistically significant (p = 0.32); on Office-Home it is 0.8% higher. PCT uses classifier weights as prototypes to avoid constructing class centers that do not scale well with large datasets.Source: §4.1 · pp. 8–9, Table 6 · p. 9

4.2 Analysis of results

  • PCT's bi-directional transport, gradient stopping, and cosine cost outperform the tested ablation alternatives on Office-31.PCT exceeds DeepJDOT by 1.7%, while the two POT variants trail PCT by 2.1% and 1.7%; gradient stopping adds 1.0% accuracy, and cosine distance gives 3.1% higher average accuracy than the radial-basis-inspired cost. Removing Lµ→t causes a larger accuracy drop and can lead to mapping target data to only a few prototypes.Source: §4.2 · p. 9, Table 7 · p. 10
  • On task A →W, PCT converges within the first 1,000 iterations, after which test accuracy fluctuates little.The authors attribute the fast, stable convergence to prototypes acting as representative samples of source features.Source: §4.2 · p. 10, Figure 4 · p. 10
  • For task A →W, the t-SNE visualization shows source and target features close to the prototypes for each class.The figure distinguishes prototypes (stars), source features (dots), and target features (crosses).Source: §4.2 · p. 10, Figure 4 · p. 10

5 Conclusion

The method uses probabilistic bidirectional transport to align target features with class prototypes, and experiments show it works across multiple datasets and adaptation settings. The broader-impact discussion notes that domain adaptation cannot eliminate dataset bias from domain shift.

5 Conclusion

  • The method works across multiple datasets, is robust to class imbalance, and adapts without direct access to source data, with competitive state-of-the-art performance and no added model parameters.The authors describe the algorithm as memory- and computation-efficient.Source: §5 · p. 10

A Broader impact

  • Domain adaptation cannot eliminate dataset bias from domain shift because of the combinatorial nature of exogenous factors; sustainable energy sources also warrant care.The authors note that the method addresses sampling variability, class imbalance, and source-data privacy through class prototypes.Source: A Broader impact · p. 16

B EM steps derivation

The derivation estimates target class proportions by maximizing the likelihood of unlabeled target data with EM. The accompanying connection shows that the continuous transport formulation corresponds to balanced K-means assignment under a divisibility condition.

B EM steps derivation

  • EM estimates target class proportions by maximizing the likelihood of unlabeled target data, treating each unobserved target label as a latent variable.The method initializes class proportions uniformly, then alternates E- and M-steps; the E-step posterior probabilities weight transport costs.Source: B EM steps derivation · p. 16
  • The prototypes are learned jointly using source cross-entropy, while a mini-batch estimates the relevant quantity in practice.A Lagrange multiplier enforces that the class proportions lie in a simplex.Source: B EM steps derivation · pp. 16–17

C Connection with K-means clustering and optimal transport

  • When M is divisible by K, continuous optimal transport with uniform data and class marginals gives an integral solution equivalent to balanced K-means assignment.The transport update is analogous to the cluster-assignment step, while the cross-entropy update is analogous to the cluster-center update.Source: C Connection with K-means clustering… · pp. 17–18

D Implementation details

Implementation uses pretrained ResNet encoders and jointly trains the feature encoder and linear layers. Training schedules, batch sizes, priors, and run counts vary by adaptation setting.

D Implementation details

  • Experiments use pretrained ResNet-50 encoders, except multi-source adaptation uses ResNet-101; feature encoders are fine-tuned and linear layers start from random initialization.The linear layers use a learning rate ten times that of the feature encoder.Source: D Implementation details · p. 18
  • The learning-rate schedule uses η0 = 0.01, γ = 0.0002, and α = 0.75, with momentum 0.9 and batch sizes N = 32 for source and M = 96 for target.The cross-entropy and transport losses are equally weighted in single- and multi-source settings.Source: D Implementation details · p. 18
  • Single-source and multi-source training run for 10,000 and 75,000 iterations, respectively; class-imbalance experiments use three seeds and setting-specific β0 values.Sub-sampled target β0 is 0.001 for Office-31 and 0.0001 for Office-Home; sub-sampled source experiments use β0 = 0.Source: D Implementation details · p. 18
  • Source-private experiments follow Liang et al., using 70 epochs for Office-31 and 100 for Office-Home, with target batch size 96.They use the original paper's seeds {2019, 2020, 2021}.Source: D Implementation details · p. 18

E Additional experimental results

The additional results visualize transport behavior, report sensitivity to β0, and show feature visualizations across Office-31 transfer tasks. They also describe the toy experiment and the scope of the error-versus-time comparison.

E Additional experimental results

  • The toy experiment samples two-dimensional Gaussian data with different domain means and the same covariance; Figure 6 plots test error against training time for task A→W.The toy setup uses 300 examples per domain and a three-layer feature encoder with output dimension 2.Source: E Additional experimental results · p. 19, Figure 6 · p. 20
  • Figure 5 contrasts transport probabilities: optimal transport assigns each point to one cluster with equal weight, while PCT probabilities reflect target-feature uncertainty.PCT's target-to-prototype probabilities have an active cell across rows, and prototype-to-target probabilities have an active cell across columns.Source: E Additional experimental results · p. 19, Figure 5 · p. 19
  • Office-31 t-SNE plots cover A→D, D→A, and W→D; source and target features lie near prototypes in class views, while domain views show the domains close together.The top row highlights class information and the bottom row domain information.Source: Figure 7 · p. 20
  • β0 works well from 0.0001 to 0.01; small values give more weight to past mini-batch average predictions, which are more stable than a single-batch average.Table 8 reports accuracy on the sub-sampled-target Office-31 task A→sW.Source: E Additional experimental results · p. 21, Table 8 · p. 21

E.1 Full experimental results

Some experiments report dataset-level average accuracy because of space constraints, while the full tables provide average accuracy for individual transfer tasks.

E.1 Full experimental results

  • Full tables provide average accuracy for individual transfer tasks where space constraints lead some experiments to report dataset-level averages.The passage does not provide the table values.Source: §E.1 · p. 21

E.1.1 Ablation study

Table 9 reports PCT accuracy on Office-31 across different variants using ResNet-50.

E.1.1 Ablation study

  • Table 9 reports accuracy (%) for PCT variants on Office-31 using ResNet-50.The supplied caption does not state variant results.Source: Table 9 · p. 21

E.1.2 Sub-sampled setting

The tables report accuracy for sub-sampled and source-private unsupervised domain adaptation settings on Office-31 and Office-Home using ResNet-50.

E.1.2 Sub-sampled setting

  • Tables 10–13 report accuracy (%) for sub-sampled source and target Office-31 and Office-Home settings using ResNet-50.Tables 10 and 12 cover sub-sampled sources; Tables 11 and 13 cover sub-sampled targets.Source: Table 10 · p. 21, Table 11 · p. 21, Table 12 · p. 22

E.1.3 Source-private setting

  • Tables 14–15 report accuracy (%) for source-private Office-31 and Office-Home settings using ResNet-50.The captions identify the datasets and source-private setting, not the results.Source: Table 14 · p. 22, Table 15 · p. 22

E.2 Additional Results

Additional results cover Cross-Digits, Office-Caltech, Image-Clef, and VisDA under different settings.

E.2.1 Single-source setting

  • Tables 16–18 report average accuracy (%) for single-source adaptation on Cross-Digits, Office-Caltech, and Image-Clef using ResNet-50.The captions identify dataset scope but do not provide accuracy values.Source: Table 16 · p. 22, Table 17 · p. 23, Table 18 · p. 23

E.2.2 Multi-source setting

  • Table 19 reports average accuracy (%) on Office-31 for ResNet50-based multi-source domain adaptation methods.The caption does not state comparative outcomes.Source: Table 19 · p. 23

E.2.3 Source-private setting

  • Table 20 reports accuracy (%) on VisDA-2017 for unsupervised domain adaptation using ResNet-50.The caption identifies the dataset and setting without reporting values.Source: Table 20 · p. 23
Loading 2110.12024v1…