PaperlayerView exact revision on arXiv

Source-linked AI summary

Unified Optimal Transport Framework for Universal Domain Adaptation

Wanxing Chang, Ye Shi, Hoang Duong Tuan, Jingya Wang

arXiv:2210.17067v2cs.CV

TL;DR

Universal Domain Adaptation must transfer knowledge when source and target label sets may differ, without known category priors. UniOT applies optimal transport to common-class detection and private-class representation learning. It achieves the best H-score on four benchmarks and exceeds other methods on H3-score in Office and Office-Home.

  • Problem

    UniDA lacks known label-set priors, while existing methods use hand-tuned thresholds and generally treat target-private samples as a single class.

    Source: §1 · p. 2

  • Method

    UniOT combines OT-based partial alignment with adaptive filling for common detection and OT-based target representation learning for private-class discovery.

    Source: §1 · pp. 2–3

  • Results

    UniOT achieves the best H-score on Office, Office-Home, VisDA, and DomainNet, and surpasses other methods by 7% and 9% in H3-score on Office and Office-Home, respectively.

    Source: §4.3 · p. 8

  • Takeaways & Limitations

    H3-score assesses both common-class accuracy and private-class clustering, providing a metric for target-private class discovery.

    Source: §1 · p. 2

Abstract

from arXiv · show

Universal Domain Adaptation (UniDA) aims to transfer knowledge from a source domain to a target domain without any constraints on label sets. Since both domains may hold private classes, identifying target common samples for domain alignment is an essential issue in UniDA. Most existing methods require manually specified or hand-tuned threshold values to detect common samples thus they are hard to extend to more realistic UniDA because of the diverse ratios of common classes. Moreover, they cannot recognize different categories among target-private samples as these private samples are treated as a whole. In this paper, we propose to use Optimal Transport (OT) to handle these issues under a unified framework, namely UniOT. First, an OT-based partial alignment with adaptive filling is designed to detect common classes without any predefined threshold values for realistic UniDA. It can automatically discover the intrinsic difference between common and private classes based on the statistical information of the assignment matrix obtained from OT. Second, we propose an OT-based target representation learning that encourages both global discrimination and local consistency of samples to avoid the over-reliance on the source. Notably, UniOT is the first method with the capability to automatically discover and recognize private categories in the target domain for UniDA. Accordingly, we introduce a new metric H^3-score to evaluate the performance in terms of both accuracy of common samples and clustering performance of private ones. Extensive experiments clearly demonstrate the advantages of UniOT over a wide range of state-of-the-art methods in UniDA.

1 Introduction

Universal Domain Adaptation addresses transfer when source and target domains may have private classes and category-set priors are unknown. UniOT uses optimal transport for common-class detection and private-class discovery, and introduces H3-score to evaluate common accuracy and private-class clustering.

1 Introduction

  • UniDA must detect common and target-private samples without prior knowledge of matched classes or target category counts.The setting generalizes closed-set domain adaptation, which assumes identical label sets across domains.Source: §1 · pp. 1–2
  • Existing UniDA methods use hand-tuned thresholds and treat target-private samples as one class, limiting diverse-ratio adaptation and private-category recognition.The paper frames automatic discovery and recognition of private target categories as a needed capability.Source: §1 · p. 2
  • Common-class detection and private-class discovery can both be formulated as distribution transportation problems, motivating optimal transport for UniDA.The paper notes OT's global mapping can reveal common/private differences and its constraints can avoid degenerate clustering solutions.Source: §1 · p. 2
  • UniOT combines OT-based partial alignment with adaptive filling for common detection and target representation learning for private-class discovery.Adaptive filling handles diverse common-class ratios without predefined thresholds; representation learning encourages global discrimination and local consistency.Source: §1 · pp. 2–3
  • H3-score evaluates both common-class accuracy and private-class clustering performance, addressing the lack of a metric for target-private representations.The paper presents UniOT as automatically discovering and recognizing target-private categories.Source: §1 · pp. 2–3

2 Related Work

Prior UniDA methods use sample-level uncertainty or clustering strategies, while earlier OT approaches largely address closed-set or label-prior settings. UniOT combines cross-domain partial alignment with target-domain representation learning to address both common detection and private discovery.

Universal Domain Adaptation

  • UniDA methods use uncertainty criteria to weight adversarial adaptation, but often rely on hand-tuned thresholds and may neglect discriminative target representations.DANCE uses neighborhood clustering, DCC enumerates cluster counts with K-Means, and MATHS uses a fixed K-Means hyperparameter.Source: §2 · p. 3

Optimal Transport in Domain Adaptation

  • Prior OT domain-adaptation methods use sample-to-sample mappings under closed-set or label-set-prior assumptions, unlike UniDA's unknown category gap.The passage discusses DeepJDOT, UOT, JUMBOT, and partial OT approaches before distinguishing UniOT's sample-to-prototype mapping.Source: §2 · p. 3

Clustering for Representation Learning

  • Many deep-clustering methods assume a known cluster count, while existing OT-based representation methods encourage global discrimination but ignore local consistency.UniOT's overview pairs global cluster discrimination with local sample consistency in target representation learning.Source: §2 · pp. 3–4, Figure 2 · p. 4

3 Methodology

UniOT addresses UniDA’s category gap through optimal-transport alignment for common-sample detection and target representation learning for private-class discovery. Its components use unbalanced transport across domains and target-sample clustering within the target domain.

3 Methodology

  • UniDA has labeled source and unlabeled target domains, potentially with private classes, and aims to classify target samples into the shared classes plus one unknown class.The common label set is the intersection of source and target labels; target-private samples are treated uniformly as unknown in this setup.Source: §3 · p. 4

3.1 Preliminary

  • Optimal transport seeks a mass-transport solution; its coupling represents joint probabilities with specified marginals, while unbalanced OT relaxes marginal conservation through soft penalties.The transport coupling can be computed with Sinkhorn’s algorithm, and the unbalanced formulation can be solved with generalized Sinkhorn’s algorithm.Source: §3.1 · pp. 4–5

3.2 Inter-domain Partial Alignment for Common Class Detection

  • Unbalanced optimal transport maps target features to source prototypes, and assignment-matrix statistics identify confident common samples while allowing private samples to receive low weights.A memory queue supplies previous target features; adaptive filling balances positive and negative proportions when either exceeds 50%.Source: §3.2 · pp. 5–6

3.3 Intra-domain Representation Learning for Private Class Discovery

  • Target features are mapped to learnable target prototypes with optimal transport, using uniform marginals to encourage global cluster discrimination and nearest-neighbor swapping for local consistency.The target prototypes are initialized randomly, and the method does not require knowing the number of target classes.Source: §3.3 · pp. 6–7

3.4 A Unified Framework for UniDA by Optimal Transport

  • UniOT combines source cross-entropy with common-class detection and private-class discovery losses; at test time, samples meeting the common-sample criterion receive a source class, while others are marked unknown.The test phase retains the feature extractor and prototype-based source classifier.Source: §3.4 · p. 7

4 Experiments

Experiments evaluate UniOT across four domain-adaptation benchmarks, with H-score and H3-score comparisons, component tests, robustness analyses, and target-feature visualizations. The reported results favor UniOT across benchmarks and show robustness to changes in private-class counts.

4.1 Experimental Settings

  • Evaluation uses H-score for common and unknown-class accuracy, and H3-score additionally incorporates NMI for target-private clusters.NMI is obtained with K-Means using the ground-truth number of private classes at inference; experiments use Office, Office-Home, VisDA, and DomainNet.Source: §4.1 · pp. 7–8

4.2 Experimental details

  • Experiments use a pretrained ImageNet ResNet-50 backbone, dataset-specific prototype counts, and 10K training steps.The implementation uses POT, batch size 36 for both domains, and a TITAN V GPU; each experiment takes about 2.5 hours.Source: §4.2 · p. 8

4.3 Results

  • UniOT achieves the best H-score on all four benchmarks and surpasses other methods by 7% on Office and 9% on Office-Home in H3-score.The authors interpret the H3-score results as balanced performance in common-class detection and private-class discovery.Source: Table 1 · p. 8, §4.3 · p. 8, Table 2 · p. 9
  • Combining global and local representation losses contributes substantially to target representation, with the global loss contributing more than the local loss.The authors also report that testing common-class detection without adaptive filling verifies the effectiveness of that design.Source: Table 4 · p. 9, §4.3 · p. 9
  • On A2W, UniOT outperforms the compared methods as source-private or target-private class counts vary, while the other methods show sensitivity to those variations.The analysis fixes the common-class set and varies either the source-private or target-private set; UniOT is reported as not sensitive to source-private variation.Source: §4.3 · p. 9, Figure 3 · p. 10
  • For Rw2Pr target features, UniOT's visualization shows global and local structure among target-private samples, while UAN does not distinguish their categories.The authors also describe UniOT's representation as having better global discrimination and local consistency than DANCE's.Source: §4.3 · p. 9, Figure 4 · p. 10

5 Conclusion

The conclusion presents UniOT as a unified OT framework for common-class detection and private-class discovery in UniDA. It reports superiority on four benchmarks and identifies more efficient OT feature filling as future work.

5 Conclusion

  • UniOT unifies common-class detection and target-private class discovery through Optimal Transport.Its partial alignment uses adaptive filling, while target representation learning encourages global discrimination and local consistency.Source: §5 · p. 10
  • Experiments on four benchmark datasets validate UniOT's reported superiority over a wide range of state-of-the-art methods.Source: §5 · p. 10
  • Future work will modify the memory model to make feature filling in OT more efficient.Source: §5 · p. 10

A.1 Evaluation metric

The appendix defines H-score as a harmonic-mean metric for common-class and unknown-class accuracy. It is high only when both component accuracies are high.

A.1 Evaluation metric

  • H-score is the harmonic mean of instance accuracy on common classes and accuracy on the unknown class.Source: §A.1 · p. 13
  • H-score is high only when both common-class and unknown-class accuracies are high.Source: §A.1 · p. 13
  • The paper describes H-score as a fair evaluation metric used in UniDA.Source: §A.1 · p. 13

A.2 Dataset split.

The dataset split defines common labels as shared by the source and target, with separate source-private and target-private label sets. Dataset-specific class ordering determines which labels occupy each set.

A.2 Dataset split.

  • The common label set C is the intersection of source and target labels; source-private and target-private sets are the respective labels outside C.Source: §A.2 · p. 13
  • Office and Office-Home use ordered class lists to assign common, source-private, and target-private labels, with 10 common classes in each dataset.Office assigns 10 source-private and 11 target-private classes; Office-Home assigns 5 source-private and 55 target-private classes.Source: §A.2 · p. 13
  • VisDA uses 6 common, 3 source-private, and remaining target-private classes; DomainNet uses 150 common, 50 source-private, and remaining target-private classes.The split follows the stated class ordering, with DomainNet's split attributed to CMU.Source: §A.2 · p. 13

B Additional results

Additional results report strong H3-score performance, stable H-score results, and analyses of common-class detection, hyperparameter sensitivity, and target representations. Qualitative visualizations illustrate partial alignment and target clustering.

H3-score on VisDA and DomainNet

  • UniOT surpasses other methods by a large margin on H3-score for VisDA and DomainNet, especially on VisDA.The comparison omits TNT and DCC because code or hyperparameters were unavailable for these datasets.Source: B Additional results · p. 13, Table S5 · p. 13

H-score with standard deviation

  • H-score standard deviations are generally close to zero across Office and Office-Home transfer tasks, indicating stable results across three runs.The reported values are averages and standard deviations after three runs.Source: B Additional results · p. 13, Table S6 · p. 14

Effect of Common Class Detection

  • CCD maintains high recall and low specificity on D2W of Office, supporting common-sample detection while avoiding alignment of target-private samples.Recall measures correctly retrieved common samples; specificity measures private samples not retrieved.Source: B Additional results · p. 13, Figure S5 · p. 14

Sensitivity to hyper-parameters

  • Performance is insensitive to γ and λ over the tested settings, while µ = 1 performs poorly and µ < 1 improves performance without sensitivity to µ.γ is the rough boundary for adaptive filling; µ controls adaptive updates to the marginal probability vector, and λ is the overall loss weight.Source: B Additional results · pp. 13–14

Effect of local neighbor for representation learning

  • Local-neighbor learning performs better than SimSiam- or SwAV-based augmentation consistency, which barely contributes when combined with domain adaptation.The paper offers source supervision as one possible interpretation of the limited contribution from those self-supervised methods.Source: B Additional results · p. 15, Table S7 · p. 15

Feature visualization for the significance of Private Class Discovery

  • PCD encourages more compact common-class representations and refines target-private representations that become disordered around source-private classes without it.The visualization notes improved common-class accuracy and acute target-private misclassification without PCD.Source: B Additional results · p. 15, Figure S6 · p. 16

Qualitative illustration for cross-domain mapping matrix

  • On A2W of Office, the UOT coupling matrix has clearer diagonal assignments for the ten common classes and only slight mappings for other classes.The visualization includes ten common classes, ten source-private classes, and eleven target-private classes.Source: B Additional results · p. 15, Figure S7 · p. 17, Figure S8 · p. 18

Qualitative results of target cluster examples

Loading 2210.17067v2…