Source-linked AI summary
Transferrable Prototypical Networks for Unsupervised Domain Adaptation
Yingwei Pan, Ting Yao, Yehao Li, Yu Wang, Chong-Wah Ngo, Tao Mei
TL;DR
Unsupervised domain adaptation seeks to transfer models from labeled source data to unlabeled target data despite domain differences. TPN remolds Prototypical Networks by aligning same-class prototypes and classifier score distributions across domains, achieving strong results across digit transfers and 80.4% mean accuracy on VisDA 2017.
Problem
Unsupervised domain adaptation must transfer knowledge from labeled source data to unlabeled target data when source and target distributions differ, while target labeling is costly.
Method
TPN assigns target examples pseudo labels from nearest source prototypes, then aligns same-class prototypes and cross-domain classifier score distributions.
Results
80.4% mean accuracy was achieved across the 12 VisDA 2017 categories, while TPN also outperformed competing methods across the reported digit-transfer settings.
Takeaways & Limitations
TPN supports unsupervised adaptation by combining class-level prototype alignment with sample-level alignment of task-specific classifier decisions.
Takeaways & Limitations
The formulation relies on the widely adopted assumption of the ex-
Abstract
from arXiv · showhide
In this paper, we introduce a new idea for unsupervised domain adaptation via a remold of Prototypical Networks, which learn an embedding space and perform classification via a remold of the distances to the prototype of each class. Specifically, we present Transferrable Prototypical Networks (TPN) for adaptation such that the prototypes for each class in source and target domains are close in the embedding space and the score distributions predicted by prototypes separately on source and target data are similar. Technically, TPN initially matches each target example to the nearest prototype in the source domain and assigns an example a "pseudo" label. The prototype of each class could then be computed on source-only, target-only and source-target data, respectively. The optimization of TPN is end-to-end trained by jointly minimizing the distance across the prototypes on three types of data and KL-divergence of score distributions output by each pair of the prototypes. Extensive experiments are conducted on the transfers across MNIST, USPS and SVHN datasets, and superior results are reported when comparing to state-of-the-art approaches. More remarkably, we obtain an accuracy of 80.4% of single model on VisDA 2017 dataset.
1. Introduction
Unsupervised domain adaptation addresses the costly need for target-domain annotations by transferring knowledge from labeled source data to unlabeled target data. TPN adapts Prototypical Networks by aligning class prototypes and classifier score distributions across domains.
- Unsupervised domain adaptation uses labeled source examples or models together with unlabeled target examples to generalize a target model.
- TPN learns an embedding in which source and target representations are intended to be invariant across domains.
- TPN assigns each target example the label of its nearest source-domain prototype as a pseudo label.
- General-purpose adaptation minimizes distances among prototypes computed from source data, pseudo-labeled target data, and combined source-target data.
- Task-specific adaptation aligns classifier score distributions across domain-specific or combined prototypes using KL-divergence.
2. Related Work
Prior unsupervised domain adaptation methods primarily reduce distribution discrepancy with MMD-based objectives or domain-confusion discriminators. TPN extends discrepancy-based adaptation with class-level prototype alignment and sample-level score-distribution alignment.
- MMD-based methods align source and target distributions using measures such as correlation distances or maximum mean discrepancy.
- Domain-discriminator methods learn domain-invariant representations by adversarially predicting whether each input belongs to the source or target domain.
- TPN belongs to domain discrepancy methods and uses unlabeled target data to learn task-specific classifiers.
- TPN introduces multi-granular discrepancy reduction through same-class prototype alignment and cross-domain score-distribution alignment.
3. Unsupervised Domain Adaptation
Unsupervised domain adaptation transfers knowledge from labeled source data to unlabeled target data by learning shared representations and classifiers. TPN remoulds Prototypical Networks to reduce domain discrepancies at class and sample levels using pseudo labels, prototype alignment, and score-distribution alignment.
- Prototypical Networks learn an embedding space where samples cluster around class prototypes and classify queries through distances to those prototypes.
- Unsupervised domain adaptation uses labeled source examples and unlabeled target examples to learn an embedding that reduces domain shifts and supports transferable representations and classifiers.
- TPN assigns each target sample the label of its nearest source-domain prototype, producing pseudo labels for subsequent adaptation.
- TPN computes prototypes from source-only, target-only, and combined source-target data, then minimizes pairwise RKHS distances between same-class prototypes.This class-level discrepancy objective is intended to bring corresponding prototypes into close proximity across domains.
- TPN aligns sample-level score distributions from classifiers built on different prototype sets using KL-divergence, complementing general-purpose feature-distribution adaptation.The score distributions are produced by applying prototype-based classifiers to source and target samples.
- Theoretical analysis decomposes TPN’s error bound into hypothesis-space domain discrepancy, joint-hypothesis error, and pseudo-label noise.The paper associates the first term with class-level prototype discrepancy and sample-level score-distribution discrepancy, while ρ denotes false pseudo-labels.
4. Experiments
TPN is evaluated on Digits and synthetic-to-real domain shifts, where it consistently outperforms competing adaptation methods and reaches 80.4% mean accuracy on VisDA 2017. Analyses associate its performance with class-level and sample-level discrepancy reduction, improved feature alignment, and decreasing pseudo-label noise during training.
- Datasets and Experimental Settings: TPN is evaluated across M →U, U →M, S →M and synthetic-to-real VisDA 2017 transfers using accuracy-based target-domain metrics.Digits experiments use MNIST, USPS and SVHN; VisDA uses synthetic training images as source and COCO validation images as target.
- Digits Image Transfer: 92.1% on M →U and 94.1% on U →M exceed the best competitor ADDA by 2.7% and 4%, respectively.TPN also improves classification on the harder S →M transfer compared with JAN.
- Digits Image Transfer: 98.6% on M →U exceeds 98.3% for S-En when TPN uses the same CNN architecture.The paper contrasts TPN’s 2-convolution-layer LeNet setting with S-En’s originally deeper 9-convolution-layer CNNs.
- Synthetic-to-Real Image Transfer: 80.4% mean accuracy across 12 VisDA categories exceeds JAN by 13.9% for TPN without data augmentation.The reported 80.4% is from a single model, whereas S-En’s 82.8% uses test-time augmentation; S-En reaches 74.2% with Mini-aug.
- Feature and Confusion-Matrix Analysis: TPN reduces source-target feature separation and confusion for most VisDA classes, achieving higher accuracy than JAN for 10 of 12 categories.The t-SNE visualization shows the adapted target distribution becoming indistinguishable from the source distribution.
- Convergence Analysis: During VisDA training, target-class discrimination improves, pseudo-label noise decreases, adaptation losses decline, and final accuracy reaches 80.4%.The analysis tracks accuracy together with class-level and sample-level domain discrepancy losses across iterations.
5. Conclusions
The paper presents TPN as an unsupervised adaptation framework that aligns domains at class and sample levels through prototypical representations. Experiments on Digits transfers and VisDA support its effectiveness, including state-of-the-art single-model performance on VisDA 2017.
- Conclusions: TPN aligns class prototypes across domains and classifier decisions across samples to learn domain-invariant representations.Its general-purpose component matches class prototypes, while its task-specific component aligns score distributions across prototypes.
- Conclusions: Experiments across MNIST, USPS and SVHN validate the proposal, while VisDA 2017 yields new state-of-the-art single-model performance.The conclusion summarizes evidence from both Digits and synthetic-to-real transfers.