Source-linked AI summary
Curriculum Manager for Source Selection in Multi-Source Domain Adaptation
Luyu Yang, Yogesh Balaji, Ser-Nam Lim, Abhinav Shrivastava
TL;DR
MS-UDA aims to transfer from labeled source domains to an unlabeled target, but source diversity and unknown domain structure make transfer challenging. The paper proposes CMSS, an adversarial agent that learns a changing source-sample curriculum without domain labels. It reports state-of-the-art results on four benchmarks and interpretable evidence about the curriculum.
Problem
MS-UDA must transfer from labeled sources to an unlabeled target, while source diversity can make alignment difficult and potentially cause negative transfer, and real data may have unknown domain labels and counts.
Source: §1 · pp. 1–2Method
CMSS is an independent adversarial agent that continually updates a curriculum to favor source samples suited to target alignment by remeasuring transferability and weighting feature-network gradients.
Source: §1 · p. 2Results
CMSS achieves state-of-the-art results on four benchmark datasets, with interpretable results showing its learned curriculum's effect on MS-UDA.
Source: §1 · p. 2, §7 · p. 14Takeaways & Limitations
The authors conclude that adversarially learned source curricula can favor better-aligning samples over training without requiring domain partitioning prior or clustering.
Source: §1 · p. 2, §7 · p. 14The task formulation assumes source-domain labels and the number of source domains are unknown a priori.
Source: Task Formulation · p. 4
Abstract
from arXiv · showhide
The performance of Multi-Source Unsupervised Domain Adaptation depends significantly on the effectiveness of transfer from labeled source domain samples. In this paper, we proposed an adversarial agent that learns a dynamic curriculum for source samples, called Curriculum Manager for Source Selection (CMSS). The Curriculum Manager, an independent network module, constantly updates the curriculum during training, and iteratively learns which domains or samples are best suited for aligning to the target. The intuition behind this is to force the Curriculum Manager to constantly re-measure the transferability of latent domains over time to adversarially raise the error rate of the domain discriminator. CMSS does not require any knowledge of the domain labels, yet it outperforms other methods on four well-known benchmarks by significant margins. We also provide interpretable results that shed light on the proposed method.
1 Introduction
The introduction frames MS-UDA as transferring from labeled source data to an unlabeled target when more source samples may bring diverse domains and negative transfer. It presents CMSS, an adversarially learned curriculum, and reports state-of-the-art results on four benchmarks.
1 Introduction
- MS-UDA uses labeled source data to learn a representation for an unlabeled target, but exact alignment of every source domain may increase training difficulty and create ambiguous decision-boundary representations that can result in negative transfer.More labeled source samples do not necessarily improve transfer because they may encompass more domains.Source: §1 · p. 1
- The practical setting may have an unconstrained number of source domains and no known domain labels, as with Internet images from potentially many users.Source: §1 · p. 2
- CMSS is an independent adversarial agent that continually remeasures latent-domain transferability and weights source gradients to favor samples suited to target alignment.It does not require prior domain partitioning or clustering.Source: §1 · p. 2
- CMSS achieves state-of-the-art results on four benchmarks, including large-scale DomainNet with approximately 0.6 million images, and provides interpretable results about its curriculum behavior.The authors describe CMSS as having a substantial effect compared with prior art.Source: §1 · p. 2
2 Related Work
Related work covers unsupervised and multi-source domain adaptation, including latent-domain discovery and methods for weighting or selecting sources. Domain-adversarial approaches train a discriminator and feature network in opposition, while CMSS uses an independently learned curriculum.
2 Related Work
- MS-UDA treats labeled source domains and an unlabeled target as a multi-modal adaptation setting; prior work includes moment alignment and DCTN's adversarial and category classifiers [32].Source: §2 · pp. 2–3
- Figure 1 depicts CMSS repeatedly favoring source samples with stronger transferability and shows its weights tending toward similarity after most samples are aligned.Source: Figure 1 · p. 3
Domain-Adversarial Training
- Domain-adversarial training pits a discriminator that distinguishes source from target features against a feature network trained to fool it; MCD instead uses two classifiers and prediction discrepancy.DANN [15] is the cited foundational approach, and Zhao et al. extend it to multiple sources.Source: Domain-Adversarial Training · p. 3
- Prior source-selection and weighting methods include domain selection with loosely labeled web images, iterative selection, hand-crafted reweighting, graph-based dependency modeling, and discriminator-based weighting.Source: §2 · p. 3, Domain-Adversarial Training · p. 4
- Figure 2 compares DANN [15], IWAN [43], and the proposed architecture, with red dotted lines marking backward passes and separate labels for domain loss and weighted domain loss.Source: Figure 2 · p. 4
- Curriculum methods for domain adaptation include loss-based weighting, prototype confidence with a threshold, and dropout discriminators that simulate increasing sample variance [20].Source: Domain-Adversarial Training · p. 4
3 Preliminaries
The preliminaries formulate MS-UDA as classification with multiple labeled source domains and an unlabeled target. They describe domain-adversarial training and explain why unknown source-domain labels preclude discriminator designs that require those labels.
Task Formulation:
- Because source-domain labels and the number of domains are unknown, the setup cannot use either separate source-target discriminators or a (k + 1)-way discriminator requiring domain labels.Source: Task Formulation · pp. 4–5
- MS-UDA seeks to train on multiple source domains and an unlabeled target to improve performance on the target test set.The task is framed as classification, with source samples carrying labels and domain labels not known a priori.Source: Task Formulation · p. 4
- Domain-adversarial training minimizes source-target feature-distribution distance by training a discriminator to distinguish the domains and a feature network to fool it while supporting classification.The model comprises a feature extractor, classifier, and domain discriminator.Source: Task Formulation · p. 5
4 CMSS: Curriculum Manager for Source Selection
CMSS learns a changing source-sample curriculum to align selected samples with the target, using discriminator loss to update sample preferences. The section also relates reweighting to an adaptation bound and describes relaxations intended to limit overfitting.
4 CMSS: Curriculum Manager for Source Selection
- The curriculum first favors samples with feature distributions similar to the target, then shifts to other samples as alignment progresses.The method uses different aspects of feature distribution as proxies for transferability.Source: §4 · pp. 5–6
- CMSS continually re-estimates source-sample preferences, using discriminator loss to prioritize samples for target alignment as the discriminator improves.It represents preferences as per-sample weights and uses an independent feature extractor so the classification task does not influence curriculum design.Source: §4 · pp. 5–6
- CMSS scores source samples and softmax-normalizes the scores into weights used to form the weighted domain-discriminator loss.The weighted loss encourages alignment of higher-weight source samples through the gradient-reversal framework.Source: §4 · p. 6
4.1 CMSS: Theoretical Insights
- The reweighted adaptation bound combines weighted source risk and weighted symmetric divergence; uniform weights recover the original bound.The section describes minimizing the bound over source-sample weights to obtain a tighter bound.Source: §4.1 · p. 8
Relaxations.
- Unrestricted bound minimization can concentrate weight on a few source samples, risking overfitting despite low training risk and domain divergence.The paper introduces relaxations: use unweighted source risk, optimize divergence weights over mini-batches, and make weights input-dependent to produce soft selection.Source: Relaxations · p. 8
5 Experimental Results
The experiments evaluate CMSS across digit recognition, DomainNet, PACS, and Office-Caltech10, and compare its sample reweighting with IWAN. CMSS reports strong average-accuracy results across these benchmarks and examines how its sample weights behave.
5 Experimental Results
- The evaluation selects each domain in turn as target, uses the remaining domains as sources, and matches architectures, batch sizes, and preprocessing across methods.Digit experiments sample 25,000 training and 9,000 test images per dataset, except that they use the full USPS dataset.Source: §5 · pp. 8–9, §5.1 · p. 9
5.1 Experiments on Digit Recognition
- 90.8% average accuracy on digit recognition, about a 3% improvement over the previous state-of-the-art approach.Experiments use MNIST, MNIST-M, SVHN, Synthetic Digits, and USPS.Source: Table 1 · p. 9, §5.1 · p. 9
5.2 Experiments on DomainNet
- 46.5% average accuracy on DomainNet; CMSS leads every experimental setting and is the only method to beat Source Only for Quickdraw.The authors report that Quickdraw has substantial domain shift and that CMSS alleviates negative transfer in this setting.Source: Table 2 · p. 10, §5.2 · p. 10
5.3 Experiments on PACS
- 89.5% average accuracy on PACS, including 82.0% on the challenging sketch target, outperforming other baselines by a large margin.PACS includes art, cartoon, photo, and sketch domains.Source: §5.3 · p. 10, Table 3 · p. 11
5.4 Experiments on Office-Caltech10
- 97.2% state-of-the-art average accuracy on Office-Caltech10.The dataset comprises Amazon, Caltech, DSLR, and Webcam domains.Source: §5.4 · pp. 10–11, Table 4 · p. 11
5.5 Comparison with other re-weighting methods
- CMSS's separate-network weighting outperforms IWAN, which reaches 43.1% average accuracy, close to DANN with combined sources.CMSS produces weights with larger variance than IWAN's weights, which stay close to 1; CMSS also converges faster and more stably in test accuracy than DANN in the single-source comparison.Source: Table 5 · p. 11, §5.5 · p. 11
- Figure 3 plots weight means and variances over training, contrasting CMSS's more varied weights with IWAN's weights near 1.The authors interpret CMSS's larger weight variance as evidence of sample selection.Source: Figure 3 · p. 11, §5.5 · p. 11
- Figure 4 counts source-domain samples whose predicted weights exceed a manually chosen threshold for each target-domain setting.Bars represent source domains, and the maximum count in each plot is highlighted in red.Source: Figure 4 · p. 12
6 Interpretations
CMSS’s interpretations examine how source selection varies by target and training stage, across domains and individual samples.
6.1 Visualizations of source selection
- CMSS favors visually similar domains for Clipart, Painting, Infograph, Sketch, and Real targets, but shows no domain preference for Quickdraw.The authors attribute Quickdraw’s exception to its substantial shift from the other domains.Source: Domain Preference · p. 12
6 Interpretations
- Figure 6 compares test accuracy over training epochs for CMSS and DANN with source domains combined as one.The caption identifies the comparison but does not report an outcome.Source: Figure 6 · p. 13
Domain Preference
- Even without aggregate domain preference for Quickdraw, CMSS shows within-domain sample preference and produces better performance on that target.The paper points to ranked source samples as an example of this finer-grained selection.Source: Domain Preference · p. 13
- CMSS ranks source images by learned weights; Figure 5 shows top- and bottom-ranked Clock samples for Clipart and Quickdraw targets.The displayed weights were obtained at inference using CMSS trained for five epochs.Source: Beyond Domain Preference · p. 13, Figure 5 · p. 13
Beyond Domain Preference
- For Clock samples, CMSS ranks examples differently by target: it favors color and cartoon-like shapes for Clipart, but structural similarity for Quickdraw.For Quickdraw, the paper notes that selection is not simply based on white backgrounds.Source: Beyond Domain Preference · pp. 13–14, Figure 5 · p. 13
6.2 Selection Over Time
- Initially, CMSS assigns varied sample weights, then most weights approach 1; target Sketch features align first with Art, then gradually with Cartoon and Photo.The paper presents the feature progression in t-SNE visualizations across training epochs.Source: §6.2 · p. 14, Figure 7 · p. 14
7 Conclusion
The conclusion presents CMSS as a curriculum-learning approach for multi-source unsupervised domain adaptation, with benchmark results and interpretations of its operation.
7 Conclusion
- CMSS iteratively favors source samples that align better with the target, using adversarial interplay with the discriminator, and achieves state-of-the-art results on four benchmark datasets.The paper also reports examining CMSS’s inner workings.Source: §7 · p. 14