Source-linked AI summary
Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach
Behnam Gholami, Pritish Sahu, Ognjen Rudovic, Konstantinos Bousmalis, Vladimir Pavlovic
TL;DR
The paper tackles the limited fit of pairwise uDA methods to settings with one labeled source and multiple unlabeled target domains. It proposes MTDA-ITA, which separates shared and private information using an information-theoretic objective, and reports improved performance across target domains and benchmark datasets.
Problem
Existing uDA work primarily addresses one labeled source and one unlabeled target, although applications may contain multiple unlabeled domains with common task-related content.
Method
MTDA-ITA jointly factorizes each domain into shared domain-invariant and private domain-specific representations using mutual-information objectives and variational lower bounds.
Results
MTDA-ITA improves performance on each target domain over independent pairwise adaptation, naive target fusion, and applicable state-of-the-art models, with results on three benchmark datasets.
Takeaways & Limitations
Multiple unlabeled target domains can be leveraged simultaneously for domain adaptation while modeling both information shared across domains and domain-specific information.
Takeaways & Limitations
The method does not directly minimize mutual information between shared and private features because computing or approximating it is intractable for the complex joint distribution.
Abstract
from arXiv · showhide
Unsupervised domain adaptation (uDA) models focus on pairwise adaptation settings where there is a single, labeled, source and a single target domain. However, in many real-world settings one seeks to adapt to multiple, but somewhat similar, target domains. Applying pairwise adaptation approaches to this setting may be suboptimal, as they fail to leverage shared information among multiple domains. In this work we propose an information theoretic approach for domain adaptation in the novel context of multiple target domains with unlabeled instances and one source domain with labeled instances. Our model aims to find a shared latent space common to all domains, while simultaneously accounting for the remaining private, domain-specific factors. Disentanglement of shared and private information is accomplished using a unified information-theoretic approach, which also serves to establish a stronger link between the latent representations and the observed data. The resulting model, accompanied by an efficient optimization algorithm, allows simultaneous adaptation from a single source to multiple target domains. We test our approach on three challenging publicly-available datasets, showing that it outperforms several popular domain adaptation methods.
1 Introduction
The paper addresses unsupervised adaptation from one labeled source to multiple unlabeled target domains that share task-related content but differ statistically. MTDA-ITA jointly exploits these domains by separating shared and private information, improving performance over independent and naive multi-target adaptation.
- Motivation: Unsupervised domain adaptation transfers knowledge from labeled source data to unlabeled target data when training and test distributions differ.Distribution shifts can arise from image noise, viewpoint changes, and related factors, degrading inference performance.
- Motivation: Most existing uDA methods address a single-source-single-target setting, despite applications involving multiple statistically distinct domains with common task-related content.Examples include object images captured by different cameras or views and facial expressions recorded from different people.
- Research gap: Simultaneous adaptation to multiple unlabeled target domains remains underexplored and is challenging because target domains lack supervision.Multi-target adaptation can enable direct knowledge sharing between the source and several targets, but its data integration strategy is critical.
- Approach: MTDA-ITA uses one source and multiple targets to factorize each domain into shared correlated subspaces and private domain-specific subspaces.The approach is designed for single-source-multi-target adaptation and can be extended to multiple source domains.
- Results: MTDA-ITA improves performance on each target domain compared with pairwise source-target adaptation, naive target fusion, and applicable state-of-the-art models.The method jointly maximizes mutual information between domain labels and private features while minimizing it between domain labels and shared features.
2 Preliminaries
Mutual information quantifies the dependence between random variables by measuring shared uncertainty reduction. The preliminaries define it through entropies and joint entropy.
- Mutual information: For random variables x and z, mutual information is defined as I(x; z) = H(x) + H(z) − H(x, z).Here, H denotes entropy and H(x, z) denotes joint entropy.
3 Method
The method factorizes each sample into shared and private latent features, using mutual-information objectives to preserve data and class information while separating domain-specific information. Separate encoders, a shared decoder, a class classifier, and a domain classifier are optimized together for simultaneous multi-target adaptation.
- Latent factorization: MTDA-ITA learns shared, domain-invariant features and private, domain-dependent features in factorized latent spaces.The latent representation is z = [zs, zp], where zs is shared and zp is private to each domain.
- Information-theoretic objective: The objective maximizes I(x; z), I(y; zs), and I(d; zp) − I(d; zs) with weighted terms λr, λc, and λd.These terms preserve information for reconstruction, support class prediction from shared features, and encourage domain information in private rather than shared features.
- Optimization: The domain classifier predicts domain labels from shared and private features, while optimization penalizes domain information in shared features and preserves it in private features.The shared encoder is trained adversarially against domain prediction, whereas the private encoder retains domain-discriminative information.
- Optimization: Variational lower bounds and deterministic encoder networks convert the mutual-information objective into a minimax optimization trained by alternating stochastic-gradient updates.The variational distributions approximate intractable posteriors, and the model alternates updates for encoders, decoder, classifiers, and variational parameters.
- Information-theoretic objective: A shared decoder reconstructs each input from shared and private features, preserving information in the latent representation.The decoder minimizes the difference between the original sample and its reconstruction from F(zs, zp).
- Optimization: The shared-feature classifier predicts source classes and reduces target prediction entropy, while balancing target labels to avoid assigning every target point to one class.The target entropy term encourages confident predictions away from decision boundaries, and the balancing term reaches ln K under uniform class probabilities.
4 Related Work
Prior work spans single-domain adversarial and distribution-alignment methods, multi-domain transfer, and information-theoretic representation learning. This paper distinguishes its approach through simultaneous feature-space adaptation across multiple domains with explicit shared/private representations.
- Distribution Alignment: MMD-, covariance-, and related alignment methods match statistical properties between source and target distributions in learned feature spaces.DAN applies MMD in an RKHS to match higher-order statistics, while CORAL matches means and covariances.
- Unsupervised Domain Adaptation: Adversarial uDA methods learn source-discriminative, domain-indistinguishable representations, including DANN, ADDA, and related frameworks.These approaches primarily address single source–target adaptation.
- Multi-Domain Adaptation: Existing uDA approaches are not designed to exploit information from multiple domains simultaneously, unlike the multi-domain setting targeted here.MDAN addresses multiple sources and one target, whereas simultaneous adaptation to multiple unlabeled targets is identified as an unexplored scenario.
- Information-Theoretic Representation Learning: Information-theoretic representation-learning methods use variational mutual-information bounds for deep models, but typically discard task-irrelevant information rather than model domain-specific features.The paper explicitly retains private features while extracting domain-invariant representations.
- Multiple Domain Transfer: Multi-domain transfer methods such as StarGAN and ComboGAN adapt representations in raw pixel space through domain-conditioned image translation.The paper instead performs domain alignment in feature space, which is presented as less limiting for larger domain shifts.
- Domain Separation Networks: Compared with DSN, MTDA-ITA couples private encoders, uses an information-theoretic domain classifier to separate private spaces, and performs better than a trivial multi-domain DSN extension.DSN requires an autoencoder per domain and uses Euclidean orthogonality, whereas MTDA-ITA supports an arbitrary number of target domains through a shared private encoder model.
5 Experimental Results
Experiments evaluate MTDA-ITA across digit, Multi-PIE, and PACS domain-adaptation benchmarks, comparing it with pairwise methods, combined-target baselines, and ablations. MTDA-ITA generally achieves the strongest performance, while removing its components degrades results.
- Benchmark setup: MTDA-ITA is evaluated on digit classification, Multi-PIE expression recognition, and PACS image recognition using classification accuracy.The experiments use publicly available datasets with multiple domains and compare methods under source-to-multiple-target adaptation settings.
- Benchmark setup: The comparison includes pairwise adaptation methods, combined-target variants, separate source-target MTDA-ITA models, and a multi-private-encoder DSN extension.The evaluated methods include CORAL, DANN, ADDA, DTN, UNIT, PixelDA, DSN, c-MTDA-ITA, s-MTDA-ITA, c-DTN, c-ADDA, c-DSN, and mp-DSN.
- Digits datasets: On the digit benchmark, naive target-domain combination can reduce performance, whereas MTDA-ITA outperforms state-of-the-art methods in most domain transformations.The authors attribute the advantage to jointly adapting related domains and using unlabeled target data during classifier learning.
- Multi-PIE dataset: MTDA-ITA achieves the best performances in most Multi-PIE settings, including challenging transfers where increasing camera angle makes views more heterogeneous.Each camera view is treated as a domain, with facial expressions used as labels.
- PACS dataset: On PACS, MTDA-ITA consistently achieves the best performance for all transfer tasks despite diverse styles and limited samples.The PACS domains are Photo, Art painting, Cartoon, and Sketch, and the models are trained from scratch on the dataset.
- Ablation studies: Ablations show average performance drops of approximately 3.5% without classifier entropy, 4.5% without reconstruction, 22% without multi-domain separation, and 2% without private features.The results support using complementary components, including explicit modeling of domain-specific features and separation of shared and private representations.
6 Conclusion
The paper introduces an information-theoretic end-to-end method for adapting one labeled source to multiple unlabeled target domains. It separates private and shared representations, achieving strong benchmark performance while remaining scalable and efficiently optimized.
- The method learns representations invariant to multiple domain shifts while remaining discriminative for the task.It explicitly separates domain-private representations from representations shared across source and target domains.
- A single private domain encoder makes the model highly scalable and compatible with established back-propagation optimization.
- Results on three benchmark image-classification datasets show superiority over state-of-the-art unsupervised domain-adaptation methods.
- The learned-feature visualization distinguishes large shared-space clusters from smaller domain-private clusters.In Figure 5, circles denote shared features and triangles denote private features.
A Network Architecture
The experimental network architecture uses standardized abbreviations to describe layer and model dimensions, convolutional operations, and residual blocks.
- N, K, S, D, and C denote neurons, kernel size, stride size, number of domains, and number of classes, respectively.
- DCONV denotes a transposed convolutional layer.
- RESBLK denotes a residual basic block in the network architecture.