Source-linked AI summary
Boosting Domain Adaptation by Discovering Latent Domains
Massimiliano Mancini, Lorenzo Porzi, Samuel Rota Bulò, Barbara Caputo, Elisa Ricci
TL;DR
Deep domain adaptation commonly assumes a single source distribution, despite datasets containing multiple and sometimes unlabeled domains. This paper introduces a CNN architecture that discovers latent domains and aligns their feature distributions, outperforming prior multi-source methods on benchmark datasets.
Problem
Deep domain adaptation commonly assumes a single source distribution, although practical datasets may contain multiple source domains with unavailable domain labels.
Method
A CNN side branch predicts source-sample assignments to latent domains, while domain-alignment layers use those assignments to align internal feature distributions.
Results
The proposed approach outperforms previous multi-source domain adaptation techniques on benchmarks including Office-31 and Office-Caltech.
Takeaways & Limitations
Automatically discovered latent domains can be exploited to reduce domain discrepancy and improve target classification in challenging multi-source adaptation tasks.
Takeaways & Limitations
The domain-prediction branch is split at a low-level layer because deeper features become increasingly domain invariant and harder to use for domain prediction.
Abstract
from arXiv · showhide
Current Domain Adaptation (DA) methods based on deep architectures assume that the source samples arise from a single distribution. However, in practice, most datasets can be regarded as mixtures of multiple domains. In these cases exploiting single-source DA methods for learning target classifiers may lead to sub-optimal, if not poor, results. In addition, in many applications it is difficult to manually provide the domain labels for all source data points, i.e. latent domains should be automatically discovered. This paper introduces a novel Convolutional Neural Network (CNN) architecture which (i) automatically discovers latent domains in visual datasets and (ii) exploits this information to learn robust target classifiers. Our approach is based on the introduction of two main components, which can be embedded into any existing CNN architecture: (i) a side branch that automatically computes the assignment of a source sample to a latent domain and (ii) novel layers that exploit domain membership information to appropriately align the distribution of the CNN internal feature representations to a reference distribution. We test our approach on publicly-available datasets, showing that it outperforms state-of-the-art multi-source DA methods by a large margin.
1. Introduction
Domain adaptation becomes difficult when visual training data combines multiple, partly unlabeled source domains. The paper proposes a deep architecture that discovers latent domains and uses them to align source and target feature distributions.
- Visual domain shift arises from differences such as resolution, camera pose, occlusions, and illumination, motivating domain adaptation.
- The framework automatically discovers multiple latent source domains and aligns their internal CNN representations with target representations for domain adaptation.
- Most domain adaptation methods focus on a single source and target, although practical vision datasets often contain multiple source distributions.
- When source-domain labels are unavailable or incomplete, latent domain discovery must jointly infer hidden domains and use them for target classification.
- The proposed deep architecture learns source-sample assignment variables and uses them to align internal CNN feature distributions to a reference Gaussian distribution.
2. Related Work
Earlier domain adaptation work includes instance-based, feature-based, and deep approaches, but multi-source methods generally require known domain labels. This paper addresses the remaining setting of deep latent-domain discovery for target classification.
- Hand-crafted-feature methods reduce source-target discrepancy through source reweighting or learning a common domain-invariant subspace.
- Multi-source methods can improve over a single-source classifier but assume domain labels for all source samples are known in advance.
- Latent-domain methods instead discover hidden source domains within training data while learning target classification models.
- Deep domain adaptation methods learn robust representations using neural networks, autoencoders, adversarial training, or distribution-alignment objectives.
- The paper presents the first deep architecture designed to discover latent source domains and exploit them for target classification.
3. Method
The method models multiple, partly unknown source domains and integrates domain prediction with multi-domain feature alignment in a CNN. It uses predicted or known membership probabilities to estimate domain-specific statistics, normalize features, and train classification and domain-prediction branches jointly.
- 3.1. Problem Formulation and Notation: The problem formulation includes labeled source samples from multiple domains, unlabeled target samples, and unknown or partially known source-domain assignments.The number of source domains is treated as a hyper-parameter, while domain labels may be available for only a subset of source samples.
- 3.2. Multi-domain DA-layers: Domain Alignment layers normalize activations separately by domain to match their first and second moments to a reference distribution.The reference distribution is generally Gaussian with zero mean and unit standard deviation; with current-batch statistics, this corresponds to domain-specific Batch Normalization.
- 3.2. Multi-domain DA-layers: Multi-domain DA-layers model each layer input as a mixture of Gaussian domain components when source-domain membership is unavailable.Each component represents one domain, with domain-specific mean and variance estimated from weighted assignments.
- 3.3. Domain prediction: The domain prediction branch supplies per-sample membership probabilities to the multi-domain alignment layers, replacing unavailable assignments with inferred weights.Known memberships can be fixed as one-hot weights, while inferred probabilities are used to estimate the alignment statistics and receive gradients through back-propagation.
- 3.3. Domain prediction: A shared low-level network splits into domain prediction and classification branches so domain assignments remain coherent across alignment layers.The split occurs at a low-level layer because deeper features become increasingly domain invariant and therefore less informative for predicting domains.
- 3.4. Training the network: Training estimates shared parameters for both branches using labeled and unlabeled data through a posterior or maximum-a-posteriori objective.The classification branch depends on the domain prediction branch through the multi-domain alignment layers, while the likelihood terms include source labels and known domains.
4. Experiments
Experiments evaluate the approach across digits, Office-31, Office-Caltech, and PACS using multiple CNN architectures and source-domain settings. Results show that automatically discovered latent domains generally improve target accuracy, though gains vary by dataset.
- Datasets and architectures: Experiments cover digits, Office-31, Office-Caltech, and PACS, using MNIST, AlexNet, and ResNet architectures.The evaluation uses several common domain-adaptation benchmarks and applies the method to three CNN architectures.
- Implementation: The digits experiments add mDA-layers and a domain-prediction side branch to the MNIST architecture.The side branch begins at conv1 and uses convolutional and fully connected layers for domain prediction.
- Digits results: On digits, the method discovers latent domains without labels, improves target classification, and remains comparable across different values of k.The authors report that it partially fills the gap between single-source models and multi-source DA, while performance changes little as k varies.
- PACS results: On PACS, the method automatically infers domain information and performs comparably to the multi-source model for Photo, Art, and Cartoon targets.For Sketch, its accuracy falls between the single-source DIAL and multi-source models.
- Office-31 results: On Office-31, latent-domain discovery only slightly improves performance over the single-source DA model.The authors associate the limited multi-source gain with relatively limited deep-feature domain shift in this dataset.
- Office-31 results: On Office-31, the proposed model outperforms all listed baselines and shallow multi-source DA methods in accuracy.This comparison is reported separately from the experiment where gains over the single-source DA model are only slight.
5. Conclusions
The paper presents a deep domain-adaptation model that discovers latent domains in visual datasets and uses them to reduce domain shift for target adaptation.
- A side branch assigns each source sample to a latent domain.
- Novel domain alignment layers exploit these assignments to align discovered-source and target feature distributions.
- Experiments demonstrate that the model can exploit discovered latent domains for challenging domain adaptation tasks.