Source-linked AI summary
Unified Deep Supervised Domain Adaptation and Generalization
Saeid Motiian, Marco Piccirilli, Donald A. Adjeroh, Gianfranco Doretto
TL;DR
The paper addresses visual recognition when target data are scarce or unavailable, where deep models and distribution-based alignment are difficult to train effectively. It introduces a Siamese CCSA framework using semantic alignment, separation, classification, and point-wise surrogates, then extends it to domain generalization. The approach is reported effective with very few labeled target samples and across both supervised adaptation and generalization.
Problem
Visual recognition often lacks sufficient labeled target data, while deep models require large labeled datasets and unsupervised adaptation expects substantial target data.
Method
A Siamese deep model learns an embedding with classification and contrastive semantic alignment losses, using point-wise surrogates and a loss modification for domain generalization.
Results
The approach works effectively with very few labeled target samples, converges quickly as samples increase, and extends to domain generalization without changing the basic architecture.
Takeaways & Limitations
Even one labeled target sample per category can significantly improve performance, while the same architecture supports effective supervised adaptation and generalization.
Takeaways & Limitations
The approach assumes covariate shift between source and target domains, with differing p(X_s) and p(X_t).
Abstract
from arXiv · showhide
This work provides a unified framework for addressing the problem of visual supervised domain adaptation and generalization with deep models. The main idea is to exploit the Siamese architecture to learn an embedding subspace that is discriminative, and where mapped visual domains are semantically aligned and yet maximally separated. The supervised setting becomes attractive especially when only few target data samples need to be labeled. In this scenario, alignment and separation of semantic probability distributions is difficult because of the lack of data. We found that by reverting to point-wise surrogates of distribution distances and similarities provides an effective solution. In addition, the approach has a high speed of adaptation, which requires an extremely low number of labeled target training samples, even one per category can be effective. The approach is extended to domain generalization. For both applications the experiments show very promising results.
1. Introduction
The paper targets visual recognition when labeled target data are scarce or unavailable, proposing one supervised framework for domain adaptation and generalization. It uses semantic alignment, cross-domain separation, classification, and point-wise surrogates to obtain effective adaptation with few labels and extend the approach to generalization.
- UDA often requires large amounts of target data, whereas scarce labeled target data make supervised domain adaptation more attractive.The paper notes that limited target data may also be less expensive to label.
- The framework supports both supervised domain adaptation and domain generalization with the same basic architecture and simple loss-function changes.The approach uses a supervised visual-recognition model and extends it from SDA to DG.
- Even one labeled target sample per category can significantly improve performance, while a few additional samples bring accuracy close to its peak.The paper describes this as a high speed of adaptation.
- CCSA combines semantic alignment with semantic separation so same-class cross-domain samples approach while different-class samples separate.The objective also includes classification to preserve recognition accuracy.
- Point-wise surrogates replace distribution distances and similarities because scarce target samples make those distributions difficult to represent.The resulting approach is reported as effective in experiments.
2. Related work
Related work addresses domain shift through mappings, shared spaces, or classifier regularization, while domain generalization learns from multiple source domains without target data. This paper uses a deep Siamese embedding that explicitly models semantic alignment and cross-domain separation for both adaptation and generalization.
- Domain adaptation addresses performance decreases when source-trained models encounter target data with a different marginal distribution.
- Traditional domain adaptation methods seek mappings, shared latent spaces, or classifier regularization between source and target distributions.Unsupervised methods occupy the first two categories, while supervised methods may use either approach.
- Siamese networks learn shared subspaces; this paper additionally introduces explicit cross-domain class-separation forces alongside semantic alignment.The method uses very few training samples and therefore relies on point-wise surrogates for difficult distribution characterization.
- Domain generalization combines multiple related source datasets to learn classifiers less sensitive to the unseen target data.
- The proposed deep Siamese approach explicitly accounts for semantic alignment and separation and is presented as the first deep method addressing both adaptation and generalization.
- In domain generalization, training uses pairwise source-domain losses to make the embedding domain invariant for unseen distributions.Figure 2 depicts colors as domain distributions and shapes as classes.
3. Supervised DA with Scarce Target Data
The paper formulates supervised domain adaptation with scarce labeled target data and learns an embedding that aligns same-class samples across domains while separating different classes. To make this feasible with very few target examples, it replaces distribution-level distances and similarities with point-wise surrogates.
- Problem: Supervised domain adaptation learns a target-performing predictor from source and limited labeled target data under covariate shift between source and target distributions.The setting may include one labeled target sample per class or no target samples for some classes.
- Motivation: UDA aligns source and target feature distributions, but small target datasets make distribution estimates unreliable and confusion alignment does not ensure same-class cross-domain proximity.The paper identifies semantic alignment as a limitation of unsupervised alignment.
- Deep SDA: The proposed Siamese embedding maps same-class samples from different domains nearby and adds separation forces for different-label cross-domain samples.The semantic alignment loss encourages nearby same-label embeddings, while the separation loss penalizes close different-label distributions.
- Deep SDA: CCSA combines source classification, semantic alignment, and separation losses, with a shared embedding function and optional target fine-tuning using the few available target samples.The classification network is trained with source data, while target adaptation can use fine-tuning on Dt.
- Handling Scarce Target Data: With scarce target data, CCSA replaces distribution distances and similarities with average pairwise point distances and similarities, allowing one target sample to pair with all source samples.The resulting losses are normalized and weighted to balance classification against contrastive semantic alignment.
- Handling Scarce Target Data: Figure 4 reports CCSA improvement over a source-only base model and average M →U accuracy across different numbers of labeled target samples per category.The supplied figure text identifies the comparison and reports significant improvement over baselines without providing numerical values.
4. Extension to Domain Generalization
For domain generalization, the method learns a domain-invariant embedding from multiple labeled source domains because the target domain is unknown during training. It applies semantic alignment and separation across every pair of source domains, with random pairing used when quadratic sample growth becomes impractical.
- Domain generalization learns from multiple labeled source domains to classify an unknown target domain unavailable during training.
- The embedding maps source domains into a domain-invariant space without adapting the classifier to the unknown target.
- Semantic alignment and separation losses are imposed on every distinct unordered pair of source domains.
- The domain-generalization architecture retains the Siamese network and the distance and similarity choices used for supervised adaptation.
- Quadratic growth in paired training samples can be handled by randomly selecting paired samples instead of processing every pair.
5. Experiments
Experiments evaluate CCSA for supervised domain adaptation and domain generalization across Office, MNIST-USPS, and VLCS benchmarks. CCSA performs well with few labeled target samples, improves over comparison methods, and achieves higher average domain-generalization performance.
- Experimental setup: CCSA is evaluated for domain adaptation and domain generalization on Office, MNIST-USPS, and VLCS benchmark datasets.The experiments compare CCSA with state-of-the-art methods under benchmark settings.
- Office dataset: CCSA achieves better Office-domain adaptation performance than [60], with larger margins on several larger domain shifts.The evaluated shifts include A →W, A →D, W →A, W →D, D →A, and D →W.
- Office dataset: CCSA transfers information from labeled classes to unlabeled target classes and remains effective when target-domain shifts are larger.This experiment trains with labeled target samples from 15 classes and tests on all 31 classes.
- Office dataset: CCSA provides improved accuracy over GFK, mSDA, RTML, and CDML on the Office-domain adaptation comparison.The comparison uses three UDA methods and one SDA method under the same settings.
- MNIST-USPS datasets: CCSA works with one labeled target sample per category, and increasing samples quickly brings accuracy near its top level on MNIST-USPS.The MNIST-USPS experiments use n from 1 to 8 labeled target samples per class, with 10 repetitions.
- MNIST-USPS datasets: Semantic alignment and separation together produce the best MNIST-USPS ablation performance, while weight sharing outperforms separate weights.For M →U with n = 4, separate weights achieve average accuracy 88.6 over 10 repetitions, below the weight-sharing result.
- VLCS dataset: CCSA achieves higher average performance than the compared domain-generalization methods on VLCS, while all DG methods outperform the base model.The DG comparison uses DeCaF-fc6 features and two fully connected layers.
6. Conclusions
The deep CCSA model addresses both supervised domain adaptation and domain generalization, using point-wise surrogates to align semantic distributions effectively with few labeled target samples.
- The CCSA loss addresses supervised domain adaptation and can be augmented for domain generalization without changing the basic model architecture.
- Point-wise surrogates of distribution distances and similarities make semantic alignment effective for both supervised adaptation and domain generalization with very few labeled target samples.
- Supervised domain adaptation accuracy converges quickly as more labeled target samples per category become available.