Source-linked AI summary
Learning to Generate Novel Domains for Domain Generalization
Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, Tao Xiang
TL;DR
Domain generalization must handle unseen domains despite limited source diversity and the absence of target data. L2A-OT synthesizes pseudo-novel domains by maximizing optimal-transport divergence while using cycle-consistency and classification losses to preserve semantics. Experiments across four benchmark datasets show its effectiveness and versatility, including superiority to current state-of-the-art methods.
Problem
Domain generalization faces limited source-domain diversity, while alignment and meta-learning methods risk overfitting to seen domains without access to target data.
Method
L2A-OT maps source images to pseudo-novel domains with a conditional generator, maximizes OT distance from source domains, and applies cycle-consistency and classification losses.
Results
L2A-OT surpasses current state-of-the-art DG methods on four benchmark datasets spanning homogeneous and heterogeneous visual-recognition tasks.
Takeaways & Limitations
Synthesizing diverse unseen-domain images provides an effective and versatile data-augmentation strategy for improving robustness to domain shift.
Takeaways & Limitations
The generator’s novelty objective alone may produce diverse images without preserving semantic content, requiring semantic constraints.
Abstract
from arXiv · showhide
This paper focuses on domain generalization (DG), the task of learning from multiple source domains a model that generalizes well to unseen domains. A main challenge for DG is that the available source domains often exhibit limited diversity, hampering the model's ability to learn to generalize. We therefore employ a data generator to synthesize data from pseudo-novel domains to augment the source domains. This explicitly increases the diversity of available training domains and leads to a more generalizable model. To train the generator, we model the distribution divergence between source and synthesized pseudo-novel domains using optimal transport, and maximize the divergence. To ensure that semantics are preserved in the synthesized data, we further impose cycle-consistency and classification losses on the generator. Our method, L2A-OT (Learning to Augment by Optimal Transport) outperforms current state-of-the-art DG methods on four benchmark datasets.
1 Introduction
The paper targets domain generalization without target-domain data, arguing that limited source diversity and source-domain alignment can leave models overfit to seen domains. L2A-OT instead synthesizes pseudo-novel domains with optimal transport while preserving structural and semantic consistency.
- Domain generalization learns models that generalize directly to unseen target domains without target-data collection or model updates.
- Existing alignment and meta-learning approaches reduce gaps among source domains, but may overfit to seen domains because target data are unavailable.
- L2A-OT increases training-domain diversity by mapping source images to pseudo-novel domains and combining both domains for task-model training.
- Optimal transport trains the generator to maximize distance between source and generated domains, producing images with distributions different from the source domains.
- Cycle-consistency and classification losses maintain structural and semantic consistency during domain translation.
- L2A-OT surpasses current state-of-the-art methods on three homogeneous and one heterogeneous DG benchmark dataset.
2 Related Work
Related work includes domain alignment, adversarial-gradient augmentation, domain randomization, and image-to-image translation. L2A-OT differs by generating unseen domains through an OT-optimized full CNN generator rather than aligning sources or applying simple image perturbations.
- Domain-alignment DG methods minimize discrepancies among source domains, although their UDA-derived theory does not apply when target data are absent.
- Adversarial-gradient augmentation produces imperceptible, simple pixel-wise effects, whereas L2A-OT uses a full CNN generator to map source images to unseen domains.
- Domain generalization: Homogeneous DG shares label spaces between training and test data, whereas heterogeneous DG uses disjoint label spaces.
- Domain randomization: Domain randomization diversifies training domains by changing colors, textures, backgrounds, and lighting through a computer simulator.
- Image-to-image translation: CycleGAN and StarGAN translate between source domains using GAN and cycle-consistency losses, while L2A-OT maps source images to unseen domains.
3 Methodology
L2A-OT synthesizes pseudo-novel domains from source images and combines them with source data to train the task model. It maximizes distributional dissimilarity while using cycle-consistency and classification losses to preserve structure and semantics.
- Generating Novel-Domain Data: L2A-OT maps source-domain images to pseudo-novel domains and augments task-model training with both original and generated images.Each source domain is paired with a randomly selected novel-domain label during training.
- Generating Novel-Domain Data: The conditional generator maps a source distribution to a novel distribution conditioned on a novel-domain label.In practice, sampled mini-batches represent the empirical source distributions.
- Distribution Divergence: The generator maximizes divergence between source and generated distributions, and between generated novel distributions, to create dissimilar and diverse domains.The diversity constraint is summed over pairs of novel distributions and supports generating multiple novel domains.
- Maintaining Semantic Consistency: Cycle-consistency reconstructs the original input after translation back to its source domain, enforcing structural consistency.The forward and backward generator passes share parameters.
- Maintaining Semantic Consistency: A cross-entropy loss requires generated data to retain the category label of the original data, enforcing semantic consistency.The loss compares predictions from a pretrained classifier on generated data with the original ground-truth labels.
- Distribution Divergence: The distribution divergence uses Wasserstein, or optimal transport, distance estimated with entropy-regularized Sinkhorn distance and an energy-distance formulation for mini-batches.The transport cost is cosine distance between CNN-derived image representations, with the CNN fixed after domain-classification training.
4 Experiments
L2A-OT is evaluated across homogeneous DG benchmarks and cross-domain person re-identification using source-only training, with additional ablations and visual analyses of its synthesized domains. It achieves strong results across the reported settings while showing that semantic constraints, source-domain count, and generated-domain diversity affect the method’s behavior.
- Evaluation on Homogeneous DG: L2A-OT is evaluated on Digits-DG, PACS, and Office-Home using leave-one-domain-out domain generalization.Digits-DG contains four digit datasets; PACS has four visual domains and seven classes; Office-Home has four domains and 65 classes.
- Results on Digits-DG: L2A-OT achieves the best performance on all Digits-DG domains and improves over CrossGrad by 2.8% on MNIST-M, 3.3% on SVHN, and 3% on SYN.It also surpasses MMD-AAE and CCSA by more than 3.5% on average.
- Results on PACS and Office-Home: L2A-OT achieves the best performance on all PACS test domains and the best overall performance on Office-Home.On PACS it beats JiGen, Epi-FCR, and CrossGrad; on Office-Home, the vanilla model is comparatively strong because the dataset is larger and its domain shift is less severe.
- Evaluation on Heterogeneous DG: L2A-OT clearly improves over the vanilla model in both cross-domain person re-identification directions and is highly competitive with UDA methods without target-domain adaptation.The protocol tests source-trained models on disjoint target identities and unseen camera views.
- Ablation Study: Removing the cycle-consistency and cross-entropy semantic constraints reduces performance from 78.1% to 73.9%, indicating that they preserve semantic content during translation.Without the semantic constraint, the generated content is reported as completely missing.
- Further Analysis: Using three source domains generally outperforms using two, while performance is not very sensitive to the number of generated novel domains and K_n = K_s is a good rule of thumb.The choice of two sources affects performance because sources differ in transferability to a target; synthesizing more domains can increase diversity but also optimization difficulty.
5 Conclusion
L2A-OT improves domain generalization by synthesizing images from diverse pseudo-novel domains and demonstrates effectiveness across four benchmark datasets and varied visual recognition tasks.
- L2A-OT synthesizes images from diverse unseen domains through a conditional generator to improve classifier robustness to domain shift.
- The generator maximizes optimal-transport distance between source and pseudo-novel domains while using cycle-consistency and classification losses to preserve structural and semantic consistency.
- Four DG benchmark datasets covering a wide range of visual recognition tasks demonstrate L2A-OT's effectiveness and versatility.