Source-linked AI summary
Learning to Diversify for Single Domain Generalization
Zijian Wang, Yadan Luo, Ruihong Qiu, Zi Huang, Mahsa Baktashmotlagh
TL;DR
Single-DG must generalize from limited diversity in one source domain to unseen target domains. L2D synthesizes complementary styles with a style-complement module and iteratively minimizes source–generated mutual information while maximizing same-category mutual information. Across three benchmark datasets, it outperforms state-of-the-art single-DG methods by up to 25.14%.
Problem
Single-DG uses only one source domain, limiting diversity for generalization to unseen target domains.
Method
L2D synthesizes complementary out-of-domain images and iteratively minimizes source–generated MI while maximizing MI among same-category samples.
Results
L2D surpasses state-of-the-art DG and Single-DG methods across three benchmark datasets, with improvements of up to 25.14%.
Takeaways & Limitations
Generating diverse styles and learning style-invariant representations improves the model's generalization capacity in Single-DG.
Takeaways & Limitations
The MI upper-bound estimator is reliable when the two distributions differ only slightly.
Abstract
from arXiv · showhide
Domain generalization (DG) aims to generalize a model trained on multiple source (i.e., training) domains to a distributionally different target (i.e., test) domain. In contrast to the conventional DG that strictly requires the availability of multiple source domains, this paper considers a more realistic yet challenging scenario, namely Single Domain Generalization (Single-DG), where only one source domain is available for training. In this scenario, the limited diversity may jeopardize the model generalization on unseen target domains. To tackle this problem, we propose a style-complement module to enhance the generalization power of the model by synthesizing images from diverse distributions that are complementary to the source ones. More specifically, we adopt a tractable upper bound of mutual information (MI) between the generated and source samples and perform a two-step optimization iteratively: (1) by minimizing the MI upper bound approximation for each sample pair, the generated images are forced to be diversified from the source samples; (2) subsequently, we maximize the MI between the samples from the same semantic category, which assists the network to learn discriminative features from diverse-styled images. Extensive experiments on three benchmark datasets demonstrate the superiority of our approach, which surpasses the state-of-the-art single-DG methods by up to 25.14%.
1. Introduction
Single-DG addresses the realistic setting where only one source domain is available, making generalization to unseen target domains especially challenging. L2D responds by synthesizing diverse complementary styles and iteratively optimizing mutual information to improve generalization.
- Motivation: Domain shift can substantially degrade models trained on source samples when target samples follow different distributions.The shift may arise from changes in illumination, object appearance, or background.
- Problem: Single-DG trains with one source domain and remains under-explored because prior DG methods lose access to multiple sources and domain identifiers.Without domain information, alignment- and augmentation-based methods cannot readily identify domain-invariant features or robust transformations.
- Approach: L2D uses a style-complement module to synthesize samples with unseen styles that complement the source distribution.Unlike image-space Euclidean diversification, it diversifies generated samples in latent feature space.
- Approach: Its min-max mutual information strategy minimizes dependence between generated and source images, then maximizes mutual information among samples sharing a semantic category.The two-step optimization enlarges distribution shifts while encouraging discriminative style-invariant representations.
- Evaluation: L2D is evaluated on digits recognition, corrupted CIFAR-10, and PACS, with results surpassing state-of-the-art DG and Single-DG methods on all datasets.The study also evaluates standard DG using a leave-one-domain-out protocol.
2. Related Work
Prior work addresses domain shift through adaptation, multi-source generalization, and augmentation, while Single-DG trains on one source domain and tests across unseen domains. L2D differs by generating larger-shift samples with novel styles, textures, and appearances that complement the source distribution.
- Domain Adaptation and Generalization: Domain adaptation uses source knowledge with unlabeled or limited labeled target data, whereas domain generalization does not access the target domain during training.DG typically learns from multiple source domains.
- Data Augmentation: Data-augmentation methods generate out-of-domain samples or auxiliary signals to train networks alongside source samples.Examples include adversarial hard samples, domain-change directions, pseudo-novel domains, and jigsaw-based self-supervision.
- Single-DG: Single-DG trains on one source domain and evaluates on multiple unseen domains, with prior methods using gradient-based augmentation and adversarial perturbations.These methods mainly express visual differences through color and texture changes.
- L2D: L2D generates novel style, texture, and appearance with a larger shift from the source distribution, making its samples complementary to the source data.This distinguishes it from existing Single-DG methods focused on smaller visual changes.
3. Methodology
L2D alternates style-diverse sample generation with discriminative, style-invariant representation learning for Single-DG. Its style-complement module diversifies source features while semantic-consistency and same-class objectives preserve useful category information.
- Style-Complement Module: The style-complement module G synthesizes out-of-domain images that retain the source image’s semantic information while adopting different styles.It uses convolution, style learning, and transposed-convolution transformations to reconstruct augmented images, whose weighted combination forms x+.
- MI Maximization: After generation, the task model maximizes MI among source and generated samples sharing a semantic category to learn discriminative features from diverse styles.The supervised contrastive loss uses same-class source and generated latent representations, while cross-entropy is applied to both source and generated images.
- Synthesizing Novel Styles: The method minimizes an MI upper-bound approximation between each source and generated latent pair to diversify generated features.A variational distribution q(z+|z) approximates the otherwise intractable conditional distribution; minimizing the resulting objective trains mean/variance shifts to complement source styles.
- Semantic Consistency: Conditional MMD constrains distribution shifts between source and augmented samples within each semantic class, reducing semantic distortion.The constraint is motivated by the risk that style generation can introduce noise or semantically unrelated images.
- Objective Function: Training alternates optimization of the style-complement module and task model using weighted combinations of the method’s generation, consistency, and task losses.The task model includes F, q, and H; α1, α2, and β balance the relevant objectives.
4. Experiments
Experiments across digits, corrupted CIFAR-10, and PACS evaluate L2D under single-domain and leave-one-domain-out generalization settings. Results, ablations, sensitivity analysis, and feature visualizations support the value of diverse style synthesis and mutual-information optimization.
- Comparisons on Digits: Digits experiments train on MNIST and test on SVHN, MNIST-M, SYN, and USPS, using 10,000 MNIST source images and LeNet.All grayscale images are resized to 32 × 32 and converted to RGB by channel duplication.
- Comparisons on Digits: 20.3%, 29.36%, 13.33%, and 14.9% improvements are reported on SVHN, MNIST-M, SYN, and overall accuracy, respectively.L2D achieves the second-best performance on USPS, where MNIST and USPS have similar stroke styles.
- Comparisons on PACS: On PACS, models train on photo and evaluate on art painting, cartoon, and sketch, with L2D achieving the best average classification accuracy among baselines.The largest performance margin occurs on sketch, described as having the largest shift from photo because of highly abstracted shapes.
- Comparisons on PACS: Without domain identifiers, L2D achieves state-of-the-art performance on PACS under leave-one-domain-out domain generalization.The training augmentation module gradually enlarges the domain shift between generated and source images.
- Empirical Analysis: Removing style modification lowers absolute performance by 1.35%, while removing mutual-information maximization creates a 3.76% gap from the full model.Removing mutual-information minimization lowers accuracy by 1.26% and encourages generated images to follow the source distribution, limiting diversity.
- Empirical Analysis: t-SNE visualizations show better class-wise separation for L2D, whereas ERM, ADA, and ME-ADA exhibit multiple sub-clusters within some classes.The visualizations use unseen target features from SVHN, SYN, USPS, and MNIST-M, with same-label features plotted in the same color.
5. Conclusion
Learning-to-Diversify (L2D) addresses single domain generalization by generating diverse out-of-domain images from one source domain and optimizing mutual information to improve representation learning. Experiments show that it outperforms state-of-the-art methods in both single-domain and standard leave-one-domain-out generalization.
- L2D generates diverse out-of-domain images from a single source domain through a style-complement module.
- An iterative min-max mutual information strategy diversifies generated images while learning style-invariant representations from same-category samples.The method minimizes a tractable MI upper bound for diversity and maximizes same-category MI for representation learning.
- L2D outperforms state-of-the-art methods on three benchmark datasets in both single-domain and standard leave-one-domain-out domain generalization.