Source-linked AI summary
Deep Domain-Adversarial Image Generation for Domain Generalisation
Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, Tao Xiang
TL;DR
Domain shift makes models trained on source domains unreliable on unseen target domains, while collecting and annotating every new domain is impractical. DDAIG synthesizes label-preserving unseen-domain images with an adversarial transformation network and improves unseen-domain generalization, beating second-best PACS methods by around 1.5% on Avg.
Problem
Domain shift violates the shared source–target distribution assumption, while collecting and annotating data for every unseen domain is impractical.
Method
DDAIG trains a domain transformation network to create label-preserving unseen-domain images while classifiers enforce class correctness and domain confusion, then augments source training data.
Results
DDAIG improves unseen-domain generalization and outperforms current state-of-the-art DG methods; on PACS, it beats second-best methods by around 1.5% (on Avg.).
Takeaways & Limitations
Results on cross-domain person re-ID demonstrate DDAIG’s versatility beyond domain generalisation.
Takeaways & Limitations
DDAIG assumes access to multiple source domains, as is typical in domain-generalisation tasks.
Abstract
from arXiv · showhide
Machine learning models typically suffer from the domain shift problem when trained on a source dataset and evaluated on a target dataset of different distribution. To overcome this problem, domain generalisation (DG) methods aim to leverage data from multiple source domains so that a trained model can generalise to unseen domains. In this paper, we propose a novel DG approach based on \emph{Deep Domain-Adversarial Image Generation} (DDAIG). Specifically, DDAIG consists of three components, namely a label classifier, a domain classifier and a domain transformation network (DoTNet). The goal for DoTNet is to map the source training data to unseen domains. This is achieved by having a learning objective formulated to ensure that the generated data can be correctly classified by the label classifier while fooling the domain classifier. By augmenting the source training data with the generated unseen domain data, we can make the label classifier more robust to unknown domain changes. Extensive experiments on four DG datasets demonstrate the effectiveness of our approach.
Introduction
Domain generalisation addresses the impracticality of collecting labelled data for every target domain by training on multiple source domains without adaptation. DDAIG tackles this problem by synthesising label-preserving images from unseen domains and augmenting source data, with experiments demonstrating effectiveness across several benchmarks.
- Problem: Source and target data often follow different distributions, making same-domain training assumptions invalid in practical deployments across cities and weather conditions.
- Motivation: Collecting and annotating large-scale data for every new target domain is prohibitively expensive and time-consuming, limiting supervised fine-tuning.
- DDAIG: DDAIG learns a domain transformation network to create unseen-domain images that maintain class labels while changing domain-related properties, then trains the label classifier on original and transformed images.
- Domain generalisation: Domain generalisation trains on multiple source domains and deploys to arbitrary unseen target domains without adaptation, but source-domain alignment offers no guarantee of unseen-domain generalisation.
- Evaluation: Experiments evaluate DDAIG on PACS, Office-Home, and digit recognition across MNIST, MNIST-M, SVHN, and SYN, covering varied recognition tasks and domain variations.
Related Work
Deep DG research is organized around domain alignment, meta learning, and data augmentation. DDAIG addresses limitations of gradient-based augmentation by learning a full CNN to synthesize more sophisticated domain shifts.
- Deep domain generalisation: Deep DG studies generally fall into domain alignment, meta learning, and data augmentation, following earlier kernel-alignment and exemplar-SVM approaches.The paper identifies these as the three main groups of current deep DG research.
- Domain alignment: Domain-alignment methods minimize source–target distribution distance or fool a domain classifier, and can be adapted from domain adaptation to domain generalisation.Examples include maximum mean discrepancy and domain-classifier-based adversarial training.
- Meta learning: Meta-learning approaches simulate domain shifts during training so models are prepared to generalize to unseen domains at test time.These methods were adapted from computer-vision few-shot learning to the DG setting.
- Data augmentation: Conventional augmentation handles simple geometric changes, whereas large domain gaps involving style variation require learning-based augmentation strategies.The paper motivates learned augmentation for domain changes that exceed flipping and rotation.
- Data augmentation: Gradient-based adversarial augmentation produces subtle perturbations that cannot capture semantic shifts such as style or font changes.The cited methods are designed for imperceptible image modifications, limiting their representativeness of real-world domain shifts.
- DDAIG motivation: In contrast, DoTNet learns a full CNN to generate more sophisticated and overt shifts, producing interpretable synthetic data with quantitative and qualitative advantages over CrossGrad.The paper presents DoTNet as a response to the simplicity and subtlety of direct gradient-based perturbations.
Methodology
DDAIG trains a differentiable domain transformation network alongside label and domain classifiers to synthesize unseen-domain data and learn a domain-invariant classifier. Its adversarial multi-class domain objective and residual perturbation design aim to avoid collapsing generated data into a single source-domain distribution.
- DDAIG framework: DDAIG trains DoTNet with label and domain classifiers, combining original and synthetic data to learn a domain-invariant classifier.DoTNet synthesizes data from unseen domains, while the label and domain classifiers recognize class labels and input domains.
- Joint optimization: Differentiable networks allow gradients to back-propagate through the label and domain classifiers to DoTNet.The label classifier receives both original and synthetic data, and the losses are cross-entropy losses for label and domain classification.
- Adversarial domain objective: The domain classifier performs multi-class source-domain classification rather than GAN-style binary real-or-fake discrimination, preventing synthetic data from being forced into one alternative domain.The domain classifier minimizes domain classification loss, while the adversarial objective maximizes that loss for generated data.
- DoTNet architecture: DoTNet uses a residual connection from input to output and may produce visually perceptible perturbations rather than imperceptible adversarial changes.The perturbation design is generic to differentiable architectures, while the illustrated architecture uses convolutional residual blocks and global context features.
Experiments
Experiments evaluate DDAIG across conventional domain-generalisation benchmarks and cross-dataset person re-identification, with ablations examining transformation strength and source–novel data aggregation. The results show consistent gains over vanilla baselines and preserve category information while altering domain-related image properties.
- Datasets and protocols: DDAIG is evaluated on Digits-DG, PACS, Office-Home, and cross-dataset person re-identification under leave-one-domain-out or cross-domain protocols.Digits-DG combines MNIST, MNIST-M, SVHN, and SYN, while Office-Home contains Artistic, Clipart, Product, and Real World domains.
- Evaluation: Performance is measured as top-1 classification accuracy averaged over five runs with 95% confidence intervals and compared against established DG methods.The comparison includes CCSA, MMD-AAE, CrossGrad, MetaReg, D-SAM, JiGen, and Epi-FCR.
- Benchmark results: DDAIG achieves clear gains on Office-Home despite a strong Vanilla baseline, while other DG methods including MMD-AAE, D-SAM, and JiGen are largely outperformed by Vanilla.The smaller Office-Home domain gap is attributed mainly to background and viewpoint variation.
- Person re-identification: DDAIG consistently improves upon the vanilla baseline on both cross-domain person re-identification settings, with noticeable margins.The comparison includes CrossGrad and the strong vanilla model.
- Ablation studies: All tested λ values outperform Vanilla, but performance is moderately sensitive to λ when the target domain has a larger gap from the source domains.Sensitivity is smaller for targets such as MNIST and SYN and greater for MNIST-M and SVHN.
- Ablation studies: Training with generated novel data alone provides no gain, indicating that DDAIG’s performance improvement comes from aggregating source and generated novel domains.The experiments also show that transformations drastically change domain-related information while preserving category-specific properties, including through instance-specific colour and texture changes.
Conclusion
DDAIG is presented as a domain-generalisation method that synthesises unseen-domain data for augmentation by learning a full transformation CNN to model domain shift. Experiments showed improved generalisation on unseen domains and versatility beyond DG in cross-domain person re-ID.
- Method: DDAIG synthesises data from unseen domains for data augmentation in domain generalisation.It is introduced as a novel DG method.
- Method: Unlike current augmentation-based DG methods, DDAIG learns a full transformation CNN to model domain shift.
- Results: Experiments on three DG datasets showed improved CNN generalisation on unseen domains and outperformance of current state-of-the-art DG methods.
- Results: Results on cross-domain person re-ID demonstrated DDAIG’s versatility beyond domain generalisation.