Source-linked AI summary
Learning to Generalize: Meta-Learning for Domain Generalization
Da Li, Yongxin Yang, Yi-Zhe Song, Timothy M. Hospedales
TL;DR
Models trained under one domain's statistics can fail in novel domains, while domain generalization seeks robust performance without target-domain data. The paper introduces MLDG, which simulates train/test domain shift through virtual domains and aligns update directions, achieving state-of-the-art vision results and promising reinforcement-learning results. Its model-agnostic procedure applies across supervised and reinforcement-learning settings without constraining the base architecture.
Problem
Domain shift causes models trained in one domain to perform poorly in target domains with different statistics, motivating domain generalization without target-domain training data.
Method
MLDG is a model-agnostic meta-learning procedure that synthesizes virtual training and testing domains within minibatches and encourages training updates that also improve virtual testing loss.
Results
MLDG achieves state-of-the-art results on a recent cross-domain image-recognition benchmark and promising results on two classic reinforcement-learning tasks.
Takeaways & Limitations
The approach provides a single training procedure for improving domain generalization across different base networks and both supervised and reinforcement-learning problems.
Takeaways & Limitations
The DG formulation assumes source and target domains share the same task, label space, and input feature space while differing in statistics.
Abstract
from arXiv · showhide
Domain shift refers to the well known problem that a model trained in one source domain performs poorly when applied to a target domain with different statistics. {Domain Generalization} (DG) techniques attempt to alleviate this issue by producing models which by design generalize well to novel testing domains. We propose a novel {meta-learning} method for domain generalization. Rather than designing a specific model that is robust to domain shift as in most previous DG work, we propose a model agnostic training procedure for DG. Our algorithm simulates train/test domain shift during training by synthesizing virtual testing domains within each mini-batch. The meta-optimization objective requires that steps to improve training domain performance should also improve testing domain performance. This meta-learning procedure trains models with good generalization ability to novel domains. We evaluate our method and achieve state of the art results on a recent cross-domain image classification benchmark, as well demonstrating its potential on two classic reinforcement learning tasks.
Introduction
Domain generalization seeks models that work in unseen domains without target-domain training data, addressing failures caused by domain shift. The paper introduces MLDG, a model-agnostic meta-learning procedure that simulates domain shift during training and reports strong results across vision and reinforcement learning.
- Motivation: The paper frames domain generalization as a way to make models robust to changed conditions and perform well immediately in novel domains.Motivating examples include changed camera types and altered agent morphology without retraining or adaptation.
- Motivation: Domain generalization trains models to function in new target domains without updating them using target-domain data.Unlike domain adaptation, DG assumes zero training examples from the target problem.
- Method: MLDG synthesizes virtual training and testing domains within each minibatch and optimizes training performance so its update direction also improves virtual testing loss.The procedure uses meta-optimization to train a base learner for greater domain generality.
- Method: MLDG is model agnostic, introduces no new parameters, and applies to both supervised and reinforcement learning settings.It does not constrain the architecture of the base learner.
- Evaluation: The method achieves state-of-the-art results on a recent cross-domain image-recognition benchmark and promising results on two classic reinforcement-learning tasks.The evaluation covers both visual recognition and reinforcement learning.
Related Work
Prior domain-generalization work primarily learns domain-invariant representations by reducing differences among source domains. The paper distinguishes its approach as the first to address domain generalization through meta-learning.
- Multi-Domain Learning: Multi-domain learning targets one model effective across known domains, whereas domain generalization targets held-out unknown domains.DG often begins with multiple source domains but evaluates generalization beyond them.
- Domain Generalization: Existing DG methods commonly learn shared representations that minimize source-domain gaps so they can perform well on a new target domain.Examples include multi-view autoencoders, mean map embeddings, and gradient-reversal domain-confusion losses.
- Meta-Learning: The paper positions MLDG as the first domain-generalization approach based on meta-learning rather than representation-alignment mechanisms.This contrasts with prior methods that seek domain-indistinguishable shared representations.
Methodology
MLDG trains a base learner for domain generalization by simulating train–test domain shifts within each iteration and optimizing performance across virtual domains. Its objective coordinates improvements on meta-train and meta-test domains, while gradient-based variants extend this idea to model-agnostic supervised and reinforcement learning settings.
- Meta-Learning Domain Generalization: Each iteration splits source domains into meta-train and virtual meta-test sets to mimic real train–test domain shifts.The virtual meta-test domains represent held-out domains during training.
- Supervised Learning: MLDG first updates model parameters using the meta-train loss, then evaluates the adapted parameters on virtual meta-test domains.The supervised procedure computes the meta-test loss after the meta-train update.
- Reinforcement Learning: The model-agnostic procedure can be applied to supervised learning and reinforcement learning, where the meta-test loss in RL is the average negative return after policy updating.RL meta-testing requires rolling out the updated policy in virtual meta-test environments.
- Analysis of MLDG: The meta-optimization objective minimizes meta-train and meta-test losses while encouraging their gradients to point in similar directions.Gradient alignment represents coordinated descent across the two domain sets rather than asymmetric optimization.
- Alternative Variants of MLDG: MLDG-GC normalizes gradients for cosine-similarity regularization, while MLDG-GN uses the intuition that meta-test gradients should approach zero after meta-train convergence.MLDG-GN requires a good initialization and is initialized with the domain aggregation baseline.
Experiments
Experiments evaluate MLDG across synthetic classification, PACS cross-domain recognition, and Cart-Pole and Mountain Car reinforcement learning. The results show strong domain-generalization performance, while the benefit of aggregating source domains and MLDG variants depends on the problem.
- Experimental settings: MLDG is evaluated on a synthetic classification task, the PACS multi-domain recognition benchmark, and two reinforcement-learning problems.The experiments compare domain-generalization methods against domain-agnostic aggregation and other alternatives across supervised and reinforcement-learning settings.
- PACS ablations: ≈2.7% margin over Deep-All is obtained when MLDG is applied to all AlexNet layers rather than using the vanilla baseline.MLDG on fully connected layers provides ≈1.6%, while applying it to convolutional layers adds a further ≈1.1%.
- PACS ablations: The meta-optimisation step provides the key benefit: setting α = 0 performs comparably with Deep-All, whereas vanilla MLDG performs best among the evaluated variants.MLDG-GC and MLDG-GN provide some benefit compared with Deep-All, but vanilla MLDG achieves the strongest result in the reported PACS variant comparison.
- Overall findings: MLDG-based meta-learning effectively alleviates domain shift across supervised and reinforcement-learning problems and always outperforms prior alternatives in the reported experiments.The paper reports state-of-the-art results on PACS and promising results on Cart-Pole and Mountain Car, with the core MLDG strategy effective across all problems.
- Overall findings: Aggregating multiple source domains is problem dependent: it works for PACS and Mountain Car but not for Cart-Pole.The paper also reports mixed results for MLDG-GC and MLDG-GN, with the stronger variant differing across experiments.
Conclusion
The paper proposes MLDG, a model-agnostic meta-learning procedure for domain generalization, and reports strong results across visual recognition and reinforcement learning.
- MLDG trains models for domain generalization through meta-optimization on simulated train/test splits with domain shift.
- Unlike prior model-based approaches, MLDG scales with the number of domains and can be applied to different base network types.
- MLDG applies to both supervised and reinforcement learning problems.
- Experiments show state-of-the-art results on a challenging visual recognition benchmark and promising results on multiple classic reinforcement learning problems.