Source-linked AI summary
Deeper, Broader and Artier Domain Generalization
Da Li, Yongxin Yang, Yi-Zhe Song, Timothy M. Hospedales
TL;DR
Domain generalization must transfer from multiple training domains to unseen domains, especially when target data are sparse and visually distinct, while prior photo-only benchmarks provide limited challenge. The paper introduces PACS and a low-rank parameterized CNN for end-to-end DG, reporting performance comparable to or better than prior approaches; its method and analysis remain bounded by stated parameter-growth and visualization limitations.
Problem
Prior DG evaluations mainly use photo-based benchmarks with limited domain distinctiveness, despite practical targets such as sketches and art styles being sparse and visually distinct.
Method
The paper develops a low-rank parameterized CNN that extends domain-agnostic modeling to end-to-end deep learning across multiple source domains.
Results
The method performs comparably to or better than prior DG approaches, while PACS provides a harder benchmark with greater domain shift than prior datasets.
Takeaways & Limitations
PACS offers a more practically motivated DG challenge, and the proposed CNN approach shows favorable performance for transferring across its diverse domains.
Takeaways & Limitations
The parameterized method's required parameters grow linearly with the number of domains, and its sharing structure is initially prescribed as equally weighted domain components.
Abstract
from arXiv · showhide
The problem of domain generalization is to learn from multiple training domains, and extract a domain-agnostic model that can then be applied to an unseen domain. Domain generalization (DG) has a clear motivation in contexts where there are target domains with distinct characteristics, yet sparse data for training. For example recognition in sketch images, which are distinctly more abstract and rarer than photos. Nevertheless, DG methods have primarily been evaluated on photo-only benchmarks focusing on alleviating the dataset bias where both problems of domain distinctiveness and data sparsity can be minimal. We argue that these benchmarks are overly straightforward, and show that simple deep learning baselines perform surprisingly well on them. In this paper, we make two main contributions: Firstly, we build upon the favorable domain shift-robust properties of deep learning methods, and develop a low-rank parameterized CNN model for end-to-end DG learning. Secondly, we develop a DG benchmark dataset covering photo, sketch, cartoon and painting domains. This is both more practically relevant, and harder (bigger domain shift) than existing benchmarks. The results show that our method outperforms existing DG alternatives, and our dataset provides a more significant DG challenge to drive future research.
1. Introduction
Domain generalization seeks models that transfer across unseen domains without target-domain data, a need amplified when target domains are sparse and visually distinct. The paper introduces PACS as a harder benchmark and proposes low-rank parameterized CNNs for end-to-end DG.
- Domain generalization learns a domain-agnostic model from multiple source domains for application to an unseen target domain.
- DG is especially valuable when target-domain data are sparse, such as sketches or particular art styles that lack enough examples for training.
- Existing photo-based benchmarks often involve camera-type or composition differences, but their domain shifts may be too small to motivate difficult DG.
- PACS covers photo, art painting, cartoon, and sketch domains across a wide spectrum of visual abstraction and more challenging domain shifts.
- The paper develops a low-rank parameterized CNN that learns domain-agnostic features and classifiers end-to-end across multiple source domains.
- The authors report that prior DG methods can lose to simple deep-learning baselines, while their low-rank method performs favorably against prior work.
2. Related work
Prior DG research uses invariant representations, domain prediction, or domain-agnostic classifiers, while related CNN work has focused mainly on domain adaptation. This paper extends domain-agnostic modeling to end-to-end CNNs and introduces more challenging non-photorealistic benchmarks.
- Existing DG methods use invariant representations, domain prediction, or domain-agnostic classifiers implemented with tools such as SVMs, subspace learning, and autoencoders.
- The paper places its method in the domain-agnostic-classifier family while extending prior shallow linear models to end-to-end CNN learning with richer sharing.
- CNN transferability varies across layers, so domain-adaptation studies often hand-design sharing structures for particular tasks rather than addressing domain generalization.
- DG resembles learning to learn by extracting common knowledge from source domains, but unlike few-shot learning it evaluates on the target domain without further learning.
- Established benchmarks include Office, OfficeCaltech, and VLCS, whose domains arise mainly from camera types or dataset-collection biases.
- The paper argues that deep features reduce VLCS domain shift, making simple multi-source fine-tuning competitive with prior DG methods.
- PACS aggregates paintings, cartoons, and sketches into a DG benchmark spanning broader visual abstraction and larger domain shifts than prior benchmarks.
3. Methodology
The method learns a domain-agnostic CNN from labeled examples across observed domains, dynamically generating domain-specific parameters while sharing domain-agnostic components. A low-rank tensor parameterization reduces model complexity and enables differentiable learning and inference for unseen domains.
- Problem setup: Domain generalization minimizes empirical error across observed domains while learning a model applicable to unseen domains without access to their examples.Inputs are assumed to share a size, and the class-label space is consistent across domains.
- Problem setup: A universal deep model f(x|Θ*) is presented as a strong baseline for unseen-domain performance.This baseline ignores domain differences rather than explicitly modeling domain-specific parameters.
- Parameterized neural network: Domain-conditioned parameters are generated from a binary domain vector that activates one domain-specific component and one shared component.The shared component is represented by the final bias bit of z(i), while a linear mapping reproduces the Undo-Bias formulation.
- Parameterized neural network: Multi-linear tensor mappings generate higher-order parameters, including fully connected and convolutional weights, from domain-conditioned representations.Fully connected weights use a tensor whose third dimension is S + 1; convolutional weights use a fifth-order tensor with the same domain dimension.
- Inference and learning: For an unseen domain, the bias-only vector z* synthesizes a domain-agnostic CNN from shared tensors.Concrete CNN parameters are synthesized for a domain and learned with standard back-propagation because the model remains differentiable.
- Low-rank parameterization: Tucker decomposition constrains parameter tensors to reduce the number of learned parameters and replace equally weighted sharing with learned factor combinations.The low-rank core and factor matrices reduce parameterization, while the domain factor produces a dense vector for combining shared factors.
- Low-rank parameterization: The method determines layer-wise Tucker ranks through a single sharing-strength hyperparameter using a 10% reconstruction-error threshold.HO-SVD is used optionally for initialization, not repeatedly during learning.
4. Experiments
Experiments evaluate PACS and VLCS, showing PACS has substantially larger domain shifts and that the proposed low-rank CNN performs especially well on it.
- Datasets and settings: PACS combines Photo, Sketch, Cartoon, and Painting domains across seven shared categories, extending domain-generalization evaluation beyond photos.The dataset is constructed from intersected classes across Caltech256, Sketchy, TU-Berlin, and Google Images.
- Characterizing benchmark shifts: PACS features are more separated by domain than VLCS features, indicating greater domain shift in the new benchmark.The comparison uses KLD-based shift analysis and two-dimensional t-SNE visualizations.
- Characterizing benchmark shifts: 20.2% versus 10.0%: average cross-domain performance drops are twice as large for PACS as for VLCS.The authors use the gap between within-domain and cross-domain accuracy as an indicator of domain-shift difficulty.
- VLCS benchmark: On VLCS, Deep-All outperforms prior domain-generalization methods, while Ours-Full improves on Deep-All only slightly.The small margin is attributed to VLCS’s relatively small domain gap and the additional parameters learned by Ours-Full.
- PACS benchmark: On PACS, Ours-Full performs best overall by combining CNN robustness with an explicit domain-generalization mechanism.The comparison includes SVM, prior DG methods, Deep-All, DSN, and Ours-Full using end-to-end or pre-extracted features as specified.
- Ablation and analysis: Each architectural component improves performance: tensor-generated agnostic models, low-rank weight-generator modeling, and applying DG strategies at every CNN layer.The ablation compares Tuning-Last, 2HE-Last, 2HE+Decomp-Last, and Ours-Full.
5. Conclusion
The paper introduces PACS and a deep learning-based domain generalization method. PACS reflects practical DG needs and has more challenging domain shifts, while the method performs comparably or better than prior approaches.
- PACS combines photo, art, cartoon, and sketch domains to provide a practical domain generalization benchmark.
- PACS exhibits more challenging domain shifts than prior datasets, making it suitable for future domain generalization research.
- The proposed method integrates a domain-agnostic classifier with robust end-to-end deep learning for domain generalization.
- The proposed method performs comparably or better than prior domain generalization approaches.