Source-linked AI summary
Domain Generalization with Domain-Specific Aggregation Modules
Antonio D'Innocente, Barbara Caputo
TL;DR
The paper addresses domain generalization: robust visual recognition across unseen domains without access to target-domain statistics. It introduces multi-branch Domain-Specific Aggregation Modules that combine generic pre-trained knowledge with separated source-domain information, and reports new state-of-the-art results on two benchmark databases.
Problem
Domain generalization seeks visual recognition that works across new domains without access to target-domain data or statistics.
Method
A multi-branch architecture uses Domain-Specific Aggregation Modules to iteratively combine generic pre-trained perceptual knowledge with source-specific information.
Results
Experiments using two pre-trained architectures on two benchmark databases achieve the new state of the art in domain generalization.
Takeaways & Limitations
The architecture supports both end-to-end classification and using its convolutional layers as features for domain generalization.
Takeaways & Limitations
The target distribution remains unknown, so the method cannot estimate target-domain statistics during training.
Abstract
from arXiv · showhide
Visual recognition systems are meant to work in the real world. For this to happen, they must work robustly in any visual domain, and not only on the data used during training. Within this context, a very realistic scenario deals with domain generalization, i.e. the ability to build visual recognition algorithms able to work robustly in several visual domains, without having access to any information about target data statistic. This paper contributes to this research thread, proposing a deep architecture that maintains separated the information about the available source domains data while at the same time leveraging over generic perceptual information. We achieve this by introducing domain-specific aggregation modules that through an aggregation layer strategy are able to merge generic and specific information in an effective manner. Experiments on two different benchmark databases show the power of our approach, reaching the new state of the art in domain generalization.
1 Introduction
Domain generalization targets visual recognition that remains robust across unseen domains without access to target-domain statistics. The paper proposes combining shared perceptual knowledge with separated source-domain information through domain-specific aggregation modules.
- Unlike domain adaptation, the setting uses multiple available source domains while forbidding access to target data during training.
- Domain generalization seeks recognition methods that work on any target domain without accessing representative target data or its statistics.
- The proposed architecture combines generic and source-specific information through multiple branches, each corresponding to an available source domain.
- Domain-Specific Aggregation Modules merge pre-trained generic knowledge with separated source-domain information through deep layer aggregation.
2 Related Works
Prior domain generalization work includes source-model weighting, domain-specific and domain-agnostic decomposition, and meta-learning with virtual testing domains. This paper instead adapts iterative deep aggregation into a multi-branch architecture that preserves source separation while retaining general representations.
- Domain adaptation research has largely addressed single-source settings through invariant features, adversarial losses, and two-step networks.
- Existing domain generalization methods include source-model weighting, domain-specific and domain-agnostic sub-models, and meta-learning with virtual testing domains.
- Deep layer aggregation research motivates systematic connectivity patterns that combine features across layers through aggregation structures.
- The paper extends iterative deep aggregation into a multi-branch architecture that retains source separation while learning general representations for domain generalization.
3 Domain Specific Aggregation Modules
The method augments a generic CNN backbone with domain-specific aggregation modules that separately model source-domain information while retaining shared features. Training specializes each module on one source domain, while validation combines module predictions while excluding the module associated with the validation domain.
- Architecture: Domain-specific aggregation modules augment a generic CNN core by modeling source-specific features separately within one coherent architecture.The backbone provides shared processing, while separate modules preserve domain-specific information.
- Aggregation Module: Each aggregation module iteratively combines outputs from multiple backbone layers through a sequence of aggregation nodes before producing classification outputs.The nodes use 1x1 convolutions followed by nonlinearities and are implemented as pluggable branches for CNN architectures.
- Training: With S source domains, the architecture uses S specialized modules alongside the shared branch Θ, and each module learns from one domain’s mini-batch.Training feeds equal-sized, domain-grouped mini-batches so the backbone processes all inputs while each module specializes in its corresponding source.
- Validation and Testing: Validation combines module probabilities while excluding the module associated with the input source domain, seeking modules that generalize to unseen distributions.The procedure addresses the limited informativeness of source-domain validation accuracy in domain generalization.
- Validation and Testing: The model is evaluated both end to end and as a feature extractor using a linear SVM on features from Θ and the aggregation modules.This provides both direct classification and feature-quality evaluations.
4 Experiments
Experiments on PACS and Office-Home evaluate D-SAMs with AlexNet and ResNet-18 under a three-source, one-target domain-generalization protocol. D-SAMs outperform prior work with AlexNet, while l2-normalized feature representations outperform corresponding end-to-end models with ResNet-18.
- Datasets and protocol: PACS contains 9,991 images across four domains and seven categories, while Office-Home contains four domains and 65 object categories.Both datasets use three source domains for training and the remaining domain as target.
- Datasets and protocol: The experiments use three source domains for training, reserve the remaining domain as target, and select the final model using a 90%/10% train-validation split.Training uses domain-specific batches and standard image augmentations under the stated optimization protocol.
- AlexNet experiments: D-SAMs outperform previous work by a sizable margin on PACS with AlexNet, including the challenging Sketch target domain.The reported comparison uses end-to-end D-SAMs coupled with AlexNet.
- ResNet-18 experiments: SVM classifiers trained on l2-normalized features always outperform the corresponding end-to-end models on PACS and Office-Home.The comparison covers ResNet-18 experiments on both databases.
- ResNet-18 experiments: Theta and Lambda features have similar performance, while Theta features outperform corresponding Deep All features without added inference computational overhead.These findings are reported for the ResNet-18 feature-representation experiments.
5 Conclusions
The paper presents a domain-generalization architecture that combines generic perceptual knowledge with source-domain-specific aggregation modules. Tested with two pretrained backbones on two benchmark databases, both end-to-end and feature-based versions achieve the reported new state of the art.
- Conclusions: D-SAMs augment a pretrained backbone with one aggregation module per source domain, while the backbone retains generic perceptual information.The architecture can operate end-to-end or provide convolutional features for a linear SVM.
- Conclusions: Both end-to-end and feature-based D-SAM approaches achieve the new state of the art on two benchmark databases with two pretrained architectures.The paper identifies scalability with respect to the number of sources and further aggregation strategies as future work.