PaperlayerView exact revision on arXiv

Source-linked AI summary

Open Domain Generalization with Domain-Augmented Meta-Learning

Yang Shu, Zhangjie Cao, Chenyu Wang, Jianmin Wang, Mingsheng Long

arXiv:2104.03620v1cs.CVcs.LG

TL;DR

OpenDG asks how to generalize from source domains with different distributions and label sets to an unknown target without annotated target data. DAML augments domains with Dir-mixup and distilled soft-labeling, then performs meta-learning across them. Across multi-domain evaluations, DAML outperforms prior methods for unseen-domain recognition, including known- and open-class accuracy.

  • Problem

    OpenDG studies generalization from arbitrary source domains with disparate distributions and label sets to an unseen target domain.

  • Method

    DAML augments source domains with feature-level Dir-mixup and label-level distilled soft-labeling, then conducts meta-learning across augmented domains.

  • Results

    DAML outperforms prior methods for unseen-domain recognition, achieving higher accuracy on both known and open classes across multi-domain datasets.

  • Takeaways & Limitations

    OpenDG provides a practical setting for learning target-domain representations when target annotations are unavailable and source domains are highly diverse.

Abstract

from arXiv · show

Leveraging datasets available to learn a model with high generalization ability to unseen domains is important for computer vision, especially when the unseen domain's annotated data are unavailable. We study a novel and practical problem of Open Domain Generalization (OpenDG), which learns from different source domains to achieve high performance on an unknown target domain, where the distributions and label sets of each individual source domain and the target domain can be different. The problem can be generally applied to diverse source domains and widely applicable to real-world applications. We propose a Domain-Augmented Meta-Learning framework to learn open-domain generalizable representations. We augment domains on both feature-level by a new Dirichlet mixup and label-level by distilled soft-labeling, which complements each domain with missing classes and other domain knowledge. We conduct meta-learning over domains by designing new meta-learning tasks and losses to preserve domain unique knowledge and generalize knowledge across domains simultaneously. Experiment results on various multi-domain datasets demonstrate that the proposed Domain-Augmented Meta-Learning (DAML) outperforms prior methods for unseen domain recognition.

1. Introduction

Open Domain Generalization learns representations from source domains with different distributions and label sets so they can classify known classes and detect open classes in an unseen target domain. DAML addresses this setting through domain augmentation and meta-learning, and reports higher accuracy than prior methods.

  • Challenges: Distribution shifts and uneven class coverage across source domains make minor classes lack domain diversity.Existing distribution-alignment methods can suffer severe class mismatch when source label sets differ.
  • Setting: DAML requires no assumptions on label-set equality, target data, or post-training on the target domain.Table 1 characterizes OpenDG as a broadly applicable cross-domain learning setting.
  • Problem: OpenDG learns from arbitrary source domains with disparate distributions and label sets to generalize to an unseen target domain.The target domain is unknown and may also differ in distribution and label set.
  • Approach: DAML augments domains with feature-level Dir-mixup and label-level distilled soft-labeling before meta-learning across domains.These augmentations compensate for missing labels and transfer knowledge from other domains.
  • Results: DAML achieves higher classification accuracy on both known and open classes in unseen targets than previous generalization methods.The reported result holds across several multi-domain datasets, including extremely diverse source domains.

2. Related Work

Prior cross-domain methods impose assumptions about shared labels or access to target data, while DAML targets open-domain generalization with disparate source labels. Its novelty is combining multi-domain Dir-mixup, distilled soft-labeling, and meta-learning to bridge domains and transfer knowledge.

  • Domain Adaptation: Domain adaptation commonly minimizes source-target distribution distance but requires the same source and target label set.Extensions to varied label sets rely on unlabeled target data, unavailable in OpenDG.
  • Related Settings: Multi-source adaptation methods often assume shared source labels, while DCN still requires the union of source labels to equal the target labels.OpenDG removes these label-set constraints.
  • Domain Generalization: Domain generalization learns from source data for unseen targets, but differing source labels can cause class mismatch.Existing approaches commonly learn domain-invariant features, which do not directly resolve this mismatch.
  • Meta-learning: Existing meta-learning domain-generalization methods do not address distinct source label sets and target open classes.Heterogeneous domain generalization has a similar goal but targets a more powerful pretrained model.
  • DAML: DAML uses Dir-mixup across multiple domains to compensate each domain with missing classes and distilled soft-labeling to transfer cross-domain knowledge.This differs from classic mixup, which combines two arbitrary samples.

3. Domain-Augmented Meta-Learning

DAML learns generalizable representations across disparate source domains by augmenting each domain and applying meta-learning to preserve domain-specific information while aggregating cross-domain knowledge.

  • The DAML Framework: DAML learns generalizable representations by meta-learning over augmented source domains.The framework combines domain augmentation with meta-learning rather than relying on a shared network alone.
  • Meta-Learning: Each source domain uses an individual feature extractor and classifier to preserve unique domain information.The networks are later optimized through meta-learning across all source domains.
  • Meta-Learning: The meta-training loss combines classification on original data, Dir-mixup data, and distilled soft-labels from other domain networks.Each source network receives one gradient update from this augmented loss before the meta-objective is applied.
  • Domain Augmentation: Unlike classic pairwise mixup, Dir-mixup covers combinations of multiple domains across the triangle area rather than only its edges.Pairwise mixing requires O(#domains×#domains) samples to obtain all domain combinations.
  • Domain Augmentation: Dir-mixup mixes feature representations and one-hot labels from multiple source domains using weights sampled from a Dirichlet distribution.This feature-level augmentation compensates for missing classes and increases inter-domain diversity.
  • Domain Augmentation: Distilled augmentation transfers other domains’ knowledge by mixing their soft-label predictions on data from the current domain.The method uses a uniform Dirichlet parameter vector across the other domains and does not back-propagate through the networks producing the soft-labels.

4. Experiments

Experiments evaluate DAML across closed-set and open-domain generalization settings using PACS, Office-Home, and a realistic Multi-Datasets scenario. DAML consistently outperforms compared methods, while ablations support contributions from domain augmentation and meta-learning.

  • Evaluation settings: Experiments cover closed-set PACS, open-domain PACS and Office-Home, and a naturally open-domain Multi-Datasets scenario.The evaluations vary domain distributions, label-set disparity, and the presence of open classes.
  • Closed-set generalization: DAML outperforms all compared methods on average in closed-set PACS and achieves at least comparable performance on every task.The comparison includes supervised learning, domain-generalization, meta-learning, and augmentation-based methods.
  • Open-domain generalization: DAML outperforms all compared methods by a large margin on both Acc and H-score in open-domain PACS, Office-Home, and Multi-Datasets evaluations.These results indicate stronger recognition of non-open classes and open classes.
  • Open-domain generalization: DAML’s Dir-mixup mixes samples from multiple domains, creating higher-variation samples than classic two-sample mixup.The paper states that this higher variation and diversity encourages more generalizable representations.
  • Ablation analysis: Ablations show that DAML benefits from Dir-mixup in both meta-training and meta-objective losses, distilled soft-labels, and meta-learning.Replacing Dir-mixup with classic mixup lowers accuracy, while removing either augmentation or meta-learning weakens performance.
  • Feature analysis: Fréchet distances between source and unseen-target features are smaller for DAML than for the AGG baseline.The comparison uses source features and the non-open-class portion of the target domain.

5. Conclusion

The paper formulates OpenDG for arbitrary source domains with disparate label sets and unseen targets, and proposes DAML with feature- and label-level domain augmentation. Experiments report more generalizable target-domain classification representations than previous methods.

  • Conclusion: The paper proposes OpenDG, which generalizes from arbitrary source domains with disparate label sets to unseen target domains.The problem is presented as applicable to real-world applications.
  • Conclusion: DAML performs meta-learning over domains augmented with Dir-mixup at feature level and distilled soft-labels at label level.The framework is designed to address disparate source-domain label sets.
  • Conclusion: Extensive experiments demonstrate that DAML learns more generalizable target-domain classification representations than previous generalization methods.

A. Experiment Details

The experiment-details section clarifies additional experimental settings because of space limitations in the main text.

  • Experiment details: Additional experiment settings are clarified because of space limitations in the main text.

A.1. Datasets

The experiments construct open-domain benchmarks from PACS, Office-Home, and multiple public datasets, varying domains, class splits, and target-domain conditions. These settings include both controlled benchmark tasks and a realistic cross-dataset scenario.

  • Dataset construction: Exact class splits are shown for each dataset.
  • PACS: PACS contains four image-style domains with the same seven classes and is evaluated in four cross-domain tasks.Open-domain variants split the label space across domains; closed-set and open-domain settings are both evaluated.
  • Office-Home: Office-Home contains Artistic, Clip art, Product, and Real-world domains with 65 classes and a large domain gap.The classes are spread among domains to create four open generalization tasks, each using one domain as target and three as sources.
  • Multi-Datasets: The Multi-Datasets scenario uses Office-31, STL-10, and VisDA2017 as source domains and DomainNet domains as unseen targets.It simulates learning from datasets obtained from different resources, with substantial distribution discrepancy and label-set disparity.

A.2. Implementation

The implementation uses PyTorch with an ImageNet-pretrained ResNet-18 backbone and trains with SGD for 30 epochs. DAML uses a fast first-order gradient approximation, while open-class detection for non-open-set methods uses a confidence threshold.

  • DAML is implemented in PyTorch with an ImageNet-pretrained ResNet-18 backbone and trained for 30 epochs using SGD.
  • DAML uses a fast first-order approximation to estimate meta-learning gradients.
  • Open-class detection for non-open-set methods uses a confidence threshold T selected by sorting prediction confidence.
  • The experiments use PyTorch 1.5, torchvision 0.6, CUDA 10, and one NVIDIA TITAN X with 256 GB memory.The machine has 32 CPUs, and each run takes approximately 2 hours.

C. Experiment Results

Additional experiments examine hyperparameter sensitivity, class-specific results, parameter sharing, and classification-result visualizations. Figure 6 specifically reports sensitivity to αmax and αmin.

  • The supplementary experiments cover hyperparameter sensitivity, class-specific results, parameter sharing, and classification-result visualization.
  • Figure 6 examines the sensitivity of the hyperparameters αmax and αmin.

C.1. Parameter Sensitivity

The parameter-sensitivity study evaluates αmax, αmin, β, and η on the DAML ArClRw-Pr task. Performance remains stable near optimal settings but declines substantially when parameters deviate far from them.

  • The study evaluates sensitivity to αmax, αmin, β, and η on the DAML ArClRw-Pr task.
  • Performance drops substantially when the hyperparameters deviate far from their optimal values.
  • The results indicate that these hyperparameters are relatively easy to tune near optimal values but remain necessary for performance.
  • Performance is fairly stable around the optimal values of αmax, αmin, and η.
  • β yields stable performance within the learning-rate range [0.0003, 0.003].β is the learning rate used to finally update the parameters.

C.2. Classes with Different Domain Variations

The experiments analyze how class-specific domain variation affects target accuracy and compare DAML with AGG and alternative network designs. DAML performs especially well for classes present in only one source domain, while separate domain networks trade computational cost for discriminative performance.

  • Classes with Different Domain Variations: The analysis divides non-open target classes by presence in one, two, or three source domains.It evaluates target accuracy on four open-domain Office-Home transfers from three source domains to the remaining domain.
  • Classes with Different Domain Variations: DAML outperforms AGG for nearly all classes, especially classes present in only one source domain.This result is reported as addressing different domain variations across classes.
  • Classes with Different Domain Variations: Classes present in fewer domains have lower target-domain accuracy than classes present in two or three domains.
  • Classes with Different Domain Variations: Using a shared network across domains is described as detrimental to discriminative power, so DAML uses separate networks for different domains.
  • Classes with Different Domain Variations: Separate networks increase training and inference time linearly with the number of domains.
  • Classes with Different Domain Variations: In the ClPrRw-Ar visualization, images only classified correctly by AGG are often accidental across random seeds, whereas DAML correctly classifies a digital clock example.
Loading 2104.03620v1…