PaperlayerView exact revision on arXiv

Source-linked AI summary

Domain Generalization Using a Mixture of Multiple Latent Domains

Toshihiko Matsuura, Tatsuya Harada

arXiv:1911.07661v1cs.CVcs.LG

TL;DR

The paper addresses domain generalization when source data mix multiple latent domains whose sample-level labels are unavailable. It clusters style features to assign pseudo domains and uses adversarial learning for domain-invariant features, achieving better performance than conventional labeled-domain methods. The approach nevertheless requires assumptions about style and a preset number of pseudo domains.

  • Problem

    Conventional domain-generalization methods require domain labels, but many datasets mix multiple latent domains with unknown boundaries and costly manual annotation.

  • Method

    The method clusters multi-scale convolutional feature statistics representing image style to assign pseudo domain labels, then trains a shared domain-invariant feature extractor through adversarial learning.

  • Results

    The method achieves better performance without domain labels and outperforms conventional domain-generalization methods using domain labels; pseudo labels also outperform human-annotated original labels in classification.

  • Takeaways & Limitations

    Style-based pseudo domains can support domain generalization when source-domain labels are unavailable, including cases where original labels are not separated by image style.

  • Takeaways & Limitations

    The method assumes latent domains are reflected in image style and requires the number of pseudo domains to be set in advance, which may differ from the number of original domains.

Abstract

from arXiv · show

When domains, which represent underlying data distributions, vary during training and testing processes, deep neural networks suffer a drop in their performance. Domain generalization allows improvements in the generalization performance for unseen target domains by using multiple source domains. Conventional methods assume that the domain to which each sample belongs is known in training. However, many datasets, such as those collected via web crawling, contain a mixture of multiple latent domains, in which the domain of each sample is unknown. This paper introduces domain generalization using a mixture of multiple latent domains as a novel and more realistic scenario, where we try to train a domain-generalized model without using domain labels. To address this scenario, we propose a method that iteratively divides samples into latent domains via clustering, and which trains the domain-invariant feature extractor shared among the divided latent domains via adversarial learning. We assume that the latent domain of images is reflected in their style, and thus, utilize style features for clustering. By using these features, our proposed method successfully discovers latent domains and achieves domain generalization even if the domain labels are not given. Experiments show that our proposed method can train a domain-generalized model without using domain labels. Moreover, it outperforms conventional domain generalization methods, including those that utilize domain labels.

Introduction

The paper frames domain generalization without domain labels as a realistic problem for datasets mixing multiple latent domains. It proposes exploiting image style to discover those domains and reports stronger performance than conventional labeled-domain methods.

  • Motivation: Domain generalization targets unseen conditions without access to target-domain samples during training.The motivation includes differing illumination and object types in autonomous-driving and robotic settings.
  • Problem: Conventional methods require domain labels, but web-crawled datasets may mix latent domains whose boundaries are unknown.Manual annotation can be costly and time-consuming, while dividing unknown underlying distributions is difficult.
  • Proposed Scenario: The proposed scenario trains a domain-generalized model from a source mixture where each sample’s domain is unknown.The method assumes image style reflects latent domains and uses style-transfer features as domain-discriminative features.
  • Results: Benchmark experiments show the proposed method is effective without domain labels and outperforms conventional domain generalization methods that use them.Pseudo domain labels obtained by clustering style features also improve classification over human-annotated original domain labels.

Related Work

Prior work includes domain adaptation, domain generalization, and style transfer, but conventional domain-generalization methods generally require domain labels. The proposed approach instead exploits latent domains without using target samples during training.

  • Domain Adaptation and Generalization: Domain adaptation uses source and target data, whereas domain generalization targets unseen domains without target samples during training.Existing approaches include matching distributions in pixel or feature space and using multiple source domains to learn domain relations.
  • Latent-Domain Methods: Mancini et al. automatically discovers latent domains for domain adaptation, but requires target samples during training.This distinguishes it from the proposed domain-generalization setting.
  • Domain Generalization: Most conventional domain-generalization methods require labels identifying the domain of each sample.JiGen avoids domain labels through jigsaw-puzzle self-supervision but does not exploit multiple latent domains in the source data.
  • Style Transfer: Style transfer represents image style using neural activation statistics, including Gram matrices and related feature representations.The paper draws on style-transfer research because style features are used as domain-discriminative features.

Multiple Latent Domains

Conventional domain generalization represents training data with inputs, labels, and known domain labels. The paper instead models a mixture of latent source domains where only inputs and object labels are available.

  • Conventional Formulation: Conventional domain generalization uses multiple source domains sharing input and label spaces but having different data distributions.Its training representation includes each image, its object-category label, and its domain label.
  • Conventional Formulation: The domain label d_i identifies the source domain to which sample x_i belongs, while N_s denotes the total number of source samples.These labels are used alongside images x_i and object-category labels y_i.
  • Latent-Domain Formulation: In the latent-domain scenario, the training dataset contains image-label pairs without domain labels because the source data mix multiple unknown domains.The formulation omits d_i from the observed dataset.

Proposed Method

The method alternates clustering style-based domain-discriminative features to obtain pseudo domain labels with adversarial training of a domain-invariant feature extractor. Entropy regularization further encourages discriminative object-category features, while label reassignment stabilizes iterative training.

  • Adversarial domain generalization: Adversarial learning trains a domain discriminator and a feature extractor whose outputs make the latent domains difficult to distinguish.This produces domain-invariant features shared across the divided domains.
  • Entropy regularization: Entropy regularization encourages low-density separation between object categories, helping extract discriminative features and improve classification accuracy.The entropy loss is applied to all labeled training samples in this method.
  • Training objective: The training objective combines classification, entropy, and adversarial losses, with λ suppressing the latter two losses’ noise during early training.The objective is optimized over the feature extractor and classifier.
  • Latent-domain discovery: The method clusters style-based domain-discriminative features to assign samples to pseudo domains, then iteratively reassigns their pseudo labels during training.It uses k-means by default, although other clustering algorithms are possible.
  • Domain-discriminative features: The method assumes latent image domains are reflected in style and stacks convolutional feature statistics from multiple layers as domain-discriminative features.The statistics include layer outputs’ means and standard deviations across spatial dimensions for each channel.
  • Pseudo-label stabilization: Cluster assignments are converted into stable pseudo domain labels by choosing a permutation that maximizes agreement with labels from the preceding epoch.The optimal permutation can be computed with the Kuhn-Munkres algorithm.

Experiments

Experiments evaluate the method on PACS and VLCS using standard domain-generalization splits, comparisons with existing methods, and ablations. Across both datasets, the method surpasses existing approaches without domain labels, while pseudo domains based on image styles improve performance.

  • Datasets: PACS contains four image-style domains and seven object categories, while VLCS combines five shared object categories across multiple datasets.PACS uses Photo, Art Paintings, Cartoon, and Sketch; VLCS includes bird, car, chair, dog, and person.
  • Experimental setup: The experiments use three source domains and one target domain, with 10% of PACS or 30% of VLCS source samples reserved for validation.Target-domain samples are used only for testing, and the model with the best validation accuracy is evaluated.
  • Baselines: Comparisons include Deep All, TF, and CIDDG, although methods other than Deep All and JiGen require domain labels and are evaluated in the labeled-domain-generalization scenario.The PACS table marks methods using domain labels with an asterisk.
  • Results: The method surpasses existing methods on all datasets regardless of the number of pseudo domains, without using domain labels.On PACS, adversarial learning is especially effective relative to Deep All; the method discovers latent domains from image styles.
  • Results: On VLCS, the method improves classification accuracy even though domain shifts occur within photo images rather than across visibly distinct styles.Assigning pseudo domain labels from image styles remains beneficial when the original dataset domains are not separated by style.
  • Ablation study: The ablation study evaluates removing adversarial loss, entropy loss, or style-statistics features for clustering on PACS with AlexNet.The reported variants isolate the contributions of adversarial learning, entropy loss, and the feature representation used for clustering.

Further Analysis

Further analyses examine the method’s components, robustness to the number of pseudo domains, and whether clustering captures domains rather than object categories.

  • Ablation Study: The adversarial loss improves domain generalization, and adding the entropy loss further improves it.These effects are reported in the ablation study with three pseudo domains.
  • Ablation Study: Iteratively reassigned pseudo domain labels improve classification accuracy over assignments fixed at the start of training.The paper attributes this to extracting domain-discriminative features with models trained on samples from each domain.
  • Ablation Study: Pseudo domain labels improve classification accuracy compared with original human-annotated domain labels.The paper suggests these labels better capture image styles through the model’s inner features.
  • Varying the Number of Pseudo Domains: There is no significant correlation between the number of pseudo domains and classification accuracy, indicating robustness to this choice.Experiments vary the number of pseudo domains on PACS, where the actual number of original domains is three; accuracy is averaged over five sets.
  • Clustering Evaluation: Clustering domain-discriminative features yields pseudo labels with large NMI to original domains and small NMI to object categories.The NMI to previous assignments gradually converges to 1.0, while t-SNE features are roughly divided by original domains.

Conclusion

The paper proposes domain generalization for mixtures of multiple latent domains without domain labels. It extracts style-based features for clustering and uses adversarial learning to train a domain-invariant extractor, achieving better performance than conventional labeled methods.

  • Conclusion: The method extracts convolutional feature statistics as image-style features, clusters them into pseudo domains, and trains a domain-invariant extractor using adversarial learning.This addresses the proposed setting without requiring domain labels.
  • Conclusion: The method achieves better performance without domain labels than conventional methods that use them.
Loading 1911.07661v1…