PaperlayerView exact revision on arXiv

Source-linked AI summary

Domain Generalization with MixStyle

Kaiyang Zhou, Yongxin Yang, Yu Qiao, Tao Xiang

arXiv:2104.02008v1cs.CVcs.LG

TL;DR

CNNs often generalize poorly when unseen domains differ from training data, while domain generalization must learn from source domains without target-domain data. MixStyle probabilistically mixes instance-level feature statistics across source domains to synthesize novel styles during CNN training. Across category classification, instance retrieval, and reinforcement learning, the paper reports new state-of-the-art results on three tasks.

  • Problem

    CNNs can suffer severe performance degradation under distribution shifts, motivating domain generalization from source domains without target-domain data.

  • Method

    MixStyle probabilistically mixes instance-level feature statistics from training instances across source domains to perturb style information and synthesize novel domains.

  • Results

    MixStyle improves over vanilla ResNet-18 and most DG methods on category classification, and experiments report new state-of-the-art results across three tasks.

  • Takeaways & Limitations

    MixStyle provides a simple mini-batch-compatible approach for increasing training-domain diversity through feature-level style mixing.

Abstract

from arXiv · show

Though convolutional neural networks (CNNs) have demonstrated remarkable ability in learning discriminative features, they often generalize poorly to unseen domains. Domain generalization aims to address this problem by learning from a set of source domains a model that is generalizable to any unseen domain. In this paper, a novel approach is proposed based on probabilistically mixing instance-level feature statistics of training samples across source domains. Our method, termed MixStyle, is motivated by the observation that visual domain is closely related to image style (e.g., photo vs.~sketch images). Such style information is captured by the bottom layers of a CNN where our proposed style-mixing takes place. Mixing styles of training instances results in novel domains being synthesized implicitly, which increase the domain diversity of the source domains, and hence the generalizability of the trained model. MixStyle fits into mini-batch training perfectly and is extremely easy to implement. The effectiveness of MixStyle is demonstrated on a wide range of tasks including category classification, instance retrieval and reinforcement learning.

1 INTRODUCTION

CNNs can degrade severely when deployment data differs from training data, motivating domain generalization from multiple source domains without target-domain data. MixStyle addresses limited source-domain diversity by mixing feature statistics that encode image style, implicitly synthesizing novel domains.

  • CNN success relies heavily on training and test data being drawn from the same distribution, while even slight violations can cause severe performance degradation.
  • Domain generalization learns from multiple source domains to produce representations that remain robust and discriminative on unseen domains without access to target-domain data.
  • More diverse source domains make learning domain-invariant representations easier, but collecting data spanning many domains is often costly or impossible.
  • Style statistics from first residual-block features separate images from four distinct domains in a 2-D t-SNE visualization.
  • MixStyle probabilistically mixes instance-level feature statistics across source domains because these statistics preserve domain-related image style while semantic content is retained.
  • The method mixes styles during CNN training to create more diverse domain and style examples without explicit image synthesis, fitting mini-batch training.

2 METHODOLOGY

MixStyle adapts style-transfer ideas into a plug-and-play CNN module that mixes feature statistics from paired or shuffled batch instances. It uses random convex combinations during training and is omitted at test time.

  • 2.1 BACKGROUND: Instance normalization computes per-instance, per-channel spatial means and standard deviations to normalize feature tensors and remove image style.
  • 2.1 BACKGROUND: AdaIN replaces normalization scale and shift parameters with feature statistics from a style input to achieve arbitrary style transfer.
  • 2.2 MIXSTYLE: MixStyle perturbs source-domain style statistics as a plug-and-play module between CNN layers, without explicitly generating images.
  • 2.2 MIXSTYLE: With domain labels, MixStyle pairs instances from two domains, swaps their batch positions, and shuffles each domain’s instances to form a reference batch.
  • 2.2 MIXSTYLE: The method applies mixed feature statistics using instance-wise convex weights sampled from a Beta distribution, with α set to 0.1 by default.
  • 2.2 MIXSTYLE: MixStyle is activated with probability 0.5 during forward passes and is not applied at test time.

3 EXPERIMENTS

Across category classification, cross-dataset person re-identification, and reinforcement learning, MixStyle consistently improves generalization to unseen domains or environments. Ablations indicate that its benefits depend on mixing styles in lower CNN layers and using convex, variably shuffled feature-statistic combinations.

  • Category classification: On PACS, MixStyle significantly improves over vanilla ResNet-18 and outperforms Mixup by 5.2% on average.It also outperforms a pixel-level Mixup variant and DropBlock across all test domains.
  • Category classification: MixStyle outperforms most state-of-the-art domain-generalization methods by a clear margin despite its simpler design.The domain-label-free version reaches 82.8% accuracy, on par with L2A-OT, while avoiding L2A-OT’s heavier computational overhead.
  • Instance retrieval: In cross-dataset person re-identification, only MixStyle consistently outperforms the strong vanilla model, while DropBlock and RandomErase provide no benefit.RandomErase has a detrimental effect in this domain-shift setting.
  • Reinforcement learning: In Coinrun, MixStyle significantly improves test performance, reduces variance, and adds gains to the stronger IBAC-SNI baseline.MixStyle-trained agents also generalize faster and better, with Baseline+MixStyle already highly competitive with IBAC-SNI.
  • Analysis: Applying MixStyle across multiple lower residual blocks generally performs better, but the best combination differs between PACS and re-identification.res123 is best on PACS, res12 is best on re-ID, and performance plunges when MixStyle is applied to res4.
  • Analysis: Convex mixing outperforms replacing one style with another, and random shuffles at different layers outperform a fixed shuffle.The authors attribute the latter improvement to increased noise and stronger regularization.
  • Analysis: Increasing α from 0.1 to 0.4 lowers PACS accuracy from 82.8% to 81.7%, while larger increases have little further effect.Results across tasks show small variance overall, supporting α ∈ {0.1, 0.2, 0.3}, with α = 0.1 as a default.

4 RELATED WORK

Domain generalization methods address out-of-distribution shifts by learning representations robust across source domains. Existing approaches include feature alignment, intermediate-domain generation, and regularization for improving generalization in visual recognition and reinforcement learning.

  • Domain generalization: Domain generalization learns models from multiple related source domains that remain robust to distribution changes in unseen domains.The goal is out-of-distribution generalization when only source data are available.
  • Feature alignment: Many domain generalization methods align features across source domains to encourage invariance to domain shift.Distribution alignment is one representative strategy in this literature.
  • Novel-domain synthesis: DLOW and L2A-OT synthesize intermediate or pseudo-novel domains using learned image-translation models.These methods explicitly generate images from domains between or beyond the observed sources.
  • Reinforcement learning: Deep reinforcement learning agents can overfit training environments and perform poorly in unseen environments with different visual patterns or levels.Regularization, including weight decay, dropout, and batch normalization, has been explored to improve generalization.

5 CONCLUSION

The paper presents MixStyle as a simple domain generalization method that mixes feature statistics to synthesize novel domains. Experiments across three task types report new state-of-the-art performance.

  • Contribution: MixStyle mixes feature statistics from two instances to synthesize novel domains.The method is inspired by the observation that feature statistics encode style or domain-related information.
  • Evaluation: Extensive experiments across a wide range of tasks demonstrate MixStyle’s effectiveness.The conclusion reports results on three different tasks.
  • Evaluation: MixStyle yields new state-of-the-art results on three different tasks.This is the paper’s stated overall experimental conclusion.

A.1 PSEUDO-CODE OF MIXSTYLE

The pseudo-code applies MixStyle during training by normalizing each input instance, optionally shuffling batch instances, and reconstructing features with mixed statistics. Domain labels can guide cross-domain pairing, while convex weights control the mixture.

  • Inputs and controls: MixStyle accepts feature tensors x with shape (B, C, H, W), where p controls application probability and alpha parameterizes the Beta distribution.The default values are p = 0.5, alpha = 0.1, and eps = 1e-6.
  • Training-time application: The procedure skips MixStyle outside training mode or when a random draw exceeds the application probability p.These conditions prevent the transformation from being applied on every input.
  • Instance normalization: For each instance, MixStyle computes spatial means and variances, derives standard deviations, detaches the statistics, and normalizes the input features.The small epsilon stabilizes the square-root operation.
  • Pairing and mixing: MixStyle samples instance-wise convex weights from Beta(alpha, alpha) and pairs statistics using a shuffled batch.When domain labels are available, the pseudo-code constructs a permutation that pairs instances across domains.
  • Feature reconstruction: The mixed mean and standard deviation are convex combinations of the original and paired statistics, and they denormalize the normalized features.The output is reconstructed as x_normed * sig_mix + mu_mix.

A.2 FURTHER ANALYSIS

Further analyses examine same-domain mixing, performance on seen and additional domains, and implementation choices. Cross-domain mixing performs better than same-domain mixing, while MixStyle improves results across Digits-DG and Office-Home without domain labels.

  • Same-domain mixing: Mixing styles between same-domain instances is about 1% better than the baseline but clearly worse than mixing instances from different domains.The comparison suggests that both instance-specific and cross-domain style variation are relevant to performance.
  • Source-domain performance: MixStyle’s source-domain accuracy on PACS is evaluated to test whether gains on unseen domains sacrifice performance on seen domains.The evaluation uses a held-out validation set from the source domains.
  • Additional datasets: On Digits-DG and Office-Home, MixStyle clearly improves over the baseline CNN and outperforms general-purpose regularization methods.The experiments use no domain labels, and the datasets contain substantially different visual domains.
  • Additional datasets: MixStyle performs comparably to L2A-OT while being much simpler to train and using fewer computing resources.This comparison is reported for the Digits-DG and Office-Home evaluations.
Loading 2104.02008v1…