PaperlayerView exact revision on arXiv

Source-linked AI summary

Reducing Domain Gap by Reducing Style Bias

Hyeonseob Nam, HyunJae Lee, Jongchan Park, Wonjun Yoon, Donggeun Yoo

arXiv:1910.11645v4cs.CV

TL;DR

CNNs often lose performance on unseen domains partly because their style bias favors domain-sensitive textures over contents. SagNets reduce this bias by disentangling style from class-related content through randomized styles and adversarial learning, improving robustness across domain generalization and adaptation settings.

  • Problem

    CNNs often degrade on unseen domains because their style-biased representations favor textures that change across domains.

    Source: §1 · p. 1

  • Method

    SagNets use content-biased and style-biased networks with style and content randomization plus adversarial learning to make predictions rely more on contents.

    Source: §1 · p. 1, §3 · p. 3

  • Results

    SagNets reduce style bias and domain discrepancy while improving performance across domain generalization, unsupervised domain adaptation, and semi-supervised domain adaptation benchmarks.

    Source: §1 · p. 1, §4.1 · p. 5, §5 · p. 8

  • Takeaways & Limitations

    Controlling CNNs’ intrinsic style bias provides a domain-label-free complement to existing domain adaptation and generalization methods.

    Source: §1 · p. 2, §5 · p. 8

Abstract

from arXiv · show

Convolutional Neural Networks (CNNs) often fail to maintain their performance when they confront new test domains, which is known as the problem of domain shift. Recent studies suggest that one of the main causes of this problem is CNNs' strong inductive bias towards image styles (i.e. textures) which are sensitive to domain changes, rather than contents (i.e. shapes). Inspired by this, we propose to reduce the intrinsic style bias of CNNs to close the gap between domains. Our Style-Agnostic Networks (SagNets) disentangle style encodings from class categories to prevent style biased predictions and focus more on the contents. Extensive experiments show that our method effectively reduces the style bias and makes the model more robust under domain shift. It achieves remarkable performance improvements in a wide range of cross-domain tasks including domain generalization, unsupervised domain adaptation, and semi-supervised domain adaptation on multiple datasets.

1. Introduction

CNNs often degrade on unseen domains because their style bias favors textures that vary across domains, whereas SagNets reduce this bias by emphasizing content and improve cross-domain robustness.

  • The paper motivates domain-shift robustness by contrasting CNNs’ sensitivity to unseen environments with humans’ ability to recognize objects across photographs, cartoons, and paintings.This contrast frames style-invariant recognition as a practical goal for reliable deployment.Source: §1 · p. 1
  • CNNs are vulnerable to domain shift because they rely more on styles such as textures, while image contents such as shapes are more stable across domains.Prior findings contrast CNNs’ style bias with humans’ content-based recognition and link style variability to domain sensitivity.Source: §1 · p. 1
  • SagNets reduce style bias by separating content-biased and style-biased networks around a shared feature extractor, then using the feature extractor and content-biased network for prediction.Style randomization encourages content-focused decisions, while adversarial style-biased learning makes styles less discriminative of class categories.Source: §1 · p. 1, Figure 1 · p. 2
  • SagNets significantly improve performance across domain generalization, unsupervised domain adaptation, and semi-supervised domain adaptation benchmarks including PACS, Office-Home, and DomainNet.The approach is reported to improve results across multiple cross-domain settings and datasets.Source: §1 · p. 1
  • Unlike methods requiring domain information, SagNets control intrinsic CNN bias without domain labels or multiple domains and can complement existing adaptation techniques.The method is presented as scalable to settings with unknown or ambiguous domain boundaries and as providing additional performance gains when combined with existing methods.Source: §1 · p. 2

2. Related Work

Related work establishes CNNs’ style bias, develops feature-statistics methods for manipulating styles, and surveys domain generalization and adaptation approaches based largely on distribution alignment.

Inductive biases of CNNs

  • Prior studies show that CNNs often rely on local textures rather than global shapes, motivating efforts to reduce style bias and clarify its practical impact on domain shift.The paper identifies the practical connection between CNN inductive bias and domain shift as an open issue it investigates.Source: §2 · p. 2

Style Manipulation

  • Feature statistics capture image style, and methods such as AdaIN, BIN, and SRM manipulate or recalibrate this information for image generation or classification.AdaIN changes style through convolutional feature-map means and variances, while BIN and SRM reduce or recalibrate unnecessary style information.Source: §2 · p. 2

Domain Generalization and Adaptation

  • Domain generalization methods learn representations that transfer from multiple source domains to unseen targets using discrepancy minimization, domain decomposition, meta-learning, or self-supervision.Examples include MMD, adversarial alignment, low-rank or aggregation modules, meta-train/meta-test simulation, and jigsaw-based regularization.Source: §2 · p. 2
  • UDA and SSDA commonly align source and target distributions, with SSDA additionally using a few target labels and regularization to avoid overfitting.Reported approaches include MMD, adversarial learning, image translation, target regularization, and MME’s minimax conditional-entropy objective.Source: §2 · pp. 2–3

3. Style-Agnostic Networks

Style-Agnostic Networks reduce CNN style bias by separating content- and style-biased learning, then use the content-focused pathway for prediction. Style randomization promotes content-based decisions, while adversarial style-biased learning discourages style-discriminative representations.

3. Style-Agnostic Networks

  • SagNets combine a feature extractor with content-biased and style-biased networks to make predictions more style-agnostic and content-focused.The final test-time predictor is Gc∘Gf, while Gs supports adversarial training.Source: §3 · p. 3

3.1. Content-Biased Learning

  • Content-biased learning interpolates feature statistics from different examples to randomize styles, encouraging predictions based on contents rather than styles.The randomized representation is trained with Gc, and the style-randomization module is removed at test time.Source: §3.1 · p. 3

3.2. Adversarial Style-Biased Learning

  • Adversarial style-biased learning randomizes contents while preserving styles, trains Gs to predict classes from styles, and trains Gf to make those style-based predictions uniform.The adversarial objective is applied to Gf's affine transformation parameters, with λadv controlling the content–style trade-off.Source: §3.2 · pp. 3–4

3.3. Implementation Details

  • SagNets retain the original CNN architecture and test-time parameter and computation costs, while training uses shuffled minibatch features to obtain arbitrary paired examples.For a ResNet, Gf uses the first stages and Gc uses the remaining network; Gs has the same structure as Gc.Source: §3.3 · p. 4

3.4. Extension to Un/Semi-Supervised Learning

  • For unlabeled target or training data, the extension minimizes mean-square inconsistency between predictions with and without style randomization, while adversarial learning remains label-free.The unsupervised-loss coefficient λunl is set to 0.01.Source: §3.4 · p. 4

4. Experiments

Experiments show that SagNets shift CNNs toward shape-based decisions, reduce measured domain discrepancy, and improve performance across domain generalization, unsupervised adaptation, and semi-supervised adaptation.

4.1. Biases and Domain Gap

  • SagNets increase shape accuracy and shape bias while reducing the A-distance between 16-class-ImageNet and cue-conflict stimuli.The measured relationship indicates that more shape-biased representations generalize better across domains.Source: §4.1 · p. 5, Figure 3 · p. 5

4.2. Domain Generalization

  • On multi-source domain generalization, SagNets outperform DeepAll and competing methods on PACS and Office-Home, with both components contributing to the gains.The method adds no test-time parameters or computations over the baseline CNN.Source: §4.2 · p. 6, Table 1 · p. 6, Table 2 · p. 6
  • Single-source SagNets substantially improve PACS generalization without domain labels or multiple source domains, outperforming applicable alternatives including JiGen and ADA5.This setting trains on one domain and evaluates on the remaining domains.Source: §4.2 · p. 7, Table 3 · p. 7

4.3. Unsupervised Domain Adaptation

  • SagNets improve unsupervised adaptation across Office-Home and DomainNet, including further gains when combined with DANN, JAN, CDAN, or SymNet.The improvements remain consistent in most DomainNet scenarios, indicating complementarity with existing adaptation methods.Source: Table 4 · p. 7, §4.3 · p. 8, Table 5 · p. 8

4.4. Semi-Supervised Domain Adaptation

  • With a few target labels, SagNets consistently improve semi-supervised adaptation across architectures and baselines, often by considerable margins.The DomainNet results are built on the MME setting and support scalability across tested conditions.Source: §4.4 · p. 8, Table 6 · p. 8

5. Conclusion

SagNets target domain shift caused by style variability by steering CNNs toward content-based decisions. Experiments support their effectiveness across broad domain-shift settings, while the underlying principle may extend beyond domain adaptation.

  • SagNets reduce style-driven domain shift by randomizing latent styles and adversarially disentangling styles from class categories, encouraging content-based decisions.Source: §5 · p. 8
  • Experiments confirm that SagNets control CNN inductive biases and reduce domain discrepancy across a broad range of problem settings.Source: §5 · p. 8
  • SagNets complement domain-distribution alignment methods and can provide further improvements without replacing those approaches.Source: §5 · p. 8
  • The paper leaves applying its intrinsic-bias principle to image-corruption robustness and adversarial defense for future investigation.Source: §5 · p. 8
Loading 1910.11645v4…