PaperlayerView exact revision on arXiv

Source-linked AI summary

Shape-Texture Debiased Neural Network Training

Yingwei Li, Qihang Yu, Mingxing Tan, Jieru Mei, Peng Tang, Wei Shen, Alan Yuille, Cihang Xie

arXiv:2010.05981v2cs.CV

TL;DR

CNNs can learn shape- or texture-biased representations depending on their training data, while both cues contribute to object recognition. The paper trains on style-transferred cue-conflict images with supervision from both cues and reports gains across recognition and robustness benchmarks.

  • Problem

    CNNs can become biased toward shape or texture depending on their training data, and the authors report that either bias weakens recognition performance.

    Source: §1 · p. 1

  • Method

    The method augments training data with style-transferred cue-conflict images and assigns soft labels using both the shape-source and texture-source labels.

    Source: §1 · p. 2, §3 · p. 5

  • Results

    Shape-texture debiased training improves performance across ImageNet, ImageNet-A, ImageNet-C, Stylized ImageNet, and FGSM evaluation; ResNet-152 gains +5.2% on ImageNet-A over vanilla.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors conclude that both shape and texture are necessary for image recognition and report that their training method boosts accuracy and robustness.

    Source: §6 · p. 9

Abstract

from arXiv · show

Shape and texture are two prominent and complementary cues for recognizing objects. Nonetheless, Convolutional Neural Networks are often biased towards either texture or shape, depending on the training dataset. Our ablation shows that such bias degenerates model performance. Motivated by this observation, we develop a simple algorithm for shape-texture debiased learning. To prevent models from exclusively attending on a single cue in representation learning, we augment training data with images with conflicting shape and texture information (eg, an image of chimpanzee shape but with lemon texture) and, most importantly, provide the corresponding supervisions from shape and texture simultaneously. Experiments show that our method successfully improves model performance on several image recognition benchmarks and adversarial robustness. For example, by training on ImageNet, it helps ResNet-152 achieve substantial improvements on ImageNet (+1.2%), ImageNet-A (+5.2%), ImageNet-C (+8.3%) and Stylized-ImageNet (+11.1%), and on defending against FGSM adversarial attacker on ImageNet (+14.4%). Our method also claims to be compatible with other advanced data augmentation strategies, eg, Mixup, and CutMix. The code is available here: https://github.com/LiYingwei/ShapeTextureDebiasedTraining.

1 INTRODUCTION

CNNs can become shape- or texture-biased depending on their training data. The authors propose cue-conflict image augmentation with supervision from both cues and report improvements across recognition and robustness benchmarks.

1 INTRODUCTION

  • Shape and texture have both been used as object-recognition cues, and prior work found that combining them can improve recognition.Source: §1 · p. 1
  • CNNs tend to learn shape- or texture-biased features depending on the training dataset, and the authors report that either bias weakens performance.Source: §1 · p. 1
  • The method augments training data with style-transferred cue-conflict images and supervises both their shape and texture information.The authors state that omitting either cue's supervision leaves models biased.Source: §1 · p. 2
  • ResNet-152 gains 1.2% on ImageNet, reaching 79.8% top-1 accuracy with shape-texture debiased training.The result is an absolute improvement over its vanilla counterpart.Source: §1 · p. 2
  • ResNet-152 improves over vanilla by +5.2% on ImageNet-A, +8.3% on ImageNet-C, +11.1% on Stylized ImageNet, and +14.4% against FGSM on ImageNet.Source: §1 · p. 2

2 SHAPE/TEXTURE BIASED NEURAL NETWORKS

The section describes how cue-conflict samples and label assignment produce shape- or texture-biased models, then uses their performance and attention patterns to argue that the cues are complementary.

2.1 MODEL ACQUISITION

  • Randomly paired training images are style-transferred to create cue-conflict samples, with informative texture patterns from another image rather than random artistic styles.Shape-only or texture-only labels direct learning toward the corresponding cue.Source: §2.1 · pp. 2–3

2.2 EVALUATION AND OBSERVATION

  • 73.9% top-5 ImageNet-200 accuracy for the shape-biased ResNet-18 is below the vanilla ResNet-18's 88.2%.The ablation uses ImageNet-200, a 200-class ImageNet subset.Source: §2.2 · p. 3
  • Class Activation Mapping shows shape-biased and texture-biased models attending to different cues, such as a cat's head versus its fur.For the tabby example, the shape-biased model focuses on the head, while the texture-biased model focuses on the lower body and front legs.Source: §2.2 · pp. 3–4, Figure 3 · p. 4
  • Shape-biased models fare better on objects with distinctive structures, while texture-biased models fare better on objects with distinctive texture patterns.The examples include obelisk for shape and brain coral for texture; either bias can leave categories hard to distinguish when its cue is insufficient.Source: §2.2 · p. 4, Figure 4 · p. 4
  • The observed complementarity motivates training models to acquire both shape and texture representations.Source: §2.2 · p. 4

3 SHAPE-TEXTURE DEBIASED NEURAL NETWORK TRAINING

The method assigns cue-conflict images soft labels combining the shape-source and texture-source labels. A coefficient controls their relative importance, and the framework is presented as potentially extendable beyond classification.

3 SHAPE-TEXTURE DEBIASED NEURAL NETWORK TRAINING

  • The training method combines the shape-source and texture-source one-hot labels into a soft label for each cue-conflict image.Source: §3 · p. 5
  • The shape-texture coefficient γ interpolates between texture-biased learning at 0 and shape-biased learning at 1, with an empirically effective intermediate setting.The authors report that an intermediate point can acquire both representations and perform well across image-recognition benchmarks.Source: §3 · p. 5
  • The authors describe the training framework as potentially extendable to other computer-vision tasks and present a semantic-segmentation example.Source: §3 · p. 5

4 EXPERIMENTS

The experiments evaluate shape-texture debiased training on ImageNet classification, robustness benchmarks, cue-specific representation tests, augmentation combinations, and semantic segmentation. Across these settings, the method improves over vanilla training and shows gains beyond simply extending the training schedule.

4.1 EXPERIMENTS SETUP

  • The setup uses ImageNet classification and PASCAL VOC segmentation, evaluates generalization and robustness benchmarks, and includes a 2×-epochs vanilla baseline to match training cost.The implementation uses separate batch-normalization paths for original and cue-conflict images.Source: §4.1 · pp. 5–6

4.2 RESULTS

  • ResNet-152 gains 1.2% ImageNet top-1 accuracy (79.8%), plus ImageNet-A (+5.2%), ImageNet-C mCE (−8.3%, 67.2% to 58.9%), Stylized-ImageNet (+11.1%), and FGSM defense (+14.4%).ImageNet-C reports mCE, for which lower is better.Source: Table 2 · p. 6, §4.2 · p. 6, Table 1 · p. 6
  • Doubling the vanilla schedule yields no additional ResNet-101 improvement, while debiased training adds 1.0% top-1; shape- and texture-biased training fail to improve over vanilla.The paper notes that the training methods differ in label-assignment strategy.Source: §4.2 · p. 6
  • Across benchmarks, debiased training improves on the vanilla baseline consistently, unlike the compared state-of-the-art methods; against a shape-texture ensemble, it is more robust and avoids 2× inference cost.The ensemble is on par on ImageNet but is inferior on the reported robustness tasks.Source: Table 3 · p. 7, §4.2 · p. 7, §4.3 · p. 7

4.3 ABLATIONS

  • On shape datasets, debiased models match shape-biased models and outperform vanilla and texture-biased models; on texture datasets, they outperform shape-biased models and are competitive with vanilla or texture-biased models.The authors conclude that the method helps networks acquire both shape and texture representations.Source: §4.3 · pp. 7–8, Table 4 · p. 7
  • Combining debiased training with Mixup or CutMix improves ResNeXt-101 ImageNet accuracy to 80.5% (+0.6%) and 81.2% (+0.7%), respectively.With CutMix, it also improves ImageNet-A (+1.4%), ImageNet-C (−5.9%, lower is better), and Stylized ImageNet (+7.5%).Source: §4.3 · p. 8
  • ResNet-50 reaches 77.0% top-1 ImageNet accuracy at γ = 0.7, while γ values from 0.5 to 0.9 improve over vanilla training.Accuracy first rises and then falls as γ increases.Source: §4.3 · p. 8

4.4 SEMANTIC SEGMENTATION RESULTS

  • For segmentation, DeepLabv3-ResNet-101 reaches 77.6% mIOU, beating vanilla by 1.1% and the 2×-epochs baseline by 0.9%.The segmentation adaptation uses object-specific texture sources, a full-image texture-source label mask, α = 0.2, and γ = 0.95.Source: §4.4 · pp. 8–9

5 RELATED WORK

The related work connects shape-texture debiased training to Mixup and style transfer. It adapts both to combine shape and texture cues while supplying supervision from each.

5 RELATED WORK

  • The method can be viewed as abstraction-level Mixup, blending image texture and shape information rather than pixels or learned features.Existing Mixup methods blend image pairs and labels at pixel or feature level.Source: §5 · p. 9
  • The authors describe shape-texture representation learning as a missing piece in existing data-augmentation methods.They state that their method guides CNNs to learn better shape and texture representations.Source: §5 · p. 9
  • Unlike artistic style transfer, the method stylizes one image using another image, enabling supervision from both shape and texture.It follows Geirhos et al. (2019) in using AdaIN, introduced by Huang & Belongie (2017).Source: §5 · p. 9

6 CONCLUSION

The conclusion presents shape-texture debiased training as a way to guide CNNs toward better representations by augmenting with cue-conflict images and supervising shape and texture together. The authors report gains in accuracy and robustness and say the approach generalizes across image-recognition tasks.

6 CONCLUSION

  • The method combines cue-conflict image augmentation with supervision from both shape and texture.The conclusion identifies both components as key to shape-texture debiased training.Source: §6 · p. 9
  • The authors report that shape-texture debiased training boosts both accuracy and robustness.They characterize the approach as conceptually simple and generalizable to different image-recognition tasks.Source: §6 · p. 9
  • The conclusion frames the method as guidance for CNNs to learn better feature representations.The authors hope the work will shed light on understanding and improving convolutional neural networks.Source: §6 · p. 9

Related papers

Loading 2010.05981v2…