PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Examples Improve Image Recognition

Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan Yuille, Quoc V. Le

arXiv:1911.09665v2cs.CV

TL;DR

Adversarial examples are often treated as threats to ConvNets, and prior work has focused on robustness rather than improving recognition accuracy with them. AdvProp separates clean and adversarial examples during training, and its EfficientNet-B8 reaches 85.5% ImageNet top-1 accuracy without extra data.

  • Problem

    Prior work largely treated adversarial examples as threats and pursued robustness, while this paper examines their use for improving recognition accuracy.

    Source: §1 · p. 1

  • Method

    AdvProp trains on clean and adversarial images as additional samples while using separate batch-normalization statistics to disentangle their distributions.

    Source: §1 · p. 2, §4.1 · p. 3

  • Results

    AdvProp improved accuracy across all tested ConvNets, with EfficientNet-B8 reaching 85.5% ImageNet top-1 accuracy without extra data.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    For clean-image performance, adversarial examples can serve as additional training samples rather than the target of robustness training.

    Source: §4.1 · p. 3

    Adversarial training still produced inferior accuracy on ResNet, including ResNet-200, and the authors leave architecture’s role for future work.

    Source: §5.4 · p. 8

Abstract

from arXiv · show

Adversarial examples are commonly viewed as a threat to ConvNets. Here we present an opposite perspective: adversarial examples can be used to improve image recognition models if harnessed in the right manner. We propose AdvProp, an enhanced adversarial training scheme which treats adversarial examples as additional examples, to prevent overfitting. Key to our method is the usage of a separate auxiliary batch norm for adversarial examples, as they have different underlying distributions to normal examples. We show that AdvProp improves a wide range of models on various image recognition tasks and performs better when the models are bigger. For instance, by applying AdvProp to the latest EfficientNet-B7 [28] on ImageNet, we achieve significant improvements on ImageNet (+0.7%), ImageNet-C (+6.5%), ImageNet-A (+7.0%), Stylized-ImageNet (+4.8%). With an enhanced EfficientNet-B8, our method achieves the state-of-the-art 85.5% ImageNet top-1 accuracy without extra data. This result even surpasses the best model in [20] which is trained with 3.5B Instagram images (~3000X more than ImageNet) and ~9.4X more parameters. Models are available at https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet.

1. Introduction

The introduction presents AdvProp as a way to use adversarial examples to improve image recognition by separating clean and adversarial distributions with two batch-normalization paths. It reports gains for EfficientNet-B7 and a state-of-the-art ImageNet result for EfficientNet-B8 without extra data.

1. Introduction

  • 85.2% ImageNet accuracy: Figure 1 reports that AdvProp beats vanilla EfficientNet-B7 by 0.7%, 6.5% lower mCE on ImageNet-C, 7.0% on ImageNet-A, and 4.8% on Stylized-ImageNet.The ImageNet-C value is 52.9% mCE, for which lower is better.Source: Figure 1 · p. 1
  • Prior work found adversarial training gains only in limited settings, while supervised training on large datasets such as ImageNet could degrade clean-image performance, leaving its effective use unresolved.The introduction frames AdvProp as a shift from defending against adversarial examples to leveraging them for accuracy.Source: §1 · p. 1
  • AdvProp addresses the hypothesized distribution mismatch by using separate batch-normalization statistics for clean and adversarial examples, disentangling their distributions at normalization layers.The authors argue that this disentangling enables adversarial examples to improve rather than degrade model performance.Source: §1 · p. 2
  • 85.2% top-1 accuracy: AdvProp-trained EfficientNet-B7 beats its vanilla counterpart by 0.8% and improves absolute accuracy by 9.0%, 7.0%, and 5.0% on ImageNet-C, ImageNet-A, and Stylized-ImageNet.The introduction identifies this as the first demonstration that adversarial examples improve fully supervised performance on large-scale ImageNet.Source: §1 · p. 2
  • 85.5% top-1 accuracy: EfficientNet-B8 with AdvProp reaches state of the art on ImageNet without extra data, surpassing the model in [24] pretrained on 3.5B Instagram images.The comparison model uses about 9.4× more parameters than EfficientNet-B8.Source: §1 · p. 2

2. Related Work

Prior work commonly presents adversarial training as a robustness tool that can reduce clean accuracy; this paper instead uses adversarial examples as complementary training data to improve recognition without extra data.

Adversarial Training

  • Adversarial training under supervised learning improves robustness but is typically associated with clean-accuracy drops, including ~10% on CIFAR-10 and ~15% on ImageNet.The passage also notes that VAT [26] and deep co-training [30] use adversarial examples in semi-supervised settings but require enormous extra unlabeled datasets.Source: §2 · p. 2
  • This work targets standard supervised learning without extra data, using adversarial examples to improve clean image recognition rather than treating them as a threat.This contrasts with prior work's usual framing of adversarial training as improving robustness while hurting clean accuracy.Source: §2 · p. 2

Benefits of Learning Adversarial Features

  • Prior studies report that adversarially trained features align better with salient characteristics and human perception [43], resist high-frequency noise [47], and emphasize shape over texture [51].AdvProp exploits the complementarity of clean images and their corresponding adversarial examples, and its results support the benefit of adversarial features for recognition.Source: §2 · p. 2

Data augmentation

  • AdvProp can be viewed as data augmentation through noise-injected samples, while earlier random-noise and adversarial-noise augmentation attempts [16] [18] failed to improve clean-image accuracy.Traditional augmentation includes flipping and cropping, masking, Gaussian noise, and mixup; learned policies have also been used for image classification and object detection.Source: §2 · p. 2

3. A Preliminary Way to Boost Performance

Preliminary experiments show that adversarial pretraining followed by clean-image fine-tuning can improve clean-image accuracy over adversarial-only training and sometimes vanilla training. The gains are not consistent across models, motivating a more effective way to use adversarial examples.

3. A Preliminary Way to Boost Performance

  • Fine-tuning after adversarial pretraining improved EfficientNet-B3 accuracy by 3.3% over Madry’s baseline and raised EfficientNet-B7 accuracy by 0.3% over vanilla training to 84.8% ImageNet top-1.The experiment pretrained with adversarial examples, then fine-tuned with clean images.Source: Figure 2 · p. 3, §3 · p. 3
  • Adversarially trained EfficientNet-B3 scored 78.2% on clean images versus 81.7% for vanilla training, consistent with prior reports of poor clean-image generalization [23] [45].Source: §3 · p. 3
  • The authors attribute the clean-image performance drop to distribution mismatch between adversarial and clean examples, and test whether clean-image fine-tuning can bridge it.Source: §3 · p. 3
  • The fine-tuning strategy did not improve performance generally: EfficientNet-B3 remained 0.2% below vanilla training.Source: §3 · p. 3

4. Methodology

AdvProp jointly trains on clean and adversarial images while using separate batch-normalization statistics to address their distribution mismatch. The method aims to retain useful features from both domains without the clean-image performance degradation associated with direct mixed training.

4.1. Adversarial Training

  • Unlike adversarial training that replaces original samples with perturbed ones, AdvProp treats adversarial images as additional samples to improve clean-image performance, addressing the lower clean-image performance reported for direct mixed training [7] [18].The authors attribute this limitation to distribution mismatch that hinders learning useful features from both domains.Source: §4.1 · pp. 3–4

4.2. Disentangled Learning via An Auxiliary BN

  • Auxiliary BNs keep normalization statistics separate for clean and adversarial images, whereas one shared set can estimate inaccurate statistics and degrade performance.The method generalizes to multiple auxiliary BNs based on the number of training sources; for clean, distorted, and adversarial data, it maintains two auxiliary BNs, and the paper reports that this can further improve performance.Source: §4.2 · p. 4

4.3. AdvProp

  • For each clean mini-batch, AdvProp generates adversarial examples using auxiliary BNs, computes losses with main BNs for clean images and auxiliary BNs for adversarial images, then jointly updates shared network parameters.Only the BNs differ between the two paths; convolutional and other layers are optimized for both domains. At inference, auxiliary BNs are dropped; on EfficientNet-B7 they add 0.5% more parameters than the baseline during training.Source: §4.3 · p. 4
  • AdvProp outperforms adversarial-training baselines [7] [18] and the fine-tuning strategy by jointly learning useful features from clean and adversarial examples.The supplied passage reports these comparisons without quantitative results.Source: §4.3 · p. 4

5. Experiments

Experiments train EfficientNet-B0–B7 with PGD-generated examples on ImageNet and evaluate clean validation accuracy plus generalization to corrupted, hard, and stylized images.

5.2. ImageNet Results and Beyond

  • 85.2% top-1 accuracy: AdvProp’s EfficientNet-B7 beats the 84.5% prior art [41] by 0.7%, with gains over vanilla training increasing for larger networks.EfficientNet-B6 also surpasses the prior art by 0.3% while using ∼2× less FLOPs than B7.Source: §5.2 · p. 5, Figure 4 · p. 5
  • 52.9% mCE on ImageNet-C, 44.7% top-1 accuracy on ImageNet-A, and 26.6% top-1 accuracy on Stylized-ImageNet: AdvProp’s EfficientNet-B7 achieves the reported best results.These are the best results reported without training on corresponding distortions [6] or using extra data [24] [46]; AdvProp also improves EfficientNet-B3 by 5.1% on ImageNet-C and 3.6% on Stylized-ImageNet, versus 0.2% on ImageNet.Source: §5.2 · pp. 5–6, Table 1 · p. 6
  • 77.6% vs. 77.1%: EfficientNet-B0 performs best with PGD1 (ϵ=1) rather than PGD5 (ϵ=4), while stronger attacks improve EfficientNet-B7 by 0.2%.The authors attribute the small-network trend to limited capacity to distill information from strong adversarial examples, even with auxiliary BNs.Source: §5.2 · p. 6, Table 2 · p. 6

5.3. Comparisons to Adversarial Training

  • AdvProp beats traditional adversarial training on ImageNet for every model, by 0.9% on EfficientNet-B0 and 0.1% on B7, as its advantage narrows with model size.The main-BN versus auxiliary-BN performance gap shrinks from 3.4% on B0 to 1.9% on B7, suggesting less distribution mismatch in larger networks; the authors attribute small-model gains mainly to auxiliary-BN disentanglement. Their reproduced adversarial training also beats vanilla B7 by 0.6%, using stronger networks and weaker attackers than prior work [18] [16].Source: §5.3 · pp. 6–7
  • 3.1% on ImageNet-C, 4.3% on ImageNet-A, and 1.5% on Stylized-ImageNet: AdvProp improves EfficientNet-B7 over adversarial training.Raising the attack from PGD5 (ϵ=4) to PGD7 (ϵ=6) yields 85.3% ImageNet top-1 accuracy with AdvProp, while traditional adversarial training reaches 85.0%.Source: §5.3 · p. 7, Table 4 · p. 7

5.4. Ablations

  • 77.9% accuracy with 5.3M parameters: fine-grained AdvProp adds a separate auxiliary BN for AutoAugment images and improves EfficientNet-B0; the cited MobileNetv3 baseline has 75.2% accuracy with 5.4M parameters [11].AutoAugment alone slightly exceeds AdvProp on small models, AdvProp wins on large models, and combining both outperforms AdvProp alone.Source: §5.4 · pp. 7–8, Table 5 · p. 8, Table 6 · p. 8
  • On ImageNet, all attackers tested with EfficientNet-B3, EfficientNet-B5, and EfficientNet-B7 improve performance over vanilla training; AdvProp improves accuracy on all ResNet models, while adversarial training degrades it.For ResNet-152, adversarial training decreases baseline performance by 2.0%, while AdvProp boosts it by 0.8%. Adversarial training remains inferior on ResNet-200.Source: §5.4 · p. 8
  • 85.5% top-1 accuracy: AdvProp raises EfficientNet-B8 from 84.8% and sets a new ImageNet state of the art without extra data, surpassing [24].The model reported in [24] used 3.5B extra Instagram images and ∼9.4× more parameters (829M vs. 88M).Source: §5.4 · p. 8

6. Conclusion

AdvProp treats adversarial examples as useful training data rather than solely a threat, using separate normalization for adversarial and clean images. The authors report accuracy gains across their ConvNet experiments and 85.5% ImageNet top-1 accuracy without extra data.

6. Conclusion

  • 85.5% top-1 accuracy on ImageNet without extra data is the authors’ reported state-of-the-art result for their best model.The authors also report that AdvProp significantly improves accuracy across all ConvNets in their experiments.Source: §6 · p. 8
  • AdvProp processes adversarial examples and clean images separately at normalization layers using auxiliary batch normalization.The method is motivated by the different underlying distributions of adversarial and normal examples.Source: §6 · p. 8
  • AdvProp reframes adversarial examples as a way to improve ConvNet accuracy, countering prior work that viewed them as a threat associated with clean-image accuracy drops.Source: §6 · p. 8

Related papers

Loading 1911.09665v2…