PaperlayerView exact revision on arXiv

Source-linked AI summary

Fixing Data Augmentation to Improve Adversarial Robustness

Sylvestre-Alvise Rebuffi, Sven Gowal, Dan A. Calian, Florian Stimberg, Olivia Wiles, Timothy Mann

arXiv:2103.01946v2cs.CVcs.LG

TL;DR

Adversarial training suffers from robust overfitting, motivating better use of data to improve robustness. The paper combines model weight averaging with heuristic and generative augmentations, achieving substantial CIFAR-10 robust-accuracy gains without external data. The approach reaches 64.20% and 80.38% robust accuracy under ℓ∞ and ℓ2 perturbations, respectively.

  • Problem

    The paper addresses robust overfitting and asks whether the original training set can be used more effectively to improve adversarial robustness.

  • Method

    The paper combines model weight averaging with heuristic augmentations and generated inputs from generative models, evaluating adversarial training under ℓp-bounded perturbations.

  • Results

    64.20% and 80.38% robust accuracy are achieved against ℓ∞ perturbations of size 8/255 and ℓ2 perturbations of size 128/255, improving upon state-of-the-art methods by +7.06% and +5.88%.

  • Takeaways & Limitations

    Combining heuristic and data-driven augmentations with weight averaging improves robust accuracy beyond prior methods without additional external data.

  • Takeaways & Limitations

    A remaining gap to the robust accuracy obtained with 500K 80M-TI images indicates a scope boundary for data-driven augmentation.

Abstract

from arXiv · show

Adversarial training suffers from robust overfitting, a phenomenon where the robust test accuracy starts to decrease during training. In this paper, we focus on both heuristics-driven and data-driven augmentations as a means to reduce robust overfitting. First, we demonstrate that, contrary to previous findings, when combined with model weight averaging, data augmentation can significantly boost robust accuracy. Second, we explore how state-of-the-art generative models can be leveraged to artificially increase the size of the training set and further improve adversarial robustness. Finally, we evaluate our approach on CIFAR-10 against $\ell_\infty$ and $\ell_2$ norm-bounded perturbations of size $ε= 8/255$ and $ε= 128/255$, respectively. We show large absolute improvements of +7.06% and +5.88% in robust accuracy compared to previous state-of-the-art methods. In particular, against $\ell_\infty$ norm-bounded perturbations of size $ε= 8/255$, our model reaches 64.20% robust accuracy without using any external data, beating most prior works that use external data.

1. Introduction

The paper targets stagnating adversarial robustness by improving use of the original training set through augmentation, model weight averaging, and generated data. On CIFAR-10, the combined approach substantially exceeds prior robust-accuracy results without external data.

  • Adversarial perturbations can induce high-confidence incorrect predictions, motivating defenses such as adversarial training.
  • Model weight averaging combined with Cutout, CutMix, or MixUp improves adversarial robustness.
  • 60.07% robust accuracy is achieved with CutMix on CIFAR-10 against ℓ∞ perturbations of size 8/255, improving upon the state of the art by +2.93%.
  • Generated samples from DDPM reach 63.58% robust accuracy against ℓ∞ perturbations of size 8/255, an improvement of +6.44% upon the state of the art.
  • 64.20% and 80.38% robust accuracy are obtained against ℓ∞ and ℓ2 perturbations, with improvements of +7.06% and +5.88%, respectively.
  • The best CIFAR-10 models beat techniques using additional data except for Gowal et al. (2020), while using no additional data.

2. Related Work

Related work develops adversarial defenses, heuristic and data-driven augmentations, and external-data strategies. These approaches establish the context for using augmentation and generated data to improve robustness.

  • Adversarial attacks: FGSM uses one normalized gradient step, R+FGSM adds randomization, and BIM uses multiple smaller gradient steps to craft adversarial examples.
  • Adversarial training: Adversarial training feeds perturbed examples into training and has been modified through attack procedures, loss functions, and architectures.
  • Heuristics-driven data augmentation: Cutout creates random occlusions, CutMix replaces image parts, and MixUp linearly interpolates between images.
  • Data-driven augmentation: AutoAugment and RandAugment learn augmentation policies from data, while DeepAugment uses image-to-image model perturbations for robustness to common corruptions.
  • Robustness improvements: Figure 2 examines external data, model weight averaging, and robust overfitting during adversarial training on CIFAR-10.

3. Preliminaries and Hypothesis

The paper formalizes adversarial training and robust overfitting, then hypothesizes that model weight averaging remains useful when overfitting is reduced. Experiments compare external data, averaging, and augmentation during training.

  • Adversarial training: Adversarial training minimizes adversarial risk over model parameters and data distributions, with perturbations constrained by an ℓp norm.
  • Adversarial training: PGD computes adversarial perturbations through K projected gradient-ascent steps within the allowed perturbation set.
  • Robust overfitting: Robust overfitting occurs when test robust accuracy declines while training robust accuracy continues rising; early stopping is a standard response.
  • Robust overfitting: Adding 500K pseudo-labeled 80M-TI images makes robust overfitting disappear in the described setting.
  • Model weight averaging: Model weight averaging uses an exponential moving average of parameters and can improve robustness across models and datasets.
  • Hypothesis: Weight averaging is hypothesized to remain beneficial without robust overfitting, and Figure 2(c) reports preserved and boosted robust performance in that setting.

4. Heuristics-driven Augmentations

Heuristics-driven augmentations can reduce robust overfitting, and their combination with model weight averaging can improve adversarial robustness. The benefit depends on the augmentation and training dynamics.

  • Data augmentation with early stopping alone does not improve robustness beyond early stopping, but it slows the decrease caused by robust overfitting.This contrasts with the stronger gains observed when augmentation is combined with model weight averaging.
  • Model weight averaging makes MixUp surpass Pad & Crop in robust accuracy while preserving MixUp’s robust accuracy during training.The comparison tests whether maintaining performance between model iterations makes averaging more beneficial.
  • Cutout, CutMix, and MixUp are evaluated as image-patching or mixing techniques intended to maintain robust accuracy and work with model weight averaging.The study also evaluates AutoAugment and RandAugment later in the paper.
  • Cutout and CutMix reach higher best robust accuracy than MixUp, with performance roughly identical to Pad & Crop without emphasizing the averaging effect.The figure compares robust-accuracy evolution with and without model weight averaging for these schemes.

5. Data-driven Augmentations

Data-driven augmentation uses generative models to expand the training distribution, addressing the limited diversity of conventional augmented views. The paper evaluates sample diversity, similarity, and robustness effects across several generative models.

  • Conventional augmentations remain close to the original image, limiting their ability to improve generalization for adversarially robust models.The paper connects this limitation to the large amount of data required for robust learning.
  • Diversity and complementarity: Samples from 80M-TI1 have neighbors well distributed between train and test sets, whereas MixUp, Cutout, and CutMix samples are close to images from the original data.The analysis treats balanced train-test proximity as desirable for augmentation.
  • Generative models are considered as augmentation candidates, but this study restricts them to models trained solely on the original training set.This preserves the setting without external data.
  • Diversity and complementarity: 10K-sample nearest-neighbor analysis in LPIPS space measures whether augmented samples resemble train, test, or other augmented images, together with entropy of these proportions.Higher entropy indicates a more balanced nearest-neighbor distribution.
  • The study compares BigGAN, VDVAE, and DDPM as three fundamentally different generative models for producing additional training samples.DDPM is described as reaching state-of-the-art FID on CIFAR-10.
  • Only DDPM-generated samples successfully prevent robust overfitting in the reported training comparison.Figure 5 tracks robust test accuracy against PGD40 with generated samples and without model weight averaging.

6. Experimental Results

Experiments show that augmentation, generative data, and their combination improve adversarial robustness on CIFAR-10, with benefits depending on model size and data setting. The combined approach reaches 64.20% robust accuracy without external data.

  • Heuristics-driven augmentations: +3.06% robust accuracy is obtained by CutMix over the Pad & Crop baseline in the setting without added data.Patch-based methods including RICAP, Cutout, and CutMix also average +1.79% clean-accuracy improvement.
  • Model size and data setting: CutMix consistently outperforms Pad & Crop by at least +2.90% robust accuracy across model sizes without added data.With external data, CutMix is worse for WRN-28-10 but improves the WRN-70-16 result to 66.56%.
  • Data-driven augmentations: A robust-accuracy gap remains between generated-data training and training with 500K images from 80M-TI.This comparison defines a remaining boundary for the data-driven augmentation results.
  • Combining augmentations: Both augmentation techniques are complementary, although combining them is most beneficial with a larger model in the ℓ∞ setting.Individually, CutMix or generated data provide significant gains under both threat models.
  • Combining augmentations: 64.20% robust accuracy is achieved by combining CutMix with DDPM-generated samples on CIFAR-10 without external data.This improves by +7.06% over the best model from Gowal et al. (2020).

7. Conclusion

The conclusion argues that model weight averaging makes heuristic augmentation effective for adversarial robustness and that generative samples add diverse augmentations. Together, these approaches extend adversarial training beyond prior state-of-the-art results.

  • 7. Conclusion: Heuristics-based augmentation combined with model weight averaging significantly improves robustness despite earlier unsuccessful attempts with augmentation.The conclusion specifically cites Cutout, CutMix, and MixUp as effective techniques in this setting.
  • 7. Conclusion: Generated samples provide greater augmentation diversity, help densify the image manifold, and allow adversarial training to go beyond state-of-the-art performance.The paper presents this as a demonstrated role for generative models trained on the original data.

A. Experimental Setup

The experiments use Wide Residual Networks with TRADES-based adversarial training and evaluate robust accuracy using a specified validation and attack protocol. Weight averaging and its decay rate are part of the setup.

  • Adversarial training: TRADES with 10 PGD steps is used for adversarial training, with larger batches and longer training when generated or external data are added.Training uses batch size 512 without added data and 1024 with added data.
  • Evaluation: Robust test accuracy is reported against a mixture of AUTOATTACK and MULTITARGETED, denoted AA+MT.Two models are trained per hyperparameter setting, and the best is selected using a separate 1024-sample validation set with PGD40.
  • Weight averaging ablation: Figure 8 varies the model-weight-averaging decay rate for WRN-28-10 models trained with CutMix or Pad & Crop.The study measures robust accuracy against AA+MT at ϵ∞= 8/255 on CIFAR-10.

B. Additional Experiments

Additional experiments show that augmentation settings interact with model weight averaging, while CutMix and generated-data approaches improve robustness across CIFAR-100 and SVHN.

  • Model weight averaging decay rate: τ = 0.999 gives CutMix its best robust accuracy, whereas Pad & Crop peaks at τ = 0.9925.The differing optima support tuning weight averaging jointly with augmentation.
  • Window length of Cutout: 18 pixels is the optimal CutOut window length whether model weight averaging is used or not.The experiment varies the fixed occlusion size on 32 × 32 CIFAR-10 images.
  • Window length of CutMix: +3.14% robust accuracy is attributed to model weight averaging with CutMix.CutMix also achieves the highest robust accuracy among MixUp, CutOut, Pad & Crop, and CutMix in this comparison.
  • CIFAR-100: 34.64% robust accuracy is achieved on CIFAR-100 against AUTOATTACK without external data.The best model improves noticeably on the state of the art in that setting.
  • SVHN: 61.09% robust accuracy is achieved on SVHN against AA+MT, improving noticeably over the baseline.The authors note that CutMix was not designed for SVHN, so its improvement is expected to be smaller.

C. Analysis of Models

The model analyses test robustness with multiple AUTOATTACK components, perturbation settings, and loss landscapes, finding converged attacks and smooth landscapes without gradient obfuscation.

  • AUTOATTACK and robustness against black-box attacks: AA+MT matches AUTOATTACK’s final robust accuracy, while SQUARE finds no additional adversarial examples.These results support the meaningfulness of the empirical robustness measurements.
  • Further analysis of gradient obfuscation: Robust accuracy gradually decreases as ϵ∞ increases from zero to 64/255 under AUTOPGD-CE.The attack uses 100 steps and one restart, and the trend indicates that PGD-based attacks find adversarial examples.
  • Further analysis of gradient obfuscation: 50 attack steps are sufficient for measured robust accuracy to converge at ϵ∞ = 8/255.The experiment varies AUTOPGD-CE from five to 1000 steps with one restart.
  • Loss landscapes: The loss landscapes are smooth and show no patterns of gradient obfuscation for CutMix, DDPM, or DDPM + CutMix models.Landscapes vary inputs along PGD40 and random Rademacher directions.

D. Details on Data-driven Augmentations

Data-driven augmentation uses generative models trained on the original data, selected pseudo-labeled samples, and mixtures of real and generated images to improve robustness and diversity.

  • Generative models: DDPM, VDVAE, and BigGAN are trained solely on CIFAR-10 data and used as complementary generative models.The reported FID values are 3.28 for DDPM, 36.88 for VDVAE, and 11.07 for BigGAN.
  • Datasets of generated samples: 1M image-label pairs are constructed by selecting the top-100K-scoring images per class from 5M samples per generative model.A pretrained non-robust WRN-28-10 assigns the scores and pseudo-labels.
  • Diversity and complementarity: Nearest-neighbor analysis compares whether generated samples are closest to train, test, or generated images in LPIPS-PCA space.The pipeline uses 10K train images, the full test set, VGG-based LPIPS features, and 100 PCA components.
  • Additional results: All generative models improve robustness when mixed optimally with original images, including a +0.93% gain from the conditional Gaussian baseline.The same trend appears on CIFAR-100 and SVHN.
  • Sufficient conditions: The sufficient-condition analysis requires an accurate pseudo-labeling classifier, unlikely sampled attacks against it, and sufficient coverage of the image manifold.These conditions motivate combining generated samples with original training data.

E.2. Sufficient Conditions

The paper identifies three sufficient conditions under which generated data can support robust classification: accurate pseudo-labeling, unlikely attacks, and broad manifold coverage.

  • Sufficient conditions: The sufficient conditions are stated in the limit of infinite capacity and compute as explanations for why generated data improves robustness.They provide a theoretical understanding of the method rather than a claim that every practical setting satisfies them.
  • Condition 1 (accurate classifier): The non-robust classifier must achieve sufficient accuracy for generated samples to receive useful labels.A perfect classifier recovers the ideal formulation when the generated and real distributions match.
  • Condition 2 (unlikely attacks): The probability of sampling generated points that are adversarial to the non-robust classifier must be low.The condition concerns off-manifold points lying within an adversarial ball of a real image.
  • Condition 3 (sufficient coverage): The generative model must assign non-zero sampling probability across the image manifold.Low coverage or mode collapse can rapidly reduce the robust classifier’s accuracy.
  • Discussion: With limited capacity or compute, robustness depends on focusing optimization on real images and keeping the generated distribution close to the true distribution.Better generative models such as DDPM are therefore associated with better robustness in practice.
Loading 2103.01946v2…