PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Robustness using Generated Data

Sven Gowal, Sylvestre-Alvise Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, Timothy Mann

arXiv:2110.09468v2cs.LGcs.CVstat.ML

TL;DR

The paper addresses whether information in the original training set can be leveraged more effectively to improve adversarial robustness without additional real data. It uses generated samples, including low-quality Gaussian samples and generative-model outputs, and provides sufficient conditions explaining when they can improve robust generalization.

  • Problem

    Models trained without additional data have lower robust accuracy than models using additional data, motivating greater use of information in the original training set.

  • Method

    The paper augments adversarial training with diverse samples generated from a class-conditional Gaussian fit or generative models trained solely on the original data.

  • Results

    Generated samples improve robustness, including a +0.93% robust-accuracy gain from low-quality Gaussian inputs on CIFAR-10 and robust accuracies of 58.15% and 59.83% with a 74.47%-accurate non-robust classifier.

  • Takeaways & Limitations

    Diversity and complementarity of generated samples can help adversarial training exceed the current state-of-the-art, while sufficient conditions show that perfect data distributions or generative models are not always necessary.

  • Takeaways & Limitations

    In capacity-limited settings, optimization must focus on realistic inputs and the generated distribution should be as close as possible to the true distribution.

Abstract

from arXiv · show

Recent work argues that robust training requires substantially larger datasets than those required for standard classification. On CIFAR-10 and CIFAR-100, this translates into a sizable robust-accuracy gap between models trained solely on data from the original training set and those trained with additional data extracted from the "80 Million Tiny Images" dataset (TI-80M). In this paper, we explore how generative models trained solely on the original training set can be leveraged to artificially increase the size of the original training set and improve adversarial robustness to $\ell_p$ norm-bounded perturbations. We identify the sufficient conditions under which incorporating additional generated data can improve robustness, and demonstrate that it is possible to significantly reduce the robust-accuracy gap to models trained with additional real data. Surprisingly, we even show that even the addition of non-realistic random data (generated by Gaussian sampling) can improve robustness. We evaluate our approach on CIFAR-10, CIFAR-100, SVHN and TinyImageNet against $\ell_\infty$ and $\ell_2$ norm-bounded perturbations of size $ε= 8/255$ and $ε= 128/255$, respectively. We show large absolute improvements in robust accuracy compared to previous state-of-the-art methods. Against $\ell_\infty$ norm-bounded perturbations of size $ε= 8/255$, our models achieve 66.10% and 33.49% robust accuracy on CIFAR-10 and CIFAR-100, respectively (improving upon the state-of-the-art by +8.96% and +3.29%). Against $\ell_2$ norm-bounded perturbations of size $ε= 128/255$, our model achieves 78.31% on CIFAR-10 (+3.81%). These results beat most prior works that use external data.

1 Introduction

The paper addresses adversarial vulnerability and the robust-accuracy gap caused by relying only on original training data. It proposes using generated data, pseudo-labeling it, and combining it with original data to train robust classifiers.

  • Adversarial perturbations can cause neural networks to make incorrect predictions, motivating robust models for deployed applications.
  • The method trains a generative model and non-robust classifier, pseudo-labels generated samples, then combines generated and original data to train a robust classifier.
  • 65.87% robust accuracy with additional data versus 57.14% without it on CIFAR-10 under ℓ∞ perturbations of size 8/255 motivates using the original data more effectively.
  • The paper studies whether generated samples trained solely on original data can improve robustness despite concerns that generative models lack useful diversity.
  • +8.96% robust accuracy is achieved without additional external data on CIFAR-10, with the best model reaching 66.10% against AA+MT.

2 Related work

Related work covers adversarial attacks, adversarial training, data-driven augmentation, and generative defenses. The paper distinguishes its approach from prior methods whose robustness to ℓp attacks is limited or unsupported by evidence.

  • Adversarial ℓp norm-bounded attacks: Adversarial attacks progressed from FGSM's single gradient step to randomized and iterative gradient-based methods.
  • Adversarial training as a defense: Adversarial training is a leading defense, with modifications to attack procedures, loss functions, architectures, and the standard-versus-robust accuracy trade-off.
  • Data-driven augmentations: Data-driven augmentation methods learn policies or generate transformed views, but their primary successes include standard classification or robustness to common corruptions.
  • Robustness to ℓp norm-bounded perturbations using generative modeling: Generative defenses such as Defense-GAN and ME-Net have been broken by adaptive attacks, while evidence that generative models improve robustness to ℓp attacks remains scarce.

3 Adversarial training using generated data

The approach augments adversarial training with generated data, motivated by the need for samples that are diverse and complementary to the original set. Even class-conditional Gaussian samples can improve robustness when mixed appropriately.

  • 3.1 Adversarial training: Adversarial training minimizes adversarial risk over data examples and allowed perturbations, with ℓp attacks restricting perturbations to norm at most ε.
  • 3.2 Generated data can improve robust generalization: Mixing 50% to 90% original CIFAR-10 data with generated samples improves robust accuracy, whereas the endpoints represent generated-only and original-only training.
  • 3.2 Generated data can improve robust generalization: Common augmentations have been unsuccessful for adversarial training because they remain close to original images and are not sufficiently complementary.
  • 3.2 Generated data can improve robust generalization: The method hypothesizes that useful additional samples should be diverse, distribution-consistent, and non-duplicative of the original training set.
  • 3.2 Generated data can improve robust generalization: +0.93% robust accuracy is obtained on CIFAR-10 under ℓ∞ perturbations of size 8/255 using low-quality random inputs from class-conditional Gaussian fits.
  • 3.2 Generated data can improve robust generalization: Class-conditional Gaussian sampling fits each CIFAR-10 class using 5K training images and generates 100K samples per class, producing a 1M-image dataset.
  • 3.3 Method: The complete pipeline trains a non-robust classifier and generative model, pseudo-labels generated samples, and trains a robust classifier on both datasets.

4 Randomness might be enough

The paper formalizes conditions under which generated data can improve adversarial robustness, including settings where the generated distribution need not exactly match the true distribution. These conditions clarify both the method’s potential and its limitations.

  • Setup: The true distribution and ground-truth classifier are unavailable, so training instead uses an approximated distribution and a pre-trained non-robust classifier.The resulting parameters are therefore sub-optimal relative to optimization using the true distribution and labels.
  • Assumptions: The framework assumes realistic inputs have non-conflicting labels within their allowed perturbation sets and that realistic regions receive sufficient sampling frequency.These assumptions define the relevant input support and constrain the ground-truth labeling function.
  • Sufficient conditions: The capacity-limited regime requires an accurate non-robust classifier and an approximated distribution equivalent to the true data distribution.These are presented as sufficient conditions for the sub-optimal parameters to match the performance of optimal parameters.
  • Practical implications: Better generative models should improve robustness in capacity-limited settings because distributional closeness to the true data distribution matters.With sufficient compute and capacity, however, the theory suggests that neither the true distribution nor a perfect generative model is necessary.
  • Sufficient conditions: In the infinite-capacity regime, accurate pseudo-labeling, unlikely adversarial samples, and sufficient coverage can replace exact distribution equivalence.The generative distribution must assign non-zero probability to every relevant region, while sampled points should not be adversarial to the non-robust classifier.
  • Practical implications: Increasing the mixing factor α improves sample realism but reduces complementarity with the original training set.This exposes a trade-off between realistic generated samples and additional information beyond the training data.

5 Generative models

The paper studies four generative models trained only on the original training set and analyzes whether their samples complement and cover the data distribution. It also examines how violations of the sufficient conditions affect robust test accuracy.

  • Generative models: The study considers BigGAN, DDPM, StyleGAN2, and VDVAE as fundamentally different generative models trained solely on the original training set.This isolates the setting without external data.
  • Analysis of complementarity and coverage: The authors sample 10K images from each model and compare complementarity using nearest neighbors in Inception feature space.Nearest neighbors are assigned to the train set, test set, or generated set itself.
  • Analysis of complementarity and coverage: Coverage is assessed by the proportion of unique neighbors in the train and test sets, alongside IS and FID comparisons.The analysis treats these measures as complementary views of generated-data quality.
  • Condition-violation experiments: Figure 4 varies pseudo-labeler accuracy, the Gaussian–BigGAN mixture, and BigGAN class coverage while reporting robust test accuracy against AA+MT.All panels use different model architectures and ℓ∞ perturbations of size 8/255.
  • Analysis of complementarity and coverage: DDPM samples appear to provide better coverage of the true data distribution than samples from the other evaluated generative models.The reported trend aligns with robust-accuracy results, with StyleGAN2-trained models performing below DDPM-trained models.

6 Experiments

Experiments show that generated data can improve adversarial robustness when its quality, coverage, classifier capacity, and mixing ratio satisfy suitable conditions. DDPM-generated data produces especially strong gains across datasets and perturbation settings.

  • Sufficient conditions: 58.15% and 59.83% robust accuracy are achieved by WRN-28-10 and WRN-70-16 using a 74.47%-accurate non-robust classifier.Both results improve upon the contemporaneous 57.14% state of the art, showing that a perfect non-robust classifier is unnecessary.
  • Sufficient conditions: 17.37% is the robust-accuracy gap between ResNet-18 and WRN-70-16 at full coverage when Gaussian samples comprise 99% of training data.The result indicates that larger capacity can compensate for an imperfect generative model when coverage is sufficient.
  • Sufficient conditions: +3.73% and +6.29% robust-accuracy improvements are obtained with optimally mixed StyleGAN2 and DDPM samples, respectively.The optimal mixing factor differs by generative model; increasing α reduces distributional discrepancy but also reduces complementarity with the original training set.
  • Sufficient conditions: Samples from all tested generative models improve robustness when mixed optimally, although significant gains are reported only for StyleGAN2 and DDPM.Figure 5 varies α, the proportion of original images in each batch, from generated-only to original-data-only training.
  • State-of-the-art robust accuracy: 66.10% robust accuracy is reached on CIFAR-10 against ℓ∞ perturbations of 8/255 using 100M DDPM samples, improving 8.96% over the state of the art.With 1M DDPM samples, models improve state-of-the-art robust accuracy by 6.44% for ℓ∞ and 3.81% for ℓ2 perturbations.
  • Generalization: 33.49% robust accuracy is achieved on CIFAR-100, while DDPM samples improve robustness by 4.07% on SVHN, 5.10% on TinyImageNet, and 0.86% on ImageNet.These evaluations extend the generated-data approach beyond CIFAR-10.

7 Conclusion.

The paper concludes that generated samples trained from the original data can provide diverse augmentations for adversarial training and improve robustness beyond the current state of the art. It also emphasizes diversity and complementarity as factors shaping robustness.

  • Conclusion: Generated samples provide greater diversity of augmentations that allow adversarial training to go beyond the current state of the art.The authors also identify diversity and complementarity as sources of insight into robustness.

A Experimental setup

The experiments use ResNet and Wide ResNet architectures with TRADES-based adversarial training, stochastic weight averaging, and standardized adversarial evaluation. Training uses large batches and long CIFAR-10-equivalent schedules.

  • Experimental setup: Most experiments use a WRN-28-10 with depth 28, width multiplier 10, and 36M parameters.The architecture follows prior work and the details used by Gowal et al.
  • Experimental setup: TRADES is optimized with SGD using Nesterov momentum, weight decay 5 × 10^-4, cosine learning-rate decay, and a 0.4 initial learning rate.Training runs for 19K steps across 32 TPUv3 cores, with model weight averaging also applied.
  • Experimental setup: Adversarial examples maximize the Kullback-Leibler divergence between clean and adversarial predictions using Adam with step-size 0.1 for 10 steps.This defines the inner minimization used during adversarial training.
  • Experimental setup: Two models are trained per hyperparameter setting, early-stopped using PGD40 on a separate 1024-sample validation set, and the best model is selected by validation robust accuracy.The evaluation follows the protocol designed by Gowal et al.

B Additional results

Additional generated data improves robust accuracy across CIFAR-100, SVHN, and TinyImageNet, while DDPM samples outperform BigGAN samples and scale favorably with dataset size. Clean-accuracy changes are not consistently aligned with robust-accuracy changes.

  • Scaling dataset size: More DDPM or BigGAN samples generally improve robustness when training only on generated CIFAR-10 samples.The evaluation compares CIFAR-10 test accuracy with accuracy on a separate set of generated samples.
  • Scaling dataset size: 58.43% versus 43.34% robust accuracy on CIFAR-10 is obtained with DDPM versus BigGAN samples using a WRN-28-10.With 1M samples, DDPM produces a 5.74-point generalization gap, whereas BigGAN produces a 51.61-point gap.
  • CIFAR-100: +2.48% absolute robust accuracy is achieved on CIFAR-100 by adding DDPM samples against ϵ∞= 8/255.The comparison uses a WRN-28-10 and varies the ratio of original to generated images.
  • SVHN: +4.07% absolute robust accuracy is achieved on SVHN by adding DDPM samples against ϵ∞= 8/255.The experiment varies the proportion of generated and original samples in training minibatches.
  • TinyImageNet: +5.10% absolute robust accuracy is achieved on TinyImageNet by adding DDPM samples against ϵ∞= 8/255.The DDPM is trained on ImageNet, while evaluation uses a WRN-28-10 on TinyImageNet.
  • Clean accuracy: Clean-accuracy improvements do not consistently correlate with robust-accuracy improvements; BigGAN, StyleGAN2, and DDPM improve clean accuracy by up to +1.27%, +3.45%, and +2.05%.VDVAE samples provide no clean-accuracy improvement.

C Analysis of models

The analysis tests whether the strongest models' robustness measurements are meaningful and whether their loss landscapes indicate gradient obfuscation. AUTOATTACK, black-box attacks, and landscape visualizations provide complementary diagnostics.

  • AUTOATTACK and robustness against black-box attacks: The AA+MT attack cascade matches AUTOATTACK's final robust accuracy, while SQUARE finds no additional adversarial examples.These results support the meaningfulness of the empirical robustness measurement.
  • Loss landscapes: Both analyzed loss landscapes are smooth and do not exhibit patterns of gradient obfuscation.The landscapes vary inputs along the PGD40 worst-attack direction and a random Rademacher direction using adversarial margin loss.

D Details on generated data

The paper generates additional training data using four generative models and a class-conditional Gaussian fit, then evaluates the samples through filtering, feature-space complementarity, and coverage analyses.

  • Generative models: Four generative models—BigGAN, VDVAE, StyleGAN2, and DDPM—are trained solely on the relevant original training set.A class-conditional multivariate Gaussian serves as a baseline.
  • Generated datasets: Pseudo-labels for Gaussian and DDPM samples come from a non-robust WRN-28-10 trained on CIFAR-10, which achieves 95.68% accuracy.The same pseudo-labeling model is used to score and filter BigGAN and VDVAE samples.
  • Generated datasets: 1M generated image-label pairs are constructed by sampling 5M BigGAN and VDVAE images, retaining the top-100K scoring images per class, and combining classes.The generated data are mixed with original examples during adversarially robust training.
  • Complementarity and coverage: Complementarity measures whether generated samples are nearest to training, test, or generated samples, while coverage measures the proportion of unique training or test neighbors.The procedure uses sampled subsets and feature mappings such as Inception features.
  • Augmented samples: Data augmentations produce samples close to the train set and far from the test set, indicating limited complementarity to the true distribution.LPIPS-based measurements broadly agree with the Inception-based analysis.
  • Complementarity and coverage: FID and IS alone can miss memorization and limited distribution coverage, motivating nearest-neighbor complementarity and coverage metrics.A memorizing model can achieve near-perfect FID and IS while matching the train set almost exclusively.

E.1 Proofs

The proofs give sufficient conditions under which generated-data training can match optimal performance in capacity-limited and infinite-capacity regimes, while the mixing factor trades realism against complementarity.

  • Capacity-limited regime: In the capacity-limited regime, Conditions 1 and 2 suffice for sub-optimal parameters to match the performance of optimal parameters.When the non-robust classifier and generated distribution match their ideal counterparts, the relevant objectives become identical.
  • Infinite-capacity regime: In the infinite-capacity regime, Conditions 1, 3, and 4 suffice for sub-optimal parameters to match optimal performance.The proof uses non-conflicting labels within perturbation sets and generated-data coverage of the true distribution.
  • Impact of the mixing factor α: Increasing the mixing factor α improves training-sample realism but reduces complementarity with the training set.The mixing formulation combines uniformly sampled training data and generated data.

G Societal impact

The paper frames robustness as important for deployed systems, trains generative models on the same data to avoid separate-dataset privacy leakage, and provides an alternative to withdrawn 80M-TI data.

  • Societal impact: Robustness and generalization matter for deployed neural networks, including applications in ranking, autonomous driving, and medical diagnostics.The paper notes that robustness can also have unforeseen consequences.
  • Privacy: Training generative models from scratch on the same data as robust-model training avoids leakage risks associated with models trained on separate datasets.The cited risk is that generative models may leak confidential or private data.
  • External-data replacement: The work matches models trained with external 80M-TI data using only original CIFAR-10 data and releases generated datasets after 80M-TI was withdrawn.The paper cites derogatory categories and offensive images as reasons for the withdrawal.
  • Generated samples: Figure 12 organizes generated CIFAR-10 samples by class and assigns each image a pseudo-label using a standard classifier trained on CIFAR-10.Rows correspond to airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck.
Loading 2110.09468v2…