PaperlayerView exact revision on arXiv

Source-linked AI summary

Understanding and Improving Fast Adversarial Training

Maksym Andriushchenko, Nicolas Flammarion

arXiv:2007.02617v2cs.LGcs.CRcs.CVstat.ML

TL;DR

Computationally efficient FGSM adversarial training can undergo catastrophic overfitting, and random initialization does not reliably prevent it. The paper analyzes this failure through gradient alignment and proposes GradAlign, which prevents catastrophic overfitting, improves robustness, and reduces the gap to multi-step PGD training.

  • Problem

    FGSM adversarial training can suddenly lose robustness through catastrophic overfitting, while random-step variants remain unreliable, especially at larger perturbation radii.

  • Method

    The paper analyzes catastrophic overfitting through local linearity and gradient alignment, then uses GradAlign to increase gradient alignment during fast adversarial training.

  • Results

    GradAlign prevents catastrophic overfitting and improves robustness compared with other fast adversarial training methods, reducing the gap to multi-step PGD training.

  • Takeaways & Limitations

    FGSM training can be improved by controlling perturbation quality through gradient alignment rather than relying on random initialization.

  • Takeaways & Limitations

    The analysis in one section ignores the projection of perturbed inputs onto the valid input domain [0, 1]^d.

Abstract

from arXiv · show

A recent line of work focused on making adversarial training computationally efficient for deep learning models. In particular, Wong et al. (2020) showed that $\ell_\infty$-adversarial training with fast gradient sign method (FGSM) can fail due to a phenomenon called "catastrophic overfitting", when the model quickly loses its robustness over a single epoch of training. We show that adding a random step to FGSM, as proposed in Wong et al. (2020), does not prevent catastrophic overfitting, and that randomness is not important per se -- its main role being simply to reduce the magnitude of the perturbation. Moreover, we show that catastrophic overfitting is not inherent to deep and overparametrized networks, but can occur in a single-layer convolutional network with a few filters. In an extreme case, even a single filter can make the network highly non-linear locally, which is the main reason why FGSM training fails. Based on this observation, we propose a new regularization method, GradAlign, that prevents catastrophic overfitting by explicitly maximizing the gradient alignment inside the perturbation set and improves the quality of the FGSM solution. As a result, GradAlign allows to successfully apply FGSM training also for larger $\ell_\infty$-perturbations and reduce the gap to multi-step adversarial training. The code of our experiments is available at https://github.com/tml-epfl/understanding-fast-adv-training.

1 Introduction

Adversarial training improves robustness but requires approximate solutions to a difficult inner maximization problem. The paper asks when fast FGSM training produces robust models and proposes GradAlign to prevent catastrophic overfitting.

  • Adversarial training is widely used because it offers empirical robustness, scales to deep networks without affecting inference time, and supports different threat models.
  • The inner adversarial maximization is NP-hard, so practical adversarial training must rely on approximate methods.
  • FGSM uses a linear approximation of the loss, but prior work disagreed about whether FGSM training remains robust beyond small ℓ∞-perturbations.
  • Catastrophic overfitting occurs when FGSM training initially gains robustness but suddenly becomes non-robust within one training epoch.
  • The paper analyzes fast adversarial training failures and introduces GradAlign, which prevents catastrophic overfitting and improves robustness for large ℓ∞-perturbations.

2 Problem overview and related work

Adversarial training balances computational efficiency against the accuracy of solving each example-specific inner maximization. FGSM is fast but can suffer catastrophic overfitting, while existing mitigations include random initialization, early stopping, and hybrid methods.

  • The ℓ∞ threat model permits changing each input coordinate by at most ε, with a separate inner maximization problem for each example.
  • Because each inner problem is non-concave and example-specific, alternating gradient descent in model parameters and ascent in perturbations cannot directly solve the overall problem.
  • PGD uses multiple projected gradient-ascent steps and is more accurate, whereas computationally efficient methods seek to solve the inner problem in fewer iterations.
  • FGSM can produce temporary robustness before a sudden drop, called catastrophic overfitting; early stopping yields non-trivial but suboptimal robustness.
  • Random initialization, dropout, and switching from FGSM to multi-step PGD have been proposed to stabilize or improve fast adversarial training.

3 The role and limitations of using random initialization in FGSM training

Random initialization does not reliably prevent catastrophic overfitting in FGSM training; its main benefit is reducing perturbation magnitude and improving the linear approximation. Reducing the standard FGSM step size can achieve comparable robustness, but may remain suboptimal to PGD training for larger evaluation radii.

  • FGSM with a random step fails to resolve catastrophic overfitting for larger ℓ∞-perturbations.With α = 1.25ε, the working regime extends only from ε = 6/255 to ε = 9/255, while ε = 10/255 yields 0% adversarial accuracy.
  • Randomness is not crucial because projecting PGD perturbations onto the corners {−ε, ε}^d preserves robustness comparable to standard PGD training.At ε = 8/255, FGSM training achieves 0.00±0.00% adversarial accuracy, whereas standard and modified PGD-10 achieve 50.48 ± 0.20% and 50.64 ± 0.23%.
  • The perturbation-norm bound is estimated by Monte Carlo sampling and becomes increasingly tight for high-dimensional inputs.Figure 2 visualizes the bound using 1,000 samples of η; the dashed line denotes α = 1.25ε.
  • A random step improves FGSM chiefly by reducing expected perturbation magnitude, thereby improving the quality of the loss’s linear approximation.Perturbations with smaller ℓ2-norm have smaller linear approximation error, and this benefit disappears when the random-step size is too large.
  • Reducing FGSM’s step size to α ≈ 7/255 matches FGSM-RS robustness without randomness and can generalize to higher ε without early stopping.For ε = 8/255, the comparison uses standard FGSM with α = 7/255 versus FGSM-RS with α = 1.25ε; the smaller-step method remains suboptimal to PGD when evaluation exceeds the training radius.

4 Understanding catastrophic overfitting via gradient alignment

The paper links catastrophic overfitting to sharply reduced gradient alignment, showing that FGSM can fail when the loss is not locally linear within the perturbation region. This behavior occurs in both deep and very simple convolutional networks.

  • Gradient alignment and FGSM: FGSM is optimal only when the loss gradient remains constant inside the ℓ∞-ball, making local linearity central to its success.Gradient alignment measures cosine similarity between gradients at x and x + η; it equals one for locally linear models and approaches zero for nearly orthogonal gradients.
  • Deep networks: Around epoch 23, FGSM training on ResNet-18 caused PGD accuracy to fall from 40.1% to 0.0% while FGSM accuracy rose from 43.5% to 86.7%.Gradient alignment simultaneously dropped from 0.95 to 0.05 within one epoch, indicating that FGSM and PGD no longer identify similar perturbations.
  • Single-layer CNNs: A single-layer CNN with four filters also exhibited catastrophic overfitting, showing that the phenomenon is not specific to deep or overparameterized networks.Its pattern matched ResNet behavior through simultaneous reductions in PGD accuracy and gradient alignment.
  • Single-layer CNNs: At initialization, gradient alignment in sufficiently wide randomly initialized CNNs is bounded below by 0.5, whereas it was observed at 0.12 after catastrophic overfitting.The bound approaches one when ε is small relative to the expected patch norm.
  • Single-layer CNNs: A dominant Laplace filter amplified small random noise, changed ReLU activation signs, and reduced gradient alignment in the single-layer CNN.The filter w4 and its outgoing weights had much larger norms than the other filters, making their contribution to predictions and gradients most significant.

5 Increasing gradient alignment improves fast adversarial training

The paper proposes GradAlign, which maximizes gradient alignment between clean and randomly perturbed inputs without an iterative inner attack. Experiments show that it prevents catastrophic overfitting and improves fast adversarial training at larger perturbation radii.

  • GradAlign: GradAlign regularizes training by maximizing gradient alignment between gradients at x and at a random point x + η inside the perturbation ball.Its objective is the expected value of one minus the cosine similarity between the two input gradients.
  • GradAlign: GradAlign uses two gradient evaluations rather than an expensive iterative procedure and is invariant to gradient magnitude.This distinguishes it from iterative local-linearity methods and penalties based on gradient norms.
  • Experimental results: For ε ≤ 6/255 on CIFAR-10 and ε ≤ 4/255 on SVHN, the evaluated adversarial-training methods performed almost equally well.At larger radii, performance varied substantially because of catastrophic overfitting.
  • Experimental results: GradAlign successfully prevented catastrophic overfitting and produced high robustness for larger ℓ∞-perturbations.Figure 8 compares accuracy and robustness across methods, datasets, and radii, averaging results over five training seeds with standard deviations.
  • Experimental results: GradAlign also improved PGD-2 robustness at ε = 16/255 on CIFAR-10, while early stopping sacrificed standard accuracy.On ImageNet, GradAlign was unnecessary at the considered ε values because standard FGSM training did not catastrophically overfit.
  • Experimental results: Training FGSM with GradAlign beyond 30 epochs slightly worsened test robustness, indicating that robust and catastrophic overfitting are distinct phenomena.The method still avoided catastrophic overfitting when trained for up to 200 epochs.

6 Conclusions and outlook

The paper identifies catastrophic overfitting as a broad problem for computationally efficient adversarial training and proposes GradAlign to improve fast-training robustness. Its main trade-off is increased runtime from double backpropagation.

  • Conclusions: Catastrophic overfitting affects standard FGSM training and other computationally efficient adversarial-training schemes, including random-step FGSM and AT for Free.Early stopping provides non-trivial but suboptimal models.
  • Conclusions: GradAlign prevents catastrophic overfitting, improves robustness over other fast methods, and narrows the gap to multi-step PGD training.The method does so by stabilizing gradients under random noise, according to the paper’s conclusion.
  • Outlook: GradAlign increases runtime because it requires double backpropagation.The authors identify finding similarly effective regularizers without double backpropagation as future work.

Broader Impact

The paper frames efficient adversarial training as a way to obtain robust models without restricting training to small perturbations. It presents a regularization approach aimed at overcoming catastrophic overfitting.

  • Broader impact: The work studies failure reasons in computationally efficient adversarial training and proposes regularization to address catastrophic overfitting.The stated broader-impact motivation is to support efficient training of robust models.
  • Broader impact: Adversarial robustness is presented as desirable because it improves the reliability of machine-learning models.The paper connects this motivation to training robust models efficiently without limiting perturbations to small sizes.

A.2 Proof and discussion of Lemma 2

The proof derives lower bounds for gradient alignment in a single-layer CNN and compares them with empirical alignment estimates across perturbation radii.

  • Proof: The proof uses large-k,m approximations to replace empirical means with expectations over patches, perturbations, weights, and output parameters.It then bounds the resulting terms using Hoeffding’s and Cauchy–Schwarz inequalities.
  • Proof: The derivation separately lower-bounds numerator and denominator terms before combining them into a lower bound for gradient alignment.
  • Empirical discussion: The empirical comparison uses k = 100 patches and m = 4 filters, alongside the limiting expression for k,m →∞ and two derived lower bounds.
  • Empirical discussion: All four quantities have gradient-alignment values in [0.7, 1.0] for ε ∈ [0, 0.1], compared with 0.12 after catastrophic overfitting at ε = 10/255.The experiments approximate expectations with 1,000 Monte-Carlo samples and use 3 × 3 × 3 patches.
  • Empirical discussion: The derived lower bounds are empirically tight enough to capture gradient-alignment behavior for small ε, although the final bound is reported for concision.

B Experimental details

The experiments specify evaluation attacks, training protocols, model settings, and GradAlign hyperparameter selection across CIFAR-10, SVHN, and ImageNet.

  • Evaluation: Adversarial accuracy is evaluated with PGD-50-10 using step size α = ε/4, while standard accuracy uses full test sets and adversarial accuracy uses 1,000 random points.Robustness evaluation uses single precision even though training uses half precision.
  • ResNet-18 training: ResNet-18 experiments use cyclic learning rates, half-precision training, early stopping based on training-set PGD accuracy, and no random PGD initialization.The authors report that this model-selection scheme can select models before catastrophic overfitting.
  • Dataset protocols: CIFAR-10 training runs for 30 epochs with maximum learning rate 0.3, while AT for free uses 96 epochs, maximum learning rate 0.04, and m = 8 minibatch replays.
  • Dataset protocols: SVHN training runs for 15 epochs, ramps ε from 0 to its target during the first 5 epochs, and uses separate settings for AT for free with 45 epochs and m = 8 replays.
  • Hyperparameters: GradAlign λ values for CIFAR-10 and SVHN are interpolated logarithmically between successful values at ε = 8 and ε = 16, with λ = 0.1 for CIFAR-10 and λ = 0.5 for SVHN in PGD-2 experiments.ImageNet uses ResNet-50 and λ values of 0.01, 0.01, and 0.1 for ε = 2, 4, and 6.

C Supporting experiments and visualizations for Sec. 3 and Sec. 4

Supporting experiments connect catastrophic overfitting to poor local linearity and gradient alignment, while showing GradAlign improves fast-training robustness relative to relevant baselines.

  • Local linearity: Linearization error depends strongly on ∥δ∥2 even when ∥δ∥∞ = ε is fixed, with the effect more pronounced for FGSM perturbations.The comparison uses FGSM and uniformly sampled corner perturbations on CIFAR-10 with ε = 8/255.
  • Single-layer CNN: A four-filter FGSM-trained CNN exhibits catastrophic overfitting around epoch 6, with filter w4 learning a noise-sensitive Laplace-like pattern in its red and green channels.
  • Single-layer CNN: After catastrophic overfitting, only filter w4 is highly sensitive to uniform noise, causing dramatic feature-map changes and small gradient alignment.The passage links this small alignment to FGSM failure as an inner-maximization solution.
  • Robustness comparisons: For CIFAR-10, FGSM + GradAlign achieves good robustness and accuracy, outperforming FGSM and FGSM-RS and performing similarly to PGD-2 near ε = 16/255.
  • Robustness comparisons: Early stopping can preserve non-trivial robustness but often significantly sacrifices standard accuracy, whereas GradAlign matches PGD-10 standard accuracy.

D.3 Results for specific ℓ∞-radii

Results across selected perturbation radii show that GradAlign prevents catastrophic overfitting in settings where one-step or two-step training fails, with dataset- and scale-dependent benefits.

  • CIFAR-10: At ε = 8/255 on CIFAR-10, FGSM + GradAlign outperforms AT for Free and other FGSM methods, while FGSM-RS achieves 45.10% adversarial accuracy.
  • CIFAR-10: At ε = 16/255 on CIFAR-10, FGSM + GradAlign is the only one-step method reported to work without early stopping.Combining GradAlign with PGD-2 helps at ε = 16/255, but not at ε = 8/255 where PGD-2 does not catastrophically overfit.
  • SVHN: On SVHN, GradAlign prevents catastrophic overfitting for ε = 8/255 and ε = 16/255, although it remains 5% below PGD-2 at ε = 8/255.
  • SVHN: At ε = 12/255 on SVHN, PGD-2 + GradAlign reaches 31.26±0.24% adversarial accuracy versus 14.30±13.34% for PGD-2.
  • ImageNet: On ImageNet, catastrophic overfitting does not occur for standard FGSM at ε ∈ {2/255, 4/255}, so GradAlign provides no additional benefit and causes approximately 3× slowdown.
  • ImageNet: At ε = 6/255 on ImageNet, catastrophic overfitting occurs around epoch 3 for FGSM-RS but not for FGSM or FGSM + GradAlign.The authors note that repeated evaluation across random seeds is computationally demanding.

D.4 Ablation studies

The ablations examine GradAlign’s sensitivity to regularization strength, training duration, and very large perturbations. GradAlign prevents catastrophic overfitting across the tested settings, while extreme perturbations can make learning degenerate.

  • GradAlign λ: Small λ values can trigger catastrophic overfitting, producing 0% or highly variable average PGD-50-10 accuracy across runs.At ε = 16/255 on CIFAR-10, some runs succeed while others fail.
  • Training epochs: The study varies the total number of training epochs to assess whether GradAlign affects robust overfitting.Robust overfitting is described as longer training hurting test-set adversarial accuracy.
  • Very high ε: The experiments test GradAlign at very high ε values beyond the previously reported ranges for CIFAR-10 and SVHN.The reported setup uses FGSM + GradAlign without early stopping on ResNet-18.
  • Very high ε: 24.04±0.31% adversarial accuracy is achieved on SVHN, close to the 18.50% majority-classifier accuracy.Increasing ε further causes the model to learn a constant classifier.
  • Very high ε: GradAlign shows no catastrophic overfitting even for very high ε on CIFAR-10.Table 7 reports robustness and accuracy averaged over three training seeds without early stopping.

D.5 Comparison of GradAlign to gradient-based penalties

The paper compares GradAlign with gradient norm and curvature penalties as stabilizers for FGSM training. GradAlign performs competitively at ε = 8/255 and gives higher average adversarial accuracy than the compared alternatives at ε = 16/255, while its computational cost motivates future speedups.

  • Compared penalties: The comparison tests ℓ2 gradient norm penalization and CURE alongside FGSM training to determine whether they prevent catastrophic overfitting.CURE penalizes the difference between input gradients at the FGSM point and the original input.
  • Results: At ε = 8/255, all three approaches prevent catastrophic overfitting, with final robustness ranging from 46.69% for the ℓ2-gradient penalty to 47.58% for GradAlign.The results come from a grid search over λ.
  • Results: At ε = 16/255, FGSM + CURE and FGSM + GradAlign prevent catastrophic overfitting with standard deviations of 0.29% and 0.70%, respectively.Both methods produce concentrated results across runs.
  • Results: 28.88% versus 25.38% adversarial accuracy favors FGSM + GradAlign over FGSM + CURE at ε = 16/255.FGSM with the ℓ2-gradient penalty is unstable, reporting 13.64 ± 11.2% adversarial accuracy.
  • Interpretation: GradAlign differs from the considered penalties by being invariant to gradient norm and using only gradient directions inside the ℓ∞-ball.The authors identify this directional treatment as the main performance difference.
  • Limitation: GradAlign’s reliance on input gradients slows training, motivating future work on parallelization, batch subsampling, or alternative regularizers.The paper explicitly identifies speedup as an open direction.
Loading 2007.02617v2…