PaperlayerView exact revision on arXiv

Source-linked AI summary

Overfitting in adversarially robust deep learning

Leslie Rice, Eric Wong, J. Zico Kolter

arXiv:2002.11569v2cs.LGstat.ML

TL;DR

Deep learning often tolerates overparameterization and prolonged training without apparent generalization harm, but the paper tests whether this holds for adversarial training. Through empirical studies of robust training, early stopping, generalization curves, and overfitting remedies, it finds substantial robust overfitting across settings and shows that early stopping can match newer methods. The paper further reports that double descent does not explain this behavior and that other remedies generally do not improve significantly upon early stopping.

  • Problem

    Deep learning commonly trains overparameterized models for long durations without apparent generalization harm, motivating the paper’s study of adversarial training.

  • Method

    The paper empirically studies adversarially trained networks, robust loss optimization with PGD-based perturbations, learning curves, early stopping, double descent, and overfitting remedies.

  • Results

    Robust overfitting substantially harms robust performance across adversarial-training settings, while early stopping matches virtually all cited algorithmic improvements.

  • Takeaways & Limitations

    Validation-based early stopping and learning-curve analysis are important for model selection because the best robust performance need not occur at convergence.

  • Takeaways & Limitations

    The paper distinguishes robust overfitting from catastrophic overfitting reported for FGSM training and notes that standard double descent does not explain the observed robust overfitting.

Abstract

from arXiv · show

It is common practice in deep learning to use overparameterized networks and train for as long as possible; there are numerous studies that show, both theoretically and empirically, that such practices surprisingly do not unduly harm the generalization performance of the classifier. In this paper, we empirically study this phenomenon in the setting of adversarially trained deep networks, which are trained to minimize the loss under worst-case adversarial perturbations. We find that overfitting to the training set does in fact harm robust performance to a very large degree in adversarially robust training across multiple datasets (SVHN, CIFAR-10, CIFAR-100, and ImageNet) and perturbation models ($\ell_\infty$ and $\ell_2$). Based upon this observed effect, we show that the performance gains of virtually all recent algorithmic improvements upon adversarial training can be matched by simply using early stopping. We also show that effects such as the double descent curve do still occur in adversarially trained models, yet fail to explain the observed overfitting. Finally, we study several classical and modern deep learning remedies for overfitting, including regularization and data augmentation, and find that no approach in isolation improves significantly upon the gains achieved by early stopping. All code for reproducing the experiments as well as pretrained model weights and training logs can be found at https://github.com/locuslab/robust_overfitting.

1. Introduction

The paper finds that adversarially trained networks exhibit robust overfitting: continued training lowers robust training loss but can substantially worsen robust test performance. This phenomenon appears across datasets and motivates early stopping and further analysis of alternative remedies.

  • Core finding: Robust overfitting occurs when continued training decreases robust training loss while increasing robust test loss.The effect is shown after learning-rate decay and is observed on CIFAR-10, SVHN, CIFAR-100, and ImageNet.
  • Early stopping: 43.2% robust test error was achieved by early-stopped PGD training on CIFAR-10, matching the 43.4% reported for TRADES under the same adversary.The comparison uses an ℓ∞ PGD adversary with radius 8/255.
  • Early stopping: Early stopping can match the robust-performance gains of virtually all recent algorithmic improvements over vanilla PGD-based adversarial training.The paper describes this as a simpler approach based on selecting an earlier checkpoint.
  • Diagnosing overfitting: Double-descent curves also occur in robust training, but changing hypothesis-class size does not mitigate robust overfitting.The paper examines whether standard deep-learning explanations of generalization transfer to the robust setting.
  • Alternative remedies: Regularization and data augmentation mitigate robust overfitting to varying degrees, yet no isolated approach outperforms simple early stopping when trained to convergence.Semi-supervised data augmentation is an exception at selected epochs, where improved robust performance can occur.

2. Background and related work

The background surveys adversarial-training methods, defenses, robustness certification, and established approaches to overfitting and generalization. It situates the paper among PGD-based training, newer robust-learning improvements, and classical or modern regularization techniques.

  • Adversarial training: Adversarial training evolved from FGSM and iterative attacks to PGD training with random restarts, followed by methods such as momentum, logit pairing, feature denoising, and TRADES.TRADES is described as balancing standard and robust errors and achieving state-of-the-art performance on several benchmarks.
  • Efficiency and threat models: Research has also reduced adversarial-training cost by simplifying gradient calculations and decreasing the number of attack iterations.These efficiency efforts address the substantially higher computational cost of PGD training compared with standard training.
  • Efficiency and threat models: Adversarial-training algorithms have been extended to image transformations, alternative distance metrics, and multiple threat models.These extensions broaden the threat settings considered by robust-learning methods.
  • Other defenses: Other proposed defenses, including distillation and adversarial-example detection, were defeated by stronger attacks in cited studies.The passage presents these methods as unsuccessful defenses rather than as the paper’s central approach.
  • Certified robustness: Robustness-certification methods seek guarantees for norm-bounded perturbations but are not always scalable to large convolutional networks.Approaches include SMT solvers and mixed-integer linear programming.
  • Overfitting and generalization: Regularization and early stopping are established methods for reducing overfitting, while double descent reports improved test performance beyond interpolation as model complexity increases.The related work connects these ideas to classical statistics, neural-network training, and robust generalization theory.

3. Adversarial training and robust overfitting

Adversarial training minimizes loss under adversarial perturbations, but unlike standard deep learning, continued training can worsen robust test performance. This robust overfitting is widespread, makes checkpoint timing crucial, and is not explained away by model complexity or learning-rate smoothing.

  • Adversarial training: Adversarial training approximately minimizes loss over perturbation sets such as ℓ2 and ℓ∞ balls, typically using projected gradient descent.Robust error measures performance under PGD attack, whereas standard error measures performance on clean data.
  • Robust overfitting: Robust overfitting occurs when further training decreases robust training loss but increases robust test loss, so optimal performance is reached before convergence.Unlike standard training, the robust test error can dip after learning-rate decay and then rise during continued training.
  • Robust overfitting: 8.2% robust error increases on CIFAR-10 and 22.8% on ImageNet under an ℓ∞ adversary illustrate the gap between best and final performance.The phenomenon spans SVHN, CIFAR-10, CIFAR-100, and ImageNet, and affects ℓ∞ and ℓ2 threats, PGD, FGSM, and TRADES.
  • Scope: Catastrophic overfitting reported for FGSM adversarial training is separate from the robust overfitting studied here.The distinction is discussed further in the paper’s appendix.
  • Early stopping: 43.4% robust test error for TRADES can degrade to 50.6% after convergence, while early stopping vanilla PGD can match newer adversarial-training methods.The TRADES learning curve makes the timing dependence concrete; the paper also reports that publicly released ImageNet models degrade with continued training.
  • Learning-rate schedules: Smoother learning-rate schedules still exhibit robust overfitting and do not match the peak performance of the standard piecewise-decay schedule.The results indicate that the peak is associated with a single discrete learning-rate jump rather than simply smoother optimization.
  • Double descent: Increasing architecture size can preserve double descent, but training longer degrades robust test performance regardless of architecture size.Thus, model-size complexity and training-time complexity have distinct effects in adversarially robust training.

4. Alternative methods to prevent robust overfitting

The paper tests classical and modern overfitting remedies in adversarial training and finds that none generally matches early stopping in isolation. Combining early stopping with semi-supervised augmentation is the only tested approach that significantly improves upon early stopping alone.

  • Comparison with early stopping: No tested regularization or augmentation technique performs as well in isolation as early stopping on CIFAR-10.The study evaluates classical and modern methods across ablations.
  • Explicit regularization: 55.2% robust test error is achieved by the best explicit ℓ2 regularizer, but it remains less effective than early stopping.The corresponding hyperparameter is λ = 5 · 10−2.
  • Explicit regularization: Explicit ℓ1 and ℓ2 regularization cannot fully remove robust overfitting without drastically over-regularizing the model.The regularization sweep covers a range of penalty strengths.
  • Data augmentation: Cutout and mixup either regularize too weakly to prevent robust overfitting or too strongly, degrading the model.The paper reports significant robust overfitting for cutout and less overfitting for mixup.
  • Semi-supervised learning: 47.1% robust test error is the average final performance of semi-supervised augmentation, similar to early stopping because of high test-error variance.The variance persists after robust training has converged.
  • Semi-supervised learning: 40.2% robust test error is achieved only by combining early stopping with semi-supervised augmentation, the sole method significantly improving upon early stopping alone.The combination avoids the variance observed with semi-supervised augmentation alone.

5. Conclusion

The conclusion emphasizes that robust overfitting is widespread across datasets, threat models, and training settings, and that most remedies do not outperform early stopping. It recommends validation-based model selection and learning-curve analysis for robust training.

  • Conclusion: Larger architectures improve test-set generalization but do not reduce the effect of robust overfitting.The conclusion also reports that most implicit and explicit regularizers over-regularize or fail to prevent it.
  • Conclusion: All tested methods in isolation fail to improve upon early stopping.This conclusion follows the broader regularization and augmentation experiments.
  • Conclusion: The authors urge practitioners to use validation sets for model selection and inspect learning curves during adversarial training.The recommendation is motivated by the prevalence of robust overfitting.
  • Conclusion: Robust overfitting occurs across the experiments spanning datasets, adversarial training algorithms, and perturbation threat models.Table 3 defines best error as the lowest checkpoint error and final error as the average over the last five epochs.

A.1. SVHN experiments

SVHN exhibits robust overfitting under both ℓ∞ and ℓ2 perturbations, with the best checkpoint substantially outperforming the final trained model for ℓ∞ training. CIFAR-100 shows the same pattern, and both robust and standard test accuracy degrade there.

  • SVHN experiments: 39.0% robust error is achieved by the best SVHN ℓ∞ checkpoint, improving 6.6% over the 45.6% error at the end of training.Robust overfitting begins before the initial learning-rate decay on SVHN.
  • SVHN experiments: SVHN robust overfitting occurs before the initial learning-rate decay, indicating that the relevant learning-rate threshold has already been passed.The finding concerns PGD-based adversarial training for both ℓ∞ and ℓ2 perturbations.
  • CIFAR-100 experiments: CIFAR-100 reproduces the CIFAR-10 pattern, with robust overfitting occurring after the initial learning-rate decay.The effect is reported for both ℓ∞ and ℓ2 perturbation models.
  • CIFAR-100 experiments: 71.9% robust error is achieved by the best CIFAR-100 ℓ∞ checkpoint, improving 6.7% over the 78.6% error at the end of training.Both robust and standard test accuracy are degraded by robust overfitting in this setting.

A.3. ImageNet experiments

ImageNet continuation experiments show that further training can initially help or leave performance unchanged, depending on the pretrained model and learning-rate state. Eventually, both threat models suffer substantial robust overfitting amid noisy learning curves.

  • ImageNet experiments: The ImageNet ℓ∞ pretrained model gradually deteriorates when trained further without learning-rate decay.The pretrained model had not yet converged for its checkpointed learning rate.
  • ImageNet experiments: The ImageNet ℓ2 pretrained model shows no significant performance change until the learning rate is decayed to 0.001.It appeared to have already converged at the checkpointed learning rate.
  • ImageNet experiments: 22.8% average robust-error increase occurs for the ImageNet ℓ∞ model, while the ℓ2 model increases by 31.8%.The models eventually suffer greatly from robust overfitting despite occasional improvements at specific checkpoints.
  • ImageNet experiments: ImageNet learning curves are smoothed over consecutive 10-epoch windows because the raw curves are unusually noisy.This noise produces the greatest variation in final robust and standard error rates among the reported datasets.

A.4. CIFAR-10 experiments

CIFAR-10 experiments show robust overfitting across adversarial-training methods and perturbation norms, while training choices can alter its severity. FGSM and TRADES still exhibit robust overfitting, and carefully selected early stopping or schedules can avoid degradation.

  • FGSM adversarial training: α = 10/255 for ℓ∞ FGSM training causes catastrophic overfitting after convergence, whereas reducing the step size to 7/255 still leaves robust overfitting.The smaller step size avoids catastrophic overfitting but does not eliminate the generalization problem.
  • FGSM adversarial training: Cyclic FGSM training can sidestep robust overfitting and reach its best checkpoint at the end, but only with careful epoch selection.Too few epochs underperform, while too many produce robust overfitting.
  • Failure modes: Robust overfitting differs from catastrophic overfitting: it worsens test performance under the training adversary and cannot be detected from training-set evaluation.Successful FGSM training can robustly overfit without catastrophically overfitting.
  • TRADES: The TRADES reproduction’s checkpoint immediately after the initial learning-rate decay has the best test performance across 100 epochs.This supports the importance of checkpoint selection in the reported TRADES behavior.
  • TRADES: For TRADES, three of four learning-rate and architecture combinations show clear robust overfitting; the exception combines a shortened schedule with a smaller architecture.The authors attribute the exception to implicit early stopping and reduced representational power.

B.1. Different types of schedules

Robust overfitting is widespread across learning-rate schedules, with most schedules reaching their best checkpoint before convergence. Tuning the post-decay learning rate can slightly improve the best checkpoint, but does not remove overfitting.

  • Schedule types: Piecewise, multiple, linear, cyclic, and cosine schedules use distinct decay patterns spanning fixed, gradual, cyclic, and cosine learning-rate changes.The primary schedule is piecewise decay, starting at 0.1 and decaying by a factor of 10 at epochs 100 and 150.
  • Schedule comparison: Cyclic learning rates improve robust performance during both growth and shrinkage phases, but robust overfitting eventually prevents further improvement.Stretching the cyclic schedule to 300 epochs worsens both best-checkpoint and final robust test error.
  • Schedule tuning: Adjusting the post-decay learning rate improves the best robust checkpoint by 0.5%, while changing the starting rate or decay epoch yields similar or worse performance.Robust overfitting remains present in the tuned schedules.
  • Schedule tuning: Figures 18–20 show learning curves for schedules tuned through the starting rate, ending rate, and decay epoch.These figures correspond to the tuning experiments summarized in Table 4.

C. Double descent: exploring architecture sizes

Increasing architecture size improves both final and best-checkpoint robust performance, yet robust overfitting persists at every tested width. Double descent occurs before robust interpolation and does not account for the overfitting pattern.

  • Architecture size: Both final models and best checkpoints benefit from increasing Wide ResNet width, while robust overfitting remains present.The architecture experiments vary width using depth-28 Wide ResNets.
  • Architecture size: At every tested architecture size, the best robust checkpoint has higher training robust error than the converged final model.Thus, lower robust test error at the best checkpoint is not accompanied by lower training robust error.
  • Double descent: Double descent occurs before robust interpolation at width factor 5, after which final-model robust test performance continues improving with larger architectures.The reported trend differs from the standard setting described by the authors.
  • Double descent: 48.8% robust test error is achieved by the width-20 final model, compared with 41.8% at its best checkpoint.Width factor 20 was the largest architecture tested on the available hardware.
  • Evaluation protocol: The reported final and best robust errors average final converged epochs or select the lowest robust test-error checkpoint, respectively.Validation-based early stopping instead uses the validation-selected checkpoint and has no standard deviation.

D.4. Data augmentation

Cutout and mixup can mitigate robust overfitting, but neither improves substantially on vanilla early stopping. Robust test performance generally still degrades during training, with the effect reduced under stronger augmentation.

  • Cutout: Cutout with patch length 14 performs slightly worse than vanilla early stopping alone and only matches it when combined with early stopping.Increasing patch length reduces the effect of robust overfitting, but does not produce a better outcome than early stopping.
  • Cutout: Robust overfitting steadily degrades robust test performance throughout cutout training, with a smaller effect as cutout patches become longer.The augmentation mitigates the degradation without surpassing vanilla early stopping.
  • Mixup: Mixup performs best at α = 1.4, but combined with early stopping only matches vanilla early stopping and otherwise converges to a worse model.Its learning curves are noisier, although robust test error steadily decreases during training.
  • Mixup: Mixup stops robust overfitting to some degree, but does not obtain significantly better performance than vanilla early stopping.This conclusion is based on the observed decrease in robust test error during mixup training and its checkpoint comparisons.

E. Semi-supervised approaches

The semi-supervised setup combines labeled CIFAR-10 data with pseudo-labeled TinyImages data, reducing severe robust overfitting but leaving robust test performance highly variable. The section also examines regularization and data augmentation through additional learning-curve and performance analyses.

  • Training setup: Semi-supervised training uses equal parts labeled CIFAR-10 and pseudo-labeled TinyImages data in batches of 128.Each epoch requires computation equivalent to two epochs of standard adversarial training.
  • Training setup: The reproduced semi-supervised model achieves a best-checkpoint error of 40.2%, about 2% higher than the 38.5% reported by Carmon et al. (2019).The difference is attributed to the smaller pre-activation ResNet18 architecture used in the reproduction.
  • Robust overfitting: Semi-supervised training does not show severe robust overfitting because smoothed learning curves remain relatively flat without significant increases in robust test error.This improvement is assessed relative to the characteristic robust-overfitting behavior in the base setting.
  • Robust overfitting: Robust test performance still spans almost 10% robust error despite relatively flat, converged training error, making selection of the best checkpoint critical.The passage describes this variability relative to training on only the original dataset.
  • Overfitting remedies: Additional analyses vary ℓ1 and ℓ2 regularization strengths, cutout patch lengths, and mixup hyperparameter α using performance and learning-curve plots.The figures separately report standard and robust train/test performance and learning curves for these interventions.
Loading 2002.11569v2…