PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Training Can Hurt Generalization

Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John C. Duchi, Percy Liang

arXiv:1906.06032v2cs.LGstat.ML

TL;DR

The paper studies why adversarial training can reduce standard accuracy even when a predictor can be both standard-accurate and robust. Using a convex construction, it shows that finite-sample statistics—not optimization failure—can create this tradeoff, and that robust self-training with unlabeled data mostly mitigates it.

  • Problem

    Existing explanations focused on objective incompatibility or limited expressivity, leaving unclear whether a tradeoff persists when one predictor is optimal for both objectives.

  • Method

    The paper constructs a convex learning problem with invariant perturbation sets and compares standard training, adversarial training, and robust self-training.

  • Results

    Adversarial training hurts standard generalization at small sample sizes despite compatible optimal objectives, while robust self-training mostly eliminates the tradeoff using unlabeled data.

  • Takeaways & Limitations

    The observed robustness–accuracy tradeoff can arise from insufficient labeled samples, and additional unlabeled data can mitigate it.

  • Takeaways & Limitations

    The construction assumes perturbations that preserve the Bayes-optimal output and centers on a specific convex learning problem.

Abstract

from arXiv · show

While adversarial training can improve robust accuracy (against an adversary), it sometimes hurts standard accuracy (when there is no adversary). Previous work has studied this tradeoff between standard and robust accuracy, but only in the setting where no predictor performs well on both objectives in the infinite data limit. In this paper, we show that even when the optimal predictor with infinite data performs well on both objectives, a tradeoff can still manifest itself with finite data. Furthermore, since our construction is based on a convex learning problem, we rule out optimization concerns, thus laying bare a fundamental tension between robustness and generalization. Finally, we show that robust self-training mostly eliminates this tradeoff by leveraging unlabeled data.

1. Introduction

The paper argues that finite-sample generalization can create a robustness–standard-accuracy tradeoff even when objectives are compatible and the hypothesis class is expressive. A convex construction attributes the tradeoff to statistics rather than optimization, while unlabeled data can mitigate it.

  • Motivation: 95.2% to 87.3%: adversarial training improves CIFAR-10 robust accuracy from 3.5% to 45.8% but lowers standard accuracy.Both training methods achieve 100% training accuracy, indicating the test discrepancy is a generalization issue.
  • Motivation: Prior constructions made standard and robust objectives incompatible even with infinite data because perturbations changed the Bayes estimator’s output.Here, the considered perturbations do not change that output, so an optimal predictor can satisfy both objectives.
  • Research question: The paper rules out limited expressivity as the explanation because adversarial training reaches 100% standard and robust training accuracy.The remaining question is whether robust-objective statistics or neural-network optimization dynamics drive worse standard generalization.
  • Contribution: A convex learning problem shows adversarial training can hurt generalization despite an optimal predictor that is both standard-accurate and robust.Convexity removes optimization concerns and supports a statistical explanation involving greater sample requirements for standard accuracy.
  • Empirical connection: On subsampled CIFAR-10, the standard-versus-adversarial accuracy gap decreases with larger training sets, while robust self-training helps mitigate the tradeoff.These trends mirror the constructed problem and extend its finite-sample interpretation to an empirical setting.

2. Convex learning problem: the staircase

The convex construction makes robust fitting statistically harder at small sample sizes because low-probability perturbations require complex staircase predictors. As samples increase, robust training eventually generalizes better, and robust self-training uses unlabeled data to mostly remove the tradeoff.

  • 2. Convex learning problem: the staircase: The construction concentrates most probability on anchor points that simple predictors fit well, while fitting their low-probability perturbations requires complex predictors.This separates easy standard fitting from difficult robust fitting.
  • 2.1. General setup: Robust training minimizes worst-case squared loss over each invariance set B(x_i), whereas standard training minimizes loss only on observed training points.Both estimators use regularized estimation in a hypothesis class containing the optimal predictor.
  • 2.1. General setup: For small n, the robust estimator has larger error than the standard estimator, although both estimators converge to the same optimal predictor as n grows.The construction therefore isolates a finite-sample gap rather than an infinite-data objective conflict.
  • 2.2. Construction: When m = 1, the robust optimal predictor resembles a staircase while remaining invariant and accurate; when m = 0, it is linear and no tradeoff occurs.The invariance sets are disjoint, permitting a nontrivial robust and accurate estimator.
  • 2.3. Simulations: With few samples, standard training learns a simple linear predictor from anchor points, whereas robust training fits unsampled low-probability perturbations and generalizes poorly.The robust fit becomes a staircase because it must enforce invariance around observed anchors.
  • 2.3. Simulations: As sample size increases, the standard–robust test-error difference shrinks, and robust training eventually achieves lower test error by fitting low-probability regions.Data augmentation shows a similar small-sample tradeoff because the augmented estimator becomes more complex.
  • 2.4. Robust self-training mostly eliminates the tradeoff: Robust self-training uses a standard estimator to pseudo-label additional unlabeled points before applying robust training.For the staircase problem, it mostly eliminates the tradeoff and achieves test error similar to standard training while remaining robust.

3. Experiments on CIFAR-10

Subsampled CIFAR-10 shows that adversarial training’s standard-accuracy gap decreases as labeled sample size increases, mirroring the staircase construction; robust self-training mitigates the tradeoff with unlabeled data.

  • Interpretation: The staircase analysis links robust predictors’ poorer standard-objective performance to their greater complexity and consequently poorer generalization.This motivates testing whether the CIFAR-10 accuracy drop has the same explanation.
  • Figure interpretation: Figure 2 plots robust-minus-standard test error after separately selecting each method’s best regularization parameter; positive values favor standard training.The shaded regions represent one standard deviation.
  • Sample-size effects: The staircase problem and subsampled CIFAR-10 both show a decreasing gap between robust and standard test errors as labeled sample size increases.The staircase comparison motivates the same finite-sample pattern observed on CIFAR-10.
  • Experimental setup: The experiment compares standard and adversarially trained CIFAR-10 models across varying numbers of labeled samples using ℓ∞ perturbations.Robust models are trained with the adversarial training procedure from [6].
  • Unlabeled-data mitigation: Robust self-training with additional unlabeled data helps mitigate the standard-accuracy tradeoff on CIFAR-10.The cited experiment reports this result as analogous to the staircase example.

4. Adversarial training can also help

Adversarial training can help when robust predictors are simple: it acts as a regularizer, lowering standard error on the m=0 staircase problem and on MNIST, with convergence as samples increase.

  • Figure interpretation: Figure 3 reports robust-minus-standard test error, so negative values indicate that robust training performs better.The plot uses the number of training samples n on the horizontal axis and shaded one-standard-deviation regions.
  • Staircase problem: When m = 0, adversarial training lowers standard error by enforcing invariance and reducing sensitivity to target noise.The optimal robust predictor is linear in this construction.
  • MNIST: On MNIST, adversarially trained models have lower test error than standard-trained models.The difference is largest at small sample sizes and closes as sample size increases.
  • MNIST: As sample size increases on MNIST, standard and adversarially trained models converge to the same small test error.The paper attributes a differing prior observation to initialization and optimization.

5. Conclusion

The paper argues that enforcing optimal-function invariance can hurt finite-sample performance when invariant predictors are complex, while additional unlabeled data can mitigate the resulting robustness–accuracy tradeoff.

  • Conclusion: Enforcing invariance respected by the optimal function can degrade performance because invariant predictors may be complex and require more samples to generalize well.The conclusion identifies insufficient samples as the source of the observed tradeoff.
  • Conclusion: The experiments support that additional unlabeled data is sufficient to mitigate the robustness–accuracy tradeoff.This conclusion is stated within the paper’s experimental scope.

A. Consistency

Under target-distribution invariance within perturbation sets, the Bayes-optimal predictor is also optimal for the robust population objective, yielding consistency of standard and robust estimators.

  • Consistency: Under the invariance condition, the minimizers of the standard and robust objectives coincide in the infinite-data limit.This is stated for the population objectives under P.
  • Consistency: When the Bayes predictor belongs to the hypothesis class, the standard and robust estimators are consistent and converge to it with infinite data.The regression result is stated for f* in F, with the classification case following analogously.
  • Proof intuition: The robust objective’s maximum is attained at the unperturbed input for an invariant Bayes predictor, making its robust and standard losses equal.For any other predictor, robust loss is at least standard loss, which is at least the Bayes predictor’s standard loss.
  • Regression: For regression, the Bayes estimator f* = argmin_f E[ℓ(f(x),y)] is also optimal for robust population squared loss when condition (7) holds.The robust objective upper-bounds the standard loss, while the Bayes estimator minimizes standard loss.
  • Classification: For classification, label invariance ensures that the Bayes-optimal classifier is also optimal for the robust population 0-1 loss.The condition prevents perturbations from changing the label that achieves the maximum while allowing distributional perturbation.
  • Invariance condition: Target-distribution invariance means points within an invariance set B(x) have the same target distribution.This condition implies consistency in both regression and classification.

B.1. Data distribution

The construction uses a one-dimensional staircase regression distribution with perturbation points, Gaussian target noise, and cubic B-splines as an expressive hypothesis class. Its behavior depends on slope, perturbation probability, and sample size: robustness can help or hurt generalization.

  • Data distribution: The data distribution samples staircase points according to w and perturbs them with probability δ to nearby points.The unperturbed point has probability 1−δ, while each neighboring perturbation has probability δ/2.
  • Data distribution: Targets are Gaussian around the rounded staircase value, and each invariance set assigns the same conditional target mean to its three points.The perturbation size is bounded so rounding remains unchanged across each invariance set.
  • Model: The hypothesis class is cubic B-splines with knots at every support point, ensuring it contains the optimal invariant predictor.The spline representation uses a kernel feature map, while regularization penalizes nonsmoothness through the second derivative.
  • Estimators: The robust estimator minimizes worst-case squared loss over invariance sets using a convex objective, whereas standard training minimizes empirical loss on observed points.The robust objective is solved directly as a pointwise maximum of squared losses.
  • Regimes: A large slope m and small perturbation probability δ create the regime where robustness requires fitting rare perturbations and can hurt finite-sample generalization.The construction can exhibit this effect even without label noise; when m≈0, robustness instead helps by cancelling noise.

C. Robust self-training algorithm

Robust self-training augments labeled data with unlabeled examples pseudo-labeled by a standard estimator, then applies robust training to the combined dataset. This procedure targets both standard and robust test error.

  • Procedure: Robust self-training performs robust training on a dataset augmented with unlabeled data.The augmented objective is intended to improve both standard and robust test error over robust training on labeled data alone.
  • Procedure: The algorithm first computes a standard estimator on the labeled data using a chosen regularization parameter λ.This estimator supplies the pseudo-labeling function for the unlabeled examples.
  • Procedure: It generates pseudo-targets for unlabeled inputs by evaluating the standard estimator on those inputs.The pseudo-targets are yu = f^std_n(Xu).
  • Evaluation: Figure 5 evaluates RST against standard and robust training using test robust MSE, selecting λ by grid search for each method and sample size.The two panels compare RST with robust training and with standard training.
  • Procedure: It returns a robust estimator trained on the labeled and pseudo-labeled datasets together.The final estimator uses the augmented dataset as robust-training data.

C.1. Results on CIFAR-10

Prior CIFAR-10 work applied robust self-training with unlabeled data using TRADES as the robust estimator. The procedure matches the algorithm described in this paper.

  • CIFAR-10 results: The cited CIFAR-10 robust self-training procedure uses unlabeled data together with the CIFAR-10 training set.Its robust estimator is a modified adversarial-training method, TRADES.

C.2. Robust self-training doesn’t sacrifice robustness

Robust self-training mitigates the standard-error tradeoff without sacrificing robustness in the convex construction and in the CIFAR-10 comparison. It improves both standard and robust performance over traditional adversarial training.

  • Convex construction: RST improves over robust training in standard test error and robust test error when robustness hurts generalization in the m=1 construction.Thus unlabeled data recovers standard-training generalization while simultaneously improving robustness.
  • CIFAR-10: RST increases both standard and robust accuracies over traditional adversarial training in the CIFAR-10 comparison.The comparison uses additional unlabeled data alongside the CIFAR-10 training set.
  • Experimental setting: The CIFAR-10 study varies labeled sample size and adversarial perturbation magnitude while averaging results across multiple training trials.Models are Wide ResNet 40-2 networks trained under an ℓ∞ attack model.

D.2. MNIST

On MNIST, the experiments use subsampled labeled data and compare standard and adversarial training. Initialization strongly affects adversarial-training convergence, while the figures show opposing standard- versus robust-MSE patterns across sample sizes.

  • Experimental setup: The MNIST experiments subsample 60,000 labeled examples across eleven factors and average results for a small 3-layer CNN over two trials.All models train for 200 epochs; standard training reaches 100% standard training accuracy, while adversarial models exceed 99% adversarial training accuracy.
  • Initialization: Xavier initialization speeds adversarial-training convergence and removes the clean-accuracy drop at the same robust accuracy level.Standard training is unaffected by initialization, whereas adversarial training is dramatically affected.
  • Robustness hurts: When robustness hurts, standard training has lower test MSE, but the gap shrinks as sample size increases.The standard estimator’s norm increases with sample size as it approaches the complex true function.
  • Robustness helps: When robustness helps, robust training has lower test MSE, with the gap again shrinking as sample size increases.The robust estimator maintains a consistent norm throughout training due to noise-cancelling behavior.
Loading 1906.06032v2…