Source-linked AI summary
Towards Deep Learning Models Resistant to Adversarial Attacks
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu
TL;DR
Deep neural networks can be fooled by imperceptible input changes, so this paper studies robustness through robust optimization. It develops PGD-based adversarial training and reports improved resistance across MNIST and CIFAR10, including more than 89% and 46% accuracy against strong adversaries.
Problem
Deep neural networks can misclassify inputs after imperceptible adversarial manipulation, raising the question of how to make them resistant to such attacks.
Method
The paper formulates robustness as a saddle-point robust-optimization problem, uses PGD as a first-order adversary, and trains networks against resulting perturbations.
Results
The trained models resisted a wide range of attacks, achieving more than 89% accuracy on MNIST and 46% on CIFAR10 against the strongest test-suite adversaries.
Takeaways & Limitations
The findings provide evidence that reliable adversarial training can make deep neural networks resistant to adversarial attacks.
Takeaways & Limitations
Robustness remains far from satisfactory, and the authors suggest larger-capacity networks as a direction for improvement.
Abstract
from arXiv · showhide
Recent work has demonstrated that deep neural networks are vulnerable to adversarial examples---inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. In fact, some of the latest findings suggest that the existence of adversarial attacks may be an inherent weakness of deep learning models. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much of the prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against any adversary. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest the notion of security against a first-order adversary as a natural and broad security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models. Code and pre-trained models are available at https://github.com/MadryLab/mnist_challenge and https://github.com/MadryLab/cifar10_challenge.
1 Introduction
Adversarial examples expose a security-critical weakness in deep neural networks, motivating robust optimization as a principled way to train models against broad classes of attacks. The paper shows that first-order methods and sufficient model capacity can yield substantially improved robustness on MNIST and CIFAR10.
- Motivation: Imperceptible input changes can fool correctly classified state-of-the-art neural networks with high confidence, raising fundamental concerns about robust concept learning.The issue is especially striking in computer vision and has important security implications.
- Motivation: Existing attack and defense methods provide limited understanding of their security guarantees because they target specific mechanisms rather than clearly defined broad adversary classes.The paper frames this limitation as uncertainty about whether an attack finds the “most adversarial” example.
- Approach: The paper formulates adversarial robustness as a natural saddle-point problem through robust optimization, specifying the broad class of attacks against which security should be achieved.This min-max formulation is intended to provide a principled alternative to defending only against known attacks.
- Contributions: First-order methods can reliably solve the non-convex, non-concave optimization problem, motivating projected gradient descent as a universal first-order adversary.The authors support this conclusion with experiments on the optimization landscape and ideas from real analysis.
- Contributions: Model capacity plays an important role in robustness: networks need larger capacity to withstand strong attacks than to classify benign examples correctly.The resulting robust decision boundary can be substantially more complicated than one that merely separates benign data points.
- Results: More than 89% MNIST accuracy and 46% CIFAR10 accuracy are achieved against the strongest test-suite adversaries, while weaker black-box/transfer attacks yield more than 95% and 64%, respectively.The models are trained by optimizing the saddle-point formulation with PGD as a first-order adversary.
2 An Optimization View on Adversarial Robustness
The paper frames adversarial robustness as a robust-optimization problem: an adversary perturbs inputs within a specified set to maximize loss, while training minimizes the resulting worst-case risk. This formulation unifies attacks and defenses and provides a quantitative robustness goal for neural networks.
- 2 An Optimization View on Adversarial Robustness: ERM can produce classifiers vulnerable to efficient adversaries that make imperceptible perturbations causing incorrect classification.An adversary transforms an example from class c1 into a nearby xadv classified as c2 ≠ c1.
- 2 An Optimization View on Adversarial Robustness: The robust-optimization approach specifies an allowed perturbation set S for each input, focusing here on ℓ∞-bounded attacks.S formalizes the adversary’s manipulative power and is chosen to capture perceptual similarity between images.
- 2 An Optimization View on Adversarial Robustness: The central saddle-point formulation replaces ordinary population risk with loss evaluated after an adversary first perturbs each sampled input.Training minimizes the adversarially modified risk, while the adversary selects perturbations that increase loss.
- 2 An Optimization View on Adversarial Robustness: A nearly vanishing robust risk gives a clear quantitative target: the corresponding model is perfectly robust to attacks specified by the attack model.The formulation therefore provides both an ideal robustness goal and a measure of robustness.
- 2 An Optimization View on Adversarial Robustness: The inner maximization interprets attacks as loss-maximization methods, with multi-step attacks corresponding essentially to projected gradient descent on the negative loss.FGSM is a one-step approximation, while adversarial training can be viewed as approximating the same inner maximization.
3 Towards Universally Robust Networks
The section argues that the robust-optimization saddle-point problem is practically solvable: PGD finds adversarial maxima with rapidly plateauing and concentrated losses. This supports treating PGD robustness as a broad first-order security guarantee against current attacks.
- Universal Robustness: Small adversarial loss guarantees that no attack allowed by the attack model can fool the network.This follows because the loss remains small for every allowed perturbation.
- Universal Robustness: PGD experiments show that the non-concave inner maximization is tractable from the perspective of first-order methods.The study explores ℓ∞-balls using many randomly restarted PGD runs on MNIST and CIFAR10 evaluation examples.
- Universal Robustness: PGD loss rises consistently, plateaus rapidly, and produces well-concentrated final losses without extreme outliers across 105 random restarts.This behavior is observed for normally trained and adversarially trained networks.
- Universal Robustness: Local maxima are widely separated yet have similar losses, with pairwise angles near 90° and distances close to those of random points in the ℓ∞ ball.Loss along segments between maxima is highest at endpoints, decreases by a constant factor in the middle, and remains considerably higher than typical natural inputs.
- Universal Robustness: PGD appears to be a universal first-order adversary because robustness against it is conjectured to imply robustness against attacks using only input gradients.Random restarts did not reveal significantly higher-loss maxima, though isolated better maxima are not ruled out.
- Universal Robustness: Training against PGD is expected to provide robustness against a wide range of attacks encompassing current approaches.Stronger adversarial training and greater network capacity also improve resistance to transfer attacks.
4 Network Capacity and Adversarial Robustness
The section argues that adversarial robustness depends critically on network capacity: robust classification requires more complex decision boundaries, while insufficient capacity can prevent meaningful learning. Experiments show that capacity improves robustness and adversarial fitting, whereas restricted adversaries can induce overfitting and stronger models reduce transferability.
- A small final loss in the robust optimization problem is necessary for accurate classification and corresponds to robustness against adversarial inputs.Solving the optimization problem alone does not guarantee robust, accurate classification.
- For a fixed perturbation set, classifier architecture determines the optimization value, and robust classification requires greater capacity for more complex decision boundaries.Adversarial perturbations change the decision boundary from a simpler separation to one that must separate perturbation regions.
- Increasing capacity improves robustness to one-step perturbations even when training uses only natural examples, with larger gains for smaller ε.Capacity also increases accuracy on natural examples.
- FGSM training overfits its restricted adversarial examples, producing poor natural-example performance and no robustness against PGD adversaries for large ε.This behavior is identified as label leaking.
- Strong-adversary training can make small-capacity networks collapse to predicting one fixed class instead of learning a meaningful classifier.Such networks sacrifice natural-example performance to obtain robustness against the strong adversary.
- Increasing capacity lowers the saddle-point value and reduces transferability, while stronger inner optimization similarly weakens transferred adversarial inputs.The transferability reduction is accompanied by less significant gradient correlation between source and transfer networks.
5 Experiments: Adversarially Robust Deep Learning Models
The experiments train adversarially robust MNIST and CIFAR10 classifiers using high-capacity networks and strong PGD adversaries, then evaluate them against diverse white-box and black-box attacks. The models show substantial robustness, but performance degrades beyond the training perturbation scale and ℓ2 robustness on MNIST is misleading.
- Training methodology: Robust training relies on sufficiently high-capacity networks and strong PGD adversaries initialized from random perturbations.The PGD procedure is described as a complete first-order adversary that efficiently maximizes loss using first-order information.
- Training methodology: Adversarial-example training steadily decreases training loss, indicating that the inner optimization problem is being solved during training.This behavior is reported for both MNIST and CIFAR10 networks trained against PGD.
- Evaluation: The trained models are evaluated against white-box PGD and CW attacks, plus black-box attacks from independently trained, naturally trained, and alternative-architecture networks.The attack families are denoted A, CW/CW+, A’, Anat, and B.
- Results: The adversarially trained networks achieve significant resistance to iterative adversaries, but their robustness remains far from satisfactory.The paper identifies larger-capacity networks and further progress along these directions as potential improvements.
- Robustness across perturbation strengths: For smaller ε than used in training, accuracy is equal or higher; MNIST robustness drops sharply for slightly larger ε, whereas CIFAR10 degrades more smoothly.The MNIST drop may result from threshold operators tuned to the training value of ε.
- Robustness across perturbation norms: On MNIST, ℓ2-bounded PGD finds no adversarial examples even at ε = 4.5, but these perturbations can change ground-truth labels, so the apparent robustness is unreliable.The paper further reports that PGD performs poorly on ℓ2-trained MNIST networks because threshold filters may mask loss gradients, while decision-based attacks do not use gradients.
6 Related Work
This section situates the paper within earlier robust optimization and adversarial machine-learning research. It contrasts the paper’s multi-step adversarial training and optimization analysis with limitations identified in prior work.
- Robust optimization predates deep learning by decades, while adversarial machine learning predates widespread deep neural-network use.
- Earlier adversarial training used a weak linearized-loss adversary, leaving models vulnerable to slightly more sophisticated iterative attacks.
- Although ImageNet work found model capacity important, this paper finds that multi-step PGD training produces resistance to such adversaries.
- Prior min-max training work claimed inner maximization could be difficult, whereas this paper finds randomly restarted PGD often reaches comparably good solutions.
- Prior transferability analysis focused near natural examples where loss is nearly linear, but large perturbations reveal a more complex adversarial landscape.
7 Conclusion
The findings provide evidence that deep neural networks can be made resistant to adversarial attacks through reliable adversarial training. MNIST networks are highly robust, while CIFAR10 results show significant improvement but remain below the same performance level.
- The findings provide evidence that deep neural networks can be made resistant to adversarial attacks using reliable adversarial training methods.This conclusion is supported by both the theoretical analysis and experiments.
- The underlying optimization task has highly concentrated local-maxima values despite maximizing a highly non-concave function with many distinct local maxima.This unexpectedly regular structure is identified as a key insight behind reliable adversarial training.
- MNIST networks achieve high accuracy against a wide range of powerful ℓ∞-bound adversaries and large perturbations, whereas CIFAR10 performance has not reached the same level.For CIFAR10, the techniques nevertheless produce a significant increase in network robustness.
A Statement and Application of Danskin’s Theorem
Danskin’s theorem establishes conditions under which the max-function is directionally differentiable and gradients at inner maximizers provide descent directions. Applied to adversarial loss maximization, it supports gradient-based optimization while acknowledging nonsmooth losses and locally optimal inner solutions.
- Theorem A.1: Under compactness and continuity assumptions, Danskin’s theorem makes the max-function locally Lipschitz continuous and directionally differentiable.The theorem assumes a nonempty compact set S, differentiability of g(·, δ), and continuity of ∇θg(θ, δ).
- Theorem A.1: When the maximizing set is a singleton, the max-function is differentiable at that parameter value.This is the theorem’s stated special case for a unique maximizer.
- Corollary A.2: For adversarial loss maximization, any nonzero negative parameter gradient evaluated at an inner maximizer is a descent direction for the outer objective.The corollary applies Danskin’s theorem to g(θ, δ) = L(θ, x + δ, y) over an ε-norm ball.
- Technical issues: ReLU and max-pooling make the loss not continuously differentiable, but the problematic discontinuities have measure zero and are assumed irrelevant in practice.The argument treats the nonsmooth points as unlikely to be encountered.
- Technical issues: Because the inner problem is nonconcave, PGD may find only local maxima; restricting S to a region where that maximum is global preserves the descent-direction conclusion there.The resulting guarantee applies when the adversary is constrained to S′, and the local maximum is a true adversarial example.
B Transferability
Adversarial examples transfer between independently trained networks, creating black-box vulnerabilities, but stronger adversarial training and greater network capacity reduce transferability. Experiments further show that transferred attacks generally weaken loss increases, although transferred PGD can outperform white-box FGSM.
- Transferability: Adversarial examples transfer between independently trained networks, even without direct access to the target network.The experiments consider same-architecture networks trained identically but with different random initializations, likely representing worst-case transferability.
- Transferability: Increasing network capacity or adversary strength during training decreases adversarial-example transferability.The paper identifies high-capacity networks combined with strong inner-optimization oracles as a way to alleviate transferability.
- MNIST: FGSM-trained MNIST networks have less correlated gradients and significantly higher transferred-example accuracy than standard networks.A very large standard network shows only a mild increase in transferred-example accuracy, whereas FGSM training produces a significant increase.
- CIFAR10: On CIFAR10, PGD training and wide architectures generally make transfer attacks less successful across FGSM and PGD configurations.The exceptions mainly involve transfers from PGD-trained source models and a few close architecture-change cases.
- CIFAR10: Transferred attacks increase loss less than white-box attacks, while transferred PGD can outperform white-box FGSM near the ε-boundary.PGD always achieves higher loss than FGSM near the boundary of the ℓ∞ box of radius ε, and transferred PGD dominates white-box FGSM for the standard network and sometimes the PGD-trained network.
C MNIST Inspection
Inspection of robust MNIST models reveals sparse, thresholding-based early layers, selectively connected later filters, and strongly nonuniform class biases. These structures were learned through adversarial training rather than manually imposed, while manual imitation failed against PGD.
- C MNIST Inspection: The inspection compares a standard model with public and secret adversarially trained models that differ only in random weight initialization.The robust MNIST model is small enough for visual inspection of most parameters.
- C MNIST Inspection: Robust models use only 3 of 32 first-layer filters, each with one nonzero weight, reducing convolution to thresholding filters.Together with bias and ReLU, each filter implements ReLU(αx −β).
- C MNIST Inspection: The first layer maps the image to three thresholded channels, and the second layer uses sparse filters with a wider value range than the standard model.Only filters interacting with those three channels are relevant.
- C MNIST Inspection: Adversarially trained networks rely on strongly nonuniform class biases, despite having output weights roughly similar to the standard network.The similar biases may reflect greater conservatism for classes vulnerable to adversarial perturbations.
- C MNIST Inspection: These architectural characteristics were learned solely through adversarial training rather than hard-coded modifications.The authors’ attempts to introduce them manually did not achieve adversarial robustness.
D Supplementary Figures
The supplementary figures examine convolutional filters, softmax parameters, PGD loss trajectories, and human-perceptible adversarial examples. They compare standard and adversarially trained networks and visualize perturbations bounded in ℓ2 norm by 4.
- Convolutional filters: Robust networks’ first-layer filters include 3 non-zero filters, while their second-layer filters use the 3 non-zero channels.For standard models, random filters are visualized because no observable difference exists among them.
- Softmax layer: Softmax weights are similar across networks, with the standard model’s weights slightly more concentrated, whereas biases are far from uniform.The two adversarially trained networks exhibit a similar bias pattern.
- PGD trajectories: Loss values over PGD iterations are visualized for 20 random restarts on random examples across standard and adversarially trained networks.The 1st and 3rd rows correspond to standard networks, while the 2nd and 4th correspond to adversarially trained ones.
- Adversarial examples: Adversarial examples with ℓ2 norm bounded by 4 have perturbations significant enough to cause misclassification by humans.The figure provides sample adversarial examples under the stated norm bound.