Source-linked AI summary
MagNet and "Efficient Defenses Against Adversarial Attacks" are Not Robust to Adversarial Examples
Nicholas Carlini, David Wagner
TL;DR
The paper asks whether proposed defenses reliably protect neural networks from adversarial examples. It attacks MagNet using transferable examples from local defense copies and attacks Efficient Defenses and APE-GAN directly. The defenses are bypassed with high success and only modest distortion increases, motivating adaptive evaluation before robustness claims.
Problem
It remains an open question how to train neural networks to be robust to adversarial examples, including under defenses such as MagNet, Efficient Defenses, and APE-GAN.
Method
The paper uses Carlini–Wagner L2 attacks, transfer attacks from local MagNet copies, and direct attacks against Efficient Defenses and APE-GAN.
Results
The defenses are bypassed with greater than 99% success for MagNet and 100% for Efficient Defenses and APE-GAN, while the full Efficient Defenses increases mean distance by 30% on MNIST and 20% on CIFAR.
Takeaways & Limitations
Robustness claims should be tested with adaptive white-box attacks or attacks targeting the specific defense through a local copy.
Abstract
from arXiv · showhide
MagNet and "Efficient Defenses..." were recently proposed as a defense to adversarial examples. We find that we can construct adversarial examples that defeat these defenses with only a slight increase in distortion.
1 Introduction
The paper challenges three proposed defenses on MNIST and CIFAR-10, showing that adversarial examples can bypass them with only a slight distortion increase. MagNet is defeated through transferability, while existing attacks defeat Efficient Defenses and APE-GAN without modification.
- 1 Introduction: The paper evaluates MagNet, Efficient Defenses, and APE-GAN as proposed defenses against adversarial examples.MagNet reforms near-manifold examples and detects farther ones; Efficient Defenses uses Gaussian augmentation and BReLU; APE-GAN projects examples with a GAN.
- 1 Introduction: The three defenses are ineffective on MNIST and CIFAR-10 under the paper’s attacks.The authors report bypassing MagNet with greater than 99% success and the latter two defenses with 100% success.
- 1 Introduction: MagNet is defeated by training a local copy of the defense, constructing adversarial examples on it, and transferring them to the defender’s model.The attack exploits adversarial-example transferability when the defender’s defense parameters are unknown.
- 1 Introduction: Existing attacks defeat Efficient Defenses and APE-GAN with 100% success without modification.The resulting adversarial examples are reported as no more visually detectable than those against an undefended network.
2 Background
The paper reviews adversarial examples, transferability, and the operating principles of MagNet, Efficient Defenses, and APE-GAN. These defenses respectively detect or reform inputs, alter activations and training data, or use a GAN-based projection.
- 2 Background: Adversarial examples remain close to normal inputs under L2 distance while forcing classification to an adversary-chosen target.The classifier is represented as C(x), derived from the network’s output probabilities.
- 2 Background: Transferability means adversarial examples built for one model often fool another model performing the same task.The effect can be amplified by constructing examples against multiple local models.
- 2 Background: MagNet detects adversarial inputs and reforms detector-approved inputs, using multiple randomly selected reformers to obscure their parameters.Its detector rejects flagged inputs, while the reformer maps accepted inputs toward the data manifold.
- 2 Background: MagNet’s auto-encoder detectors compare reconstruction error, and on CIFAR may also compare Jensen-Shannon divergence between classifier outputs.The detector threshold is selected to minimize false positives; reforming applies the auto-encoder to the input.
- 2 Background: Efficient Defenses replaces standard ReLU with bounded BReLU and trains on Gaussian-noise-augmented examples.The noise is freshly sampled for each training instance, with σ = 0.3 on MNIST and σ = 0.05 on CIFAR.
- 2 Background: APE-GAN trains a generator to make adversarial examples appear nonadversarial and project normal and adversarial inputs toward the data manifold.Unlike a standard GAN, its generator receives an adversarial example rather than a noise vector.
- 2 Background: The experiments use author code for MagNet and APE-GAN, reimplement Efficient Defenses, and apply the Carlini–Wagner L2 attack.The authors state that they release their code.
3 Breaking MagNet
To break MagNet, the authors construct transferable targeted adversarial examples against local copies of its detectors and reformers. The attack achieves near-universal grey-box success, including 99% on MNIST and 100% on CIFAR.
- 3 Breaking MagNet: The attack trains 32 local auto-encoders, differing from the defender’s 16, as substitute detectors and reformers.These local models support transferable adversarial-example generation against MagNet’s hidden defensive parameters.
- 3 Breaking MagNet: Table 1 reports attack success and mean distance to the nearest targeted adversarial example over the first 1,000 test instances.Targets are selected uniformly at random among incorrect classes.
- 3 Breaking MagNet: The optimization incorporates local reformers R_j and detectors D_j, with c and d selected simultaneously by binary search.The detector threshold is calibrated so 1% of validation images are detected, while κ = 1 encourages transfer.
- 3 Breaking MagNet: The attack uses 10,000 gradient-descent iterations at learning rate 10^-2 without hyperparameter search.The authors state that improved parameter search could produce lower-distortion adversarial examples.
- 3 Breaking MagNet: 99% grey-box success is achieved on MNIST targeted source/target pairs, with the 7 → 6 attack failing to transfer.The figure reports the result for MagNet-protected MNIST images.
- 3 Breaking MagNet: 100% grey-box success is achieved on CIFAR targeted source/target pairs.The reported examples are evaluated on the MagNet-secured network.
4 Breaking “Efficient Defenses...”
The authors apply the Carlini–Wagner L2 attack directly to the defended network and find that Efficient Defenses provides only modest distortion increases. Gaussian augmentation and BReLU together do not improve robustness under these attacks.
- 4 Breaking “Efficient Defenses...”: The authors directly apply the Carlini–Wagner L2 attack to the defended network without further modification.They evaluate mean distortion over the first 1,000 test instances with randomly selected incorrect targets.
- 4 Breaking “Efficient Defenses...”: 30% on MNIST and 20% on CIFAR are the increases in mean distance to the nearest adversarial example from the full defense.The paper contrasts these increases with the significantly larger distortion increase reported for adversarial retraining.
- 4 Breaking “Efficient Defenses...”: BReLU increases distortion when used without Gaussian augmentation but does not help when combined with it.This result is reported for the evaluated MNIST and CIFAR settings.
5 Breaking APE-GAN
The paper breaks APE-GAN by applying a Carlini–Wagner L2 attack directly to the defended network, accounting for manifold projection before classification. The recovered adversarial examples are less similar to the originals than the initial adversarial examples.
- Carlini and Wagner’s L2 attack defeats APE-GAN by incorporating manifold projection into the loss function before classification.The authors otherwise use the same minimization formulation.
- The evaluation measures mean distortion over the first 1000 test instances, with targets selected randomly from incorrect classes.
- Table 3 reports that APE-GAN does not significantly increase robustness on MNIST or CIFAR-10.
- 5.8 mean distortion separates recovered instances from originals, compared with 4.3 between adversarial examples and originals.The recovered adversarial examples are therefore less similar to the original inputs.
- The attack fools generator G into producing reconstructions less similar to the originals than the adversarial examples themselves.Figure 4 shows faint lines becoming more pronounced after reconstruction.
6 Conclusion
The paper demonstrates that the evaluated defenses are defeated by adversarial examples and provide only approximately the same, limited robustness increase. It recommends testing defenses with adaptive attacks tailored to their threat model before claiming robustness.
- Approximately 30%: MagNet, Gaussian data augmentation with BReLU, and the other evaluated defenses provide similar robustness increases against adversarial examples.The paper reports that these defenses take different approaches but achieve approximately the same increase in robustness.
- MagNet is not robust to transferable adversarial examples, while Gaussian augmentation and BReLU do not significantly improve robustness against strong iterative attacks.
- The paper recommends adaptive white-box attacks, or defense-specific source models for grey-box evaluation, before claiming robustness.Lack of knowledge of exact parameters does not imply that transfer from an unsecured model is the strongest available attack.
- The figures illustrate attacks against Efficient Defenses... with BReLU alone and with Gaussian noise plus BReLU, and against APE-GAN through reconstructed adversarial examples.