The Robust Manifold Defense: Adversarial Training using Generative Models

Ajil Jalal; Andrew Ilyas; Constantinos Daskalakis; Alexandros G. Dimakis

Source-linked AI summary

The Robust Manifold Defense: Adversarial Training using Generative Models

Ajil Jalal, Andrew Ilyas, Constantinos Daskalakis, Alexandros G. Dimakis

arXiv:1712.09196v5Dec 26, 2017cs.CVcs.CRcs.LGstat.ML

TL;DR

The paper addresses whether generative-model manifolds can yield stronger attacks and defenses for image classifiers. It searches over latent-code pairs, circumvents DefenseGAN, and combines the attack with adversarial training to improve MNIST robustness.

Problem
The paper asks whether a spanner can provide improved attacks or defenses for DNN classifiers.
Method
The method searches over pairs of latent codes in a low-dimensional spanner space to generate nearby images with different classifier outputs, then uses the attack in min-max adversarial training.
Results
The attack fully circumvents DefenseGAN and improves the state of the art for DefenseGAN, while the combined defense obtains the most robust known MNIST classifier.
Takeaways & Limitations
Generative-model attacks can strengthen adversarial training and produce robust classifiers, while also exposing vulnerabilities in manifold-based defenses.
Takeaways & Limitations
The approach relies on having a good spanner or generative model for the domain of interest.

Abstract

from arXiv · show

We propose a new type of attack for finding adversarial examples for image classifiers. Our method exploits spanners, i.e. deep neural networks whose input space is low-dimensional and whose output range approximates the set of images of interest. Spanners may be generators of GANs or decoders of VAEs. The key idea in our attack is to search over latent code pairs to find ones that generate nearby images with different classifier outputs. We argue that our attack is stronger than searching over perturbations of real images. Moreover, we show that our stronger attack can be used to reduce the accuracy of Defense-GAN to 3\%, resolving an open problem from the well-known paper by Athalye et al. We combine our attack with normal adversarial training to obtain the most robust known MNIST classifier, significantly improving the state of the art against PGD attacks. Our formulation involves solving a min-max problem, where the min player sets the parameters of the classifier and the max player is running our attack, and is thus searching for adversarial examples in the {\em low-dimensional} input space of the spanner. All code and models are available at \url{https://github.com/ajiljalal/manifold-defense.git}

1 Introduction

The paper uses low-dimensional generative-model ranges to construct stronger attacks and defenses for image classifiers. Its overpowered latent-space attack circumvents DefenseGAN and improves adversarial training for MNIST.

A spanner is a low-dimensional DNN whose output range approximates the dataset; examples include GAN generators and autoencoder decoders.
The paper asks whether spanners can improve attacks or defenses of DNN classifiers.
The overpowered attack searches for nearby generated images from latent-code pairs whose classifier outputs differ substantially.
The attack fully circumvents DefenseGAN, resolving an open challenge posed by Athalye et al.
The Robust Manifold Defense combines the attack with adversarial training in a min-max problem, reducing the max player’s search from image space to latent space.
The reported MNIST robustness result is adversarial accuracy of 96.26%, compared with 91.88% for TRADES under white-box attacks with bounded ℓ2 norm= 1.5.
The evaluation reports robustness on original test-set images, enabling direct comparison with prior results.

2 Related work

The related work covers adversarial training and generative-model defenses, positioning this paper most closely against DefenseGAN and PixelDefend. The paper’s main distinction is using spanners to generate attacks and train defenses.

Prior defenses modify training data, network architectures, or training procedures such as defensive distillation and adversarial training.
GAN- and VAE-based defenses use generative models in ways that differ from this paper’s projection-based adversarial training approach.
DefenseGAN is the closest related work, while PixelDefend is similar but uses PixelCNN generators; both defenses are discussed as circumventable.

3 An (Ineffective) First Take at Adversarial Defense with Spanners

The preliminary Invert-and-Classify defense projects inputs onto a spanner’s range before classification. Its non-differentiable projection does not prevent white-box attacks from circumventing the defense.

Invert-and-Classify projects an input onto the range of a GAN generator or VAE decoder and applies the classifier to that projection.
The procedure finds a latent code by minimizing the reconstruction distance between G(z) and the input.
Inputs whose projection is at least η away are rejected as unnatural; otherwise, the classifier outputs a label for the projection.
Although projection non-differentiability blocks direct gradients through that step, INC belongs to a class of defenses circumvented by white-box adversaries.

4 The Overpowered Attack

The overpowered attack searches for nearby generated images with sharply different classifier outputs, exposing weaknesses in projection-based defenses and reducing DefenseGAN accuracy under standard perturbation budgets.

The overpowered attack reduces the INC defense’s accuracy to 0% and fully circumvents DefenseGAN.
The attack identifies latent-code pairs whose generated images are close while their classifier outputs differ substantially.
The optimization problem upper-bounds the worst loss that an overpowered adversary can cause, including projection-step suboptimality in INC.
3% DefenseGAN accuracy is achieved under the same perturbation budget and hyperparameter settings as the earlier comparison.
The latent-space attack alone need not produce images close to real data, so additional constraints are required for standard real-image perturbation evaluation.

5 The Robust Manifold Defense

The Robust Manifold Defense embeds the overpowered attack in adversarial training, making the inner search lower-dimensional and stronger while relying on a faithful spanner. On MNIST, it raises adversarial accuracy from 91.88% to 96.26% under an ℓ2 norm bound of 1.5.

The defense uses a min-max formulation whose outer player sets classifier parameters and whose inner player searches for overpowered attacks.
The objective mixes attack loss with training-set classification loss through a hyperparameter µ.
The inner search operates over latent-code pairs in a typically lower-dimensional space than image-space adversarial training.
The overpowered attack can search for close adversarial image pairs beyond the training set, while the spanner makes such image-pair searches meaningful.
A good spanner is required: poor approximation can make the attacker overly powerful and reduce robustness to real images.
96.26% adversarial accuracy improves on TRADES’ 91.88% for MNIST under white-box attacks with bounded ℓ2 norm = 1.5.

6 Experiments

The experiments adapt the latent-space attack to break DefenseGAN and combine it with adversarial training. The resulting classifier improves MNIST robustness while the attack fools DefenseGAN on most tested images.

Breaking DefenseGAN: The DefenseGAN attack searches for a latent code whose generated image remains close to the original while changing the classifier’s prediction.The procedure uses a GAN inverse, restarts, gradient descent-ascent, and noise averaging to account for DefenseGAN’s inversion procedure.
Breaking DefenseGAN: The attack is formulated as a max-min optimization using a Lagrange multiplier and solved by gradient descent-ascent.The optimization targets a Carlini-Wagner loss while enforcing the image-distance constraint.
Breaking DefenseGAN: 97% of MNIST test-set attacks fool DefenseGAN while satisfying its perturbation constraint.DefenseGAN is robust to only 3% of the attacks under Athalye et al.'s settings.
Adversarial Training: The adversarial-training loop alternates overpowered latent-space attacks with five-epoch runs of Madry et al.’s adversarial training.Each attack iteration supplies latent-code pairs for a classifier update using cross-entropy loss.
Adversarial Training: The trained model retains substantial robustness when the test-set perturbation budget increases from δ = 1.5 to δ = 2.5.The paper reports that robustness does not fall significantly at the larger budget, although the model was trained at δ = 1.5.

7 Conclusion

The paper introduces generative-model attacks that search pairs of images in a spanner’s range and uses them to improve adversarial robustness. The attack improves DefenseGAN and supports robust classifiers with natural uncertainty around decision boundaries.

Generative models enable an overpowered attack that searches pairs of images in the spanner range.
The attack improves the state of the art for DefenseGAN and resolves a challenging problem.
Generated attack images can boost existing adversarial training and produce robust classifiers with natural uncertainty near decision boundaries.
Figure 4 shows the robust adversarially trained classifier reducing confidence on borderline images generated by the attack.

A Hyperparameters for DefenseGAN break

This section documents the implementation resources and optimization initialization used for the DefenseGAN attack.

The generator, classifier architecture, checkpoint, and attack implementation are available in the project repository.
The latent search initializes z with the closest generator-range reconstruction of x and uses Adam for z and SGD for λ.The initialization sets λ to −1000, with learning rates 0.05 for z and 10000 for λ.

A.1 Hyperparameters for DefenseGAN

DefenseGAN is evaluated under two published hyperparameter settings, with the combined overpowered attack and EOT reducing its accuracy substantially in both settings.

DefenseGAN is evaluated under the Athalye et al. and Samangouei et al. hyperparameter settings.
3% accuracy remains under the Athalye et al. threat model after the combined overpowered attack and EOT.This setting uses 20,000 projection steps and 20 random restarts.
5% accuracy remains under the Samangouei et al. threat model after the combined overpowered attack and EOT.This setting uses 200 projection steps and 10 random restarts.

B Hyperparameters for Boosting Madry’s MNIST

The MNIST boosting procedure combines classifier training with periodic overpowered attacks, using specified optimization, dataset, and model configurations. Attack frequency is tuned because excessive generated samples can destabilize training.

The max-min optimization uses Yellowfin for z and λ, while the classifier uses Adam; both use learning rate 10^-4.The overpowered attack runs for 500 updates.
The VAE and classifier train on MNIST images 1–55,000, validate on 55,000–60,000, and select the classifier with maximum robust accuracy.The standard MNIST test set is used for testing.
The MNIST classifier follows Madry et al.’s model and is trained against PGD with bounded ℓ2 norm 1.5.The generative model is a VAE with a decoder whose final output width is 784.
Classifier and generator model definitions and checkpoints are provided in the project repository.
Overpowered-attack frequency is crucial because too many generated samples can cause overfitting to generator artifacts and random guessing.
Five epochs of regular PGD training followed by one batch of overpowered-attack samples produced the best results among the tested schedules.Models were selected using validation-set robustness.

C Appendix for Adversarial Training on CelebA

The CelebA procedure attacks the invert-and-classify defense by searching for nearby generated images with different classifications, then trains the classifier adversarially against this attack. The resulting robust classifier preserves clean accuracy while sharply reducing classification divergence on attack-generated inputs.

Attack setup: The overpowered attack searches for latent codes z and z′ whose generated images are close but whose classifier outputs differ substantially.The attack is formulated as a max-min optimization using classifier-output distance and cross-entropy loss.
Training procedure: Adversarial training combines dataset cross-entropy with an adversariality loss generated by 500-step min-max attacks.The final ℓ2 distance between classification outputs is added to the cross-entropy objective.
Attack behavior: 100% of attack-generated images were valid, while 22% induced different classifications and the average KL divergence was 0.08 after 10,000 iterations.These results indicate a softened decision boundary for the classifier under attack.
Robustness: The robust classifier was effective against the overpowered latent-space attack crafted for the INC-protected classifier.The cited evaluation compares the robust classifier with the attack targeting the invert-and-classify defense.
Robustness: The adversarially trained classifier retained 97% accuracy on normal inputs, while average classification divergence fell to 0.007 and inconsistent classifications to 18%.The evaluation used inputs generated by the min-max attack against the initial classifier.

C.1 Architecture

The CelebA architecture uses a BEGAN generator and a modified CIFAR10 convolutional classifier for binary gender prediction. The section also tracks attack-output divergence and training-loss decay, while Table 2 organizes accuracy across defenses and attack types.

Architecture: The generator is a BEGAN model used to produce the CelebA attack images.The implementation uses the cited TensorFlow BEGAN repository.
Architecture: The classifier is a modified CIFAR10 deep convolutional network whose final layer has two nodes for binary gender classification.It was adapted to the CelebA dataset.
Diagnostics: Figure 5 reports average classifier-output distance for latent-code pairs found by the attack.The plotted quantity is ||C(G(z)) − C(G(z′))||2.
Diagnostics: Figure 6 tracks the cross-entropy and adversarial components of the loss as training continues.Both components are shown as decaying during training.
Evaluation: Table 2 compares CelebA accuracy under the overpowered attack, FGSM, BIM, and PGD at multiple powers ε for non-robust and robust INC settings.NR denotes the non-robust classifier, NR+INC the non-robust INC-protected classifier, and R+INC the robust INC-protected classifier.