Source-linked AI summary
Generative Adversarial Trainer: Defense to Adversarial Perturbations with GAN
Hyeungill Lee, Sungyeob Han, Jungwoo Lee
TL;DR
Neural networks are vulnerable to adversarial examples, motivating a method that combines adversarial training with a GAN framework. The method alternates a perturbation-generating network with classifier retraining, and experiments report stronger robustness and generalization, alongside higher training cost and GAN hyperparameter sensitivity.
Problem
Deep neural networks are vulnerable to adversarial examples, which can change predictions while remaining visually indistinguishable from original images.
Method
The method alternately trains a generator to produce classifier-fooling perturbations from image gradients and a classifier to label original and adversarial images correctly.
Results
50.44 ± 0.56 test accuracy is reported for Adv. training (GAT), compared with 45.16 ± 0.73 for Adv. training (FG, L∞).
Takeaways & Limitations
The proposed method is reported to improve robustness to adversarial examples and classifier generalization accuracy.
Takeaways & Limitations
Training takes 3 to 4 times longer than conventional fast-gradient training, and each network's hyperparameters require careful tuning.
Abstract
from arXiv · showhide
We propose a novel technique to make neural network robust to adversarial examples using a generative adversarial network. We alternately train both classifier and generator networks. The generator network generates an adversarial perturbation that can easily fool the classifier network by using a gradient of each image. Simultaneously, the classifier network is trained to classify correctly both original and adversarial images generated by the generator. These procedures help the classifier network to become more robust to adversarial perturbations. Furthermore, our adversarial training framework efficiently reduces overfitting and outperforms other regularization methods such as Dropout. We applied our method to supervised learning for CIFAR datasets, and experimantal results show that our method significantly lowers the generalization error of the network. To the best of our knowledge, this is the first method which uses GAN to improve supervised learning.
1 Introduction
Deep neural networks remain vulnerable to adversarial examples that fool predictions while appearing unchanged to humans. The paper proposes adversarial training with a GAN-style alternating classifier–generator framework and reports improved robustness and generalization.
- Adversarial examples can fool neural networks into changing predictions while remaining indistinguishable from original images to humans.
- Adversarial training retrains networks to predict correct labels for generated adversarial examples, so its inner-loop generation must be fast.
- GANs train generative models through a two-network minimax game that gradually approaches an optimal solution.
- The proposed method alternately trains a generator to fool the classifier and a classifier to label original and generated adversarial images correctly.The procedure is intended to improve robustness to adversarial perturbations.
- The method outperforms fast-gradient adversarial training and yields lower generalization errors than regularization methods such as dropout.
2 Backgrounds
The background reviews fast-gradient adversarial training and GANs as foundations for the proposed framework. Adversarial training combines original and perturbed-image losses, while GANs formulate competition between generative and discriminative models.
- 2.1 Adversarial Training: Fast-gradient adversarial training generates perturbations from the input-loss gradient and trains networks against an adversarial objective.The fast-gradient sign method uses the sign of the input gradient, while an L2 variant normalizes the gradient.
- 2.1 Adversarial Training: The adversarial-training loss combines original cross-entropy with loss on perturbed inputs, weighted by hyperparameter α.The combined objective connects robust optimization with regularization and stabilizes networks near training points.
- 2.2 Generative Adversarial Networks: GANs train a generative model and a discriminative model in a two-player minimax game.The generator maps noise toward the data distribution, while the discriminator distinguishes generated samples from training data.
- 2.2 Generative Adversarial Networks: The proposed framework applies this adversarial-training logic by using a GAN-style trainer to generate perturbations and training the classifier on original and adversarial images.
3 Proposed Method
The proposed framework alternates a generative adversarial trainer with a classifier to produce and defend against classifier-specific adversarial perturbations. The method balances perturbation effectiveness with bounded magnitude while jointly training on original and perturbed images.
- 3 Proposed Method: The framework alternates training a classifier network and a generative adversarial trainer (GAT) to improve robustness to adversarial perturbations.The GAT generates perturbations intended to fool the classifier, while the classifier learns from both original and adversarial images.
- 3 Proposed Method: The GAT uses each image’s gradient to generate classifier-specific perturbations rather than relying only on fast-gradient sign or normalization functions.Its objective is to find perturbations that lower the classifier’s predicted probability for the correct class.
- 3 Proposed Method: The GAT loss combines reducing the classifier’s class probability with restricting perturbation power.The hyperparameter c_g controls the ratio: values that are too low can produce excessive perturbations, while values that are too high can yield zero perturbations.
- 3 Proposed Method: The classifier is trained to classify correctly both original images and adversarial images generated by the GAT.Its loss combines the classifier loss on original inputs with the loss on perturbed inputs.
- 3 Proposed Method: The method alternately optimizes the generator for k steps and the classifier for one step instead of fully optimizing the generator in every inner loop.The experiments use α = 0.5 and k = 1; full inner-loop optimization is described as computationally expensive and potentially prone to overfitting.
4 Experiments
Experiments on CIFAR-10 and CIFAR-100 evaluate GAT’s perturbation strength, robustness under direct and indirect attacks, and regularization effect. GAT generates strong adversarial images at low perturbation power and improves classification robustness and accuracy relative to baseline and other regularization methods.
- Experimental setup: Experiments used CIFAR-10 and CIFAR-100, splitting the original training data into 45,000 training and 5,000 validation samples.Each dataset contains 50,000 training samples and 10,000 test samples.
- Experimental setup: The classifier used a small AllConvNet variant, while the generator used six 3 × 3 and two 1 × 1 convolutional layers.The generator received an input gradient image computed from the classifier.
- Perturbations generated by GAT: GAT generated stronger adversarial images than the fast gradient methods at low perturbation power, although it required several iterations for optimization.Full generator optimization was unnecessary during adversarial training because the generator and classifier were optimized alternately.
- Direct attack: GAT achieved the best classification accuracy without perturbation and was more robust than fast-gradient adversarial training across perturbation powers in direct attacks.The comparison generated adversarial examples separately from each network’s parameters.
- Indirect attack: Indirect attacks generated perturbations from another baseline network to test robustness when attackers lack each model’s internal structure.The experiment compared the proposed method with baseline and robustified networks under this transfer-style setting.
- Regularization effect: GAT improved accuracy by 4 −6% over the baseline, compared with approximately 1 −2% improvements from existing regularization techniques.The study compared GAT with dropout, random perturbation, and fast-gradient adversarial training.
5 Discussion
The method combines adversarial training with GAN-based adaptive perturbations, improving robustness and generalization over conventional approaches. Its main costs are longer training and careful hyperparameter tuning.
- 50.44 ± 0.56% test accuracy was achieved by Adv. training (GAT), exceeding Adv. training (FG, L∞) at 45.16 ± 0.73%.Dropout + GAT reached 50.71 ± 0.49%.
- Adaptive adversarial examples address differing image-wise robustness that fixed-size fast-gradient perturbations do not accommodate.The method does not normalize the gradient vector and evolves adaptively during classifier training.
- Because the classifier is nonlinear, GAT can detect nonlinear patterns and generate better perturbations than the fast gradient method.This provides a mechanism for producing more effective adversarial examples during training.
- Training takes 3 to 4 times longer than the conventional fast gradient method, and each network’s hyperparameters require careful tuning.The duration depends on generator capacity, while the tuning requirement follows from GAN-training properties.
6 Conclusions
The paper presents GAN-based adversarial training in which a generator finds classifier weaknesses and the classifier relearns the correct labels. The method is reported as robust, practical, and effective for regularization, while broader applicability remains open.
- The generator finds classifier weaknesses, while the classifier relearns generated images with their original labels.Alternating learning makes the classifier more robust to adversarial images.
- The method does not require expensive inner-loop optimization to find optimal adversarial images, making the adversarial training procedure practical.
- The classifier is reported to be highly robust to adversarial examples and the method effective at regularizing neural networks.
- Further work is needed to determine generator capacity, identify useful information beyond gradients, and test larger networks such as Inception.These questions define the stated scope for future improvement and evaluation.