Source-linked AI summary
Curriculum Adversarial Training
Qi-Zhi Cai, Min Du, Chang Liu, Dawn Song
TL;DR
Adversarial training had not achieved high empirical worst-case accuracy on complex tasks such as CIFAR-10 and SVHN. The paper proposes curriculum adversarial training, using adversarial examples of varied strengths with batch mixing and quantization, and reports substantial improvements over prior art while largely preserving non-adversarial performance.
Problem
Adversarial examples hinder security-sensitive deep-learning applications, while prior adversarial training had not been shown to extend effectively from MNIST to harder tasks such as CIFAR-10 and SVHN.
Method
Curriculum adversarial training uses adversarial examples generated by attacks of varied strengths, with batch mixing and quantization to mitigate catastrophic forgetting and improve attack generalization.
Results
CAT consistently improves empirical worst-case accuracy over prior art, from 46.18% to 69.27% on CIFAR-10 and from 40.38% to 75.66% on SVHN, while non-adversarial accuracy decreases by at most 6%.
Takeaways & Limitations
CAT has potential for practical deployment as a robust model and suggests further investigation of adversarial-training frameworks.
Takeaways & Limitations
Models trained with weaker attacks may not generalize to stronger attacks, causing accuracy to drop significantly beyond the strongest training attack.
Abstract
from arXiv · showhide
Recently, deep learning has been applied to many security-sensitive applications, such as facial authentication. The existence of adversarial examples hinders such applications. The state-of-the-art result on defense shows that adversarial training can be applied to train a robust model on MNIST against adversarial examples; but it fails to achieve a high empirical worst-case accuracy on a more complex task, such as CIFAR-10 and SVHN. In our work, we propose curriculum adversarial training (CAT) to resolve this issue. The basic idea is to develop a curriculum of adversarial examples generated by attacks with a wide range of strengths. With two techniques to mitigate the forgetting and the generalization issues, we demonstrate that CAT can improve the prior art's empirical worst-case accuracy by a large margin of 25% on CIFAR-10 and 35% on SVHN. At the same, the model's performance on non-adversarial inputs is comparable to the state-of-the-art models.
1 Introduction
Adversarial examples hinder security-sensitive deep-learning applications, while prior defenses showed strong robustness on MNIST but limited results on harder datasets. CAT extends adversarial training with attack-strength curricula and optimizations, improving empirical worst-case accuracy on CIFAR-10 and SVHN.
- Adversarial examples hinder deep learning in security- and safety-sensitive applications.
- CAT combines a curriculum of attacks with different strengths, batch mixing, and quantization.
- 46.18% to 69.27% on CIFAR-10 and 40.38% to 75.66% on SVHN, while non-adversarial accuracy decreased by at most 6%.
- Prior defenses have largely been broken, although adversarial training achieved a robust MNIST model.
2 Adversarial examples and adversarial training
Adversarial examples are misclassified inputs constrained to remain close to benign inputs, and attacks approximate loss maximization under that constraint. Adversarial training alternates model optimization with adversarial-example generation, but vanilla training performs poorly on CIFAR-10 and SVHN.
- 2.1 Adversarial examples: An adversarial example is an input that misleads the model while remaining within a specified distance of the original input.
- 2.1 Adversarial examples: This work measures image perturbations with the L∞ distance metric.
- 2.2 Adversarial example attacks: Attacks optimize a loss function under a perturbation constraint, with stronger attacks approximating the optimum more closely.
- 2.2 Adversarial example attacks: PGD(k) uses k iterations, so larger k represents a stronger attack.
- 2.3 Adversarial training: Adversarial training alternates optimizing model parameters with finding worst-case adversarial examples.
- 2.3 Adversarial training: Vanilla adversarial training cannot exceed 46% empirical worst-case accuracy on CIFAR-10 and 40% on SVHN.
3 Curriculum adversarial training
CAT trains models through progressively stronger attacks, then uses batch mixing and quantization to address forgetting and attack generalization. Quantization reduces the adversarial-example space but trades resilience against model effectiveness.
- 3.1 Basic curriculum adversarial training: CAT trains first with weak attacks and increases attack strength after validation performance stops improving.
- 3.1 Basic curriculum adversarial training: The basic curriculum can forget examples generated by weaker attacks, preventing high empirical worst-case accuracy.
- 3.2 Batch mixing: Batch mixing combines adversarial examples generated by PGD attacks at multiple strengths within each training batch.
- 3.2 Batch mixing: With batch mixing, models trained with strong attacks remember most adversarial examples from weaker attacks and achieve better overall accuracy.
- 3.3 Quantization: Quantization converts each input dimension to a b-bit integer at inference time, reducing the space of adversarial examples.
- 3.3 Quantization: Smaller b reduces the adversarial-example space but may remove information; evaluation found b = 4 performed best.
4 Evaluation Setup
The evaluation compares CAT with prior art on CIFAR-10 and SVHN using standard image-classification architectures and multiple white-box and black-box attacks. Empirical worst-case accuracy counts test inputs that survive all evaluated attacks.
- CAT is evaluated against Madry et al. on CIFAR-10 and SVHN using L∞ perturbations.
- The experiments use ResNet-50 and DenseNet-161, with mini-batches of 200.
- The CIFAR-10 bound is 8/255 with K = 7, while the SVHN bound is 12/255 with K = 10.
- Empirical worst-case accuracy is the percentage of test inputs not successfully misled by any evaluated attack.
- The attack suite includes Carlini–Wagner, PGD with multiple iteration counts, and black-box attacks.
5 Evaluation results
CAT improves empirical worst-case resilience across CIFAR-10 and SVHN while addressing overfitting, catastrophic forgetting, and attack generalization through curriculum training, batch mixing, and quantization.
- CAT effectiveness: CAT surpasses AT after lesson l = 4 and continues increasing beyond 60% test resilience.The result is presented as evidence that CAT overcomes AT’s overfitting problem.
- Forgetting phenomenon: After lesson l = 7, PGD(6) and PGD(7) accuracy approaches 100%, but PGD(6) falls below 10% during lesson l = 9.This decline documents catastrophic forgetting under basic curriculum adversarial training on SVHN.
- Batch mixing: Batch mixing improves ˜l-accuracy over Basic by 5.88%–68.3% and can outperform AT on this measure.On SVHN, Basic’s empirical worst-case accuracy is around 10%, largely because of forgetting.
- Attack generalization: Basic+MIX has empirical worst-case accuracy below 6%, reflecting poor generalization from weaker training attacks to stronger attacks.Quantization reduces the need for the resilience trade-off introduced by MIX and contributes to CAT’s better overall resilience.
- Quantization: Quantization improves performance across almost all tested model, dataset, and training combinations, including vanilla adversarial training.It reduces the attack space at inference time, limiting the additional solutions stronger attacks can find after pixel values are quantized.
- Regular accuracy: Non-adversarial accuracy drops around 1% on SVHN and 5%–6% on CIFAR-10.The authors attribute the limited loss to including non-adversarial inputs in the curriculum and batch-mixing training.
6 Conclusion
The paper presents curriculum adversarial training as a defense framework that combines varied attack strengths with batch mixing and quantization to improve resilience.
- Curriculum adversarial training uses adversarial examples generated by attacks with various strengths.
- Batch mixing and quantization are proposed as optimizations to improve the algorithm’s resilience.
- 25% to 35%: combining the techniques improves over previous state-of-the-art by this margin.