Source-linked AI summary
Early Methods for Detecting Adversarial Images
Dan Hendrycks, Kevin Gimpel
TL;DR
Adversarial perturbations expose a gap between human perception and machine classification and can give attackers leverage over deep learning systems. The paper develops three detectors and a saliency map; its strongest finding is that adversarial images abnormally emphasize low-ranked PCA components, while bypass attempts fail or require larger modifications.
Problem
Adversarial perturbations can change classifier predictions while remaining difficult for humans to notice, exposing classifier frailties and enabling attacks.
Method
The paper deploys three methods to detect adversarial images, including PCA-whitened coefficient analysis, softmax-distribution analysis, and reconstruction-based detection, plus a saliency map.
Results
Adversarial images abnormally emphasize low-ranked PCA components, and attempts to bypass the detectors either fail or require larger image modifications.
Takeaways & Limitations
Detector ensembles may be a feasible avenue for defending against adversarial images because adversaries must contort images to evade detection.
Abstract
from arXiv · showhide
Many machine learning classifiers are vulnerable to adversarial perturbations. An adversarial perturbation modifies an input to change a classifier's prediction without causing the input to seem substantially different to human perception. We deploy three methods to detect adversarial images. Adversaries trying to bypass our detectors must make the adversarial image less pathological or they will fail trying. Our best detection method reveals that adversarial images place abnormal emphasis on the lower-ranked principal components from PCA. Other detectors and a colorful saliency map are in an appendix.
1 INTRODUCTION
Adversarial images exploit a gap between human perception and machine classification, creating security risks for deep learning systems. The paper presents three detection methods and a saliency map, with bypass attempts either failing or requiring larger image modifications.
- Adversarial perturbations can change classifier predictions while remaining barely noticeable to humans.
- Attackers can exploit adversarial images to misread checks or evade malware and spam detectors without exact system knowledge.
- The paper presents three adversarial-image detectors and a saliency map for making network classification decisions more understandable.
- Bypassing the detectors either fails or forces adversaries to make larger modifications to clean images.
2 DETECTING ADVERSARIAL IMAGES
The paper detects adversarial images using PCA-whitened coefficient variance and related visual or statistical differences. Adversarial images emphasize low-ranked components, and attempts to normalize this variance fail.
- PCA whitening transforms an input x into Σ−1/2U^Tx, whose later entries correspond to principal components with smaller eigenvalues.
- Figure 1 illustrates abnormal emphasis on low-ranked principal-component coefficients, while Figure 2 shows visible differences between ZCA-whitened adversarial and clean images.
- Adversarial images show different, typically higher-variance coefficients in low-ranked principal components than clean images.
- The first detector uses variance from late PCA-whitened coefficient entries as its sole detection feature across Tiny-ImageNet, CIFAR-10, and MNIST.
- Fast gradient sign adversarial images have coefficient variance beyond ten standard deviations from the clean mean, including 100% of MNIST images beyond 10 billion standard deviations.
- Attempts to force adversarial images to have typical coefficient variances fail.
A DETECTOR EVALUATION
The paper evaluates detectors with threshold-independent metrics rather than accuracy, because threshold choice depends on the trade-off between false positives and false negatives. AUROC measures ranking quality without selecting a threshold.
- Detection accuracy is omitted because its value depends on a threshold chosen according to false-negative and false-positive trade-offs.
- AUROC is used as a threshold-independent evaluation metric for adversarial-image detectors.
- AUROC can be interpreted as the probability that an adversarial example receives a greater detector score than a clean example.
- A random detector corresponds to 50% AUROC, while a perfect classifier corresponds to 100%.
- AUPR also avoids threshold selection, but AUROC is less ideal when positive and negative classes have greatly differing base rates.
B.1 SOFTMAX DISTRIBUTIONS OF ADVERSARIAL IMAGES
The softmax detector adapts an out-of-distribution observation to adversarial images. Softmax metrics can be manipulated, but doing so makes the adversarial image less pathological.
- Clean and many out-of-distribution examples tend to produce different softmax output distributions.
- The method applies softmax-distribution differences to detect adversarial images.
- Softmax distribution metrics can be gamed, but constrained adversarial generation makes the resulting image less pathological.
B.2 RECONSTRUCTING ADVERSARIAL IMAGES WITH LOGITS
The reconstruction-based detector compares inputs with decoder reconstructions that incorporate classifier logits. Adversarial images reconstruct atypically, enabling strong separation from clean examples.
- The model attaches an auxiliary decoder to an MNIST classifier and feeds logits into a 10-neuron bottleneck.The hidden layer has 256 units, and the decoder reconstructs the input from the bottleneck representation.
- Adversarial reconstructions are lower quality and more atypical than clean reconstructions.This reconstruction difference supplies the detector’s distinguishing signal.
- 96.2% AUROC and 96.6% AUPR distinguish adversarial from clean examples, against 50% baselines.The detector scores the mean difference between each input image and its reconstruction.
C A SALIENCY MAP
The paper introduces a saliency map based on positive gradient propagation to visualize classification evidence more clearly than guided backpropagation. Examples reveal image regions and color changes associated with misclassification logits.
- The saliency map uses positive gradient values to display image regions contributing to network classifications.It is applied to a pretrained VGG-16 model without modifying network weights.
- The method extends beyond standard backpropagation and also works with other nonlinearities, including GELU.The paper describes modified backward propagation rules and reports similar saliency maps for another nonlinearity.
- For a desert scene classified as a lakeside, bluer clouds and a more orange sunlit sky would increase the logits.The map associates the clouds with lakeside water and the sky with the shore.
- Compared with guided backpropagation, the saliency map reveals more detailed and clearer sources of saliency.The paper’s Figure 4 caption states the comparison, while the examples describe additional scene-level evidence.
D DISCUSSION AND CONCLUSION
The paper concludes that adversarial-image defense should combine diverse detectors because individual statistics can be bypassed. Future work includes attention-based detection and preprocessing defenses.
- Future ad-hoc attacks may completely bypass each individual detector.The softmax-distribution detector can already be bypassed.
- An ensemble of strong but imperfect predictors may force adversarial perturbations to become increasingly conspicuous.The discussion contrasts combined detectors with relying on one narrow, gameable metric.
- A proposed attention-based detector would flag abnormal or scattered attention when a street image is misclassified as a war tank.The proposal relies on attacks appearing nonlocalized.
- Preprocessing can hinder attacks, but anticipating it may require a 35% average increase in the new adversarial image’s ℓ2 norm.The preliminary CIFAR-10 procedure squares pixels, applies slight Gaussian blur, then takes the square root.