PaperlayerView exact revision on arXiv

Source-linked AI summary

PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples

Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, Nate Kushman

arXiv:1710.10766v3cs.LG

TL;DR

Imperceptible adversarial perturbations can seriously mislead image classifiers, raising the question of why these inputs are unusual. The paper uses generative-model likelihoods and statistical testing to detect them, then purifies images toward the training distribution with PixelDefend. The method improves robustness across attacks while remaining independent of the classifier and attack method.

  • Problem

    The paper asks why imperceptible perturbations fool neural classifiers and examines whether adversarial examples occupy low-probability regions of the training distribution.

  • Method

    PixelDefend uses PixelCNN likelihoods and statistical testing to detect adversarial inputs, then purifies them toward higher-probability training-distribution regions without changing the classifier.

  • Results

    PixelDefend achieves state-of-the-art results across many attacks, including improving strongest-adversary CIFAR-10 accuracy from 32% to 70%.

  • Takeaways & Limitations

    Because PixelDefend is model-agnostic and attack-agnostic, it can protect deployed classifiers and combine with other defenses without modifying the classification model.

  • Takeaways & Limitations

    Exact constrained optimization of PixelCNN likelihood is computationally intractable, and gradient-based optimization can decrease likelihood for most random initializations.

Abstract

from arXiv · show

Adversarial perturbations of normal images are usually imperceptible to humans, but they can seriously confuse state-of-the-art machine learning models. What makes them so special in the eyes of image classifiers? In this paper, we show empirically that adversarial examples mainly lie in the low probability regions of the training distribution, regardless of attack types and targeted models. Using statistical hypothesis testing, we find that modern neural density models are surprisingly good at detecting imperceptible image perturbations. Based on this discovery, we devised PixelDefend, a new approach that purifies a maliciously perturbed image by moving it back towards the distribution seen in the training data. The purified image is then run through an unmodified classifier, making our method agnostic to both the classifier and the attacking method. As a result, PixelDefend can be used to protect already deployed models and be combined with other model-specific defenses. Experiments show that our method greatly improves resilience across a wide variety of state-of-the-art attacking methods, increasing accuracy on the strongest attack from 63% to 84% for Fashion MNIST and from 32% to 70% for CIFAR-10.

1 INTRODUCTION

The paper investigates why imperceptible perturbations fool neural classifiers, proposing that adversarial examples occupy low-probability regions of the training distribution. It uses PixelCNN likelihoods and statistical testing to detect such inputs, then purifies them by moving them toward higher-probability images.

  • 1 INTRODUCTION: Imperceptible input modifications can cause neural classifiers to produce incorrect labels, motivating defenses for safety-critical deployments.The paper highlights applications such as self-driving cars.
  • 1 INTRODUCTION: Adversarial examples largely lie in low-probability regions of the distribution that generated the training data, despite small deviations from clean images.The paper interprets this as a form of covariate shift.
  • 1 INTRODUCTION: PixelCNN likelihoods are used to estimate the training distribution and are typically several orders of magnitude lower for adversarial inputs than for training and test images.The model is selected for strong image-density modeling and tractable likelihood evaluation.
  • 1 INTRODUCTION: Statistical hypothesis testing ranks a novel image among training-image density values to obtain a p-value for testing whether it came from the training distribution.The method detects adversarial images across many attacks, including perturbations differing by only a few pixel values.
  • 1 INTRODUCTION: PixelDefend purifies an input by searching within a bounded image neighborhood for a more probable image, without requiring knowledge of the attack or targeted classifier.The procedure limits L∞ changes so purified images remain visually imperceptible and is approximated with greedy decoding because exact optimization is intractable.
  • 1 INTRODUCTION: 32% to 70% accuracy against the strongest CIFAR-10 adversary demonstrates the reported improvement from the purification-based defense.The paper presents PixelDefend as a defense that can achieve state-of-the-art results across many attacking techniques.

2 BACKGROUND

The background defines the attack, defense, dataset, classifier, and generative-model settings used to evaluate PixelDefend. Attacks are constrained by L∞ perturbation budgets, while PixelCNN models clean image distributions for likelihood-based analysis.

  • 2.1 ATTACKING METHODS: Adversarial attacks seek a perturbation within an L∞ budget that changes a classifier’s prediction while remaining bounded per pixel.The attack parameter ϵ_attack sets the maximum integer perturbation for each pixel.
  • 2.1 ATTACKING METHODS: Random perturbation provides the simplest and weakest attack baseline, using element-wise uniformly sampled integer changes.The uniform distribution is defined over the interval [a, b].
  • 2.1 ATTACKING METHODS: FGSM changes pixels in the direction of the loss gradient, whereas BIM applies a smaller-step gradient-sign update repeatedly.BIM clips the result within the ϵ_attack-ball and is also called projected gradient descent in the cited work.
  • 2.1 ATTACKING METHODS: DeepFool iteratively linearizes the decision boundary, while Carlini-Wagner optimizes for adversarial examples with small perturbations.Both outputs are clipped so their perturbations do not exceed ϵ_attack.
  • 2.2 DEFENSE METHODS: Defense methods either modify model architecture or training, or modify adversarial examples to reduce their harm.The comparison includes adversarial training, label smoothing, and feature squeezing.
  • 2.3 EXPERIMENT METHODOLOGIES: Experiments use Fashion MNIST and CIFAR-10 with ResNet and VGG classifiers, alongside PixelCNN generative models trained on clean images.PixelCNN factorizes the joint pixel distribution into conditional distributions and provides tractable image likelihoods.
  • 2.3 EXPERIMENT METHODOLOGIES: Bits per dimension represent PixelCNN image likelihood normalized by image dimensions, channels, and log 2.The quantity is defined as BPD(X) ≜ −log p_CNN(X)/(I × J × K × log 2).

3 DETECTING ADVERSARIAL EXAMPLES

The paper evaluates whether adversarial and random perturbations move images outside the training distribution, using PixelCNN likelihoods and p-value-based statistical testing. These distributional measures detect many adversarial inputs, while random perturbations can also shift distributional likelihood without equally harming accuracy.

  • Adversarial examples can transfer across classifiers, suggesting they possess properties not tied to a single model.
  • A PixelCNN trained on CIFAR-10 uses log-likelihood as an approximation to the underlying image probability density.
  • Even a 3% perturbation systematically decreases PixelCNN log-likelihood, without the model observing the attack method or target ResNet.
  • Random perturbations also push images outside the training distribution, but they do not have the same adverse effect on accuracy.
  • The detection test compares a novel input's PixelCNN probability rank against training-image probabilities to assess whether it came from the training distribution.
  • For CIFAR-10, p-value distributions are nearly uniform for clean test images, while several attacks show significant deviations from uniformity, especially RAND, FGSM, and BIM.

4 PURIFYING IMAGES WITH PIXELDEFEND

PixelDefend purifies adversarial images by moving them toward higher-probability regions of the training distribution, then classifies them with an existing model. Its greedy procedure improves robustness across attacks, while excessive purification can damage clean-image performance.

  • 4 PURIFYING IMAGES WITH PIXELDEFEND: PixelDefend purifies inputs before classification, improving performance against advanced attacks while retaining an unmodified classifier.The method is designed to work across classifiers and attacks and can be combined with other defenses.
  • 4 PURIFYING IMAGES WITH PIXELDEFEND: PixelDefend searches for a high-probability image within an ϵdefend-ball around the input, using a PixelCNN approximation of the training distribution.The optimization is constrained because large ϵdefend may change image meaning, whereas small ϵdefend may not restore the correct distribution.
  • 4 PURIFYING IMAGES WITH PIXELDEFEND: Exact constrained optimization of pCNN(X) is computationally intractable, so PixelDefend uses greedy decoding instead.L-BFGS-B often lowered PixelCNN probability, whereas the greedy method increased it more effectively on FGSM examples.
  • 4.1 RETURNING IMAGES TO THE TRAINING DISTRIBUTION: After purification, p-values become less effective for detecting adversarial images, and purified examples become closer to the training distribution.For DeepFool and CW examples, purification makes adversarial images barely distinguishable from normal samples by the reported detection analysis.
  • 4.2 ADAPTIVE PIXELDEFEND: If ϵdefend is too large, purification can modify clean images toward the mode image and cause misclassifications.An adaptive thresholding variant uses attack knowledge, so it is no longer strictly attack-agnostic.
  • 4.3 PIXELDEFEND RESULTS: 76% to 85% and 63% to 84%: PixelDefend improves Fashion MNIST accuracy on the strongest attacks while clean-image performance declines only slightly.On CIFAR-10, reported strongest-attack improvements include 74% to 81%, 32% to 70%, and 6% to 56% for ϵattack = 2, 8, and 16.

5 RELATED WORK

Related work detects or mitigates adversarial examples through uncertainty estimates, classifier modifications, statistical testing, and density-based purification. PixelDefend differs by emphasizing single-example testing and model-agnostic purification.

  • 5 RELATED WORK: Many detection methods add outlier-class modules or rely on kernel density and Bayesian uncertainty, but they can require classifier changes or be bypassed.These approaches are therefore not uniformly model-agnostic or robust to adaptive attacks.
  • 5 RELATED WORK: PixelDefend’s statistical testing differs from an earlier kernel-based two-sample test because it requires only one suspicious input.The related method also focused mainly on first-order attacks, whereas PixelDefend is presented across a broader attack setting.
  • 5 RELATED WORK: Moving data points from low- to high-density regions also appears in manifold denoising, density-gradient estimation, and denoising autoencoders.The paper identifies these methods as related ideas that could potentially be adapted to amortize purification.

6 CONCLUSION

The paper finds that neural density models detect small perturbations and uses this sensitivity to detect and purify adversarial examples. PixelDefend returns inputs toward high-probability training-distribution regions while remaining model- and attack-agnostic.

  • Neural density models such as PixelCNN detect small perturbations with high sensitivity across perturbations generated by different methods.
  • Permutation testing provides exact p-values that serve as an uncertainty measure for detecting outlier perturbations.
  • PixelDefend purifies adversarial examples by returning them to high-probability training-distribution regions, significantly decreasing their damage to classifiers.
  • PixelDefend is model-agnostic and attack-agnostic, so it can combine with other defenses without modifying the classification model.

APPENDIX A ON RANDOM PERTURBATIONS

Random perturbations can also occupy low-density regions and substantially reduce accuracy for a classifier constructed to be sensitive to them, despite strong clean-image performance.

  • Random perturbations have very low p-values and therefore lie outside the high-density area, although many classifiers remain robust to random noise.
  • 9874 of 10000 random images have bits per dimension larger than 4.5, compared with 175 of 10000 test images.The ResNet achieves 92.0% accuracy on clean CIFAR-10 test images and 87.3% on randomly perturbed images with ϵ_attack = 8.
  • 90.6% accuracy on clean test images falls to 11.0% on randomly perturbed images for a classifier constructed from density-sensitive routing.

APPENDIX B EXPERIMENTAL SETTINGS

The experiments compare adversarial defenses using specified perturbation sampling, feature-squeezing, classifier, and adaptive-threshold settings across Fashion MNIST and CIFAR-10.

  • Adversarial Training: Adversarial training uses FGSM and BIM examples generated from predicted rather than true labels to prevent label leaking.
  • Adversarial Training: Fashion MNIST samples ϵ_attack from a truncated absolute normal distribution with δ = 8 or 25, while CIFAR-10 fixes δ = 8.
  • Feature Squeezing: Feature squeezing reduces colors to 8 on Fashion MNIST and 32 on CIFAR-10, followed by a 2 × 2 median filter with reflective padding.
  • Models: The experiments use 62-layer ResNet and 16-layer VGG classifiers, with Fashion MNIST using one quarter as many feature maps.
  • Adaptive Threshold: Adaptive bits-per-dimension thresholds are 1.8 for Fashion MNIST and 3.2 for CIFAR-10, selected using validation data.

APPENDIX C IMAGE CLASSIFIER ARCHITECTURES∗

The same classifier architecture is used for CIFAR-10 and Fashion MNIST, but Fashion MNIST uses a different number of feature maps.

  • The same architecture is used for CIFAR-10 and Fashion MNIST, with different numbers of feature maps for Fashion MNIST.

APPENDIX D SAMPLED IMAGES FROM PIXELCNN

Figures 8 and 9 compare true dataset images with images generated by PixelCNN for Fashion MNIST and CIFAR-10.

  • Figure 8 contrasts true Fashion MNIST images sampled from the dataset with PixelCNN-generated images.
  • Figure 9 contrasts true CIFAR-10 images sampled from the dataset with PixelCNN-generated images.

APPENDIX E SAMPLED PURIFIED IMAGES FROM PIXELDEFEND

Figures 10 and 11 compare FGSM adversarial images with corresponding images purified by PixelDefend under specified attack and defense perturbation limits.

  • Figure 10 shows FGSM adversarial images above and corresponding PixelDefend-purified images below, with ϵ_attack = 25 and ϵ_defend = 32.
  • Figure 11 shows FGSM adversarial images above and corresponding PixelDefend-purified images below, with ϵ_attack = 8 and ϵ_defend = 16.
Loading 1710.10766v3…