PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics

Xin Li, Fuxin Li

arXiv:1612.07767v2cs.CV

TL;DR

Adversarial examples can alter deep-network predictions despite being visually indistinguishable, raising the problem of detecting inputs that differ from normal examples. The paper detects them with simple statistics of convolutional-layer outputs and a cascade classifier, finding cross-mechanism detection and recovery of many corrupted predictions through a small average filter.

  • Problem

    The paper addresses how to detect visually indistinguishable adversarial examples that can substantially change deep-network predictions and threaten applications.

  • Method

    The paper extracts simple statistics from convolutional-filter outputs across layers and uses them in a cascade classifier to discriminate normal from adversarial examples.

  • Results

    The detector generalizes from LBFGS-generated adversarials to EA-adversarials, detects more than 85% of adversarial examples, and many corrupted predictions are recovered by a 3 × 3 average filter.

  • Takeaways & Limitations

    Convolutional-layer output statistics provide a simple basis for adversarial detection, while the findings motivate studying how deep networks process local image structure.

  • Takeaways & Limitations

    A tail-distribution classifier can exceed 99% accuracy, but an adversary aware of the defense can deliberately remove its tail-distribution footprints.

Abstract

from arXiv · show

Deep learning has greatly improved visual recognition in recent years. However, recent research has shown that there exist many adversarial examples that can negatively impact the performance of such an architecture. This paper focuses on detecting those adversarial examples by analyzing whether they come from the same distribution as the normal examples. Instead of directly training a deep neural network to detect adversarials, a much simpler approach was proposed based on statistics on outputs from convolutional layers. A cascade classifier was designed to efficiently detect adversarials. Furthermore, trained from one particular adversarial generating mechanism, the resulting classifier can successfully detect adversarials from a completely different mechanism as well. The resulting classifier is non-subdifferentiable, hence creates a difficulty for adversaries to attack by using the gradient of the classifier. After detecting adversarial examples, we show that many of them can be recovered by simply performing a small average filter on the image. Those findings should lead to more insights about the classification mechanisms in deep convolutional neural networks.

1. Introduction

Adversarial examples can look unchanged to humans yet sharply alter deep-network predictions, motivating detection of inputs outside the normal distribution. The paper proposes using intermediate convolutional-filter statistics and a cascade classifier, and reports that simple average filtering can recover many corrupted predictions.

  • Motivation: Adversarial optimization can produce nearly imperceptible image changes that substantially alter deep-network predictions.Such examples have been reported across different deep networks and motivate this paper’s search for a defense.
  • Motivation: Adversarial examples pose safety risks for autonomous driving, robotics, and automatic identification systems when incorrect predictions affect real applications.The paper therefore emphasizes identifying whether an input comes from a normal or adversarial distribution.
  • Background: Distinguishing interpolation from extrapolation is difficult in the high-dimensional spaces used by deep learning because data are inherently sparse.The paper notes that generative-adversarial approaches may be confined to unsupervised settings or unavailable for some convolutional-network domains.
  • Approach: The proposed discriminative detector uses simple statistics from intermediate convolutional-filter outputs rather than treating the network as a black box at only its final layer.A cascade classifier combines features from multiple layers and can approach good accuracy with limited training examples.
  • Results: Experiments report that convolutional-filter statistics separate normal and adversarial examples well, including adversarials generated by a different mechanism from the training examples.The introduction also reports that many corrupted predictions can be recovered using a simple average filter.

2. Deep Convolutional Neural Networks

Deep convolutional networks transform inputs through convolutional filters and nonlinear layers, but small adversarial perturbations can drastically change their outputs. PCA and filter-output statistics reveal distributional differences between normal and adversarial examples, including contrasting behavior across informative and tail directions.

  • Architecture: A convolutional neural network applies learned filters to intermediate feature maps, followed by nonlinear transformations such as ReLU.The filters may be smaller than the feature-map input, and other common layers include pooling and normalization.
  • Adversarial Optimization: Small-norm perturbations can make an image receive a different predicted category while remaining indistinguishable to human observers.The adversarial construction trades off proximity to the original image against classification loss for a chosen target label.
  • Spectral Analysis: PCA at layer 14 of a VGG network was used to compare 50,000 normal ImageNet validation images with 4,000 adversarial examples.The analysis treats the layer as a nonlinear activation over a linear transformation, making PCA a tool for examining the learned representation.
  • Spectral Analysis: The first two principal directions do not separate adversarial from normal examples, whereas tail directions reveal adversarial deviations as large as 5 or 10 standard deviations.The tail analysis examined the 3,547-th and 3,844-th eigenvectors, where normal projections resemble Gaussian samples.
  • Spectral Analysis: Adversarial examples show lower extremal values and standard deviations on the first 500 −700 eigenvectors but higher values on the last 1,000 −1,500 eigenvectors.The reported contrast indicates that adversarial and normal examples differ across both informative and tail projection ranges.
  • Detection Limits: Tail statistics can separate adversarial from normal examples with more than 99% accuracy, but adversaries can deliberately remove tail footprints after learning the defense.Because the tail contributes little to classification, relying on it alone creates an exploitable weakness.

4. Identifying Adversarial Examples

The paper detects adversarial examples using statistics of convolutional-layer outputs and combines layer-specific classifiers in a cascade. The features are designed to be efficient and difficult to attack with gradients.

  • Feature Collection: Convolutional-layer outputs are treated as samples from a K-dimensional distribution D_m over feature vectors at each spatial location.For a W × H × K layer output, each pixel contributes a K-dimensional convolutional feature.
  • Feature Collection: The collected statistics include normalized PCA coefficients, minimum and maximum values, and the 25th, 50th, and 75th percentiles for each feature.PCA statistics are obtained after centering, projecting, and normalizing by per-dimension standard deviations.
  • Feature Collection: The resulting features are non-subdifferentiable, making gradient-based attacks against the detector difficult.The paper notes that derivative-free attacks do not currently scale to realistic image sizes.
  • Classifier Cascade: A cascade assigns a classifier to each convolutional layer, passing examples classified as positive to later stages while directly outputting negatives.The cascade is motivated by the finding that more than 80% of normal VGGNet examples were identified at the first layer with 100% precision.
  • Classifier Cascade: Each stage trains an SVM on normal examples and perturbed training examples, then removes normal predictions above a selected threshold from the remaining pool.The cascade multiplies stage-wise error rates, while thresholds are chosen to preserve high true-positive rates: 97% for AlexNet and 98% for VGG.

5. Related Work

Related work frames adversarial examples as products of optimization and gradient mechanisms, while also distinguishing detection, mitigation, abstention, and generative approaches. These approaches address robustness through different objectives and intervention points.

  • Adversarial Generation: Prior work formulates adversarial generation as optimization that changes a network’s category prediction while keeping the image close to its original input.A gradient-sign method further attributes this vulnerability to the magnifying effect of network linearities.
  • Adversarial Generation: Generative adversarial networks use a generator-discriminator game to create increasingly natural-looking images that evade classification.The cited work reports that such generated images resemble natural images.
  • Detection and Mitigation: Other approaches detect or mitigate adversarial examples through foveation, open-set networks, universum classifiers, or related mechanisms.The foveation approach assumes the input is already known to be adversarial, whereas open-set methods create an abstain option.
  • Detection and Mitigation: Self-aware learning and abstention differ from robust learning, which typically optimizes minimax loss directly under adversarial conditions.The cited literature also includes additional classification methods with an abstain option.

6. Experiments

The experiments evaluate convolutional-filter statistics and a cascade classifier for distinguishing normal images from adversarials across attack methods and network architectures. The detector generalizes from L-BFGS adversarials to EA-adversarials, which are especially easy to distinguish.

  • Experimental setup: The study evaluates L-BFGS and EA-adversarials on AlexNet and VGG-16, using non-overlapping training and testing sets.Training uses 2,000 L-BFGS adversarials and 45,000 normal images; testing includes held-out normals and adversarials from both methods.
  • Single-layer results: Single convolutional-layer statistics have discriminative power, but are not extremely effective for separating normal images from adversarials.The experiments therefore proceed beyond isolated-layer statistics to the cascade classifier.
  • Single-layer results: 97.34% overall classification accuracy is reached using only the first three convolutional layers for EA-adversarials, despite training only on L-BFGS adversarials.The authors attribute this separation to differences between EA-adversarial and natural-image statistics.
  • Cascade detection: 83.4% average accuracy and 90.7% AUC are obtained on AlexNet, outperforming OpenMax by over 9% in AUC and 11% in accuracy.The comparison uses the same training set for learning the Weibull distribution required by OpenMax.
  • Cascade detection: 90.665% average accuracy is obtained on VGG-16 over 20 random trials.The authors suggest that VGG-16’s greater depth imposes more constraints on layer statistics, aiding detection.
  • Cascade detection: 97.3% accuracy and 98.2% AUC are obtained on EA-adversarials, with more than 96% accuracy at a 0 false positive rate without training on them.The result indicates that the cascade classifier is rarely fooled by EA-adversarials in this experiment.
  • Statistics analysis: PCA, extremal-value, and median statistics differ more dramatically for EA-adversarials than for L-BFGS-adversarials relative to normal images.L-BFGS-adversarials show smaller extremal values than normal images, while EA-adversarial statistics visibly diverge from normal-image statistics.

7. Discussions

The discussion frames detection as self-aware learning with an abstain option and examines recovery by filtering detected adversarials. A simple average filter can recover many examples, while the findings motivate broader receptive fields and alternative normalization strategies.

  • 7.1. Self-Aware Learning with an Abstain Option: Self-aware learning allows a classifier to abstain instead of always making a prediction, while not requiring that it never make a mistake.The framework is presented as related to prior abstention-based approaches but differs from some by avoiding that requirement.
  • 7.1. Self-Aware Learning with an Abstain Option: The framework models testing inputs as a mixture of the training distribution P(x, y) and an adversarial distribution Q(x, y).The mixture weight Ω is unknown, and the classifier estimates whether an input belongs to the adversarial component.
  • 7.1. Self-Aware Learning with an Abstain Option: Abstaining is preferable to predicting on adversarial examples when its fixed cost e_a is lower than the prediction error e_q, while e_a must remain large enough to permit confident predictions.The decision balances the cost of abstention against the expected error on adversarial inputs.
  • 7.2. Image Recovery: A small 3 × 3 average filter recovers a large proportion of detected adversarial examples, while more complex foveation-based cancellation can achieve better results.The recovery experiment is intentionally simple and follows adversarial detection.
  • 7.2. Image Recovery: The recovery result suggests that current deep convolutional networks are locally focused because a simple 3 × 3 filter can cancel perturbations that affect the network’s overall result.The discussion proposes enlarging receptive fields and exploring classification approaches without softmax-type normalization.

8. Conclusion

The paper detects adversarial examples using convolutional-layer output statistics and a cascade classifier, then reports cross-mechanism detection and recovery with a 3 × 3 average filter.

  • The cascade classifier uses simple statistics on convolutional-layer filter outputs and detects more than 85% of adversarial examples.
  • Training on L-BFGS adversarials still enables detection of EA-adversarials, demonstrating transfer across generation methods.
  • A simple 3 × 3 average filter successfully recovered most corrupted images after adversarial detection.

1. Results on DeepFool

Using ResNet-50 and DeepFool adversarials, the cascade detector maintained more than 90% AUC, indicating that DeepFool adversarials remained detectable by the approach.

  • The experiment used 5,000 DeepFool adversarial images and an equal number of real images from the ILSVRC2012 validation set with ResNet-50 as the backbone.

2. Results on Self-Aware Learning

Self-aware learning uses abstention to avoid uncertain or adversarial inputs, trading retained predictions against the number of examples rejected.

  • The experiment varied the abstention parameter to seek thresholds that rejected adversarials while retaining predictions for real examples.
  • Lower thresholds abstained from all adversarial predictions but also rejected many real examples, while higher thresholds retained more real predictions but allowed some adversarials through.
  • The system also abstained from some normal examples on which the classifier was not confident.
  • With a high abstain ratio, prediction accuracy on normal examples was higher because uncertain normal examples were also rejected.

3. Images Classified Correctly and Incorrectly

The paper presents examples of correctly and incorrectly classified adversarial images, alongside detection performance and self-aware-learning behavior; visible trends in the images were difficult to identify.

  • The authors report that they could not observe particular visible trends in correctly and incorrectly classified images, possibly because adversarial changes are subtle.
  • The ROC curve for detecting DeepFool adversarials in ResNet-50 maintained more than 90% AUC.
  • In a mixture of half real and half adversarial examples, discarding nothing produced accuracy slightly below 50%, while abstaining on more examples substantially improved accuracy.
  • Figures 3 and 4 show examples from normal and L-BFGS-adversarial datasets, including adversarial images classified into categories such as water buffalo, ostrich, shower cap, and Angora.
  • Figures 5 and 6 show misclassified EA images and examples classified correctly by the classifier, with labels including dome, green mamba, traffic light, goblet, vase, and medicine chest.
  • Figure 7 shows images misclassified by the OpenSet method but correctly classified by the authors’ classifier, including L-BFGS-adversarial examples labeled American black bear and Angora.
Loading 1612.07767v2…