PaperlayerView exact revision on arXiv

Source-linked AI summary

Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification

Xiaoyu Cao, Neil Zhenqiang Gong

arXiv:1709.05583v4cs.CRcs.LGstat.ML

TL;DR

DNNs are vulnerable to testing-time evasion attacks that can cause incorrect predictions in safety-critical applications. The paper proposes region-based classification, which aggregates predictions within a centered hypercube, and evaluates it on MNIST and CIFAR-10. It reports unchanged benign-example accuracy and significantly greater robustness than existing methods across various attacks.

  • Problem

    DNNs can be manipulated at testing time with small carefully crafted noises, threatening their use in safety- and security-critical applications.

  • Method

    Region-based classification predicts an example’s label by aggregating DNN predictions for sampled points in a hypercube centered at that example.

  • Results

    The method achieves the same benign-example accuracy as point-based classification and is significantly more robust to various evasion attacks on MNIST and CIFAR-10.

  • Takeaways & Limitations

    Region-based classification mitigates evasion attacks without sacrificing classification accuracy on benign examples.

  • Takeaways & Limitations

    The evaluation identifies generating adversarial examples that remain robust to random noise as an interesting future direction.

Abstract

from arXiv · show

Deep neural networks (DNNs) have transformed several artificial intelligence research areas including computer vision, speech recognition, and natural language processing. However, recent studies demonstrated that DNNs are vulnerable to adversarial manipulations at testing time. Specifically, suppose we have a testing example, whose label can be correctly predicted by a DNN classifier. An attacker can add a small carefully crafted noise to the testing example such that the DNN classifier predicts an incorrect label, where the crafted testing example is called adversarial example. Such attacks are called evasion attacks. Evasion attacks are one of the biggest challenges for deploying DNNs in safety and security critical applications such as self-driving cars. In this work, we develop new methods to defend against evasion attacks. Our key observation is that adversarial examples are close to the classification boundary. Therefore, we propose region-based classification to be robust to adversarial examples. For a benign/adversarial testing example, we ensemble information in a hypercube centered at the example to predict its label. In contrast, traditional classifiers are point-based classification, i.e., given a testing example, the classifier predicts its label based on the testing example alone. Our evaluation results on MNIST and CIFAR-10 datasets demonstrate that our region-based classification can significantly mitigate evasion attacks without sacrificing classification accuracy on benign examples. Specifically, our region-based classification achieves the same classification accuracy on testing benign examples as point-based classification, but our region-based classification is significantly more robust than point-based classification to various evasion attacks.

1 INTRODUCTION

DNNs perform strongly across AI tasks but remain vulnerable to small testing-time perturbations that can cause harmful misclassification. The paper proposes region-based classification and evaluates it as a defense against evasion attacks.

  • Motivation: Small crafted noises can turn correctly classified benign examples into adversarial examples while preserving their human-perceived label.These vulnerabilities constrain safety-critical uses such as self-driving cars, where misrecognized signs can cause collisions.
  • Evaluation: <20% and 7% targeted CW success rates were achieved on MNIST and CIFAR-10, respectively, versus 100% for three comparison methods on both datasets.The comparison methods were standard point-based classification, adversarial training, and defensive distillation.
  • Implications: The paper recommends evaluating future powerful evasion attacks against region-based rather than standard point-based classification.This recommendation is stated as a future research direction after the reported robustness results.
  • Contributions: The paper measures adversarial examples and proposes region-based classification as a defense against evasion attacks.The method ensembles predictions from sampled points in a hypercube centered on each testing example, unlike point-based classification.
  • Evaluation: Region-based classification maintains benign-example accuracy while improving robustness against existing and strategically adapted evasion attacks.The evaluation uses MNIST and CIFAR-10 and compares the method with standard point-based classification and other defenses.

2 BACKGROUND AND RELATED WORK

This section defines DNN classification and evasion attacks, then describes representative targeted attacks and their distance or optimization objectives. It distinguishes targeted attacks, which seek a specified wrong label, from untargeted attacks, which seek any wrong label.

  • DNN Classification: A DNN predicts the label with the largest softmax probability, and its classifier partitions input space into regions associated with labels.The classifier is represented as C(x) = argmax_i F_i(x), while each class region contains points receiving that class label.
  • Evasion Attacks: Evasion attacks add noise to benign testing examples to induce incorrect predictions, moving examples between class regions.Targeted attacks seek a particular incorrect label, whereas untargeted attacks seek any incorrect label.
  • Targeted Evasion Attacks: Targeted attack formulations constrain the classifier to predict a chosen target label while minimizing a distance between benign and adversarial examples.The paper discusses L0, L2, and L∞ distances, measuring changed dimensions, Euclidean distance, and maximum coordinate change.
  • Targeted Evasion Attacks: T-FGSM and T-IGSM use gradient-sign updates to construct targeted adversarial examples under L∞ constraints.T-FGSM searches over ϵ, while T-IGSM iteratively adds small noise and clips examples to an L∞ϵ-neighborhood.
  • Targeted Evasion Attacks: T-JSMA, T-CW-L2, T-CW-L0, and T-CW-L∞ target different perturbation structures or norms through iterative or optimization-based procedures.The CW variants are tailored to L2, L0, and L∞ noise objectives, while T-JSMA targets small L0-norm noise.

2.3 Defenses Against Evasion Attacks

Prior defenses either detect and reform adversarial examples or retrain DNNs, while the paper positions region-based classification as a different defense direction. Existing approaches include detection, denoising, adversarial training, and distillation.

  • Detection and Denoising: Adversarial-example detectors add a binary classifier that distinguishes benign from adversarial inputs.Attackers can strategically modify attacks to evade both the original classifier and the detector.
  • Detection and Denoising: MagNet combines detection with denoising by rejecting inputs predicted adversarial and reforming inputs predicted benign.Its reformer is described as denoising the testing example before classification.
  • Defense Directions: The paper contrasts these defense lines with designing a new robust DNN classification method.The related-work overview identifies training new DNN methods as another defense direction.
  • Training-based Defenses: Adversarial training augments the training set with generated adversarial examples before learning the DNN.The supplied passage notes that adversarial training is not robust to adversarial examples unseen during training.
  • Training-based Defenses: Defensive distillation retrains a DNN using confidence-score vectors produced by an initially trained network as soft labels.A distillation temperature controls confidence scores during the two training sessions.

3 DESIGN GOALS

The paper seeks a defense that preserves benign-example accuracy while improving robustness to powerful evasion attacks. It explicitly does not aim to eliminate adversarial examples completely.

  • Accuracy: The first design goal is to avoid sacrificing classification accuracy on testing benign examples.The defense should maintain the high accuracy of the underlying DNN classifier.
  • Robustness: The second design goal is greater robustness than conventional DNN classifiers against state-of-the-art evasion attacks.The paper specifically cites CW attacks as an example of the attacks used to assess robustness.
  • Scope: The paper does not aim to completely eliminate adversarial examples, but to increase robustness without sacrificing benign-example accuracy.It states that existing methods do not satisfy both goals simultaneously.

4 MEASURING EVASION ATTACKS

Measurements on MNIST show that hypercubes around benign and most adversarial examples overlap most with the true-label class region, motivating region-based classification.

  • The study samples MNIST digit-0 benign and adversarial examples targeting labels 1–9 using T-CW-L2, T-CW-L0, and T-CW-L∞ attacks.Each attack generates one adversarial example for every target label.
  • For each example, 10,000 points are sampled uniformly from a length-r hypercube, classified by the DNN, and summarized in a label histogram.The experiments set r = 0.3.
  • Almost all sampled points around the benign example receive its true label, 0.
  • For most adversarial examples, a majority of sampled points are also assigned label 0, the examples’ true label.
  • The measurements indicate that hypercube ensembling can preserve correct predictions for benign examples and correct most adversarial examples.

5 OUR REGION-BASED CLASSIFICATION

Region-based classification predicts a testing example from the class region occupying the largest part of a surrounding hypercube, estimated by random sampling. The method selects the hypercube length using benign validation accuracy and evaluates robustness under white-box transfer and adaptive attacks.

  • 5 OUR REGION-BASED CLASSIFICATION: Region-based Classification (RC) transforms a point-based DNN classifier into a classifier that ensembles information from a region around each testing example.
  • 5.1 Region-based Classification: RC predicts the label whose class region has the largest intersection with the hypercube B(x,r) centered at the testing example.The intersection area is denoted Ai(x,r), and the prediction is argmax_i Ai(x,r).
  • 5.1 Region-based Classification: RC approximates class-region intersections by uniformly sampling m points from B(x,r) and majority-voting over the point-based classifier’s predictions.
  • 5.1 Region-based Classification: The hypercube length r controls the tradeoff between robustness to adversarial examples and benign-example accuracy, so RC searches for a maximal validation-preserving length.
  • 5.2 Evasion Attacks to Our RC Classifier: The threat model gives the attacker full knowledge of RC, while attacks are generated on the differentiable point-based component C and transferred to non-differentiable RC.
  • 5.2.2 New evasion atacks: An adaptive attack scales existing perturbation δ by constructing x′′ = x + (1 + α)δ, then clips the result to [0,1]^n.For L2 and L∞ norms, the perturbation norms increase by the factor 1 + α.

6.1 Experimental Setup

The evaluation uses MNIST and CIFAR-10, trains point-based DNN baselines, and compares them with region-based and other classifiers while varying the region length on MNIST.

  • Datasets: Evaluations use the standard MNIST and CIFAR-10 image datasets.
  • Datasets: For each dataset, 5,000 predefined training examples are sampled uniformly as the validation set for learning r.
  • Compared methods: The standard point-based DNN uses the Carlini–Wagner architecture for MNIST and a He et al. architecture for CIFAR-10.
  • Parameter setting: Figure 4 compares standard point-based and region-based DNN classification accuracies on MNIST validation data as r increases.
  • Parameter setting: For CIFAR-10, the region length is r = 0.02 and intersection areas are estimated with m = 1,000 sampled points.

6.2 Classification Accuracies

On benign test examples from MNIST and CIFAR-10, region-based DNN classification matches standard point-based accuracy, whereas adversarial training and distillation achieve lower accuracy.

  • The region-based DNN achieves the same benign-testing classification accuracy as the standard point-based DNN on both MNIST and CIFAR-10.
  • Adversarial-training and distillation DNNs achieve lower benign-testing accuracy than the standard point-based DNN.The accuracy differences are smaller for MNIST.
  • The authors report that adversarial training and distillation sacrifice benign-example classification accuracy for robustness.

6.3 Robustness to Existing Evasion Attacks

Across targeted, untargeted, high-confidence, and combined attacks, region-based classification generally lowers attack success and/or requires larger perturbations than point-based classification. The defense preserves benign accuracy, but its effectiveness varies across attack types and reported success rates can overestimate real high-confidence attack success.

  • Targeted Evasion Attacks: Region-based classification has significantly lower success rates than point-based classification for each targeted evasion attack.Some attacks produce larger noise and others smaller noise under the region-based classifier.
  • Targeted Evasion Attacks: 19% and 53% are the MNIST success rates of T-CW-L0 and T-JSMA, respectively, versus 100% for both attacks against point-based classification.The passage links smaller point-based attack noise with lower region-based success rates.
  • Targeted Evasion Attacks: Minimum-noise attacks such as CW attacks have low success rates against region-based classifiers because their adversarial examples remain close to the classification boundary.The observation contrasts their effectiveness against standard point-based classifiers with their lower success against region-based classifiers.
  • Targeted Evasion Attacks: L0-norm attacks achieve the highest success rates among existing attacks against region-based classifiers, but their added image spots may be easier for humans to perceive.The proposed explanation is that these examples may lie farther from the classification boundary.
  • High-confidence T-CW-L2: 70% more noise on MNIST and 100% more noise on CIFAR-10 are required for high-confidence T-CW-L2 to reach 100% success against region-based classification.On MNIST, the 100% success rate occurs around k = 20, where average noise is 70% larger than at k = 0.
  • High-confidence T-CW-L2: Reported high-confidence T-CW-L2 success rates are upper estimates because some generated examples satisfy the second success condition but not the first.Several examples in Figure 5 changed the true label and were hard for humans to recognize, so real success rates are lower than Figure 6 reports.
  • Combined and Untargeted Attacks: 55% and 76% are the MNIST and CIFAR-10 success rates of T-CA-L0 against region-based classification, versus 100% for all compared classifiers.For untargeted combined attacks, success rates remain high, although the region-based defense still requires larger noise; U-CA-L0 on CIFAR-10 requires around twice as much.

6.4 Robustness to New Evasion Attacks

The paper adapts CW attacks to test robustness against the region-based classifier, finding that increased noise can move targeted adversarial examples across class regions and reduce attack success.

  • The adapted CW attacks move adversarial examples farther from the classification boundary by increasing the noise parameter α.The parameter α controls the tradeoff between added noise and attack success rates.
  • The adapted targeted CW attacks cannot achieve 100% success rates for any setting of α.Success rates first increase and then decrease as excessive noise moves examples into regions with different labels.
  • 64% is the largest MNIST success rate for T-CW-L2-A at α = 1, when the average added noise is doubled.At a 50% success rate, the attack requires 25% more noise.
  • 85% is the CIFAR-10 success rate for T-CW-L∞-A at α = 1.8; α = 0.75 yields a 50% success rate.
  • For MNIST examples generated at α = 1, the region-based classifier predicts the target label, while some examples almost change the benign examples’ true labels.The results indicate that some examples satisfying the attack’s second success condition do not satisfy the first, making real success rates lower.

7 DISCUSSIONS

The discussion interprets region-based classification as randomized regional ensembling and identifies alternative regions, aggregation methods, and robust-adversarial-example generation as future directions.

  • The method can improve robustness against evasion attacks without sacrificing generalization performance on benign examples.
  • Other types of regions: The hypercube region is equivalent to an l∞-norm ball, motivating exploration of l_p-norm balls, hyperspheres, and manifold intersections.
  • Other types of regions: Majority vote ensembles regional predictions, while weighted aggregation is proposed as an alternative.
  • Randomization based defenses: Region-based classification can be interpreted as adding small random noise, classifying the noisy examples pointwise, and taking a majority vote.
  • Generating robust adversarial examples: State-of-the-art adversarial examples are not robust to small added noise, whereas benign examples retain classification accuracy under such noise.The paper proposes generating adversarial examples that are robust to random noise as future work.

8 CONCLUSION

The paper measures adversarial examples, proposes region-based DNN classification using a surrounding hypercube, and evaluates robustness on MNIST and CIFAR-10.

  • The study observes that adversarial examples are close to classification boundaries and their surrounding hypercubes significantly intersect true-label regions.
  • Region-based classification ensembles information in a hypercube around an example to predict its label.
  • Evaluations on MNIST and CIFAR-10 find greater robustness to various evasion attacks without sacrificing benign-example classification accuracy.
  • Future work includes different regions, alternative regional ensembling methods, and attacks that generate robust adversarial examples.
Loading 1709.05583v4…