PaperlayerView exact revision on arXiv

Source-linked AI summary

One pixel attack for fooling deep neural networks

Jiawei Su, Danilo Vasconcellos Vargas, Sakurai Kouichi

arXiv:1710.08864v7cs.LGcs.CVstat.ML

TL;DR

The paper examines whether deep neural networks can be fooled by modifying only one pixel, a low-dimensional alternative to perturbing every pixel. It proposes a differential-evolution black-box attack using probability-label feedback and reports successful attacks across multiple network types and datasets, including 16.04% of ImageNet test images.

  • Problem

    Prior work explored natural-image neighborhoods mainly by limiting perturbation length, while one-pixel attacks offer a contrasting low-dimensional view of CNN input space.

  • Method

    The paper uses differential evolution to generate one-pixel adversarial perturbations in a black-box setting requiring only probability labels, without gradients or network structures.

  • Results

    16.04% of ImageNet test images were successfully attacked by changing one pixel, while Kaggle CIFAR-10 attacks achieved 68.71%, 71.66%, and 63.53% success rates across three network structures.

  • Takeaways & Limitations

    One-dimensional perturbation vectors can find adversarial images for many natural images, and the vulnerability extends across network structures and image resolutions.

Abstract

from arXiv · show

Recent research has revealed that the output of Deep Neural Networks (DNN) can be easily altered by adding relatively small perturbations to the input vector. In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified. For that we propose a novel method for generating one-pixel adversarial perturbations based on differential evolution (DE). It requires less adversarial information (a black-box attack) and can fool more types of networks due to the inherent features of DE. The results show that 67.97% of the natural images in Kaggle CIFAR-10 test dataset and 16.04% of the ImageNet (ILSVRC 2012) test images can be perturbed to at least one target class by modifying just one pixel with 74.03% and 22.91% confidence on average. We also show the same vulnerability on the original CIFAR-10 dataset. Thus, the proposed attack explores a different take on adversarial machine learning in an extreme limited scenario, showing that current DNNs are also vulnerable to such low dimension attacks. Besides, we also illustrate an important application of DE (or broadly speaking, evolutionary computation) in the domain of adversarial machine learning: creating tools that can effectively generate low-cost adversarial attacks against neural networks for evaluating robustness.

I. INTRODUCTION

The paper studies whether DNNs can be fooled by changing only one pixel and proposes a differential-evolution attack requiring only probability-label feedback. It reports effectiveness across CIFAR-10 and ImageNet models while framing few-pixel perturbations as low-dimensional ways to probe DNN input spaces.

  • Contribution: One-pixel attacks target AllConv, NiN, and VGG networks trained on CIFAR-10 using only probability-label feedback.The proposed method is presented as a black-box attack without gradients or network structures.
  • Effectiveness: 68.71%, 71.66% and 63.53% success rates were obtained on Kaggle CIFAR-10 across three network structures.On the original CIFAR-10 dataset, success rates were 22.60%, 35.20% and 31.40%.
  • Effectiveness: 16.04% of ImageNet test images were attacked by changing one pixel in the BVLC AlexNet model.The ImageNet experiments used images with resolution 227x227.
  • Motivation: Few-pixel perturbations explore low-dimensional slices of the high-dimensional DNN input space, contrasting with perturbations distributed across many pixels.The paper presents one-pixel attack as an extreme case of several-pixel attack and contrasts it with universal perturbation, which modifies every pixel.
  • Motivation: Limiting modifications to very few pixels is proposed as an empirical way to reduce perceptibility without imposing additional constraints or complex cost functions.The paper notes that previous work did not guarantee complete imperceptibility.

II. RELATED WORKS

Prior work established DNN sensitivity to adversarial perturbations, developed gradient-based and black-box attacks, and proposed defenses. The paper positions one-pixel attacks as a more extreme setting connected to geometric analysis of DNN decision boundaries and input spaces.

  • Adversarial attacks: Gradient-based methods use back-propagation, linearity assumptions, or Jacobian saliency maps to construct adversarial perturbations.These approaches identify effective perturbation directions or exploit assumed properties of DNN decision boundaries.
  • Defenses: Network distillation, adversarial training, and image-processing methods were proposed to mitigate DNN vulnerability to adversarial perturbations.Adversarial training adds adversarial images to training data to improve robustness against known adversarial images.
  • Black-box attacks: Black-box attacks avoid internal target-system information such as gradients, while earlier one-pixel work used one-pixel modification only as a starting point for attacks changing about 30 pixels.The paper distinguishes its one-pixel scenario from that prior approach.
  • Geometric analysis: Geometric studies examine DNN boundaries, finding that adversarial robustness evaluation can provide insight into otherwise difficult high-dimensional behavior.Related work also reports flat regions around natural images with only a few sensitive directions.

III. METHODOLOGY

The methodology formulates adversarial-image generation as a constrained optimization problem and restricts perturbations to a small number of input dimensions. One-pixel attacks therefore search axis-parallel, one-dimensional slices, while allowing arbitrary modification strength along the selected dimension.

  • Problem Description: Adversarial-image generation seeks an additive perturbation e(x) that changes a correctly classified image toward a target class under a maximum-modification constraint L.The classifier receives an n-dimensional image x, and f_t(x) denotes the probability of its original class.
  • Problem Description: The optimization must determine which dimensions to perturb and the modification strength assigned to each selected dimension.The proposed formulation differs by restricting the number of modified dimensions to a small value d.
  • Few-pixel formulation: For one-pixel attack, d = 1 and all other components of e(x) remain zero.Previous approaches commonly modify part or all of the dimensions, whereas this formulation selects only d dimensions.
  • Geometric interpretation: One-pixel perturbation moves an image along one axis-parallel direction, while three- and five-pixel attacks move within three- and five-dimensional cubes.Few-pixel attacks therefore search low-dimensional slices of the original input space.
  • Geometric interpretation: Unlike conventional attacks that constrain accumulated modification across many pixels, few-pixel attacks constrain the number of modified pixels without limiting modification strength.Figure 4 illustrates one- and two-dimensional search slices in a three-dimensional input space.

B. Differential Evolution

Differential evolution is used as a gradient-free, population-based optimizer for one-pixel adversarial attacks, requiring only probability-label feedback. Its diversity-preserving search and simple classifier-independent formulation support constrained perturbation generation.

  • Differential evolution is a population-based optimizer for complex multimodal problems that evolves children from parent solutions and retains fitter candidates.Its selection mechanism simultaneously preserves population diversity and improves fitness.
  • DE avoids gradients and differentiability requirements, enabling optimization when the objective is unknown, noisy, dynamic, or nondifferentiable.This broadens applicability beyond gradient-based methods.
  • DE is relatively less subject to local minima than gradient descent or greedy search, a useful property under the strict one-pixel constraint.
  • The attack requires only probability labels and no gradients or network structures, while directly increasing target-class probability values.The approach is independent of the classifier used.
  • Each candidate solution encodes a fixed number of pixel perturbations as coordinate-and-RGB tuples, with 400 parents and 400 children generated per iteration.One perturbation modifies one pixel.
  • The search runs for at most 100 iterations, with early stopping at target-class probability above 90% for targeted CIFAR-10 attacks or true-class probability below 5% for non-targeted ImageNet attacks.CIFAR-10 coordinates use U(1, 32), ImageNet coordinates use U(1, 227), and RGB values use N(µ=128, σ=127).

IV. EVALUATION AND RESULTS

The evaluation measures attack effectiveness on CIFAR-10 and ImageNet using success rate, confidence, target-class coverage, and original-target class pairs. Experiments use three CIFAR-10 networks, targeted and non-targeted settings, and comparisons between Kaggle and original CIFAR-10 data.

  • The evaluation uses CIFAR-10 and ImageNet datasets and introduces multiple metrics for measuring attack effectiveness.
  • Success rate measures arbitrary-class misclassification for non-targeted attacks and probability of reaching a specific target class for targeted attacks.
  • Adversarial probability-label confidence averages the target-class probability across successful perturbations, measuring the classifier’s confidence in misclassification.
  • Number of target classes counts how many destination classes each natural image can reach, including images that cannot be perturbed to another class.
  • Number of original-target class pairs counts how often each source-to-destination class pair is attacked.
  • Three CIFAR-10 classifiers—All convolution network, Network in Network, and VGG16—are evaluated with targeted and non-targeted attacks on 500 randomly selected natural images per attack.
  • The Kaggle CIFAR-10 test set contains visually inspected modifications such as duplication, rotation, clipping, blurring, and random bad pixels; original CIFAR-10 results are also reported for comparison.

B. ImageNet

The one-pixel attack generalizes to ImageNet’s larger images under the same differential-evolution settings, despite a search space 50 times larger than CIFAR-10. On BVLC AlexNet, it perturbed 16.04% of sampled test images, with 22.91% target-class confidence on average.

  • Experimental setting: The ImageNet experiment used the same differential-evolution parameter settings as CIFAR-10 despite a 50-times-larger search space.The evaluation kept the same number of evaluations rather than increasing them proportionally.
  • Experimental scope: The ImageNet results were conducted only with one-pixel attacks to test whether tiny modifications could fool larger images while remaining computationally tractable.The images had resolution 227x227, compared with CIFAR-10 images at 32x32.
  • Interpretation: In each successful attack, the target class had the highest probability label, although the average confidence remained relatively low.The remaining 999 classes formed an almost uniform soft-label distribution.

2) Number of Target Classes (Non-targeted Attack Results):

One-pixel perturbations can reach multiple target classes, while increasing the perturbation dimensionality to three or five pixels substantially expands the reachable classes. Vulnerability varies by original-target class pair and network, with some directional asymmetries and class-specific robustness.

  • Reachable target classes: One-pixel attacks can perturb a fair number of natural images to two, three, or four target classes.Increasing the number of modified pixels makes perturbation to more target classes highly probable.
  • Class-pair vulnerability: Some original-target class pairs are much more vulnerable than others; cat images, for example, more easily reach dog than automobile.The heat maps encode original and target CIFAR-10 classes for one-, three-, and five-pixel attacks.
  • Effect of perturbation dimensionality: Increasing perturbations from one to three and five pixels significantly increases both attack success rates and reachable target classes.The paper describes this as breaking robustness along additional perturbation dimensions.
  • Class-pair asymmetry: The heat maps are approximately symmetric overall, but exceptions include ship when attacking NiN and deer when attacking AllConv with one pixel.Some class directions are easier in one direction than the reverse.
  • Security implication: Vulnerable original classes may be exploitable by malicious users, but the exceptions are not shared across networks, making exploitation hard under the given attacks.The paper attributes this boundary-related phenomenon as not shared between the evaluated networks.

4) Time complexity and average distortion:

The study measures attack cost using model evaluations and single-channel pixel distortion, and compares differential evolution with random search under equal evaluation budgets. Differential evolution achieves higher attack accuracy, especially on VGG16, while vulnerable pixels are common in some networks.

  • Cost measures: Attack cost is measured by the number of model evaluations and the average modification across the three color channels of the attacked pixel.The authors avoid the Lp norm because of its limited effectiveness for this setting.
  • Comparison setup: 80,000 evaluations were allocated equally to differential evolution and random search.The comparison used Kaggle CIFAR-10 and one-pixel non-targeted attacks.
  • Differential evolution versus random search: Differential evolution was 19.01%, 29.94%, and 47.96% more efficient than random search on AllConv, NiN, and VGG16, respectively.The largest efficiency difference occurred for VGG16.
  • Illustrative case: A dog image was perturbed to all other nine classes with approximately 100% confidence in the AllConv one-pixel attack.The example illustrates an extreme case of multi-class vulnerability.
  • Random-search behavior: Random search still found label-changing pixels 49.70% and 41.72% of the time on AllConv and NiN, respectively.For VGG16, random search achieved only 15.57%, indicating less common vulnerable pixels there.

6) Change in fitness values:

During differential-evolution attacks, fitness sometimes drops abruptly and sometimes decreases smoothly, while average fitness decreases monotonically across generations. BVLC AlexNet is harder to fool because its fitness decreases less.

  • Fitness evolution: Average fitness decreases monotonically over generations, indicating that the evolution progresses as expected.Fitness is the probability label of the true class, which the non-targeted attack minimizes.
  • Fitness evolution: Individual fitness curves can drop abruptly between generations or decrease smoothly.The figure tracks 30 successful Kaggle CIFAR-10 attacks and 15 successful ImageNet attacks.
  • Network differences: BVLC AlexNet is harder to fool because its fitness values show smaller decreases during evolution.This comparison concerns the average fitness behavior across the evaluated networks.

V. RESULTS ON ORIGINAL CIFAR-10 TEST DATA

On original CIFAR-10, one-pixel attacks were harder than on the noisier Kaggle dataset, yet vulnerable images could still reach multiple target classes and showed recurring class-pair patterns.

  • The original CIFAR-10 evaluation used 500 random non-targeted-attack images and 300 targeted-attack images, with modified network structures and retrained classifiers.An early-stop criterion was also used during attacks.
  • Original CIFAR-10 networks showed lower one-pixel attack rates and confidence than on Kaggle CIFAR-10, consistent with higher classification accuracy and confidence.Network robustness varied across datasets; Network in Network remained comparatively vulnerable, while VGG became more vulnerable in this setting.
  • Targeted attacks commonly perturbed vulnerable images to more than one class, suggesting proximity to multiple class boundaries.Figure 11 reports the percentage of natural images successfully perturbed to different numbers of target classes.
  • Successful perturbations repeatedly involved vulnerable original-target pairs, including dog-to-cat, with most pair counts approximately symmetric but ship-to-airplane asymmetric.The ship-to-airplane direction was frequent, whereas the reverse direction was not comparably frequent.
  • Some classes were frequently both original and target classes, and these class-level patterns were similar or identical to those found on Kaggle CIFAR-10.The paper notes that vulnerability as an original class often accompanied vulnerability as a target class.

VI. DISCUSSION

The discussion argues that one-pixel perturbations probe DNN decision boundaries along very few dimensions and demonstrate vulnerability across architectures and image sizes, while exposing robustness and detection trade-offs.

  • A. Adversarial Perturbation: One-pixel attacks show that considerable label changes can occur by moving natural images along only a few input dimensions, rather than relying on perturbations across many dimensions.The authors state that this challenges the necessity of explaining sensitivity through accumulation of small perturbations across many dimensions.
  • A. Adversarial Perturbation: The proposed vulnerability generalized across different CNN structures and image sizes, according to the experimental results.The comparison table is described as suggesting that one pixel can create adversarial images from most natural images.
  • A. Adversarial Perturbation: Attack success rates could improve with more differential-evolution iterations, larger initial candidate sets, or more advanced evolutionary algorithms.The experiments used a low number of iterations and a relatively small initial candidate set to mimic an attacker.
  • B. Robustness of One-pixel Attack: One-pixel attacks are not expected to provide significantly better detection robustness than other L0 attacks against methods that detect adversarial perturbations.The paper frames this as a trade-off of a low-cost, easy-to-implement attack.
  • B. Robustness of One-pixel Attack: Detection schemes can increase response time because they require preprocessing and reclassification, and their effect on classification accuracy remains incompletely understood.These costs may matter in real-time applications such as security cameras and autonomous driving.
  • B. Robustness of One-pixel Attack: Detection does not solve the underlying issue that neural networks may fail to recognize visually similar images when small adversarial perturbations are present.The authors present new attacks as a way to emphasize different vulnerabilities and improve understanding.

VII. FUTURE WORK

Future work includes improving evolutionary search, studying evolutionary computation more broadly, and extending one-pixel attacks to other domains.

  • Recent developments in evolutionary strategies, including Adaptive DE and CMA-ES, could improve the efficiency and accuracy of the current attack.The paper identifies these as variants or related developments within evolutionary strategies.
  • Evolutionary computation may address adversarial machine-learning vulnerabilities through flexible models, including neuroevolution of network weights and topology.The discussion also mentions adaptive self-organizing and novelty-organizing classifiers.
  • The one-pixel attack could potentially be extended to natural-language processing and speech recognition, which the paper leaves for future work.
Loading 1710.08864v7…