PaperlayerView exact revision on arXiv

Source-linked AI summary

Sparse and Imperceivable Adversarial Attacks

Francesco Croce, Matthias Hein

arXiv:1909.05040v1cs.LGcs.CRcs.CVstat.ML

TL;DR

Sparse adversarial attacks can threaten safety because they alter few pixels while often making large, detectable pixelwise changes. The paper introduces a black-box l0 attack and constrained PGD variants using image-adaptive bounds, reporting stronger or competitive sparse attacks and less perceivable perturbations. It also applies the constrained PGD framework to adversarial training.

  • Problem

    Sparse attacks are potentially dangerous, but unconstrained pixel changes are often clearly visible, while global componentwise bounds can be too restrictive in homogeneous regions.

  • Method

    The paper develops a randomized logits-based black-box attack and PGD extensions for l0, l0+l∞, and l0+σ-map attack models.

  • Results

    The proposed black-box attack outperforms existing l0-attacks, while CornerSearch achieves the smallest mean and median modification on Restricted ImageNet but has lower success and higher runtime than SparseFool.

  • Takeaways & Limitations

    Locally constrained sparse attacks can produce perturbations that are harder to perceive, and constrained PGD enables adversarial training against sparse and imperceivable manipulations.

  • Takeaways & Limitations

    Global componentwise bounds are image-agnostic and must be very small to remain inconspicuous in homogeneous regions; the paper therefore uses local bounds and avoids axis-aligned edges.

Abstract

from arXiv · show

Neural networks have been proven to be vulnerable to a variety of adversarial attacks. From a safety perspective, highly sparse adversarial attacks are particularly dangerous. On the other hand the pixelwise perturbations of sparse attacks are typically large and thus can be potentially detected. We propose a new black-box technique to craft adversarial examples aiming at minimizing $l_0$-distance to the original image. Extensive experiments show that our attack is better or competitive to the state of the art. Moreover, we can integrate additional bounds on the componentwise perturbation. Allowing pixels to change only in region of high variation and avoiding changes along axis-aligned edges makes our adversarial examples almost non-perceivable. Moreover, we adapt the Projected Gradient Descent attack to the $l_0$-norm integrating componentwise constraints. This allows us to do adversarial training to enhance the robustness of classifiers against sparse and imperceivable adversarial manipulations.

1. Introduction

Neural networks can make wrong decisions after very small adversarial changes, creating safety risks in systems such as autonomous driving and medical diagnosis. This paper focuses on sparse attacks and proposes black-box and constrained methods to make them both effective and less visible.

  • Motivation: Very small adversarial changes can cause correctly classified inputs to receive wrong decisions, threatening safety-critical applications.The passage names autonomous driving and medical diagnosis as examples where such vulnerabilities may lead to fatal failures.
  • Prior work: Existing attacks include white-box methods with model access and black-box methods using classifier outputs or confidence scores.Common distance measures include l∞ and l2, while other attacks use loss or confidence objectives within an ϵ-ball.
  • Problem: Sparse attacks seek to change the smallest number of pixels, but their unconstrained pixel changes are often clearly visible.The paper motivates combining sparsity with componentwise constraints to reduce detectability.
  • Contributions: The paper proposes a randomized black-box local-search attack that it reports as outperforming existing l0-attacks.It also develops projection algorithms for l0-balls and constrained intersections to extend PGD.
  • Contributions: The proposed framework combines l0 sparsity with locally adaptive componentwise constraints rather than image-agnostic global bounds.The local constraints account for image structure and target sparse perturbations that are harder to perceive.

2. Sparse and imperceivable adversarial attacks

The paper formulates sparse attacks as finding adversarial inputs that change a classifier’s decision while modifying as few pixels as possible. It adds image-specific constraints that favor changes in high-variation regions, preserve color properties, and avoid axis-aligned edges, making sparse perturbations less perceivable.

  • Sparse l0-attack: An l0-attack seeks the smallest number of pixel changes that alter the classifier’s decision.For grayscale images, pixels are represented in [0, 1]^d; for color images, each pixel has three RGB channels.
  • Sparse and imperceivable attack: Unlike l∞ attacks with small per-pixel changes, unconstrained l0 attacks can produce pixels with colors that differ sharply from their surroundings and are easily visible.The paper motivates a sparse-but-imperceivable attack model as a more realistic alternative to perturbing every pixel slightly.
  • Sparse and imperceivable attack: Global componentwise bounds are image-agnostic, requiring a very small δ to remain inconspicuous even in homogeneous regions such as sky.The paper therefore replaces these global bounds with local bounds that account for image structure.
  • Sparse and imperceivable attack: The constraints avoid changes along coordinate-aligned edges and limit color changes toward intensity adjustment while approximately preserving saturation.For color images, the construction operates across channels so pixels can be modified without substantially changing their color character.
  • Sparse and imperceivable attack: The proposed l0 + σ-map attack uses local channel variation, computed from each pixel and its immediate horizontal and vertical neighbors, to set image-specific change bounds.These local statistics enlarge the allowable adversarial-example space in variable regions while preventing perturbations in zero-variance areas.

3. Algorithms for sparse (and imperceivable) attacks

The paper develops randomized black-box and PGD-based methods for generating l0, l0 + l∞, and l0 + σ attacks. The black-box procedure ranks one-pixel changes and samples increasingly large multi-pixel candidates until classification changes.

  • Two complementary methods generate l0, l0 + l∞, and l0 + σ attacks: a randomized logits-based black-box attack and a PGD generalization.The paper derives projection algorithms for the corresponding constraint sets.
  • Randomized black-box attack: One-pixel modifications are evaluated first, and the attack stops immediately when a single changed pixel alters the decision.Otherwise, the method uses the resulting rankings to guide later sampling.
  • Attack-specific modifications: CornerSearch tests eight RGB-cube corners per pixel for l0 attacks, while l0 + l∞ uses corners of the componentwise [−ϵ, ϵ] cube.Grayscale variants test the two extreme values or xi ± ϵ, respectively.
  • Randomized black-box attack: Multi-pixel candidates sample from the top N ranked one-pixel changes using a distribution biased toward higher-ranked modifications.Candidates target each class separately plus an untargeted change, and the process repeats Niter times.
  • Randomized black-box attack: The number of modified pixels increases gradually up to kmax until the candidate receives a classification different from the original class.This search targets adversarial examples differing from the original image in as few pixels as possible.

4. PGD for sparse and imperceivable attacks

The paper extends PGD beyond the usual l∞ constraint by projecting onto sparse and componentwise-constrained perturbation sets. These projections select feasible pixel changes with the largest gains, enabling l0, l0 + l∞, and l0 + σ-map PGD attacks.

  • PGD formulation: The PGD objective uses the loss of the correct label over a constrained set C(x), with cross-entropy given as the usual loss choice.The paper presents this formulation as the basis for projected attacks under the new constraint sets.
  • Motivation and extension: PGD is commonly used for adversarial training with an l∞-ball, but the paper extends it to l0 and l0 + l∞ attack models.The extension requires efficient projection algorithms for the relevant allowed sets.
  • Motivation and extension: The extended PGD framework also supports l0 + σ-map constraints through projection onto the intersection of the l0-ball and componentwise bounds.The resulting attack is called σ-PGD and produces sparse, imperceivable perturbations.
  • Projection algorithm: Projection first clips each pixel independently to its componentwise bounds, then ranks pixels by their gains.The final projected point changes the k pixels, or fewer when fewer than k pixels have positive gain, with the largest gains.
  • Projection sets: The projection set C(x) combines an l0-ball, an l∞-ball around x, and the valid-input box [0, 1]d×3.This defines the feasible perturbations for the l0 + l∞ model.

5. Experiments

The experiments evaluate sparse attacks, constrained sparse-and-imperceivable attacks, PGD-based attacks, and adversarial training across MNIST, CIFAR-10, and Restricted ImageNet. CornerSearch generally reduces changed pixels, σ-CornerSearch improves imperceptibility, and l0-focused training improves robustness in supported settings.

  • Evaluation of l0-attacks: CornerSearch achieves the lowest median number of changed pixels among compared l0-attacks, although its fixed pixel budget can prevent success on every example.The comparison includes LocSearchAdv, Pointwise Attack, Carlini-Wagner l0, SparseFool, and JSMA.
  • Evaluation of l0-attacks: 0.89% of MNIST pixels and 0.2% of CIFAR-10 pixels need changing for at least half of the test images attacked by CornerSearch.These percentages refer to the median number of changed pixels relative to the image sizes.
  • Evaluation of l0-attacks: PGD0 outperforms SparseFool and is the best evaluated attack on MNIST for k ≥4, while approaching CornerSearch on CIFAR-10.PGD0 uses projections onto an l0-ball with radius k and the image box constraint, requiring separate evaluation for each k.
  • Sparse and Imperceivable manipulations: σ-CornerSearch produces sparse, imperceivable modifications, whereas unconstrained l0 changes are easiest to recognize and l0+l∞ changes can remain visible.The σ-map restricts changes to regions where they are difficult to perceive and avoids certain axis-aligned edge changes.
  • Adversarial training: On CIFAR-10, the l0-at model is more than 3 times more resistant to l0-attacks than the other trained models, while l0+σ-at is least vulnerable to l0+σ attacks.On MNIST, l2- and l0-trained models are most robust against l0-attacks; l∞ training does not improve l0 robustness.

A.2. Gray-scale images

This section describes gray-scale image handling and reports model accuracy for the paper’s introduced models.

  • A.2. Gray-scale images: Gray-scale images are included in the paper’s projection-based attack treatment.
  • A.2. Gray-scale images: The projection problem is equivalent to intersecting an l0-ball with box constraints.
  • A.2. Gray-scale images: Table 4 reports test-set accuracy for the models introduced in the paper.

B. Experiments

The experiments use trained neural-network models whose test or validation accuracies are reported for the models introduced in the paper.

  • B. Experiments: Test accuracy, or validation accuracy for Restricted ImageNet, is reported for every model introduced in the paper.

B.1. Evaluation of l0-attacks

The evaluation compares sparse attacks, reports their runtime, and examines the stability of CornerSearch across repeated runs.

  • B.1. Evaluation of l0-attacks: PGD0 runs fastest per image among the listed attacks on MNIST and CIFAR-10, at 0.06s and 0.19s per threshold, respectively.On ImageNet, PGD0 takes 13s per threshold, compared with 953s for CornerSearch and 17s for SparseFool.
  • B.1. Evaluation of l0-attacks: MNIST and CIFAR-10 attacks show stable repeated-run performance, with success-rate variation of ±0.13 and ±0.12 percentage points, respectively.

B.2. Sparse and Imperceivable manipulations

The experiments show that σ-map constraints can make sparse adversarial modifications less perceptible by avoiding uniform regions, non-homogeneous colors, and axis-aligned edges.

  • B.2. Sparse and Imperceivable manipulations: Axis-aligned edge changes remain evident and easy to detect, motivating the heuristic for selecting imperceivable perturbation regions.
  • B.2. Sparse and Imperceivable manipulations: The l0 + σ-map attack produces sparse, imperceivable perturbations, unlike l0 and l0 + l∞ attacks that visibly alter backgrounds or neighboring colors.
  • B.2. Sparse and Imperceivable manipulations: On MNIST, σ-CornerSearch avoids modifying background pixels and uniform-color interior regions of the digit.The comparison includes CornerSearch and l0 + l∞ attacks, with δ = 0.2 for the latter.

B.3. Adversarial training

The experiments use convolutional classifiers trained with Adam, with distinct architectures and training setups for MNIST and CIFAR-10. CIFAR-10 adversarial training against the l0-norm uses k = 20 pixels.

  • MNIST: MNIST uses two convolutional layers with max-pooling, followed by two dense layers, and classifiers are trained for 100 epochs with Adam.The plain and l∞-adversarially trained models come from Madry et al., while the l2-adversarially trained model is trained using the plain gradient direction for PGD.
  • CIFAR-10: CIFAR-10 uses an eight-layer convolutional network with feature-map widths from 96 to 384, followed by dense layers of 1200 and 10 units.ReLU is used after each layer except the final one, with random crops and mirroring for data augmentation.
  • CIFAR-10: CIFAR-10 adversarial training with respect to the l0-norm uses k = 20 as the number of pixels to be changed.

C. Adversarial examples of σ-PGD

The section compares σ-CornerSearch and σ-PGD by visualizing their adversarial images and rescaled perturbations. σ-PGD can produce more visible changes at the same κ, but lowering κ improves visibility at the cost of success rate.

  • Comparison setup: The comparison displays original images, adversarial images, and perturbations rescaled to [0,1] for σ-CornerSearch and σ-PGD.For σ-PGD, results with a smaller κ are also reported.
  • Visibility: At the same κ, σ-PGD gives more visible manipulations than σ-CornerSearch.The authors attribute this partly to σ-PGD always using the full budget of k pixels, unlike σ-CornerSearch.
  • Visibility: Decreasing κ can make σ-PGD changes less visible, but reduces its success rate.The authors also associate σ-PGD’s visibility with maximizing loss and potentially modifying neighboring pixels in opposite directions.

D. Propagation of sparse perturbations

The paper visualizes sparse perturbations across MNIST, CIFAR-10, and Restricted ImageNet, comparing attack outputs, perturbation maps, and modified-pixel locations. It also examines how sparse adversarial examples alter representations through network layers.

  • Propagation through the network: Figure 11 tracks differences between original and adversarial examples across network units, with vertical segments separating layers from input to output.Each color represents an image in the test set.
  • Cross-dataset visualizations: Figures 6–8 compare CornerSearch, l0 + l∞-attack, and σ-CornerSearch across MNIST, CIFAR-10, and Restricted ImageNet.The visualizations include adversarial examples, perturbations or zooms, and maps showing which pixels were modified.
  • σ-attack comparisons: Figures 9 and 10 compare σ-CornerSearch with σ-PGD using adversarial images and perturbations rescaled to [0,1].The MNIST comparison uses κ = 0.8 and 0.6 for σ-PGD with sparsity k = 50, while the CIFAR-10 comparison uses κ = 0.4 and sparsity k = 100.
Loading 1909.05040v1…