PaperlayerView exact revision on arXiv

Source-linked AI summary

Accurate, reliable and fast robustness evaluation

Wieland Brendel, Jonas Rauber, Matthias Kümmerer, Ivan Ustyuzhaninov, Matthias Bethge

arXiv:1907.01003v2stat.MLcs.CRcs.CVcs.LGcs.NE

TL;DR

Even a few-pixel input manipulation can derail deep neural network predictions, threatening deployed models and exposing a gap between human and machine perception. The attacks combine gradient-based boundary estimates with iterative steps along the adversarial boundary, using norm-specific optimization. 1.15 versus 3.24 median L2 perturbation sizes on Madry-MNIST in untargeted attacks for the proposed attack and C&W; overall, the attacks outperform current state-of-the-art methods across L0, L1, L2, and L∞ in attack success, query efficiency, and reliability.

  • Problem

    Even a few-pixel input manipulation can derail deep neural network predictions, threatening deployed models and exposing a gap between human and machine perception.

    Source: §1 · p. 1

  • Method

    The attacks combine gradient-based boundary estimates with iterative steps along the adversarial boundary, using norm-specific optimization.

    Source: §1 · pp. 1–2, §3 · p. 3

  • Results

    1.15 versus 3.24 median L2 perturbation sizes on Madry-MNIST in untargeted attacks for the proposed attack and C&W; overall, the attacks outperform current state-of-the-art methods across L0, L1, L2, and L∞ in attack success, query efficiency, and reliability.

    Source: §5.1 · p. 5, §6 · p. 9

  • Takeaways & Limitations

    More reliable robustness evaluation can better distinguish effective defenses from ineffective ones, providing more signal for research toward robust models.

    Source: §6 · p. 9

    The approach assumes the adversarial boundary can be described by a differentiable equality constraint.

    Source: §3 · p. 3

Abstract

from arXiv · show

Throughout the past five years, the susceptibility of neural networks to minimal adversarial perturbations has moved from a peculiar phenomenon to a core issue in Deep Learning. Despite much attention, however, progress towards more robust models is significantly impaired by the difficulty of evaluating the robustness of neural network models. Today's methods are either fast but brittle (gradient-based attacks), or they are fairly reliable but slow (score- and decision-based attacks). We here develop a new set of gradient-based adversarial attacks which (a) are more reliable in the face of gradient-masking than other gradient-based attacks, (b) perform better and are more query efficient than current state-of-the-art gradient-based attacks, (c) can be flexibly adapted to a wide range of adversarial criteria and (d) require virtually no hyperparameter tuning. These findings are carefully validated across a diverse set of six different models and hold for L0, L1, L2 and Linf in both targeted as well as untargeted scenarios. Implementations will soon be available in all major toolboxes (Foolbox, CleverHans and ART). We hope that this class of attacks will make robustness evaluations easier and more reliable, thus contributing to more signal in the search for more robust machine learning models.

1 Introduction

Reliable robustness evaluation is difficult because existing attacks can miss minimal adversarial perturbations, including when gradients are masked. The paper introduces gradient-based attacks that follow the adversarial boundary and reports stronger attack success, query efficiency, and reliability than current state-of-the-art methods.

1 Introduction

  • Existing attacks can miss minimal adversarial perturbations because of poor hyperparameters, too few optimization steps, or masked gradients.This makes reliable evaluation of model robustness difficult.Source: §1 · p. 1
  • The attacks combine ideas from decision-based boundary attacks with gradient-based estimates of the boundary.They start from a distant adversarial image and move along the boundary toward the clean input.Source: §1 · p. 1, Figure 1 · p. 2
  • In almost all tested cases, the attacks outperform current state-of-the-art methods in attack success, query efficiency, and robustness to suboptimal hyperparameter settings.The evaluation covers L0, L1, L2, and L∞, targeted and untargeted attacks, and six models.Source: §1 · p. 2

2 Related work

Gradient-based attacks are widely used because they combine efficiency with a strong success rate, while different methods target different norms and update strategies.

2 Related work

  • Gradient-based attacks are widely used for robustness evaluation because of their efficiency and success rate relative to attacks with less model information.The class includes methods such as PGD, C&W, FGSM, and DeepFool.Source: §2 · p. 2
  • PGD and C&W are commonly treated as state of the art for L∞ and L2 attacks, respectively.Other methods address norms such as L0 and L1 or are described as weaker.Source: §2 · p. 2
  • Some PGD variants aim to improve effectiveness or query efficiency by changing the update rule to Adam or momentum.Source: §2 · p. 2

3 Attack algorithm

The attack starts from an adversarial input and iteratively moves along the boundary toward the clean input. At each step, it uses a local boundary estimate to choose a norm-specific step subject to trust-region and pixel constraints.

3 Attack algorithm

  • The attack is inspired by the decision-based boundary attack but uses gradients to estimate the local boundary between adversarial and non-adversarial inputs.Source: §3 · p. 2
  • Starting from an adversarial input, the algorithm seeks a step that reduces distance to the clean input while staying near the boundary.The step is constrained by a trust-region radius and valid pixel bounds.Source: §3 · p. 3
  • The step is chosen through a quadratic trust-region optimization problem that places the updated input approximately on the adversarial boundary.The local boundary estimate supplies the boundary constraint.Source: §3 · p. 3
  • Efficient iterative algorithms derived from the optimization problem support L0, L1, L2, and L∞ attacks; the added optimization has little runtime impact relative to PGD.Off-the-shelf solvers were considered impractical because of runtime and numerical instability in high dimensions.Source: §3 · p. 3
  • The adversarial boundary is assumed to be expressible as a differentiable equality constraint, with criteria for targeted or untargeted classification.Targeted criteria compare the ground-truth and target class log-probabilities; untargeted criteria use the minimum margin against another class.Source: §3 · p. 3
  • Experiments initialize from a distant adversarial point, binary-search between it and the clean input for at most 10 steps, then descend along the boundary.The starting point is selected as the closest differently classified or target-class sample in L2 distance.Source: §3 · p. 3

4 Methods

The evaluation compares the proposed attacks with state-of-the-art methods across models, attack criteria, and perturbation norms. It uses extensive hyperparameter tuning and evaluates success with measures suited to each norm.

4 Methods

  • The evaluation spans six models, targeted and untargeted scenarios, and L0, L1, L2, and L∞ metrics, with broad model and defense coverage.The comparison includes large-scale hyperparameter tuning for the attacks.Source: §4 · pp. 3–4
  • The attack comparison includes state-of-the-art methods across L0, L1, L2, and L∞, including PGD, AdamPGD, C&W, DDN, EAD, JSMA, and SparseFool.Source: §4 · p. 4
  • All attacks are tested across the models regardless of whether each model was defended against the norm being optimized.The stated goal is to assess applicability across a broad set of models.Source: §4 · p. 4
  • Each attack-model combination and sample is run five times with a range of hyperparameter settings, from one for SparseFool to 96 for C&W.Source: §4 · p. 5
  • The evaluation uses attack success rate for L∞ and median perturbation size for L0, L1, and L2.Smaller model accuracy or median perturbation sizes indicate better performance under the respective measures.Source: §4 · p. 5
  • Results use 1,000 validation samples, and L∞ thresholds are chosen so the best attack reaches roughly 50% accuracy for each model and scenario.Source: §4 · p. 5

5 Results

The attacks generally outperform state-of-the-art methods across tested models and both targeted and untargeted scenarios, with one stated exception. They also compare favorably in query efficiency and show limited sensitivity to hyperparameter choices.

5.1 Attack success

  • The attacks surpass state-of-the-art methods across tested models in both scenarios, except for Logitpairing in targeted L∞.On Madry-MNIST, untargeted L2 median perturbation size is 1.15 versus 3.24 for C&W; targeted L2 is 1.70 versus 4.79. For L1 and L0, the attack reaches close to 100% success on all models, with adversarials up to one to two orders smaller; EAD is up to six times worse on L1.Source: §5.1 · pp. 5–6

5.2 Query efficiency

  • On L2, the attack is more query-efficient than C&W and at least on par with DDN, while L∞ generally surpasses PGD and AdamPGD after around 10 queries.The L2 attack generally gets close to convergence in about 10 to 20 queries, while C&W often needs several hundred iterations; the first few queries for L∞ typically find an initial adversarial-boundary point, giving PGD a slight early advantage.Source: §5.2 · p. 8

5.3 Hyperparameter robustness

  • Using one hyperparameter instead of eight causes only a 4% performance drop; using one repetition instead of five causes around a 15%–19% drop.A shared trust-region hyperparameter across models causes no further performance drop, whereas C&W loses a further 16%. Shifting the optimal trust region two orders of magnitude in either direction changes performance by less than 15%, compared with almost 50% deterioration for C&W after a one-order shift.Source: §5.3 · p. 9

6 Discussion & Conclusion

The attacks outperform current state-of-the-art across four perturbation norms and may help distinguish effective defenses from ineffective ones. Their criteria can extend beyond classification when the boundary is expressible as a differentiable equality constraint.

6 Discussion & Conclusion

  • More reliable robustness evaluation has the potential to distinguish effective defenses from ineffective ones and accelerate progress toward robust models.The paper identifies the lack of reliable evaluation tools as an important obstacle; fewer than a handful of roughly two hundred proposed defenses are widely accepted as effective.Source: §6 · p. 9
  • The attacks outperform current state-of-the-art in attack success, query efficiency, and reliability on L0, L1, L2, and L∞.Moving along the adversarial boundary keeps them in a region with fairly reliable gradient information.Source: §6 · p. 9
  • The attacks can extend to other metrics and adversarial criteria when the optimization is efficient and the boundary has a differentiable equality constraint.This can make them suitable beyond targeted or untargeted classification tasks.Source: §6 · p. 9
  • The authors present the methods as a new standard intended to help practitioners and researchers find more robust machine learning models.Source: §6 · p. 9

Related papers

Loading 1907.01003v2…