PaperlayerView exact revision on arXiv

Source-linked AI summary

Distributionally Adversarial Attack

Tianhang Zheng, Changyou Chen, Kui Ren

arXiv:1808.05537v3cs.LGcs.CRstat.ML

TL;DR

PGD generates attacks independently for each example, which the paper says does not necessarily support generalization-risk optimization. DAA instead optimizes an adversarial data distribution, and the authors report that it outperforms PGD and Momentum PGD on robust defenses, including MadryLab leaderboard models.

  • Problem

    PGD generates attack samples independently for each data sample, and the paper notes this does not necessarily lead to good generalization in risk optimization.

    Source: Introduction · p. 1

  • Method

    DAA optimizes an adversarial data distribution over probability measures to maximize a model’s generalization risk.

    Source: Introduction · p. 1

  • Results

    DAA outperforms PGD and Momentum PGD on adversarially trained and provable defenses; 50-restart DAA reduces MadryLab’s secret MNIST accuracy to 88.79%.

    Source: Introduction · p. 2, Conclusion · p. 7

  • Takeaways & Limitations

    The authors describe DAA as the first first-order l∞ attack they know of that outperforms PGD, especially on robust adversarially trained models.

    Source: Discussion · p. 7

Abstract

from arXiv · show

Recent work on adversarial attack has shown that Projected Gradient Descent (PGD) Adversary is a universal first-order adversary, and the classifier adversarially trained by PGD is robust against a wide range of first-order attacks. It is worth noting that the original objective of an attack/defense model relies on a data distribution $p(\mathbf{x})$, typically in the form of risk maximization/minimization, e.g., $\max/\min\mathbb{E}_{p(\mathbf(x))}\mathcal{L}(\mathbf{x})$ with $p(\mathbf{x})$ some unknown data distribution and $\mathcal{L}(\cdot)$ a loss function. However, since PGD generates attack samples independently for each data sample based on $\mathcal{L}(\cdot)$, the procedure does not necessarily lead to good generalization in terms of risk optimization. In this paper, we achieve the goal by proposing distributionally adversarial attack (DAA), a framework to solve an optimal {\em adversarial-data distribution}, a perturbed distribution that satisfies the $L_\infty$ constraint but deviates from the original data distribution to increase the generalization risk maximally. Algorithmically, DAA performs optimization on the space of potential data distributions, which introduces direct dependency between all data points when generating adversarial samples. DAA is evaluated by attacking state-of-the-art defense models, including the adversarially-trained models provided by {\em MIT MadryLab}. Notably, DAA ranks {\em the first place} on MadryLab's white-box leaderboards, reducing the accuracy of their secret MNIST model to $88.79\%$ (with $l_\infty$ perturbations of $\epsilon = 0.3$) and the accuracy of their secret CIFAR model to $44.71\%$ (with $l_\infty$ perturbations of $\epsilon = 8.0$). Code for the experiments is released on \url{https://github.com/tianzheng4/Distributionally-Adversarial-Attack}.

Introduction

DNNs can make high-confidence errors on adversarially perturbed inputs, while PGD-trained classifiers are described as robust against a wide range of first-order attacks. The paper proposes DAA to optimize an adversarial data distribution and increase generalization risk.

Introduction

  • PGD is characterized as a universal first-order attack, and PGD-adversarial training as robust against a wide range of first-order attacks.FGSM, PGD, and Carlini & Wagner are listed among proposed first-order attack algorithms.Source: Introduction · p. 1
  • DAA lifts attack optimization from individual samples to probability distributions, seeking an adversarial distribution that maximally increases generalization risk.With testing data approximating the unknown data manifold, generated adversarial samples depend explicitly on one another.Source: Introduction · p. 1
  • PGD generates adversarial samples independently per example, which the paper identifies as a limitation for global optimality and risk-based optimization.The paper frames this as a lack of statistical interpretation for PGD’s underlying optimization problem.Source: Introduction · p. 1
  • With 50 random restarts, DAA lowers MadryLab’s public and secret MNIST accuracy to 88.7% and 88.79%; with 10 restarts, it lowers secret CIFAR10 accuracy to 44.71%, outperforming listed leaderboard attacks.Source: Introduction · p. 2

Preliminaries

The section reviews Wasserstein gradient flows and standard adversarial attacks before framing DAA as optimization over distributions and outlining particle-based attack generation.

Wasserstein Gradient Flows

  • The second-order Wasserstein metric measures distance between probability distributions through transport cost, and its gradient flow evolves distributions using an energy's first variation.The flow's velocity is the negative gradient of the energy's first variation.Source: Wasserstein Gradient Flows · p. 2

Adversarial Sample

  • FGSM takes one gradient step, PGD iterates constrained FGSM updates, and Momentum PGD can exceed MI-FGSM in strength when its step size is appropriate.A step size that is too small can trap adversarial samples in poor local maxima.Source: Adversarial Sample · pp. 2–3

Adversarial Training

  • Adversarial training generates minibatch adversarial samples with PGD during training, using the inner attack maximization and outer loss minimization.The cited experiments report robustness to PGD and CW∞ attacks.Source: Adversarial Training · p. 3

Adversarial Attack as WGFs

  • DAA seeks a nearby adversarial-data distribution that increases an energy functional, rather than optimizing a separate adversarial sample for each input.The formulation starts from a loss-based distribution and seeks a distribution close to it that raises the energy.Source: Adversarial Attack as WGFs · p. 3

Energy Functional

  • The energy combines system loss with KL divergence from the original distribution and remains convex with respect to the adversarial distribution.The hyperparameter c balances the loss and divergence terms.Source: Energy Functional · p. 4

Adversarial-Distribution Optimization and Adversarial-Sample Generation

  • DAA approximates the adversarial distribution with particles and generates constrained adversarial samples using DAA-BLOB or DAA-DGF updates.DAA-BLOB substitutes the PGD gradient with the gradient from Eq. 17; DAA-DGF uses a combination of Eqs. 19 and 20.Source: Adversarial-Distribution Optimization… · pp. 4–5
  • Because updating all particles costs O(M^2) per particle update, DAA randomly partitions test samples into minibatches and updates them sequentially over multiple rounds.The proposed subsampling procedure is described as unbiased and computationally feasible.Source: Adversarial-Distribution Optimization… · p. 5
  • DAA reduces to PGD when c = 0 or when the data manifold is approximated by a single sample, M = 1.In the M = 1 case, the DAA gradient's interaction term vanishes or reduces to the stated kernel term.Source: Adversarial-Distribution Optimization… · p. 5

Experiment

Experiments compare DAA with PGD and Momentum PGD on defended models across four datasets, while also examining perturbation structure and attack accuracy.

Setup

  • The evaluation compares DAA, PGD, and Momentum PGD across MNIST, FMNIST, CIFAR10, and ImageNet in a white-box setting with full target-model access.The targets include MadryLab models for MNIST and CIFAR10, a PGD-trained FMNIST model, and an adversarially trained Inception ResNet-v2 model for ImageNet.Source: Setup · pp. 5–6

Adversarial Perturbation Analysis

  • For ten samples, DAA attacks the defense model at 0.3/1.0 l∞ perturbation where PGD with 50 random starts cannot; DAA perturbations focus around target digits while PGD perturbations scatter.Figure 1 compares the PGD and DAA perturbations.Source: Figure 1 · p. 6, Adversarial Perturbation Analysis · p. 6

Empirical Results

  • Approximately 88.79% accuracy on MadryLab’s secret MNIST model under 200-step attacks with 50 random starts was achieved by DAA-BLOB, outperforming attacks on the white-box leaderboard.The table caption specifies 0.3/1.0 l∞ perturbations.Source: Table 1 · p. 6, Empirical Results · p. 6
  • On MadryLab’s adversarially trained MNIST model, DAA consistently outperforms other methods across l∞ perturbation levels and both losses in single runs.Figure 2 separates cross-entropy and CW∞ loss results.Source: Figure 2 · p. 6, Empirical Results · p. 6
  • DAA-based methods outperform PGD on the adversarially trained FMNIST model, and DAA-BLOB and DAA-DGF are reported as comparable.The worst-case accuracy comparison uses 10 random starts; the table caption specifies 100-step attacks and 0.2/1.0 l∞ perturbations.Source: Empirical Results · p. 6, Table 2 · p. 7
  • 14.43% accuracy remained under DAA versus 14.59% under 100-step PGD on MadryLab’s adversarially trained CIFAR10 model with 16/255 l∞ perturbations.The authors conjecture that complex, sparse data impair DAA’s particle approximation of the true data manifold, making its results similar to PGD.Source: Empirical Results · p. 7, Table 3 · p. 7
  • 16.43% accuracy remained under 50-step DAA on the ensemble adversarially trained ImageNet model, versus approximately 70% under targeted Rand+FGSM.Both attacks used the least likely class as the target; Rand+FGSM used 2/255 l∞ perturbations.Source: Empirical Results · p. 7

Discussion

The authors position DAA as the first first-order l∞ attack to outperform PGD in white-box settings. They note that further attacks may require costly higher-order information and caution against comparing l1 or l2 attacks with l∞ attacks on l∞-trained models.

Discussion

  • DAA is presented as the first-and-only first-order l∞ attack algorithm to outperform PGD, especially on robust adversarially trained models.The authors qualify this claim as being to their knowledge and specifically under the white-box setting.Source: Discussion · p. 7
  • Further attacking adversarially trained models with small l∞ perturbations might require higher-order information, which is usually computationally expensive.Source: Discussion · p. 7
  • Few-pixel attacks with large l∞ perturbations can retain small l1 or l2 distance, but may cause human misclassification and are unfair comparisons with l∞ attacks (Sharma and Chen 2018).Source: Discussion · p. 7

Conclusion

DAA generalizes PGD to optimize over data distributions, using a new energy functional in the WGF framework. Evaluations report that it outperforms PGD and Momentum PGD on adversarially trained and provable defense models.

Conclusion

  • DAA learns an adversarial data distribution to maximally increase a model’s generalization risk by optimizing over distributions rather than individual samples.Source: Conclusion · p. 7
  • A new energy functional is defined to better reflect the discriminative data manifold in the WGF framework.Source: Conclusion · p. 7
  • With particle approximation, generating adversarial samples amounts to solving corresponding WGF problems, yielding an algorithm closely related to standard PGD.Source: Conclusion · p. 7
  • Extensive evaluations show DAA outperforms PGD and Momentum PGD on adversarially trained and provable defense models.Source: Conclusion · p. 7

Related papers

Loading 1808.05537v3…