NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks

Yandong Li; Lijun Li; Liqiang Wang; Tong Zhang; Boqing Gong

Source-linked AI summary

NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks

Yandong Li, Lijun Li, Liqiang Wang, Tong Zhang, Boqing Gong

arXiv:1905.00441v3May 1, 2019cs.LGcs.CRcs.CVstat.ML

TL;DR

The paper addresses the need for powerful attacks to evaluate DNN robustness and defenses. It learns input-specific adversarial-example distributions in a black-box setting, reporting strong performance across vanilla and defended networks and limited transferability across defended DNNs.

Problem
Powerful attacks are needed to understand adversarial examples, assess DNN robustness, and thoroughly test defense techniques.
Method
The method learns a probability density on a small ℓp-ball around each clean input, so samples are likely adversarial without accessing model layers or weights.
Results
Tested against two vanilla DNNs and 13 defended ones, the approach outperforms state-of-the-art black-box or white-box attacks in most cases and is on par in the remainder.
Takeaways & Limitations
Adversarial training remains one of the best defenses, while attacks transfer less readily across defended DNNs than across vanilla DNNs.
Takeaways & Limitations
The attack is evaluated under an ℓp-distance constraint with p = 2 or ∞, limiting the stated setting to these perturbation bounds.

Abstract

from arXiv · show

Powerful adversarial attack methods are vital for understanding how to construct robust deep neural networks (DNNs) and for thoroughly testing defense techniques. In this paper, we propose a black-box adversarial attack algorithm that can defeat both vanilla DNNs and those generated by various defense techniques developed recently. Instead of searching for an "optimal" adversarial example for a benign input to a targeted DNN, our algorithm finds a probability density distribution over a small region centered around the input, such that a sample drawn from this distribution is likely an adversarial example, without the need of accessing the DNN's internal layers or weights. Our approach is universal as it can successfully attack different neural networks by a single algorithm. It is also strong; according to the testing against 2 vanilla DNNs and 13 defended ones, it outperforms state-of-the-art black-box or white-box attack methods for most test cases. Additionally, our results reveal that adversarial training remains one of the best defense techniques, and the adversarial examples are not as transferable across defended DNNs as them across vanilla DNNs.

1. Introduction

The paper introduces a universal black-box attack that learns adversarial-example distributions around inputs rather than searching for individual optimal examples. It reports strong performance across vanilla and defended DNNs while highlighting adversarial training and limited transferability across defended models.

Contribution: The method is designed to attack different DNNs and defense techniques with one algorithm, avoiding defense-specific tailoring.The authors suggest this may generalize better to new defense methods.
Contribution: The proposed black-box algorithm learns a probability density over a small region around each input, from which likely adversarial examples are sampled without accessing internal layers or weights.The region is centered on the benign input and can be bounded by an ℓp constraint.
Results: Across tests of two vanilla DNNs and 13 defended DNNs, the approach outperforms state-of-the-art black-box or white-box attacks in most cases and matches them in the remainder.The method is therefore reported as both strong and universal across the evaluated networks.
Results: The study identifies adversarial training as one of the best defenses and finds lower attack transferability across defended DNNs than across vanilla DNNs.These findings differ from the stronger transferability commonly reported for vanilla networks.
Method: The distribution is optimized in a lower-dimensional parameter space, while adversarial examples remain high-dimensional, and the objective can avoid direct gradient estimation.A constrained NES formulation smooths the loss through a distribution over the ℓp-ball.
Implications: Sampling many adversarial examples from a learned distribution may be more efficient for adversarial training than finding examples through gradient-based optimization.The paper presents this as a potential benefit rather than an experimentally established result.

2. Approach

The approach formulates black-box adversarial-example search as optimizing a probability distribution over a constrained ℓp-ball, avoiding direct gradient estimation and enabling personalized, lower-dimensional optimization. N ATTACK samples from the learned distribution, with transformations, projection, normalization, and initialization used to improve stability and runtime.

Problem formulation: The attack seeks an adversarial example xadv within Sp(x), an ℓp-ball satisfying ∥x−xadv∥p≤τp for p=2 or ∞, while excluding initially misclassified inputs.The target classifier must initially predict the ground-truth label correctly, and the adversarial example must receive a different label.
N ATTACK formulation: Unlike NES-based projected-gradient attacks, N ATTACK optimizes a smoothed objective defined by a probability density πS(x′|θ) over S.The expected loss under this distribution is minimized so that samples are likely to be adversarial.
N ATTACK formulation: The smoothed objective is optimized without estimating ∇f(xt), avoiding the instability caused by non-smooth neural networks in NES gradient estimation.The paper reports that the earlier method’s performance varies across DNNs because non-smooth networks can produce unstable gradient estimates.
Distribution construction: N ATTACK represents the distribution in a lower-dimensional parameter space, with θ=(µ,σ^2), and learns personalized distributions whose support depends on each input’s ball Sp(x).The mean µ is optimized with NES, while the bandwidth σ is selected by grid search; the transformation g maps latent normal samples into input space and projection enforces S.
Distribution construction: The variable transformation maps normal samples through g, projects them onto S, and uses clipping to enforce the ℓp constraint before evaluating losses.For CIFAR10, g0 is identity; for ImageNet, it uses bilinear interpolation before range transformation and offset clipping.
Optimization and acceleration: N ATTACK z-score-normalizes mini-batch losses, updates µ using sampled perturbations, and then samples repeatedly until obtaining x′ with C(x′)≠C(x).A learned regression network can provide the initial mean µ0; with good initialization, adversarial examples are often found in fewer than 100 iterations.
Optimization and acceleration: The method differs from Ilyas et al. by incorporating projection into function evaluation and allowing an arbitrary transformation g, which the paper reports is more stable in practice.The alternative ordering treats projection as part of evaluating the objective rather than as part of an estimated projected gradient.

3. Experiments

Experiments evaluate N ATTACK against 13 defended and two vanilla DNNs, comparing attack success, efficiency, runtime, iteration dynamics, and transferability. N ATTACK performs strongly across defenses, while adversarial training is comparatively robust and defended-model attacks transfer poorly.

Attack success rates: N ATTACK achieves 100% success on six of 13 defenses and over 90% on five others.Success rate is the evaluation metric, with higher values indicating stronger attacks.
Attack success rates: As one black-box algorithm, N ATTACK is better than or comparable to powerful white-box attacks, especially on defended DNNs.It also significantly outperforms ZOO, QL, and a decision-based black-box attack.
Runtime and query efficiency: N ATTACK finds CIFAR10 adversarial examples in about 30s and ImageNet examples in 71s without, or 48s with, the regression network.The regression network reduces the reported ImageNet runtime.
Attack dynamics: ADV-TRAIN, ADV-BNN, THERM-ADV, and CAS-ADV are more difficult to attack than the other evaluated DNNs.All four employ adversarial training; ADV-TRAIN and ADV-BNN remain among the best defenses, while adversarial-training variants have weaker clean-input performance and higher computational cost.
Runtime and query efficiency: N ATTACK is more query efficient than QL on six defenses under most attack success rates, with the difference increasing at higher success rates.For SAP, N ATTACK performs better when the desired success rate exceeds 80%.
Transferability: Transferability is asymmetric and lower across defended DNNs than across undefended ones, with defended-to-vanilla transfer easier than the reverse.Architecture similarity improves transfer, whereas defense techniques can reduce it; BPDA transfers better than N ATTACK.

4. Related Work

Related work spans white-box attacks with full model access and black-box attacks that use substitutes, queries, or decision outputs. Existing black-box methods were commonly tested on vanilla DNNs, whereas this work evaluates them on defended networks as well.

White-Box Attacks: White-box attacks assume full access to the target DNN and generate adversarial examples using objectives such as L-BFGS or gradient signs.Prior methods target different perturbation metrics, including ℓ0, ℓ2, and ℓ∞.
Black-Box Attacks: Black-box attacks hide some DNN components and exploit transferability, substitute models, soft outputs, or hard-label decisions.Representative approaches include substitute-model attacks, ZOO, and decision-based methods.
Black-Box Attacks: Most existing black-box methods are tested against vanilla DNNs, while this work tests them on defended DNNs alongside N ATTACK.The related-work discussion focuses on the most relevant attacks and defenses rather than surveying the full literature.

5. Conclusion and Future Work

The paper concludes that N ATTACK learns input-centered adversarial distributions and defeats many defended networks without model internals. It also finds limited transferability across defended DNNs and identifies distribution design and efficient adversarial sampling as open questions.

Conclusion: N ATTACK learns a probability density on the ℓp-ball around a clean input and can reduce attack dimensionality through an arbitrary transformation g(·).The learned density is used as the core black-box attack representation.
Conclusion: N ATTACK defeats 13 defended DNNs and performs better than or comparably to state-of-the-art white-box attacks.The reported conclusion summarizes experiments across the defended models.
Conclusion: Adversarial examples are difficult to transfer across defended DNNs, unlike the high transferability commonly observed across vanilla DNNs.A majority of examples that fail one defended DNN cannot defeat the others.
Future Work: Future work includes identifying suitable distribution families and developing efficient adversarial training through sampling from learned distributions.The paper frames these as open questions for further investigation.

Supplementary Materials for N ATTACK: Learning the Distributions of Adversarial Examples for an

The supplementary document provides defense descriptions, regression-network architecture details, and runtime analyses for N ATTACK and BPDA.

Supplementary contents: Section A describes the 13 defense methods studied.The defenses are documented as supplementary material.
Supplementary contents: Section B presents the regression neural network architecture used to initialize N ATTACK.The initialization role is specified in the supplementary overview.
Supplementary contents: Section C analyzes the runtime of N ATTACK and BPDA.The runtime comparison supports the main experimental discussion.

A. More Details of the 13 Defense Methods

The paper surveys 13 defense methods spanning input transformations, randomness, denoising, pruning, Bayesian modeling, generative perturbations, and adversarial training.

Thermometer encoding transforms inputs non-differentiably and non-linearly before a modified conventional DNN input layer.
Adversarial-training defenses include standard PGD-based training, cascade training, adversarially trained Bayesian networks, and GAN-generated adversarial examples.
Stochastic activation pruning randomly drops neurons with probabilities proportional to their absolute values.
Input-processing defenses include denoising, image transformations, bit-depth reduction, JPEG compression, pixel deflection, and wavelet filtering.
Randomization defenses add stochasticity through noisy prediction ensembles or randomized image resizing, padding, and selection.

B. Architecture of the Regression Network

The regression network uses a fully convolutional architecture adapted from a pretrained PASCAL VOC segmentation model to output adversarial perturbations.

The regression network adapts a PASCAL VOC-pretrained FCN by changing its last two layers.
Its output is an adversarial perturbation of size 32 × 32 × 3.
The network is trained using mean square loss.

C. Run Time Comparison

N ATTACK’s runtime depends on the comparison method, image resolution, initialization, and attack difficulty; it can be faster on CIFAR-10 but slower on ImageNet.

N ATTACK may take longer than BPDA because BPDA quickly finds local optima using approximate gradients.
N ATTACK can succeed faster than BPDA on CIFAR-10 but runs slower on ImageNet.
High-resolution ImageNet attacks can take about 60 minutes for hard cases, despite finding adversarial examples for nearly 90% of test images.
82% success with random initialization increased to 91.9% with N ATTACK-R on ImageNet.N ATTACK-R also reduced attack time by 22.5s per image relative to random initialization.