Source-linked AI summary
ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, Cho-Jui Hsieh
TL;DR
The paper addresses black-box attacks when a DNN's internal configurations are unavailable, a setting where adversarial robustness matters for security-critical applications. It proposes ZOO, which directly estimates gradients from model queries using zeroth-order optimization and acceleration techniques. Across MNIST, CIFAR10, and ImageNet, ZOO matches state-of-the-art white-box performance and outperforms substitute-model black-box attacks.
Problem
DNNs can be fooled by barely noticeable adversarial images, while practical black-box attackers can query inputs and outputs but cannot access internal configurations.
Method
ZOO directly estimates gradients of a targeted DNN through zeroth-order optimization, using coordinate descent, dimension reduction, hierarchical attacks, and importance sampling.
Results
ZOO is as effective as the state-of-the-art white-box C&W attack and significantly outperforms substitute-model black-box attacks across MNIST, CIFAR10, and ImageNet.
Takeaways & Limitations
Direct zeroth-order attacks avoid substitute-model training and the associated loss in attack transferability, while extending black-box attacks to large ImageNet models.
Takeaways & Limitations
ZOO requires more iterations than white-box attacks because gradient approximation and pseudo back propagation add computation.
Abstract
from arXiv · showhide
Deep neural networks (DNNs) are one of the most prominent technologies of our time, as they achieve state-of-the-art performance in many machine learning tasks, including but not limited to image classification, text mining, and speech processing. However, recent research on DNNs has indicated ever-increasing concern on the robustness to adversarial examples, especially for security-critical tasks such as traffic sign identification for autonomous driving. Studies have unveiled the vulnerability of a well-trained DNN by demonstrating the ability of generating barely noticeable (to both human and machines) adversarial images that lead to misclassification. Furthermore, researchers have shown that these adversarial images are highly transferable by simply training and attacking a substitute model built upon the target model, known as a black-box attack to DNNs. Similar to the setting of training substitute models, in this paper we propose an effective black-box attack that also only has access to the input (images) and the output (confidence scores) of a targeted DNN. However, different from leveraging attack transferability from substitute models, we propose zeroth order optimization (ZOO) based attacks to directly estimate the gradients of the targeted DNN for generating adversarial examples. We use zeroth order stochastic coordinate descent along with dimension reduction, hierarchical attack and importance sampling techniques to efficiently attack black-box models. By exploiting zeroth order optimization, improved attacks to the targeted DNN can be accomplished, sparing the need for training substitute models and avoiding the loss in attack transferability. Experimental results on MNIST, CIFAR10 and ImageNet show that the proposed ZOO attack is as effective as the state-of-the-art white-box attack and significantly outperforms existing black-box attacks via substitute models.
1 INTRODUCTION
DNNs are vulnerable to barely noticeable adversarial examples, creating security concerns while practical black-box settings deny access to internal model configurations. ZOO addresses this gap by directly attacking queried models with zeroth-order optimization instead of substitute models.
- Motivation: Slight image modifications can fool well-trained DNN classifiers with high confidence, raising security concerns for applications such as autonomous-driving traffic-sign identification.The paper highlights adversarial robustness as a security issue for mission-critical systems.
- Attack settings: White-box attacks use back propagation to compute input-output gradients, but most real-world systems withhold network structure and weights.Without internal configurations, standard gradient-based attacks cannot be used in practice.
- Attack settings: In the practical black-box setting, attackers can query a DNN's input-output behavior but cannot access its internal configurations.The targeted model is an image classifier that returns confidence scores for each class.
- Prior work: Existing black-box approaches commonly train substitute models, relying on transferability from surrogate-generated adversarial examples.This approach is presented as the prevailing methodology under the practical query-access setting.
- Proposed approach: ZOO uses zeroth-order optimization and coordinate descent to estimate gradients directly from queries, with dimension reduction, hierarchical attacks, and importance sampling improving scalability.The method avoids training a substitute model and targets large networks including ImageNet's Inception-v3.
2 RELATED WORK
Black-box attacks traditionally train substitute models because target internals are inaccessible, making transferability central. ZOO instead estimates target gradients directly and matches C&W while outperforming a substitute-model attack.
- Earlier adversarial-attack research includes attacks on SVMs and multilayer perceptrons, establishing robustness analysis as a broader machine-learning security concern.
- Black-box DNN attacks typically train substitute models, then transfer white-box adversarial attacks to the target without internal access.Their effectiveness therefore depends heavily on transferability from substitute to target models.
- ZOO provides a general zeroth-order framework that estimates objective-function gradients without back propagation on the targeted DNN.The paper demonstrates it using the C&W attack formulation.
- ZOO matches the original C&W attack's effectiveness but requires longer processing time for pseudo back propagation.
- ZOO significantly outperforms a substitute-model black-box attack by directly attacking the target and avoiding transferability loss.
3 ZOO: A BLACK-BOX ATTACK WITHOUT TRAINING SUBSTITUTE MODELS
ZOO adapts the C&W attack to a black-box setting by using only model outputs and replacing internal logits and back propagation with output-based loss functions and zeroth-order gradient estimates. Stochastic coordinate descent, Hessian estimates, and importance sampling reduce the cost of optimizing image pixels.
- Black-box setting: ZOO targets CNN classifiers that accept an image and return class confidence scores, without exposing network structure, weights, or internal states.The framework can in principle apply to other classifiers with the same input-output relationship, but the paper focuses on DNN image classifiers.
- C&W formulation: The C&W formulation minimizes perturbation size plus a target-misclassification loss, while enforcing valid image values through a box constraint or a tanh reparameterization.The Euclidean perturbation term encourages similarity to the original image, and the loss measures unsuccessful targeted attacks.
- Black-box reformulation: ZOO replaces the white-box C&W loss with a hinge-like loss using observable output confidence scores and replaces back propagation with finite-difference gradient estimation.The logarithm preserves confidence ordering while lessening the dominance of highly skewed class-probability distributions.
- Zeroth-order optimization: Symmetric finite differences estimate each coordinate gradient with error O(h^2), but estimating all p coordinates naively requires 2p objective evaluations.The paper notes that exact gradient values are not always necessary for successful attacks and that one additional evaluation can provide coordinate-wise Hessian estimates.
- Zeroth-order optimization: Stochastic coordinate descent updates randomly selected variables, using estimated gradients and Hessians to choose coordinate steps and ADAM for first-order updates.Coordinate descent avoids updating every pixel at every iteration, which is valuable when black-box function evaluations are expensive.
- Importance sampling: Importance sampling prioritizes pixels near the main object by assigning probabilities from recent absolute pixel changes, reducing the cost of estimating updates for every pixel.The sampling probabilities are formed by regional max pooling, up-sampling, normalization, and periodic updates.
4 PERFORMANCE EVALUATION
ZOO is evaluated against white-box C&W and substitute-model black-box attacks on MNIST, CIFAR10, and ImageNet. It achieves near-white-box performance on smaller datasets and about 90% success on untargeted Inception-v3 attacks despite a reduced attack space and iteration budget.
- Experimental setup: ZOO is compared with C&W white-box and substitute-model black-box attacks using success rate, L2 distortion, and attack time.The evaluation uses C&W-based experimental settings and substitute-model implementations for comparison.
- MNIST and CIFAR10: On MNIST and CIFAR10, ZOO achieves nearly 100% success rate with L2 distortions close to C&W.The resulting adversarial examples have similar quality to the white-box approach in the visual comparisons.
- MNIST and CIFAR10: ZOO significantly outperforms substitute-model attacks, especially for targeted attacks, while maintaining reasonable average attack time.FGSM can achieve better success in some transfers by using ϵ = 0.4, which introduces substantially more noise than C&W.
- Inception network with ImageNet: On ImageNet, substitute-model attacks do not easily scale to Inception-v3 because they require a high-capacity substitute and extensive Jacobian data augmentation.This motivates direct black-box optimization for the large target network.
- Inception network with ImageNet: About 90% success rate is achieved for untargeted Inception-v3 attacks within 1,500 iterations and 20 minutes per image using a 32 × 32 × 3 attack space.Average L2 distortion is about 3 times larger than the white-box attack, while the adversarial images remain visually indistinguishable.
- Inception network with ImageNet: For a difficult targeted Inception-v3 example, 20,000 iterations with 128 pixel updates per iteration are reported as sufficient for the attack.The attack reduces the original-class probability from 97% to about 0.6% and raises the target-class probability from 0.0006% to over 0.6%.
- Optimization techniques: Resetting ADAM states after the first valid attack lets optimization focus on reducing L2 distortion, and omitting the reset increases final distortion and loss.The loss combines a class-probability term with L2 distortion; importance sampling finds the first valid attack over 2,000 iterations earlier after attack-space expansion.
5 CONCLUSION AND FUTURE WORK
ZOO enables black-box attacks without substitute models, matching white-box performance while outperforming substitute-model attacks. The paper also identifies efficiency and broader-domain extensions as future directions.
- Conclusion: ZOO performs black-box attacks without training a substitute model and achieves performance comparable to Carlini and Wagner’s white-box attack.The method uses zeroth order optimization to deploy pseudo back propagation on the targeted black-box DNN.
- Conclusion: ZOO significantly outperforms substitute-model black-box attacks in attack success rate and distortion by avoiding transferability loss.
- Conclusion: Acceleration techniques extend ZOO to large ImageNet DNNs, whereas substitute-model attacks are described as hardly scalable to Inception-v3.
- Future work: ZOO requires more optimization iterations than white-box attacks because it approximates gradients and performs pseudo back propagation.The paper proposes further computation-acceleration and data-driven approaches to improve efficiency.
- Future work: Future work includes using ZOO for adversarial training and adapting it to speeches, time series, graphs, recurrent networks, and other settings.The paper also suggests incorporating dataset side information and existing adversarial examples.
APPENDIX
Figure 7 presents additional ImageNet adversarial examples generated by the untargeted attack.
- APPENDIX: Figure 7 displays additional ImageNet adversarial examples produced by the untargeted attack.