Source-linked AI summary
Universal Adversarial Attack on Attention and the Resulting Dataset DAmageNet
Sizhe Chen, Zhengbao He, Chengjin Sun, Jie Yang, Xiaolin Huang
TL;DR
Black-box attacks have limited transfer across model families, with existing approaches relying on similarity or extensive queries. The paper proposes Attack on Attention (AoA), replacing cross-entropy with attention loss and combining it with transfer-enhancement methods. AoA achieves over 85% transfer rate on 12 black-box DNNs, and DAmageNet records error rates above 85% on undefended models and above 70% on adversarial-trained models.
Problem
Black-box attacks rely on massive queries or surrogate–victim similarity, and small-perturbation transfer across model families remains limited.
Source: §1 · p. 1, §2.2 · p. 2Method
Results
DAmageNet samples yield error rates over 85% for undefended models and over 70% for adversarial-trained models.
Source: §5 · p. 6Takeaways & Limitations
DAmageNet provides a benchmark for evaluating DNN robustness using elaborately crafted adversarial samples.
Source: §5 · p. 6
Abstract
from arXiv · showhide
Adversarial attacks on deep neural networks (DNNs) have been found for several years. However, the existing adversarial attacks have high success rates only when the information of the victim DNN is well-known or could be estimated by the structure similarity or massive queries. In this paper, we propose to Attack on Attention (AoA), a semantic property commonly shared by DNNs. AoA enjoys a significant increase in transferability when the traditional cross entropy loss is replaced with the attention loss. Since AoA alters the loss function only, it could be easily combined with other transferability-enhancement techniques and then achieve SOTA performance. We apply AoA to generate 50000 adversarial samples from ImageNet validation set to defeat many neural networks, and thus name the dataset as DAmageNet. 13 well-trained DNNs are tested on DAmageNet, and all of them have an error rate over 85%. Even with defenses or adversarial training, most models still maintain an error rate over 70% on DAmageNet. DAmageNet is the first universal adversarial dataset. It could be downloaded freely and serve as a benchmark for robustness testing and adversarial training.
1 INTRODUCTION
Black-box attacks struggle to transfer across DNNs because existing methods rely on model similarity or extensive queries. AoA attacks shared attention, while DAmageNet provides adversarial samples for evaluating robustness.
1 INTRODUCTION
- Black-box transfer remains limited because query-based attacks need many queries, while transfer-based attacks depend on surrogate–victim similarity and often require similar structures.The authors note that high attack rates were reached only when two DNNs had similar structures.Source: §1 · p. 1
- Replacing cross-entropy with attention loss increases transferability by 10% to 15%, and combining AoA with transfer-enhancement methods achieves over 85% transfer rate on 12 black-box DNNs.The authors target attention heat maps, which they say are similar across different DNNs.Source: §1 · p. 1
- In the salamander example, a slight image change scatters the attention heat map and causes the listed well-trained DNNs to misclassify the sample.The attack is carried out on VGG19; the figure’s heat map is calculated by DenseNet121.Source: §1 · p. 1, Figure 1 · p. 2
- AoA generates 50,000 adversarial samples that raise error rates on many DNNs to over 85%; DAmageNet is presented as the first dataset of black-box adversarial samples.The samples may help improve networks through adversarial training and robustness certification.Source: §1 · pp. 1–2
2 RELATED WORK
The related work reviews adversarial attack and defense methods, black-box attack strategies, attention visualization, and ImageNet variants. It distinguishes DAmageNet’s robustness focus from datasets aimed at generalization.
2.1 Adversarial attack and its defense
- Attacks alter inputs to increase loss, using gradient-based or optimization-based methods in image, feature, or encoder/decoder spaces; defenses include adversarial training, network design, and preprocessing.The paper notes adversarial training is effective but consumes several-fold time.Source: §2.1 · p. 2
2.2 Black-box attack
- Black-box attacks use victim queries to estimate gradients or attack a surrogate model, but query volume risks detection and small-perturbation transfer across model families remains limited.Prior strong transfer results were mainly within the same model family or among models sharing blocks,,,,.Source: §2.2 · p. 2
2.3 Attention heat map
- Attention heat maps can be computed by forward visualization, which may be slow and random, or backward relevance propagation from output to input.Backward methods include LRP, CLRP, and SGLRP.Source: §2.3 · p. 3
2.4 ImageNet and its variants
- ImageNet variants test generalization, whereas DAmageNet is framed as a robustness dataset because its samples resemble ImageNet originals and expose DNN oversensitivity.ImageNet-A and ObjectNet use challenging natural images, ImageNet-C adds corruptions, and ImageNet-P adds non-attack perturbation sequences.Source: §2.4 · p. 3
3 ATTACK ON ATTENTION (AOA)
AoA attacks attention heat maps rather than model outputs, exploiting similarities across different DNN architectures to target a shared semantic feature. It iteratively changes a sample’s attention using gradients from a white-box surrogate, with attention-loss variants and prediction-loss combination improving transferability.
3 ATTACK ON ATTENTION (AOA)
- Attention heat maps are similar across different architectures, motivating AoA’s attack on attention rather than the output.The paper uses this shared feature as a target for black-box transfer attacks.Source: §3 · p. 3, Figure 2 · p. 3
- AoA uses SGLRP to calculate class-specific attention heat maps and back-propagates their gradients to update the adversarial sample iteratively.The update process minimizes the AoA loss while keeping the perturbation within an ℓ∞ bound ε.Source: §3 · pp. 3, 5, Figure 3 · p. 4
- AoA’s attention-loss designs suppress correct-class attention, distract it toward irrelevant regions, or reduce its gap from the second-most-probable class.The logarithmic boundary loss uses a ratio because heat-map magnitudes vary across models.Source: §3 · pp. 3–4
- The logarithmic boundary loss transfers better than suppress and distract losses and CW/PGD; combining attention and cross-entropy further increases transferability.Table 1 reports transfer rates from ResNet50 to other networks, but the supplied passage gives no cell values.Source: Table 1 · p. 4, §3 · pp. 4–5
- For the distract loss, the illustrated attention shifts from the correct region to irrelevant regions, and all DNNs shown misclassify the resulting sample.The VGG19 heat map is described as clearer than the others.Source: §3 · p. 4, Figure 4 · p. 4
- Because AoA changes the loss, existing optimization- and input-modification techniques for improving transferability can also be applied.The paper reports further improved transfer performance with these enhancements.Source: §3 · p. 5
4 EXPERIMENTS
The experiments assess AoA’s black-box transfer against other attacks, test defenses, and evaluate DAmageNet on well-trained models.
4.1 Setup
- Experiments used ImageNet validation images and Keras models; five runs of 200 images yielded 1,000 samples, with RMSE capped at 7 and perturbation bounded by 0.1*255.Samples incorrectly predicted by the victim model were skipped.Source: §4.1 · p. 5
4.2 Transferability of AoA
- Across tested surrogate-victim settings, AoA outperformed CW and PGD in black-box transfer; adding SI made SI-AoA the best-performing tested method on ResNet50.SI was also helpful for PGD and CW. The compared enhancement methods were MI-DI, MI-TI, and SI.Source: §4.2 · pp. 5–6
4.3 AoA under Defenses
- Preprocessing defenses lowered error rates by about 5% to 10%, while SI-AoA maintained the highest transfer rate; adversarially trained models were more robust, though SI-AoA beat SI-PGD and SI-CW.Random smoothing had much longer inference time, making it an unfair comparison; defeating adversarial training was out of scope.Source: §4.3 · p. 6
4.4 DAmageNet
- SI-AoA attacks all 50,000 ImageNet validation images to build DAmageNet; filenames preserve original sample matching, and average RMSE is 7.23.The samples can be paired with their original ImageNet validation images and labels.Source: §4.4 · p. 6
- On DAmageNet, all 13 listed undefended models had top-1 error over 85%, and five adversarially trained models had error over 70%.The dataset produced almost no error-rate drop across four tested defenses; feature denoising was promising, but combining it with preprocessing defenses did not work well.Source: §4.4 · p. 6
5 CONCLUSION
AoA attacks attention heat maps, improving black-box transferability and combining with transfer-enhancement methods such as SI. SI-AoA yields DAmageNet, a robustness benchmark with high error rates on undefended and adversarially trained models.
5 CONCLUSION
- AoA attacks attention rather than classification output, and replacing cross-entropy with attention loss increases transferability; SI can be combined with AoA for state-of-the-art performance.The paper attributes AoA’s transferability to semantic features shared across DNNs.Source: §5 · p. 6
- SI-AoA produces DAmageNet, the first adversarial dataset with small perturbations and error rates over 85% on undefended models and over 70% on adversarially trained models.The dataset is proposed as a benchmark for evaluating DNN robustness.Source: §5 · p. 6
- Attention is one shared semantic feature; the paper identifies attacks on other shared semantic features as a promising direction for transferability.Source: §5 · p. 6