Source-linked AI summary
Structure Invariant Transformation for better Adversarial Transferability
Xiaosen Wang, Zeliang Zhang, Jianping Zhang
TL;DR
Transfer-based adversarial attacks can be used in black-box settings, but existing attacks often transfer less effectively than white-box attacks. SIA applies random transformations to image blocks and uses the resulting gradients to update adversarial examples. The paper reports better transferability than existing state-of-the-art attacks across CNN-based and transformer-based models.
Problem
Existing adversarial attacks often have superior white-box performance but poor transferability, limiting their effectiveness in black-box settings.
Source: §1 · p. 1Method
SIA randomly transforms image blocks while preserving image structure and uses gradients from the transformed images to update adversarial examples.
Source: §3.2 · p. 4, §5 · p. 9Results
Takeaways & Limitations
The authors conclude that locally applying transformations offers a direction for generating more diverse images while maintaining image structure.
Source: §1 · p. 2, §5 · p. 9The supported premise is limited to transformations that preserve semantic information and image structure, illustrated with block scaling.
Source: §3.1 · p. 3, §3.2 · p. 4
Abstract
from arXiv · showhide
Given the severe vulnerability of Deep Neural Networks (DNNs) against adversarial examples, there is an urgent need for an effective adversarial attack to identify the deficiencies of DNNs in security-sensitive applications. As one of the prevalent black-box adversarial attacks, the existing transfer-based attacks still cannot achieve comparable performance with the white-box attacks. Among these, input transformation based attacks have shown remarkable effectiveness in boosting transferability. In this work, we find that the existing input transformation based attacks transform the input image globally, resulting in limited diversity of the transformed images. We postulate that the more diverse transformed images result in better transferability. Thus, we investigate how to locally apply various transformations onto the input image to improve such diversity while preserving the structure of image. To this end, we propose a novel input transformation based attack, called Structure Invariant Attack (SIA), which applies a random image transformation onto each image block to craft a set of diverse images for gradient calculation. Extensive experiments on the standard ImageNet dataset demonstrate that SIA exhibits much better transferability than the existing SOTA input transformation based attacks on CNN-based and transformer-based models, showing its generality and superiority in boosting transferability. Code is available at https://github.com/xiaosen-wang/SIT.
1. Introduction
The introduction identifies limited transferability as a gap for black-box adversarial attacks and argues that globally transformed inputs lack diversity. It presents SIA, which applies transformations locally while preserving image structure, and reports better transferability across CNN-based and transformer-based models.
- Existing attacks can perform well white-box but transfer poorly, while global input transformations may limit transformed-image diversity.Source: §1 · pp. 1–2
- SIA locally transforms different image parts to increase image diversity while preserving global structure, then uses transformed-image gradients to update adversarial examples.Source: §1 · p. 2
- SIA outperforms baselines on ImageNet across CNN-based and transformer-based models, demonstrating generality in boosting transferability.Source: §1 · p. 2
2. Related Work
Related work situates transfer-based attacks as a black-box approach and surveys methods to improve transferability, defenses, and data augmentation. The paper distinguishes its use of image-block transformations for gradient calculation from augmentation strategies used during training.
2. Related Work
- Transfer-based attacks do not access target-model information, making them applicable to models in the physical world.Source: §2 · p. 2
2.1. Adversarial Attack
2.2. Adversarial Defense
- Defenses include adversarial training, input pre-processing, learned purification, and certified methods such as randomized smoothing [8].Source: §2.2 · p. 3
2.3. Data Augmentations
- Unlike training-time augmentation methods such as Mixup [75] and Cutmix [73], this work transforms image blocks to create diverse inputs for gradient calculation.Source: §2.3 · p. 3
3. Methodology
The methodology motivates SIA by linking transformed-image diversity with transferability, then applies local block transformations while preserving image structure. It also distinguishes this attack-time transformation from AutoAugment’s training-time role.
3.1. Motivation
- The paper hypothesizes that more diverse transformed images improve transferability, provided semantic information is preserved.It uses LPIPS to assess similarity and reports that similarity decreases as transferability increases.Source: §3.1 · p. 3
- LPIPS similarity decreases as transferability increases across the input-transformation attacks summarized in Table 1.The table evaluates transferability using Inception-v3 on adversarial examples generated on ResNet-18.Source: §3.1 · p. 3, Table 1 · p. 3
3.2. Structure Invariant Attack
- SIA applies different transformations to image blocks, then calculates gradients on multiple transformed images rather than the untransformed input alone.The approach is intended to increase image diversity while preserving critical semantic information and global structure.Source: §3.2 · p. 4
3.3. SIT vs. AutoAugment
- SIT targets adversarial transferability, whereas AutoAugment targets training generalization; SIT samples transformations randomly and applies them locally.AutoAugment searches for dataset-specific policies and applies two transformations sequentially; SIT’s fine-grained transformations generate more diverse images.Source: §3.3 · p. 5
4. Experiment
On ImageNet, SIA is evaluated across single-model, ensemble, model-specific, and defense settings, alongside ablations and parameter studies. Across these evaluations, it consistently improves transferability over the compared input-transformation attacks.
4.1. Experimental Setting
- Experiments use 1,000 correctly classified ImageNet validation images and evaluate CNNs, transformers, and several defense methods.The attacks follow MI-FGSM settings with perturbation budget ϵ = 16, T = 10 iterations, and SIA settings s = 3 and N = 20.Source: §4.1 · p. 5
4.3. Integration to model-specific approaches
- Integrated with SGM and LinBP, SIA improves transferability over the best baseline by average margins of 3.3% and 3.6%, respectively.Figures 4 and 5 report attack success rates across eight models for SGM and LinBP integration.Source: Figure 4 · p. 6, Figure 5 · p. 6, §4.3 · p. 6
4.2. Attacking a Single Model
- SIA beats the strongest baseline by 14.3% on average and by at least 2.8% on each of eight models in single-model transfer tests.The models include CNNs and transformers.Source: §4.2 · p. 6
4.4. Attacking Ensemble Models
- In ensemble tests, SIA exceeds the runner-up by 6.5% and achieves at least 88.3% success on all eight models.When crafting examples on CNNs, it reaches 94.9% on ViT and 99.3% on Swin.Source: Figure 6 · p. 7, §4.4 · p. 7
4.5. Attacking Defense Methods
- Against defenses, SIA achieves 78.2% attack success on randomized smoothing and at least 89.9% on the other eight methods.The evaluated defenses include the denoising method NRP.Source: Figure 7 · p. 7, §4.5 · p. 7
4.6. Ablation Studies
- Combining any two transformations improves transferability over single transformations, and removing any one transformation decreases it.The single-transformation results include a 67.3% average attack success rate with DCT, compared with 59.7% for MI-FGSM.Source: §4.6 · pp. 7–8, Figure 8 · p. 8
- SIA outperforms SIAr, which applies transformations to the whole image without block partitioning; both outperform MI-FGSM.The comparison tests whether applying transformations to image blocks contributes beyond random whole-image transformation.Source: Figure 9 · p. 8, §4.6 · p. 8
4.7. Parameter Studies
- The studies select s = 3 to balance image diversity against gradient variance and N = 20 to balance attack performance against computation cost.Increasing N beyond 20 adds computation without improving performance.Source: §4.7 · pp. 8–9, Figure 10 · p. 9
5. Conclusion
The conclusion links greater diversity in transformed images with better transferability and presents blockwise random transformation as a way to preserve global image structure. The authors report that SIA improves transferability over existing state-of-the-art attacks.
5. Conclusion
- The paper identifies an association between more diverse transformed images and better transferability.Source: §5 · p. 9
- SIT independently applies random transformations to image blocks, and SIA averages gradients over several SIT-transformed images to update perturbations.Source: §5 · p. 9
- The authors conclude that block-level transformations offer a direction for generating more transferable adversarial examples while preserving global structure.Source: §5 · p. 9
A. Implementation details
The implementation describes ten random transformations applied to image blocks, including spatial rearrangements and pixel-value changes.
A. Implementation details
- VShift and HShift roll a block along an axis; VFlip and HFlip reverse it across an axis; Rotate turns it clockwise by 180°.Source: A. Implementation details · p. 14
- Scale multiplies pixels by a random factor, while Add Noise, Resize, DCT, and Dropout modify pixel values or frequency content.Add Noise clips perturbed pixels to [0, 1]; Resize returns the block to its original size, DCT masks the top 40% of frequencies, and Dropout zeros 10% of pixels.Source: A. Implementation details · p. 14