PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhancing the Transferability of Adversarial Attacks through Variance Tuning

Xiaosen Wang, Kun He

arXiv:2103.15571v3cs.AI

TL;DR

Transfer-based black-box attacks often underperform white-box attacks, particularly against defended models. The paper introduces variance tuning, which uses previous-iteration neighborhood gradient variance to stabilize updates, and reports improved transferability, reaching 90.1% average success against nine advanced defenses in the multi-model setting.

  • Problem

    Existing adversarial attacks have high white-box effectiveness but weak transferability to black-box models, especially models with defense mechanisms.

  • Method

    Variance tuning tunes the current gradient with gradient variance from the neighborhood of the previous data point during iterative gradient-based optimization.

  • Results

    90.1% average success rate is achieved against nine advanced defense models, outperforming state-of-the-art attacks by 6.6% on average.

  • Takeaways & Limitations

    Variance tuning is generally applicable to iterative gradient-based attacks and can be combined with ensemble attacks and input transformations to improve transferability.

Abstract

from arXiv · show

Deep neural networks are vulnerable to adversarial examples that mislead the models with imperceptible perturbations. Though adversarial attacks have achieved incredible success rates in the white-box setting, most existing adversaries often exhibit weak transferability in the black-box setting, especially under the scenario of attacking models with defense mechanisms. In this work, we propose a new method called variance tuning to enhance the class of iterative gradient based attack methods and improve their attack transferability. Specifically, at each iteration for the gradient calculation, instead of directly using the current gradient for the momentum accumulation, we further consider the gradient variance of the previous iteration to tune the current gradient so as to stabilize the update direction and escape from poor local optima. Empirical results on the standard ImageNet dataset demonstrate that our method could significantly improve the transferability of gradient-based adversarial attacks. Besides, our method could be used to attack ensemble models or be integrated with various input transformations. Incorporating variance tuning with input transformations on iterative gradient-based attacks in the multi-model setting, the integrated method could achieve an average success rate of 90.1% against nine advanced defense methods, improving the current best attack performance significantly by 85.1% . Code is available at https://github.com/JHL-HUST/VT.

1. Introduction

Adversarial attacks work well in white-box settings but often transfer poorly to black-box models, particularly defended ones. Variance tuning improves iterative gradient-based attacks by stabilizing updates and substantially increasing transferability.

  • White-box attacks are effective, but transfer-based black-box attacks still show a substantial performance gap, especially against defended models.
  • Variance tuning enhances iterative gradient-based attacks by using neighborhood gradient variance from the previous iteration to tune the current gradient.The strategy stabilizes the update direction and helps escape poor local optima during the search process.
  • More than 20% higher success rates are achieved over the momentum-based attack when adversarial examples are generated on Inc-v3.The method also maintains similar success rates to momentum-based attacks in the white-box setting.
  • 90.1% average success rate is achieved against nine advanced defense methods in the multi-model setting, exceeding baselines by 6.6%.In the single-model setting, the integrated method reaches 67.0% average success and outperforms baselines by 17.5%.

2. Related Work

Related work improves adversarial transferability through gradient calculation, multiple-model attacks, and input transformations, while defenses seek to suppress adversarial effects. The paper builds on these attack families and their combinations.

  • Adversarial attacks include gradient-based, optimization-based, score-based, and decision-based approaches, with this work focusing on transfer-based attacks.
  • Gradient-based Attacks: FGSM crafts an adversarial example by maximizing loss with a single gradient-sign update.I-FGSM extends this approach iteratively with a small step size, while MI-FGSM adds momentum and improves transferability.
  • Gradient-based Attacks: NI-FGSM uses Nesterov’s accelerated gradient by evaluating the update at a look-ahead point to further improve MI-FGSM transferability.
  • Input Transformations: DIM, TIM, and SIM improve transferability through random resizing and padding, translated-image gradients, and gradients over scaled inputs, respectively.
  • Input Transformations: The composite transformation method combines DIM, TIM, and SIM and is described as the strongest current transfer-based black-box attack method.These transformations can also be integrated with gradient-based attacks.
  • Adversarial Defenses: Adversarial defenses include adversarial training, input transformations, denoising, feature squeezing, compression, randomized smoothing, and representation purification.

3. Methodology

The methodology treats transfer-based adversarial attack generation as an optimization process and introduces variance tuning to stabilize gradient updates. The approach defines neighborhood gradient variance, uses it to tune iterative attacks, and integrates with momentum, Nesterov acceleration, and input transformations.

  • 3.1. Motivation: Variance tuning treats iterative gradient-based attack generation as an SGD-like optimization process and targets transferability rather than convergence speed.The method differs from SGD variance-reduction methods by using variance from examples near the input and by improving MI-FGSM and NI-FGSM.
  • 3.2. Variance Tuning Gradient-based Attacks: Existing iterative attacks may overfit the source model because they use gradient information only along the optimization path.The method instead uses neighborhood information around the previous adversarial data point.
  • 3.2. Variance Tuning Gradient-based Attacks: Gradient variance measures the difference between neighborhood gradients and the gradient at the current input within a bounded neighborhood.The neighborhood radius is set as ϵ′ = β · ϵ, where β is a hyper-parameter and ϵ bounds perturbation magnitude.
  • 3.2. Variance Tuning Gradient-based Attacks: Because the neighborhood expectation is impractical to compute directly, the method approximates variance by sampling N examples around the input.The sampled perturbations use a uniform distribution over the β · ϵ neighborhood.
  • 3.2. Variance Tuning Gradient-based Attacks: At each iteration, variance from the previous adversarial point tunes the current gradient before momentum accumulation, stabilizing updates and helping escape poor local optima.The resulting VMI-FGSM method extends to VNI-FGSM and can be combined with DIM, TIM, and SIM input transformations.
  • 3.3. Relationships among Various Attacks: The attack family forms a hierarchy in which setting β = 0 removes variance tuning, µ = 0 removes momentum, and T = 1 reduces I-FGSM to FGSM.The attacks can additionally be integrated with DIM, TIM, and SIM to enhance transferability.

4. Experiments

Experiments on ImageNet evaluate variance-tuned attacks across single-model, ensemble, input-transformation, defense-model, and hyper-parameter settings. Across these settings, variance tuning generally improves black-box transferability while retaining strong white-box performance.

  • Experimental setup: Experiments use 1,000 ImageNet validation images, seven networks, nine advanced defense models, and momentum-based MI-FGSM and NI-FGSM baselines.The proposed variants are evaluated with DIM, TIM, SIM, and CTM input transformations under fixed attack hyper-parameters.
  • Single-model attacks: VMI-FGSM achieves 71.7% on Inc-v4 and 32.8% on Inc-v3ens3, versus 43.6% and 13.1% for MI-FGSM, while all methods reach 100% on white-box Inc-v3.These results compare adversarial examples crafted on Inc-v3 and tested on other models.
  • Input transformations: Variance tuning improves black-box success rates by 10% ∼30% across attack algorithms and source models, including when combined with DIM, TIM, or SIM.The comparison covers adversarial examples generated on Inc-v3, with results for other source models reported separately.
  • Ensemble and CTM attacks: VNI-CT-FGSM reaches 92.3% ∼95.5% against three adversarially trained models in the multi-model setting, while CTM further improves transferability.The results indicate particularly strong gains against adversarially trained models.
  • Advanced defense models: Against nine advanced defense models, VMI-CT-FGSM and VNI-CT-FGSM achieve average success rates of 66.5% and 67.0% singly, and 88.9% and 90.1% with ensembles.These methods outperform the corresponding baselines by more than 13.5% and 17.5% in the single-model setting, and more than 3.8% and 6.6% in the multi-model setting.
  • Hyper-parameter ablations: β strongly affects transferability, peaking for normally trained models at β = 3/2, while increasing N beyond 20 has little effect and raises computation cost.The experiments therefore select β = 3/2 and N = 20 as a transferability–cost trade-off.

5. Conclusion

The paper introduces variance tuning to improve transferability in iterative gradient-based attacks while preserving similar white-box success rates. It applies the method to ensemble attacks and input transformations, achieving strong performance against advanced defenses.

  • Variance tuning defines gradient variance as the difference between neighborhood mean gradient and the input’s own gradient.
  • The method uses the previous iteration’s gradient variance to tune the current gradient, improving transferability while maintaining similar white-box success rates.
  • Variance tuning is generally applicable to iterative gradient-based attacks, including attacks on ensemble models and integrations with DIM, TIM, and SIM.
  • 90.1% average success rate against nine advanced defense models was achieved by the integrated method, outperforming state-of-the-art attacks by 6.6% on average.

Appendix

The appendix adds visualizations and supplementary results for variance-tuned attacks combined with input transformations on three additional normally trained models.

  • Supplementary experiments cover Inc-v4, IncRes-v2, and Res-101 with methods integrated with DIM, TIM, or SIM.

A. Visualizations on Adversarial Examples

The appendix visualizes adversarial examples generated by four attacks on Inc-v3 and reports that they are imperceptible to humans.

  • Eight randomly selected benign images and corresponding adversarial examples were generated on Inc-v3 using MI-FGSM, NI-FGSM, VMI-FGSM, and VNI-FGSM.
  • The crafted adversarial examples are described as human-imperceptible.

B. More Attacks with Input Transformations

Additional experiments evaluate variance-tuned attacks with DIM, TIM, and SIM across three normally trained models. The results indicate especially strong transferability improvements against adversarially trained models.

  • B. More Attacks with Input Transformations: The appendix reports results for Inc-v4, IncRes-v2, and Res-101 using variance-tuned attacks combined with DIM, TIM, and SIM.
  • B. More Attacks with Input Transformations: Variance-tuned methods remarkably improve the transferability of input transformations, especially against adversarially trained models.
  • A. Visualizations on Adversarial Examples: Figure 6 visualizes adversarial examples generated by MI-FGSM, NI-FGSM, VMI-FGSM, and VNI-FGSM on Inc-v3 with maximum perturbation ϵ = 16.
  • B. More Attacks with Input Transformations: Figures 7, 8, and 9 show success rates on seven models for attacks enhanced by DIM, TIM, and SIM, respectively.
Loading 2103.15571v3…