PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Model Ensemble in Transfer-based Adversarial Attacks

Huanran Chen, Yichi Zhang, Yinpeng Dong, Xiao Yang, Hang Su, Jun Zhu

arXiv:2303.09105v2cs.CV

TL;DR

Transfer-based attacks can target black-box models without query access, but ensemble approaches have lacked an in-depth account of why they improve transferability. The paper defines common weakness through loss flatness and proximity to each model’s local optimum, then proposes CWA to promote both. Experiments report improved transferability across image classification and object detection, with additional analysis and black-box Bard attacks reported in the paper context.

  • Problem

    Existing ensemble attacks average model outputs, while the reasons ensemble methods improve adversarial transferability remain insufficiently analyzed.

    Source: §1 · p. 2

  • Method

    CWA combines methods that promote a flat loss landscape and proximity to each surrogate model’s local optimum.

    Source: §1 · p. 2

  • Results

    Experiments report improved transferability across image classification and object detection, and analyses associate CWA with flatter landscapes and closer model optima.

    Source: §5 · p. 9, §4.4 · p. 9

  • Takeaways & Limitations

    The paper demonstrates that its approach can be combined with prior attacks and reports successful black-box attacks on Google’s Bard.

    Source: §1 · p. 2

    The authors identify potential misuse for attacking large commercial models, with possible toxic-content generation or privacy leakage.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

It is widely recognized that deep learning models lack robustness to adversarial examples. An intriguing property of adversarial examples is that they can transfer across different models, which enables black-box attacks without any knowledge of the victim model. An effective strategy to improve the transferability is attacking an ensemble of models. However, previous works simply average the outputs of different models, lacking an in-depth analysis on how and why model ensemble methods can strongly improve the transferability. In this paper, we rethink the ensemble in adversarial attacks and define the common weakness of model ensemble with two properties: 1) the flatness of loss landscape; and 2) the closeness to the local optimum of each model. We empirically and theoretically show that both properties are strongly correlated with the transferability and propose a Common Weakness Attack (CWA) to generate more transferable adversarial examples by promoting these two properties. Experimental results on both image classification and object detection tasks validate the effectiveness of our approach to improving the adversarial transferability, especially when attacking adversarially trained models. We also successfully apply our method to attack a black-box large vision-language model -- Google's Bard, showing the practical effectiveness. Code is available at \url{https://github.com/huanranchen/AdversarialAttacks}.

1 INTRODUCTION

Transfer-based attacks use surrogate models to target black-box victims without query access, but defenses and diverse architectures degrade transferability. This paper defines a model ensemble’s common weakness and proposes methods to exploit it, reporting improvements across attack settings.

1 INTRODUCTION

  • Transfer-based attacks need no query access to black-box victims, but defenses and diverse architectures can degrade their transferability.These attacks craft examples for white-box surrogate models and use them to mislead black-box models.Source: §1 · p. 1
  • CWA targets an ensemble solution with a flat loss landscape that lies close to each model’s local optima.The paper motivates these properties through a quadratic approximation of the expected attack objective and reports that they are especially relevant with few ensemble models.Source: §1 · p. 2
  • MI-CWA surpasses MI with logits ensembling by 30% against state-of-the-art defenses, while experiments also cover object detection and black-box attacks on Bard.The evaluation includes 31 image-classification victim models across architectures and training settings.Source: §1 · p. 2
  • The experiments report that CWA-based methods improve transferability in image classification and object detection, including adversarially trained settings and Bard attacks.The paper also reports that its methods can be combined with prior attacks such as MI, VMI, and SSA.Source: §1 · p. 2

2 RELATED WORK

Prior transfer-based attacks borrow optimization and augmentation ideas from model training, while ensemble methods combine surrogate models. The related work also identifies limits in model count, gradient-variance focus, and self-ensembled surrogates.

2 RELATED WORK

  • These approaches generally frame adversarial transferability by analogy with deep-learning generalization.The section groups the existing methods into three classes.Source: §2 · p. 2

Gradient-based optimization

  • Optimization-based attacks use momentum, Nesterov acceleration, gradient averaging, or neighborhood variance tuning to avoid poor optima or stabilize updates.The cited examples include MI, NI, EMI, and VMI.Source: §2 · p. 3

Input transformation

  • Input-transformation attacks diversify classifier inputs using resizing and padding, translated-image gradients, or multiple image scales.The cited methods are DI, TI, and SI.Source: §2 · p. 3

Model ensemble methods

  • Ensemble attacks average surrogate losses, probabilities, or logits, while other approaches create surrogate variants or reduce gradient variance.The paper notes that ensemble sizes are usually small, variance reduction need not improve generalization, and self-ensembled surrogates can be less effective than standard ones.Source: §2 · p. 3

3 METHODOLOGY

The methodology connects transferability to loss flatness and proximity to each model’s local optimum, then develops separate procedures for promoting those properties and combines them in CWA.

3 METHODOLOGY

  • The methodology defines common weakness using a flat landscape and proximity to each model’s local optimum.CWA is designed to optimize these two properties separately before combining them.Source: §3 · p. 3, §3.2 · p. 4

3.1 PRELIMINARIES

  • Transfer-based attacks optimize an adversarial example against a set of image classifiers, using a few training classifiers to approximate the broader objective.The setup uses an ℓ∞ constraint and describes loss-ensemble and logits-ensemble approximations.Source: §3.1 · p. 3

3.2 MOTIVATION OF COMMON WEAKNESS

  • A quadratic approximation motivates seeking low loss at model optima, a flat landscape, and small distances to each optimum; the theorem assumes zero covariance between Hessian norm and distance.The paper focuses on the latter two terms because it says there may be little room to improve loss at individual optima.Source: §3.2 · p. 4

3.3 SHARPNESS AWARE MINIMIZATION

  • SAM promotes flatness through an inner ascent and outer descent, modified for the attack’s ℓ∞ space and applied to the model ensemble.The SAM step can be combined with MI; repeated updates are described as converging to a flatter landscape and improving transferability.Source: §3.3 · pp. 4–5

3.4 COSINE SIMILARITY ENCOURAGER

  • CSE encourages closeness to model optima by using gradient similarity as a surrogate objective and successively updating with each model’s gradient.The method normalizes gradients by their ℓ2 norm, which the paper says maximizes cosine similarity and makes CSE compatible with SAM.Source: §3.4 · pp. 5–6

3.5 COMMON WEAKNESS ATTACK

  • CWA combines procedures for loss-landscape flatness and proximity to model optima, and can be integrated with MI, VMI, and SSA.The combined algorithm substitutes CSE for SAM’s second step to support parallel gradient backpropagation and manage time complexity.Source: §3.5 · p. 6

4 EXPERIMENTS

Experiments evaluate the methods across image classification, object detection, and large vision-language models, then analyze loss landscapes and computational efficiency. The reported results span black-box attack success, adversarial-patch transfer, and Bard attack outcomes.

4.1 ATTACK IN IMAGE CLASSIFICATION

  • 16 normally trained models lead in performance, while the methods significantly empower attacks on 8 adversarially trained RobustBench models.Against FastAT and PGDAT, the reported attack-success-rate improvement over existing methods is 40%.Source: Table 1 · p. 7, §4.1 · p. 8
  • MI-CSE is nearly 30% higher than MI against ViT-B/16; MI-SAM gains consistently, and MI-CWA further improves attack success.VMI-CWA and SSA-CWA outperform their vanilla versions, with SSA-CWA exceeding 99% attack success on most normal models.Source: §4.1 · p. 7

4.2 ATTACK IN OBJECT DETECTION

  • 20%: Adam-CWA exceeds Loss Ensemble mAP reduction on average across models, and reaches lower mAPs on the two white-box detectors than white-box attacks.Adam-CWA reports mAPs of 2.32 on YOLOv3 and 2.06 on YOLOv5, versus 12.35 and 7.98 for white-box attacks, respectively.Source: §4.2 · p. 8, Table 2 · p. 8

4.3 ATTACK IN LARGE VISION-LANGUAGE MODELS

  • SSA-CWA increases Bard misdescription by 8% over SSA and has a substantially higher rejection rate under both tested perturbation budgets.The reported Bard behaviors include misdescription, blindness, and refusal.Source: Figure 3 · p. 8, §4.3 · p. 9, Table 3 · p. 9

4.4 ANALYSIS AND DISCUSSION

  • MI-CWA produces a flatter loss landscape and closer model optima than MI around the crafted adversarial example.The paper presents these observations as support for CWA's flatness and local-optimum-closeness objectives.Source: §4.4 · p. 9, Figure 4 · p. 9
  • At equal iteration counts, CWA costs twice as much computation as MI, yet outperforms other algorithms across tested iteration counts, including T = 5.The paper attributes this effectiveness to capturing models' common weakness rather than using more iterations.Source: §4.4 · p. 9, Figure 4 · p. 9

5 CONCLUSION

The paper analyzes model ensembles for black-box adversarial attacks and proposes common weaknesses as a basis for finding more transferable examples. Experiments in image classification and object detection show that the algorithm enhances transferability.

5 CONCLUSION

  • Experiments in image classification and object detection show that the proposed algorithm finds common weaknesses and enhances adversarial-example transferability.The analysis relates transferability to Hessian Frobenius norm and distance from each model’s local optimum.Source: §5 · p. 9

ETHICS STATEMENT

The ethics statement identifies potential misuse of the method, while the appendix relates loss-landscape flatness and local-optimum proximity to transferability. Under a Gaussian assumption about surrogate optima, it connects these properties across training and testing.

ETHICS STATEMENT

  • Malicious attackers could use the method against large commercial models, potentially leading to toxic content generation or privacy leakage.The paper also notes that black-box attacks need not know specific details of the target models.Source: ETHICS STATEMENT · p. 10

A.1 PROOF OF THEOREM 3.1

  • The spectral and Frobenius norms of the Hessian correspond to loss-landscape flatness, motivating sharpness-aware optimization.Source: §A.1 · p. 16

A.2 THE GENERALIZATION ABILITY OF COMMON WEAKNESS

  • The appendix’s training objective seeks an ensemble optimum with smaller average Hessian Frobenius norm and distance to the surrogate models’ local optima.Source: §A.2 · p. 16
  • The appendix links smaller training-objective terms to smaller test error and argues that flatness and local-optimum proximity relate to transferability.It states that expected distances from optima increase monotonically with the Gaussian variance parameter.Source: §A.2 · pp. 16–17
  • Under the Gaussian local-optimum assumption, closer surrogate optima suggest closer target-model optima.The appendix states this relationship as a tendency, not a guarantee.Source: §A.2 · p. 17

A.3 PROOF OF THEOREM 3.2

The proof motivates cosine similarity between model gradients as a safer alternative to maximizing their dot product. The appendix then describes sharpness-aware optimization and its computational trade-offs.

A.3 PROOF OF THEOREM 3.2

  • Maximizing gradient dot-product similarity can cause gradient explosion, so the method instead maximizes cosine similarity.The proof states that identical local optima correspond to zero dot-product similarity between model gradients.Source: §A.3 · p. 18

B.1 SHARPNESS AWARE MINIMIZATION ALGORITHM UNDER INFINITY NORM

  • Using the Hessian Frobenius norm as an alternative flatness measure would require computationally expensive third-order derivatives with respect to p.Source: §B.1 · p. 18
  • The infinity-norm SAM procedure finds a worst-case perturbation, then optimizes the loss difference between perturbed and original inputs.The appendix says its objective emphasizes landscape flatness without simultaneously minimizing L(x).Source: §B.1 · pp. 18–19

B.2 DERIVATION OF COSINE SIMILARITY ENCOURAGER

The appendix develops a cosine-similarity update and a generalized CWA that can use different optimizers. Experiments report improved transfer attacks across diverse and less diverse surrogate settings, including unseen targets.

B.2 DERIVATION OF COSINE SIMILARITY ENCOURAGER

  • The cosine-similarity encourager is designed to improve gradient cosine similarity without directly computing costly second-order derivatives.The derivation says the update enhances gradient cosine similarity.Source: §B.2 · p. 19

B.3 GENERALIZED COMMON WEAKNESS ALGORITHM

  • CWA can be combined with arbitrary optimizers and attackers, using three optimizers that may be identical or different.Source: §B.3 · p. 20

C.1 MAIN EXPERIMENT SETTINGS

  • The experiments evaluate attacks on 31 black-box models, including normally trained, adversarially trained, and other defended models.Source: §C.1 · p. 21

C.2 COMPARISON WITH NASEER ET AL. (2021)

  • By an average of approximately 20%, the method surpasses previous techniques when attacking unseen target models.The comparison uses ensembles of similar surrogate models; Table C.1 reports attack success rate.Source: Table C.1 · p. 21, §C.2 · p. 22

C.3 ATTACKING USING LESS DIVERSE SURROGATE MODELS

  • With only ResNet-family surrogates, SSA-CWA still outperforms other attackers and generalizes to previously unseen targets.Source: §C.3 · p. 22

C.4 ATTACKING USING THE SURROGATE IN DONG ET AL. (2018)

SSA-CWA is evaluated with surrogate models chosen to test transfer to dissimilar targets. It outperforms prior methods on normally trained and defended models, including a 94.6% attack success rate against DiffPure.

C.4 ATTACKING USING THE SURROGATE IN DONG ET AL. (2018)

  • The evaluation uses four surrogates from Dong et al. (2018), with most target models dissimilar to those surrogates.This setup is intended to assess transfer to diverse, previously unseen models.Source: §C.4 · p. 22
  • SSA-CWA significantly outperforms previous methods when attacking normally trained and defended models.Table C.3 reports black-box attack success rates on NIPS2017.Source: §C.4 · p. 22, Table C.3 · p. 23
  • 94.6% attack success rate against DiffPure demonstrates SSA-CWA’s performance against a challenging defense.The reported comparison also says SSA-CWA significantly outperforms previous methods on normally trained and defended models.Source: §C.4 · pp. 22–23, Table C.3 · p. 23

C.5 EXPERIMENTS ON ϵ = 4/255

At ϵ = 4/255, the methods retain an advantage on most targets, especially adversarially trained models. For object detectors, CWA’s universal patch differs from the loss-ensemble patch and improves on the previous state of the art.

C.5 EXPERIMENTS ON ϵ = 4/255

  • The ϵ = 4/255 experiment matches a threat model commonly used to evaluate adversarially trained models and defenses.The authors chose this budget for consistency with those evaluations.Source: §C.5 · p. 23
  • About 5% average improvement over previous methods on adversarially trained models holds at ϵ = 4/255.The methods achieve superior results on most target models under this perturbation budget.Source: §C.5 · p. 23, Table C.4 · p. 24

C.6 VISUALIZATION OF ADVERSARIAL PATCHES

  • The loss-ensemble patch resembles YOLOv5’s, while Adam-CWA produces a markedly different patch that captures the models’ common weakness.The authors hypothesize the loss ensemble relies nearly solely on YOLOv5 because it is more vulnerable than YOLOv3; they report their patch is more effective against the detectors.Source: §C.6 · pp. 23–24, Figure C.1 · p. 23
  • 4.26% improvement over the enhanced-baseline loss ensemble yields 4.69% mAP on eight testing models.The universal patch is generated by ensembling the models in Huang et al. (2023).Source: §C.6 · p. 24

C.7 ABLATION STUDIES

The ablations show that CWA’s performance depends on tuning its inner and reverse steps, while additional analyses examine the approximation, loss landscapes, and gradient similarity underlying the approach.

C.7 ABLATION STUDIES

  • As β increases, the regularization term’s contribution and attack success rate rise, but excessive β increases algorithm error and can flatten or reduce success.The authors recommend choosing β to balance the regularization effect and overall performance.Source: §C.7 · pp. 24–25
  • Attack success initially rises as 16/255/r increases, then falls when the reverse step is too large; further reducing it makes MI-CWA converge toward MI-CSE.The authors attribute the initial decline to an optimization direction opposing the forward step.Source: §C.7 · p. 25
  • When α < 0.01, the reverse step exceeds the forward step and efficacy drops sharply; for α > 0.01, success remains relatively consistent.This pattern is reported as resilience to hyperparameter variations above the threshold.Source: §C.7 · p. 25
  • Without MI-FGSM, the methods still outperform FGSM and BIM, but performance falls significantly relative to MI-FGSM-combined results.The passage describes MI-FGSM as a popular plug-and-play module for improving attack algorithms.Source: §C.7 · p. 25, Table C.5 · p. 25

D.1 VERIFICATION OF THE APPROXIMATION IN CSE

  • The NIPS17 heatmap shows normalized-gradient matrix diagonals near 1 and off-diagonal elements near 0.The matrix is downsampled to 256 × 256 for visualization.Source: §D.1 · p. 26, Figure D.1 · p. 26

D.2 VISUALIZATION OF LANDSCAPE

  • MI-CWA produces a flatter loss landscape and closer model optima than MI around the crafted adversarial example.The visualization uses six surrogate models and tests against RobustBench defense models.Source: §D.2 · p. 26

D.3 COSINE SIMILARITY OF GRADIENTS

  • MI-CWA increases gradient cosine similarity between surrogate and black-box models compared with MI.The authors interpret this as indicating that surrogate-model common weakness tends to generalize to testing models.Source: §D.3 · p. 26

D.4 TIME COMPLEXITY ANALYSIS

The time-complexity analysis considers both function evaluations and measured runtime. The authors report efficient operation with only slight computational overhead relative to baseline attacks.

D.4 TIME COMPLEXITY ANALYSIS

  • MI-SAM, MI-CSE, and MI-CWA are reported as efficient by number of function evaluations, where n is the number of surrogate models.Table D.2 lists the NFEs and real-time cost of the methods.Source: §D.4 · pp. 26–27, Table D.2 · p. 27
  • When combined with previous attacks, the methods maintain efficiency while enhancing efficacy.The authors note potential for integration with leading-edge attacks and optimizers.Source: §D.4 · p. 27
  • The methods incur only slight computational overhead in real-time adversarial-example generation compared with baseline attacks.The evaluation uses the surrogate models described in Section 4.1.Source: §D.4 · p. 27

Related papers

Loading 2303.09105v2…