PaperlayerView exact revision on arXiv

Source-linked AI summary

On Evaluating Adversarial Robustness of Large Vision-Language Models

Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin

arXiv:2305.16934v2cs.CVcs.CLcs.CRcs.LGcs.MM

TL;DR

Visual inputs raise safety concerns because adversaries may evade image-grounded generation, making VLM robustness to black-box targeted attacks an important question. The paper evaluates such attacks by crafting examples with pretrained CLIP and BLIP and transferring them to other VLMs; its findings indicate that system robustness depends on its most vulnerable input modality.

  • Problem

    Visual inputs raise safety concerns because adversaries may use imperceptible perturbations to evade image-grounded text generation, motivating evaluation under black-box targeted attacks.

    Source: §1 · p. 2

  • Method

    The authors evaluate open-source VLMs under black-box access by crafting targeted adversarial examples with pretrained CLIP and BLIP surrogates and transferring them to other VLMs.

    Source: §1 · p. 2, §3.2 · p. 5

  • Results

    The findings indicate that multimodal system robustness is highly dependent on its most vulnerable input modality.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors advocate comprehensive examination of VLM security defects before deployment.

    Source: §1 · p. 2

    The work focuses on digital inputs fed directly into models; evaluating attacks on images captured in real time in physical environments remains future work.

    Source: §5 · p. 10

Abstract

from arXiv · show

Large vision-language models (VLMs) such as GPT-4 have achieved unprecedented performance in response generation, especially with visual inputs, enabling more creative and adaptable interaction than large language models such as ChatGPT. Nonetheless, multimodal generation exacerbates safety concerns, since adversaries may successfully evade the entire system by subtly manipulating the most vulnerable modality (e.g., vision). To this end, we propose evaluating the robustness of open-source large VLMs in the most realistic and high-risk setting, where adversaries have only black-box system access and seek to deceive the model into returning the targeted responses. In particular, we first craft targeted adversarial examples against pretrained models such as CLIP and BLIP, and then transfer these adversarial examples to other VLMs such as MiniGPT-4, LLaVA, UniDiffuser, BLIP-2, and Img2Prompt. In addition, we observe that black-box queries on these VLMs can further improve the effectiveness of targeted evasion, resulting in a surprisingly high success rate for generating targeted responses. Our findings provide a quantitative understanding regarding the adversarial vulnerability of large VLMs and call for a more thorough examination of their potential security flaws before deployment in practice. Code is at https://github.com/yunqing-me/AttackVLM.

1 Introduction

The paper evaluates targeted adversarial attacks on open-source VLMs in a black-box setting and finds that their robustness depends on their most vulnerable input modality. Transfer-based attacks achieve high success, and query-based attacks can improve evasion further.

1 Introduction

  • Image-grounded text generation raises heightened safety concerns because human-imperceptible visual perturbations can evade models used in complex, safety-critical environments.Potential consequences include bypassing safety or privacy checkers, injecting malicious code, or accessing APIs and manipulating devices without authorization.Source: §1 · p. 2
  • Transfer-based attacks from pretrained CLIP [65] [80] and BLIP [41] achieved high targeted-response success on open-source VLMs, and black-box queries with transfer priors improved evasion further.The attacks transfer to MiniGPT-4 [109], LLaVA [46], UniDiffuser [5], BLIP-2 [42], and Img2Prompt.Source: §1 · p. 2
  • The findings indicate that multimodal-system robustness depends strongly on its most vulnerable input modality, motivating more comprehensive security examination before deployment.Source: §1 · p. 2

2 Related work

Prior work studied adversarial robustness in language models and vision-language tasks, but earlier VLM attack studies largely used conventional models and less realistic settings. This work evaluates advanced VLMs under black-box access and targeted goals without human labor.

Language models (LMs) and their robustness

Vision-language models (VLMs) and their robustness

  • Earlier VQA and image-captioning attacks mostly targeted conventional CNN-RNN models with white-box access or untargeted goals and required human interaction; this work evaluates advanced VLMs black-box and targeted.The evaluation is quantitative and free of human labor.Source: §2 · p. 3
  • Vision-language models apply language-model knowledge to vision-language tasks [26] [33] [84] [93] [101], and prior adversarial work addressed visual question answering [6] [11] [37] [38] [43] [77] [89] and image captioning [1] [14].Source: §2 · p. 3

3 Methodology

The methodology formalizes image-grounded generation and a black-box targeted threat model, then describes surrogate-based transfer attacks and query-based refinement.

3.1 Preliminary

  • The threat model targets a predefined response through a small, imperceptible image perturbation under black-box victim access, while keeping the input text fixed.The model maps an image and input text to output text, and this mapping may be probabilistic or non-deterministic [5] [98].Source: §3.1 · pp. 3–4

3.2 Transfer-based attacking strategy

  • Transfer attacks use white-box surrogate encoders to optimize adversarial images toward target responses through cross-modal MF-it or generated-image MF-ii matching.MF-it can be optimized with projected gradient descent; MF-ii compares image features with a target image from a text-to-image generator, which requires no gradients and can be accessed through APIs.Source: §3.2 · pp. 5–6

3.3 Query-based attacking strategy

  • Query-based attacks maximize textual similarity between victim outputs and the target, estimating gradients with random gradient-free sampling and refining transfer-crafted images.The gradient estimate uses N random perturbations; its approximation becomes unbiased as σ → 0 and N → ∞.Source: §3.3 · pp. 6–7

4 Experiment

The experiments evaluate targeted adversarial attacks on open-source VLMs and find that transfer-based examples can mislead them, with query-based refinement further improving target similarity. Further analyses examine perturbation visibility, budget tradeoffs, and how attacks redirect model attention.

4.1 Implementation details

  • The experiments use ImageNet-1K validation images, MS-COCO target captions, pretrained VLM weights, and default ϵ = 8 with an ℓ∞ bound of 8.Transfer attacks use 100-step PGD; query attacks use N = 100 queries and 8-step PGD updates.Source: §4.1 · pp. 7–8

4.2 Empirical studies

  • Adversarial images mislead MiniGPT-4 into targeted and unexpected descriptions despite its concrete, correct descriptions of clean images.For UniDiffuser, a text-to-image then image-to-text generation chain can produce content different from the original description.Source: §4.2 · p. 9
  • Query-based MF-tt refinement of MF-ii transfer examples makes generated responses significantly more similar to targets, while MF-it transfers worse than MF-ii, suggesting overfitting.Both MF-it and MF-ii increase target-text similarity against surrogate models in white-box tests; MF-ii transfer lays the foundation for black-box evasion.Source: §4.2 · p. 9

4.3 Further analyses

  • Successful targeted perturbations and their captions show no semantic information associated with the target, indicating that large VLMs are inherently vulnerable.Figure 5 visualizes perturbations and generated captions; the paper reports similar conclusions using other models.Source: Figure 5 · p. 8, §4.3 · p. 9
  • Increasing ϵ trades image fidelity against targeted-response success, so the perturbation budget must be chosen appropriately.The study evaluates ϵ values of 2, 4, 8, 16, and 64 and measures clean-to-adversarial perceptual distance with LPIPS (↓).Source: Figure 6 · p. 8, §4.3 · p. 9
  • Under a fixed total perturbation budget of 8, a proper combination of transfer-based and query-based attacks achieves the best performance.The study varies ϵt and ϵq subject to ϵt + ϵq = 8 and evaluates UniDiffuser.Source: Figure 7 · p. 9, §4.3 · p. 10
  • GradCAM visualizations indicate that adversarial images suppress relevance to the original description and mimic the targeted image’s attention map.The paper cautions that GradCAM has known limitations as a feature-attribution method.Source: §4.3 · p. 10, Figure 8 · p. 10

5 Discussion · Appendix

The discussion argues that automated visual adversarial attacks expose worst-case risks in open-source VLM deployments, while noting misuse potential and the digital-only scope; the appendix adds implementation details, additional analyses, and failure cases.

5 Discussion

  • Visual-input manipulation can be automated to fool entire VLM systems, and its adversarial effect can persist across multi-round interaction.The authors point to Figure 3 as showing this persistence.Source: §5 · p. 10
  • Open-source models including MiniGPT-4 [109] and LLaVA [46] [45] have unexamined worst-case behavior, posing potential risks when used as product plugins.Source: §5 · p. 10
  • The black-box, targeted-response threat model could enable misuse against deployed APIs such as GPT-4 with visual inputs or Midjourney, especially when integrated as product plugins.The authors identify potential negative societal impacts from such misuse.Source: §5 · p. 10
  • The study focuses on digital inputs that feed directly into models, unlike future deployments in robots or autonomous driving that capture real-time images from physical environments.Source: §5 · p. 10
  • Evaluating physical-world adversarial attacks is identified as a future direction for assessing VLM security.Source: §5 · p. 10

Appendix

  • The appendix presents implementation details, additional experiment results and analyses, and failure cases to clarify the proposed attacks’ capabilities.Source: Appendix · p. 18

A Implementation details

The appendix specifies data construction, target-image generation, attack hyperparameters, and a two-stage MF-ii + MF-tt procedure for reproducing the experiments.

Examples of how the datasets are utilized

  • ImageNet-1K validation images serve as clean inputs, paired with randomly selected MS-COCO [44] captions as irrelevant target texts.Stable Diffusion [72] generates corresponding target images for the illustrated clean image–target text–target image examples.Source: A Implementation details · p. 18, Figure 9 · p. 18

Text-to-image models for targeted image generation

  • Stable Diffusion [72], Midjourney, and DALL-E [67] [68] generate target images for flexible target texts; generated and real MS-COCO targets perform comparably qualitatively and quantitatively.This design allows adversaries to define target texts adaptively rather than restrict them to a specific dataset.Source: A Implementation details · p. 18

Hyperparameters

  • PGD uses ϵ = 8, step size 1, pixel clamping to [0, 255], and ∆ = 0 initialization; Gaussian initialization gives comparable results.MF-tt uses σ = 8 and δ ∼N(0, I), and adversarial images are saved as PNG to avoid compression or loss.Source: A Implementation details · pp. 18–19

Attacking algorithm

  • MF-ii + MF-tt combines substitute-model image-gradient optimization with black-box response queries because MF-it transfers poorly to large VLMs.MF-ii increases embedding similarity between the adversarial image and the generated target image; MF-tt estimates a pseudo-gradient from queried outputs and updates the perturbation.Source: A Implementation details · p. 19

Amount of computation

  • Experiments can run on a single GPU, and Table 3 reports computation and CO2 emissions for each experiment.The appendix describes the computational demand as low.Source: A Implementation details · p. 19

B Additional experiments

The paper reports that its main experiments covered six cutting-edge large VLMs across varied datasets and setups, and this section adds experiments, visualizations, and analyses to supplement those findings.

  • The additional experiments section supplements the main paper’s results from six cutting-edge large VLMs across various datasets and setups with further experiments, visualizations, and analyses.Source: B Additional experiments · p. 19

B.1 Image captioning task by BLIP-2

Adversarial examples caused pretrained BLIP-2 to generate captions resembling predefined target text, even when the clean images depicted unrelated scenes.

B.1 Image captioning task by BLIP-2

  • BLIP-2 generated captions resembling predefined targets from adversarial images, although the corresponding clean images depicted unrelated scenes.For example, a vaporwave-computer target elicited “A cartoon drawn on the side of an old computer” from an image of flowers and sky; a mushroom target elicited “A stuffed white mushroom sitting next to leaves” from an image of a girl playing with bubbles.Source: §B.1 · p. 19

B.2 Joint generation task by UniDiffuser

In UniDiffuser’s joint-generation task, crafted adversarial examples elicit targeted responses across generation paradigms, redirecting image generation away from the clean image’s text-conditioned semantics.

B.2 Joint generation task by UniDiffuser

  • Crafted adversarial examples elicit a targeted hamburger response and image from a sports-car description in UniDiffuser’s joint-generation task [5].The clean image is generated conditioned on “A pencil drawing of a cool sports car,” while the adversarial example produces a close-up hamburger response and a hamburger image.Source: §B.2 · p. 20

B.3 Visual question-answering task by MiniGPT-4 and LLaVA

In multi-round VQA, minimal image perturbations can steer MiniGPT-4 and LLaVA toward responses semantically similar to predefined targets.

B.3 Visual question-answering task by MiniGPT-4 and LLaVA

  • With a minimal perturbation budget (e.g., ϵ = 8), MiniGPT-4 [109] and LLaVA [46] produce multi-round VQA responses semantically similar to targets, including recognizing a monkey warrior as an astronaut riding a horse in space.The predefined target is “An astronaut riding a horse in the sky”.Source: §B.3 · p. 20

B.4 Interpretability of the attacking mechanism against large VLMs · C Additional discussion · C.1 Sensitivity of adversarial examples to random perturbation

The discussion uses GradCAM visualizations to show how adversarial images redirect attention while eliciting targeted responses, and finds that stronger Gaussian noise gradually weakens those responses. It also frames sensitivity and failure cases as limitations to examine.

B.4 Interpretability of the attacking mechanism against large VLMs

  • Crafted adversarial images can generate targeted responses with semantic meanings different from the clean image’s text description.Additional visualizations are presented in Figure 14.Source: §B.4 · p. 20
  • For questions about clean-image content, GradCAM highlights the corresponding clean-image area but ignores that same area in the adversarial image.The passage illustrates this with the question “How many people in this iamge?”Source: §B.4 · p. 20
  • For questions about targeted text, GradCAM highlights an adversarial-image area similar to the targeted image.The passage illustrates this with “where is the corn cob?”; additional visualizations appear in Figure 14.Source: §B.4 · p. 21

C Additional discussion

C.1 Sensitivity of adversarial examples to random perturbation

  • Adversarial examples are reasonably insensitive to Gaussian noise, but increasing noise amplitude diminishes their effectiveness and shifts responses back toward the original.The perturbations were zero-mean Gaussian noise added to adversarial images before response generation.Source: §C.1 · p. 21
  • With σnoise = 0, the response resembles the targeted text; at 0.025 it changes, and ultimately degrades to a response about the original sunflowers.The examples progress from “A red and black bird sitting on top of a tree branch” to “A red and black bird is sitting on top of a sunflower” and then “A large painting of three sunflowers in a field”.Source: §C.1 · p. 21

C.2 Failure cases

Although the attacks do not always achieve their targets, failure cases show responses falling between descriptions of the clean image and the predefined target. Increasing attack steps can address this without increasing the perturbation budget.

C.2 Failure cases

  • The attacks’ success rate is not 100%, and the reported failure cases are left for future work to improve performance.The paper presents these cases in Figure 16.Source: §C.2 · p. 21
  • Failure cases produced responses between descriptions of the clean image and the predefined target.The examples use LLaVA as the conversation platform, with similar observations reported for other large VLMs.Source: §C.2 · p. 21, Figure 16 · p. 28
  • Increasing attack steps beyond the 100 used in the main experiments could address these failures while keeping the perturbation budget unchanged (e.g., ϵ = 8).The figure caption describes this as an effective way to address the intermediate responses.Source: Figure 16 · p. 28

Related papers

Loading 2305.16934v2…