On Evaluating Adversarial Robustness of Large Vision-Language Models

Yunqing Zhao; Tianyu Pang; Chao Du; Xiao Yang; Chongxuan Li; Ngai-Man Cheung; Min Lin

Source-linked AI summary

On Evaluating Adversarial Robustness of Large Vision-Language Models

Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin

arXiv:2305.16934v2May 26, 2023cs.CVcs.CLcs.CRcs.LGcs.MM

TL;DR

Large VLMs expose safety risks because adversaries can manipulate vulnerable visual inputs to induce targeted outputs. The paper evaluates this threat by transferring surrogate-crafted adversarial images to open-source VLMs and refining them through black-box queries, finding high targeted-evasion effectiveness and deeper vulnerability across multimodal systems.

Problem
Evidence remains limited on whether large VLMs resist targeted adversarial manipulation when attackers have only black-box access and can alter visual inputs.
Method
The paper crafts targeted examples on pretrained CLIP and BLIP surrogates, transfers them to large VLMs, and strengthens evasion using query-based attacks.
Results
Transfer-based attacks already induce targeted responses with a high success rate, while query-based attacks further improve targeted evasion across evaluated VLMs.
Takeaways & Limitations
The findings provide quantitative understanding of large VLM vulnerability and show that multimodal robustness depends strongly on the most vulnerable input modality.
Takeaways & Limitations
The evaluation focuses on digital settings where images are fed directly into models, leaving physical-world attacks for future work.

Abstract

from arXiv · show

Large vision-language models (VLMs) such as GPT-4 have achieved unprecedented performance in response generation, especially with visual inputs, enabling more creative and adaptable interaction than large language models such as ChatGPT. Nonetheless, multimodal generation exacerbates safety concerns, since adversaries may successfully evade the entire system by subtly manipulating the most vulnerable modality (e.g., vision). To this end, we propose evaluating the robustness of open-source large VLMs in the most realistic and high-risk setting, where adversaries have only black-box system access and seek to deceive the model into returning the targeted responses. In particular, we first craft targeted adversarial examples against pretrained models such as CLIP and BLIP, and then transfer these adversarial examples to other VLMs such as MiniGPT-4, LLaVA, UniDiffuser, BLIP-2, and Img2Prompt. In addition, we observe that black-box queries on these VLMs can further improve the effectiveness of targeted evasion, resulting in a surprisingly high success rate for generating targeted responses. Our findings provide a quantitative understanding regarding the adversarial vulnerability of large VLMs and call for a more thorough examination of their potential security flaws before deployment in practice. Code is at https://github.com/yunqing-me/AttackVLM.

1 Introduction

Large VLMs offer powerful visual interaction but introduce serious safety risks because adversaries can manipulate vulnerable visual inputs. This work evaluates black-box targeted attacks and finds that transfer- and query-based methods can induce targeted responses across open-source VLMs.

Motivation: Visual inputs create serious safety concerns because imperceptible perturbations can mislead image-grounded text generation in safety-critical environments.Potential consequences include bypassing safety or privacy checkers, injecting malicious code, and accessing APIs.
Evaluation scope: The study evaluates open-source VLM robustness under black-box access, targeted response goals, and visual-input manipulation.Target models include MiniGPT-4, LLaVA, UniDiffuser, BLIP-2, and Img2Prompt.
Findings: Transfer-based attacks already induce targeted responses with a high success rate across large VLMs.The attacks craft examples using pretrained CLIP and BLIP surrogate models before transferring them to victim VLMs.
Findings: Query-based attacks using transfer-based priors further improve targeted evasion effectiveness against these VLMs.The effect is illustrated for BLIP-2, UniDiffuser, and MiniGPT-4.
Implications: The findings quantify large VLM vulnerability and indicate that robustness in multimodal systems depends heavily on their most vulnerable input modality.The paper advocates comprehensive security examination before deployment.

2 Related work

Prior robustness research examined language and vision-language attacks, but commonly assumed white-box access, untargeted goals, conventional architectures, or human interaction. This paper addresses the more challenging black-box targeted setting for large VLMs.

Language-model robustness: Language-model research established that textual systems can be evaded through adversarial and natural-language attacks.Examples include attacks against sentiment analysis, textual entailment, and BERT.
VLM development: Large language models provide the knowledge used to support modern vision-language tasks.The related-work discussion connects advances in large LMs with VLM development.
VLM robustness: Early VLM attack studies focused mainly on conventional CNN-RNN models for visual question answering and image captioning.Many assumed white-box access or untargeted adversarial goals and required human interaction.
Research gap: The paper differs from much prior VLM robustness work by studying targeted attacks with black-box victim access.Its threat model also restricts the adversary to manipulating visual inputs.

3 Methodology

The methodology combines surrogate-model transfer attacks with query-based refinement against black-box VLMs. It matches adversarial images to target representations, estimates victim-model gradients from queries, and evaluates transferability under constrained perturbations.

3.1 Preliminary: The victim model pθ(x; cin) maps an image and input text to generated output text for captioning or visual question answering.The mapping may be probabilistic or non-deterministic.
3.1 Preliminary: The threat model assumes black-box victim access, targeted responses, imperceptible image perturbations, and no manipulation of input text.The perturbation budget is constrained by a small image-space bound.
3.2 Transfer-based attacking strategy: Transfer-based attacks craft adversarial images on accessible surrogate encoders and then feed them to black-box victim models.The approach uses image and text encoders to optimize similarity with a predefined target response.
3.2 Transfer-based attacking strategy: MF-ii generates a target image from ctar and maximizes image-image feature similarity between that image and the adversarial example.The target-image generator requires only black-box access because its gradients are not needed.
3.2 Transfer-based attacking strategy: Table 1 reports CLIP similarity between crafted images and targeted texts for MF-it and MF-ii against surrogate models.MF-ii reaches similarity comparable to generated target images, while MF-it achieves higher similarity by directly matching cross-modality features.
3.3 Query-based attacking strategy: MF-tt maximizes textual similarity between the victim output and ctar using random gradient-free estimation from black-box queries.The estimator uses random perturbations, sampling variance σ, and N queries; it becomes unbiased as σ → 0 and N → ∞.
3.3 Query-based attacking strategy: Transfer-based adversarial examples initialize query-based optimization, combining MF-ii and MF-tt to strengthen black-box evasion.The combined strategy is empirically evaluated against large VLMs.
3.3 Query-based attacking strategy: Table 2 compares clean and adversarial outputs using CLIP scores against predefined targets across victim models and attack variants.The comparison includes MF-it, MF-ii, and MF-ii + MF-tt, with a fixed textual input prompt.

4 Experiment

The experiments evaluate targeted adversarial attacks against open-source large VLMs using clean images, targeted captions, surrogate models, and black-box victim models. Results show transferability, query-based improvement, perceptual trade-offs, and attention shifts toward targeted content.

4.2 Empirical studies: Adversarial images consistently mislead models such as BLIP-2 and MiniGPT-4 into generating responses with the targeted semantics despite clean images receiving concrete descriptions.MiniGPT-4 additionally produces unexpected descriptions not shown in the targeted text.
4.2 Empirical studies: MF-ii and MF-it increase target-text similarity on surrogate models, but MF-it transfers worse to large VLMs, suggesting overfitting during optimization.The surrogate-model evaluation uses CLIP, BLIP, and ALBEF with 50K ImageNet-1K validation images.
4.3 Further analyses: ϵ = 8 is used by default because larger perturbation budgets degrade image quality while targeted-response effectiveness saturates.The trade-off is measured using LPIPS distance between clean and adversarial images.
4.3 Further analyses: A proper combination of transfer-based and query-based attacks achieves the best performance under the fixed constraint ϵt + ϵq = 8.The victim model in this analysis is UniDiffuser.
4.3 Further analyses: GradCAM shows that adversarial images suppress relevance to the original description and mimic the attention map of the targeted image.The mechanism is visualized for Img2Prompt using clean, adversarial, and targeted images.

5 Discussion

The discussion argues that visual-input attacks can automate evasion of large multimodal systems and should be considered before deployment. It also identifies physical-world deployment as an important boundary for future evaluation.

5 Discussion: Manipulating visual inputs can be automated, unlike textual-input attacks that may require human-in-the-loop prompt engineering, enabling effective fooling of the large VLM.The discussion frames worst-case evaluation through red teaming or adversarial attacks as essential before deployment.
5 Discussion: The attack strategies could be misused against deployed vision-language APIs, especially when those APIs operate as plugins in other products.The threat model assumes black-box access and targeted responses.
5 Discussion: The work focuses on digital settings where images feed directly into models, not physical environments captured by cameras in real time.Physical-world attacks are identified as a future direction for evaluating VLM security.
5 Discussion: The appendix adds implementation details, additional experiments, analyses, and failure cases supporting the proposed attack methods.These materials are intended to clarify attack capability and empirical results.

A Implementation details

The implementation combines transfer-based image attacks with query-based refinement under black-box access, using generated target images and bounded pixel perturbations. The appendix specifies the algorithm, datasets, hyperparameters, and computational requirements.

A Implementation details: Clean images come from ImageNet-1K, target captions from MS-COCO, and Stable Diffusion generates target images for flexible target-text selection.The setup avoids requiring a corresponding real image for each predefined target caption.
A Implementation details: The default perturbation budget is ϵ = 8 with pixel values clamped to [0, 255], unit PGD step size, and zero perturbation initialization.Gaussian initialization is reported to produce comparable results.
A Implementation details: The implemented algorithm uses MF-ii followed by MF-tt because MF-it has poor transferability on large VLMs.The method initializes perturbations at zero and applies bounded updates.
A Implementation details: The attack takes a clean image, surrogate and victim models, a target text, a text-to-image generator, and its generated target image as inputs.The surrogate may use a CLIP visual encoder, while UniDiffuser is an example victim model.
A Implementation details: MF-ii optimizes perturbations by maximizing embedding similarity between the adversarial image and the generated target image.The image and target embeddings are normalized before similarity is computed and gradients update the perturbation.
A Implementation details: MF-tt refines the perturbation through victim-model queries, estimating a gradient from outputs on randomly perturbed images.The final procedure clamps the perturbation and image values before returning the queried captions and adversarial image.
A Implementation details: The experiments can run on a single GPU, and the appendix reports computation and CO2 emissions for each experiment.The authors characterize the computational demand as low in practice.

B Additional experiments

The appendix provides additional results, visualizations, and analyses to supplement the main paper’s experiments on large VLM attacks.

B Additional experiments: The additional-experiment section supplements the main paper with further results, visualizations, and analyses.It builds on experiments involving six cutting-edge large VLMs across varied datasets and setups.

B.1 Image captioning task by BLIP-2

Crafted adversarial examples caused pretrained BLIP-2 to generate text sufficiently similar to predefined targeted responses. The experiments were primarily based on pretrained models and used modest computational resources.

The experiments primarily used pretrained surrogate, generation, and victim models.This made the computational requirements feasible for individual practitioners to reproduce the results.
Table 1 reports 126 GPU hours and 9.45 CO2 emission values for the repeated experiment.

B.2 Joint generation task by UniDiffuser

Against UniDiffuser, crafted adversarial examples elicited targeted responses across multiple joint-generation paradigms. The resulting generation could depart entirely from the original text description’s semantic meaning.

Crafted adversarial examples elicited targeted responses in UniDiffuser across multiple generation paradigms.
A clean sports-car description led to a hamburger-related response and a resulting hamburger image.The generated image was completely different from the semantic meanings of the original text description.

B.3 Visual question-answering task by MiniGPT-4 and LLaVA

With a minimal perturbation budget, adversarial examples caused MiniGPT-4 and LLaVA to produce responses semantically similar to predefined targeted text in multi-round VQA.

MiniGPT-4 and LLaVA generated responses semantically similar to predefined targeted text under a minimal perturbation budget.The passage gives ϵ = 8 as an example of the perturbation budget.
In one example, a monkey warrior acting as Jedi was recognized as an astronaut riding a horse in space.This response was close to the targeted text “An astronaut riding a horse in the sky.”

B.4 Interpretability of the attacking mechanism against large VLMs

GradCAM visualizations indicate that adversarial images alter which image regions guide targeted responses. Additional experiments also examine attack sensitivity to random Gaussian noise and failure cases.

Interpretability: Compared with clean images, adversarial images produced targeted responses with semantic meanings different from the clean image’s text description.
Interpretability: For content-related questions, GradCAM highlighted corresponding regions in clean images but ignored those regions in adversarial images.
Interpretability: For target-related questions such as “where is the corn cob?”, GradCAM highlighted adversarial-image regions resembling the targeted image.
Sensitivity and limitations: The section also examines failure cases to clarify limitations of the attacks.
Sensitivity and limitations: The analysis evaluates attack sensitivity by adding zero-mean Gaussian noise with standard deviation σnoise to adversarial images.The authors report that the adversarial examples are reasonably insensitive to this perturbation.

C.2 Failure cases

The appendix reports that adversarial attacks on large VLMs can fail or produce responses between the clean-image description and the targeted text. It also shows that attack-step increases can mitigate some failures, while strong Gaussian noise degrades targeted responses.

Failure cases: Attack success is not 100%, with failure cases left for future performance improvements.The authors state that these failures were discovered during experiments and refer to Figure 16 for specifics.
Observed responses: Failure examples include models producing clean-image descriptions despite questions targeting different content.For example, one response identifies a fish and states that no boy appears when asked what the boy is doing.
Robustness to post-processing: Adversarial examples are relatively insensitive to Gaussian noise, but larger noise makes their effectiveness marginal and targeted responses gradually revert toward correct responses.The appendix presents this sensitivity analysis using the standard deviation σ_noise of noise added to x_adv.
Failure cases: Some adversarial responses fall between the clean image description and the predefined targeted text.The figure uses LLaVA as the conversation platform, with similar observations reported for other large VLMs.
Failure cases: Increasing the number of adversarial-attack steps can effectively address intermediate-response failures.The main experiments used 100 steps, while the appendix reports improvement when more steps are used.