PaperlayerView exact revision on arXiv

Source-linked AI summary

Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt

Zonghao Ying, Aishan Liu, Tianyuan Zhang, Zhengmin Yu, Siyuan Liang, Xianglong Liu, Dacheng Tao

arXiv:2406.04031v2cs.CVcs.CR

TL;DR

Existing LVLM jailbreaks often perturb only visual inputs, although aligned models fuse visual and textual features. BAP optimizes both modalities, and the paper reports that it significantly outperforms other methods across evaluations on datasets and LVLMs.

  • Problem

    Existing LVLM jailbreaks predominantly perturb visual inputs, while aligned models fuse visual and textual features for generation.

    Source: §1 · p. 1, §7 · p. 9

  • Method

    BAP optimizes visual and textual prompts together, using query-agnostic image perturbations and intent-specific textual optimization.

    Source: §1 · p. 2, §4.1 · p. 5, §4.2 · p. 6

  • Results

    BAP significantly outperforms other methods across evaluations on various datasets and LVLMs.

    Source: §7 · p. 9

  • Takeaways & Limitations

    The paper reports that BAP can jailbreak commercial LVLMs and can be used to evaluate LVLM bias and robustness.

    Source: §1 · p. 2

    Textual prompt optimization requires significant computational resources because it involves multiple interactions with LVLMs and LLMs.

    Source: §7 · p. 10

Abstract

from arXiv · show

In the realm of large vision language models (LVLMs), jailbreak attacks serve as a red-teaming approach to bypass guardrails and uncover safety implications. Existing jailbreaks predominantly focus on the visual modality, perturbing solely visual inputs in the prompt for attacks. However, they fall short when confronted with aligned models that fuse visual and textual features simultaneously for generation. To address this limitation, this paper introduces the Bi-Modal Adversarial Prompt Attack (BAP), which executes jailbreaks by optimizing textual and visual prompts cohesively. Initially, we adversarially embed universally harmful perturbations in an image, guided by a few-shot query-agnostic corpus (e.g., affirmative prefixes and negative inhibitions). This process ensures that image prompt LVLMs to respond positively to any harmful queries. Subsequently, leveraging the adversarial image, we optimize textual prompts with specific harmful intent. In particular, we utilize a large language model to analyze jailbreak failures and employ chain-of-thought reasoning to refine textual prompts through a feedback-iteration manner. To validate the efficacy of our approach, we conducted extensive evaluations on various datasets and LVLMs, demonstrating that our method significantly outperforms other methods by large margins (+29.03% in attack success rate on average). Additionally, we showcase the potential of our attacks on black-box commercial LVLMs, such as Gemini and ChatGLM.

1 Introduction

LVLMs can produce untruthful or harmful outputs, while prior jailbreaks often perturb only images and can be limited when models fuse visual and textual features. BAP perturbs visual and textual prompts together, and the paper evaluates it on open-source and commercial LVLMs.

1 Introduction

  • LVLMs combine visual and textual information and have shown performance across tasks including image captioning, visual question answering, and image retrieval.The introduction also notes that their behavior can be misaligned with creators’ intended goals, producing untruthful or potentially harmful outputs.Source: §1 · p. 1
  • Existing LVLM jailbreaks often perturb only images, although aligned models use visual and textual features together and may refuse an adversarial image without a text prompt.The paper gives LLaVA as an example of such a refusal.Source: §1 · p. 2
  • BAP is designed to jailbreak LVLMs by perturbing visual and textual modalities together, addressing the limits of attacks that perturb only visual inputs.The method uses a query-agnostic corpus for visual perturbations and LLM chain-of-thought reasoning to refine text prompts for a specific harmful intent.Source: §1 · p. 2
  • Experiments on popular open-source LVLMs in white- and black-box settings report successful jailbreaks on commercial LVLMs and potential use in evaluating model bias and robustness.The cited commercial systems include Gemini and ChatGLM in the paper abstract, but that detail is not included in this passage.Source: §1 · p. 2

2 Related Work

Related work covers prompt-based jailbreaks for LLMs and LVLMs, as well as adversarial examples. The paper distinguishes its focus on exploiting the interaction of visual and textual modalities from prior LVLM attacks centered on visual prompts.

2 Related Work

  • LLM jailbreak approaches are grouped into hand-designed, model-generation, and search-generation prompt designs.The passage cites [44] for hand-designed, [45] [46] for model-generation, and [39] [47] for search-generation approaches.Source: §2 · p. 3
  • Prior LVLM jailbreaks described here use typographic images, convert harmful semantics into visual prompts, hide harmful semantics in adversarial examples, or optimize visual prompts with harmful corpora.The cited approaches include Gong et al. [27], Liu et al. [37], Shayegani et al. [9], Qi et al. [49], and Niu et al. [50].Source: §2 · p. 3
  • The paper frames its focus as exploiting visual-textual interaction, since current LVLM attacks primarily construct visual adversarial prompts despite models fusing both modalities.The related-work passage also distinguishes adversarial examples, which aim to induce model malfunction rather than a harmful response.Source: §2 · p. 3

3 Preliminaries

The preliminaries describe LVLMs as fusing image and text inputs to generate responses, then frame jailbreaks as perturbations that seek harmful outputs. They also specify the attacker’s goal and knowledge assumptions.

3.1 Large Vision Language Models

  • An LVLM encodes an image with a visual module, fuses its representation with text through a connector, and uses a textual module for understanding and generation.The response is modeled as a probability distribution conditioned on the image-text pair, with alignment intended to prevent harmful outputs.Source: §3.1 · p. 3

3.2 Problem Definition

  • A jailbreak modifies a benign image-text query to elicit a harmful response, framed as maximizing the model’s log-likelihood for that response.The objective is expressed as log p(y*|A(xv, xt)).Source: §3.2 · p. 3
  • Visual-only attacks may be limited because LVLMs fuse visual and textual features; BAP first uses an adversarial image to encourage positive responses, then adaptively optimizes text prompts.Figure 2 describes query-agnostic image perturbation and intent-specific textual optimization as the framework’s two modules.Source: §3.2 · p. 4, Figure 2 · p. 4

3.3 Threat Model

  • The attacker’s goal is to bypass alignment and elicit harmful content, and the assumed attacker may know model architecture or weights without knowing system prompts or training data.The passage gives examples of harmful outputs including violating human values and revealing methods for creating illegal items.Source: §3.3 · p. 4

4 Bi-Modal Adversarial Prompt

BAP combines query-agnostic visual perturbation with intent-specific textual optimization to address the limits of attacks that perturb only visual inputs. The visual prompt is designed to elicit positive responses across harmful intents, while the text prompt is refined for a specific intent.

4 Bi-Modal Adversarial Prompt

  • BAP combines query-agnostic image perturbation and intent-specific text optimization to exploit visual-text interaction in attacks on LVLMs.The framework is designed to address limits of prior attacks that perturb only visual inputs.Source: §4 · p. 4

4.1 Query-Agnostic Image Perturbing

  • Prior visual prompts embedding specific harmful semantics can make attacks query-dependent when a text query concerns a different harmful intent.The passage gives a bomb-making image prompt paired with an illegal-drug query as an example of a failed jailbreak.Source: §4.1 · p. 5
  • The image module uses a few-shot corpus of affirmative prefixes and negative inhibitions to optimize a universal perturbation that encourages positive responses across textual intents.The prompt is optimized to elicit the corpus's target sentences; the textual prompt is nullified during optimization to avoid its influence.Source: §4.1 · p. 5
  • The resulting visual adversarial prompt is intended to elicit a positive response regardless of the specific intent of the textual prompt.The proposed mechanism creates a conflict between the model's safety objective and its instruction-following objective.Source: §4.1 · p. 5

4.2 Intent-Specific Text Optimization

  • With the visual adversarial prompt fixed, BAP optimizes the text prompt to strengthen the attack for a specific harmful intent.The target intent itself is used as the initial textual prompt, and the paired inputs produce an initial LVLM response.Source: §4.2 · pp. 5–6
  • An LLM uses chain-of-thought reasoning to analyze failed jailbreaks and refine textual prompts through feedback and iteration.The process can use contextual deception and semantic rephrasing, and continues until success or the iteration limit.Source: §4.2 · pp. 5–6
  • A judging LLM assesses each LVLM response as a jailbreak success or failure, providing feedback for subsequent prompt refinement.The LVLM prediction is the response evaluated by the judge.Source: §4.2 · p. 6

5 Experiment and Evaluation

The experiments evaluate BAP on open-source and commercial LVLMs, comparing query-dependent and query-agnostic white-box attacks and testing its components. BAP performs strongly across white-box settings, has some effect on commercial models, and its visual and textual components each contribute to attack performance.

5.1 Experimental Setups

  • The study evaluates three open-source LVLMs on SafetyBench and AdvBench, using ASR with automatic and manual evaluation.AdvBench's 521 harmful behaviors were deduplicated and incorporated into SafetyBench by scenario; higher ASR indicates a more effective attack.Source: §5.1 · p. 6

5.2 White-box Attacks on LVLMs

  • In white-box MiniGPT4 tests, BAP achieves the best performance in both QD and QA settings, outperforming others by up to 49.30% in QD and 52.98% in QA.The QA tests use visual prompts from two scenarios with harmful text prompts from other scenarios.Source: Table 1 · p. 7, §5.2 · p. 7

5.3 Black-box Attacks on LVLMs

  • On four commercial black-box LVLMs, BAP has some jailbreak effect, but its ASR is 22.50% lower on average than on black-box open-source models.Testing was limited to three scenarios and 50 related queries per model; the authors speculate that additional commercial-model mitigations may contribute to the difference.Source: §5.3 · p. 8

5.4 Ablation Studies

  • Removing the visual prompt lowers ASR; the original visual prompt and corpus outperform alternatives, while images without effective semantics perform worse.The tested alternatives included white, noisy, and giant-panda images, and different corpora [77] [78].Source: §5.4 · p. 8
  • Removing textual prompts significantly lowers ASR, and the original LLM-plus-CoT strategy outperforms three alternative text-generation methods.ASR initially rises with iteration count and stabilizes after N exceeds 5; increasing temperature generally helps until performance stabilizes or slightly decreases.Source: §5.4 · pp. 8–9

6 Evaluation on Bias and Adversarial Robustness

The section tests whether BAP can evaluate bias and adversarial robustness in LVLMs. It reports bias induction after jailbreaks and high ASR for adversarial attacks.

  • BAP achieves +30.34% bias induction ASR on average after successful jailbreaks, and over 85% ASR on the described adversarial attacks.Bias tests used three harmful scenarios and context-sensitive follow-up prompts; robustness tests used targeted horse and untargeted animal-description corpora.Source: §6 · p. 9

7 Conclusion and Future Work

BAP jointly optimizes visual and textual prompts, and the paper reports that it outperforms other methods across evaluations on datasets and LVLMs. The conclusion also identifies gradient-free visual-prompt construction and the computational cost of textual optimization as areas for future work.

7 Conclusion and Future Work

  • BAP jointly optimizes visual and textual prompts and significantly outperforms other methods across evaluations on datasets and LVLMs.The paper frames this as addressing limits of attacks that perturb only visual inputs.Source: §7 · p. 9
  • Gradient-free visual-prompt construction for black-box settings and the computational cost of repeated LVLM/LLM interactions remain areas for further investigation.The paper proposes more efficient prompt optimization as future work.Source: §7 · p. 10

A Algorithm of Intent-Specific Text Optimization

  • Text optimization iterates on a harmful request and visual adversarial prompt to produce a textual adversarial prompt.Algorithm 1 also takes judging and CoT prompt templates and an iteration number as inputs.Source: A Algorithm of Intent-Specific Text… · p. 15

B Judging Prompt Template

  • The judging template evaluates whether an LVLM response constitutes a successful jailbreak, with criteria adjusted for different scenarios.Criteria include illegal or unethical content, avoiding political topics, or absence of a disclaimer or risk warning, depending on the scenario.Source: B Judging Prompt Template · p. 16

C CoT Prompt Template

  • The CoT template helps ChatGPT analyze prior jailbreak failures using the harmful intent, textual prompt, and LVLM response.In practice, ChatGPT can also be instructed to produce optimized prompts in a designated format such as Markdown or JSON.Source: C CoT Prompt Template · p. 16

D Ablation Study for BAP

The ablations examine visual-prompt presence, image quality and corpus content, as well as textual-prompt use and optimization. Across these comparisons, the reported results favor BAP's visual and textual adversarial prompts and its CoT-based text optimization.

D.1 Visual Adversarial Prompt

  • Removing visual adversarial prompts lowers ASR, while the visual adversarial prompt outperforms white, noisy, and giant-panda image prompts.The authors report that image prompts lacking effective semantics yield inferior results.Source: Figure 7 · p. 17, §D.1 · p. 17
  • 59.04%, 56.48%, and 89.35% ASR are reported for BAP's original corpus in scenarios IA, PH, and LO, respectively, exceeding the two alternative corpora.The authors attribute the alternatives' lower effectiveness to semantic complexity that impedes generation of key semantics during visual-prompt optimization.Source: Figure 8 · p. 17, §D.1 · p. 18

D.2 Textual Adversarial Prompt

  • Removing textual adversarial prompts reduces ASR by up to 49.50% across scenarios.The comparison uses the original harmful queries in place of textual adversarial prompts.Source: §D.2 · p. 18
  • Among four text-prompt cases, CoT-based optimization achieves the highest ASR, while direct harmful queries achieve the lowest.LLM synonym rewriting and a prompt template without CoT fall between those cases; synonym rewriting is described as unstable.Source: §D.2 · p. 18

D.3 Hyper-parameters.

The hyperparameter study reports how iteration number and temperature relate to ASR, while the following white-box subsection points to results for LLaVA and InstructBLIP.

D.3 Hyper-parameters.

  • ASR rises most when N = 1, initially increases as N grows, and changes gradually once N exceeds 5.The study uses N = 5 empirically; Fig. 10a presents the iteration-number results.Source: §D.3 · p. 18, Figure 10 · p. 19
  • ASR generally rises with temperature but does not increase continuously beyond a threshold such as 0.6 for MiniGPT4.The study uses each model's default temperature; Fig. 10b presents the temperature results.Source: §D.3 · pp. 18–19, Figure 10 · p. 19

E White-box attacking other LVLMs

  • The white-box evaluation subsection presents LLaVA and InstructBLIP jailbreak results in Tables 3 and 4.The passage identifies Table 3 for LLaVA and Tables 3 and 4 for InstructBLIP.Source: E White-box attacking other LVLMs · p. 19

F Query-agnostic white-box attacking.

The query-agnostic white-box subsection identifies attacks by Liu et al. and Qi et al. and reports BAP's leading performance in both query-dependent and query-agnostic settings for LLaVA and InstructBLIP.

F Query-agnostic white-box attacking.

G Jailbreking commercial LVLMs

In black-box scenarios, BAP successfully jailbreaks commercial LVLMs, with examples spanning Gemini, Qwen, ChatGLM, and ERNIE Bot.

G Jailbreking commercial LVLMs

Loading 2406.04031v2…