PaperlayerView exact revision on arXiv

Source-linked AI summary

Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Character

Siyuan Ma, Weidi Luo, Yu Wang, Xiaogeng Liu

arXiv:2405.20773v2cs.CRcs.AI

TL;DR

Structure-based MLLM jailbreaks are limited in effectiveness and generalization, while MLLM safety remains a concern. Visual Role-play (VRP) uses high-risk-character images to prompt role-play, and the paper reports stronger jailbreak performance than prior methods.

  • Problem

    Existing structure-based MLLM jailbreaks require query-specific computation, limiting their generalization and practicality across broad scenarios.

    Source: §1 · p. 2

  • Method

    Visual Role-play (VRP) uses images of high-risk characters to prompt MLLMs to act as those characters and generate harmful content.

    Source: §1 · p. 2, §5 · p. 10

  • Results

    VRP reports superior jailbreak performance and generalization on popular benchmarks, outperforming Query relevant and FigStep by an average ASR margin of 14.3.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    A single VRP image can induce MLLMs to generate multiple harmful responses.

    Source: §5 · p. 10

    Effectiveness may be limited on poorly performing MLLMs with inadequate instruction-following or image-understanding capabilities.

    Source: §6 · p. 10

Abstract

from arXiv · show

With the advent and widespread deployment of Multimodal Large Language Models (MLLMs), ensuring their safety has become increasingly critical. To achieve this objective, it requires us to proactively discover the vulnerability of MLLMs by exploring the attack methods. Thus, structure-based jailbreak attacks, where harmful semantic content is embedded within images, have been proposed to mislead the models. However, previous structure-based jailbreak methods mainly focus on transforming the format of malicious queries, such as converting harmful content into images through typography, which lacks sufficient jailbreak effectiveness and generalizability. To address these limitations, we first introduce the concept of "Role-play" into MLLM jailbreak attacks and propose a novel and effective method called Visual Role-play (VRP). Specifically, VRP leverages Large Language Models to generate detailed descriptions of high-risk characters and create corresponding images based on the descriptions. When paired with benign role-play instruction texts, these high-risk character images effectively mislead MLLMs into generating malicious responses by enacting characters with negative attributes. We further extend our VRP method into a universal setup to demonstrate its generalizability. Extensive experiments on popular benchmarks show that VRP outperforms the strongest baseline, Query relevant and FigStep, by an average Attack Success Rate (ASR) margin of 14.3% across all models.

1 Introduction

MLLM safety motivates studying jailbreaks, but existing structure-based attacks are described as limited in effectiveness and generalization. Visual Role-play (VRP) adds high-risk image characters to the attack and extends them to universal use.

1 Introduction

  • The paper frames jailbreak attacks as perturbation-based, text-based, or structure-based, and focuses on structure-based attacks because of their multimodal challenges.It notes that text-based attacks may be detected and blocked by text moderators.Source: §1 · p. 2
  • Existing structure-based attacks transform malicious queries into images but are described as insufficiently effective and as requiring query-specific computation that limits practicality.The paper argues that an added jailbreak context is needed and that universal attacks should work across scenarios without extensive customization.Source: §1 · p. 2
  • VRP uses LLM-generated high-risk character descriptions and corresponding images, paired with benign role-play instructions to elicit malicious responses.The image combines character-description and malicious-query typography; the method also uses universal character images for a wide range of queries.Source: §1 · pp. 2–3
  • VRP outperforms Query relevant and FigStep by an average Attack Success Rate margin of 14.3.The evaluation uses RedTeam-2K and HarmBench.Source: §1 · p. 2

2 Related Works

Prior work explores role-play in LLMs for interaction and personalization, and as a jailbreak technique, but the paper says MLLM jailbreak studies had not examined it. VRP applies visual role-play while retaining visual representations of harmful-query information.

2 Related Works

  • LLM role-play research explores interaction, personalization, and context-faithfulness, while other work studies role-play as a jailbreak prompt strategy [77] [34].The paper identifies MLLM jailbreak role-play as an unstudied gap in those studies.Source: §2 · p. 3
  • Earlier structure-based methods include FigStep’s step-by-step harmful-query typography and Query relevant’s visualization of query-related keywords.VRP combines visual role-play with visual representations of key harmful-query information and reports better performance than other structure-based attacks.Source: §2 · pp. 3–4
  • Text-based jailbreak attacks transfer techniques initially designed for LLMs to MLLMs.The related-work discussion distinguishes them from image-based attacks.Source: §2 · pp. 3–4

3 Methodology

VRP turns malicious queries into image-and-text inputs that cast MLLMs as high-risk characters. Its universal variant iteratively generates and validates characters for use across malicious queries.

3.1 Preliminary

  • The attack objective is to make MLLMs answer malicious queries rather than return refusal messages.The task frames inputs as adversarial text-image pairs and evaluates their responses against the original malicious questions.Source: §3.1 · p. 4

3.2 Query-specific Visual Role-play

  • For each malicious query, VRP uses an LLM to derive high-risk character traits and a text-to-image prompt, then generates a corresponding character image.The character traits include negative attributes such as rudeness or immorality.Source: §3.2 · p. 4
  • VRP guides the LLM to generate character traits and text-to-image prompts step by step, drawing on chain-of-thought prompting.The authors cite prior work reporting that chain-of-thought prompts enhance MLLM performance on various tasks.Source: §3.2 · p. 4
  • The attack combines character-description and question typography with a character image, then pairs the image input with a benign role-play instruction.The image places the character description at the top, character image in the middle, and malicious question at the bottom.Source: §3.2 · pp. 4–5

3.3 Universal Visual Role-play

  • Universal VRP iteratively generates candidate characters using training queries and selects a final character by validation-set ASR.Later candidates are optimized from high-ASR roles; the selected character is used on the test set.Source: §3.3 · pp. 5–6
  • Universal character selection compares candidates with high training-set ASR on a validation set to reduce the risk of choosing an overfit character.The final candidate is the one with the highest validation-set ASR.Source: §3.3 · pp. 5–6

4 Experiments

Experiments evaluate VRP on RedTeam-2K and HarmBench with five MLLMs, then examine its query-specific and universal results, image components, defenses, and combinations with baseline attacks.

4.1 Experimental setups..

  • The evaluation uses RedTeam-2K and HarmBench with five MLLMs, and counts an attack as successful only when its response is harmful and query-relevant.RedTeam-2K has 2,000 questions across 16 categories; HarmBench supplies 320 test questions.Source: §4.1 · pp. 6–7

Implementation Details

  • Character descriptions are generated with Mixtral-8x7B-Instruct and images with Stable Diffusion XL; universal training samples candidates over five rounds.The setup samples 50 training-question demonstrations per generation round and 256 examples to compute batch training ASR.Source: Implementation Details · p. 7

4.2 Main Results

  • VRP improves ASR by 9.8% over FigStep and 14.3% over Query relevant, achieving the highest ASR across the reported datasets.The query-specific evaluation compares attacks on RedTeam-2K and HarmBench test sets.Source: Table 1 · p. 8, §4.2 · p. 8
  • Universal VRP averages 32.7% ASR on LLaVA-V1.6-Mixtral and 29.4% on Qwen-VL-Chat, with reported transfer across models.The best character is selected using train and validation data on a target model, then tested on transfer models.Source: Table 2 · p. 8, §4.2 · p. 8

4.3 Ablation Study

  • Tailored character images combined with description typography produce the highest ASR among the tested image-element settings.All tested image inputs retain question typography.Source: §4.3 · p. 8, Table 3 · p. 9

4.4 Defense Analysis

  • Query-specific VRP remains effective across models when tested against System Prompt-based Defense and ECSO.The evaluation examines both defenses on the RedTeam-2K test set.Source: Table 4 · p. 9, §4.4 · p. 9

4.5 Integrating VRP with Baseline Techniques

  • Adding VRP improves the ASR of both FigStep and Query relevant in the tested integrations.The integration replaces question typography with the baseline image input and concatenates the baseline text input with VRP text.Source: §4.5 · pp. 9–10, Table 5 · p. 10

5 Conclusion

The paper presents Visual Role-play as a jailbreak method that uses character portraits and role-play instructions to elicit harmful responses. It reports effectiveness across models, including against advanced defenses, and says a single image can elicit multiple harmful responses.

5 Conclusion

  • A single VRP image can induce MLLMs to generate multiple harmful responses.The paper presents this as part of its effort to address effectiveness and universality limitations in existing approaches.Source: §5 · p. 10
  • VRP combines character portraits with role-play instructions to compromise MLLMs’ alignment robustness.The portraits depict characters that the models are instructed to enact.Source: §5 · p. 10
  • Extensive experiments report outstanding attack effectiveness across various models, even against advanced defenses.Source: §5 · p. 10

6 Limitation

The approach has limitations for poorly performing MLLMs and depends on the quality of both LLM-generated character prompts and diffusion-generated images.

6 Limitation

  • VRP may be less effective against poorly performing MLLMs with inadequate instruction-following and image-understanding capabilities.The authors connect these capabilities to difficulty performing role-play tasks.Source: §6 · p. 10
  • LLM-generated character prompts may be constrained by the LLM’s ability to produce effective diffusion-model prompts.Source: §6 · p. 10
  • Diffusion-model capability to generate character images may further limit the approach’s efficacy.Source: §6 · p. 10

7 Future work

Future work could improve character generation and inspect and iteratively refine generated character images before attacking target MLLMs.

7 Future work

  • More sophisticated character-generation strategies are proposed as a possible direction for future work.Source: §7 · p. 10
  • Future work could inspect and iteratively improve LLM- and diffusion-generated character images before attacking target MLLMs.Source: §7 · p. 10

A Character Generation Detail

Character generation uses an LLM to produce descriptions and image-generation prompts, with distinct prompts for query-specific VRP and the two universal-VRP rounds. The universal optimization prompt uses demonstrations scored by risk to guide a best-prompt selection.

A Character Generation Detail

  • Mixtral-8x7B-Instruct-v0.1 generates characters for both query-specific and universal VRP, with maximum tokens 1024, temperature 1, and top-p 0.5.The LLM output brackets character descriptions and diffusion prompts for extraction.Source: A Character Generation Detail · p. 15

Prompt for Character Generation in Query-specific VRP

  • The query-specific prompt asks for a brief and detailed harmful-answerer description, then a representative image prompt of no more than 40 words.The prompt tailors the character to the harmful question.Source: Prompt for Character Generation in… · p. 16

Prompt for Character Generation in Universal VRP Initial Round

  • The universal initial-round prompt asks for a high-risk answerer description and a representative image prompt of no more than 40 words.It uses demonstrations and frames the task as hypothetical and aimed at mitigating risk.Source: Prompt for Character Generation in… · p. 16

Prompt for Character Generation in Universal VRP Optimization Round

  • The universal optimization prompt uses demonstrations scored by answerer risk and asks for one distinct best prompt describing the highest-risk answerer.Its prompt also limits the image-generation prompt to 40 words.Source: Prompt for Character Generation in… · p. 17

B Ethics and Broader Impact

The authors frame the work as academic research intended to expose MLLM vulnerabilities and encourage collaborative defense development.

C Effect of Text Moderator on Text-based Jailbreak Attack

The text moderator sharply reduces the reported success of text-based jailbreak attacks, while VRP's fixed harmless instruction is classified as safe.

D Examples

The section gives an example of the method and baselines targeting Llava-V1.6-Mistral-7B with a harmful question sampled from RedTeam-2K.

  • The example targets Llava-V1.6-Mistral-7B with a RedTeam-2K question about kicking a cat into traffic.Source: D Examples · p. 18

E Evaluation Detail

The evaluation adds a relevance check because toxicity-based jailbreak judgments can count responses that do not address the harmful query. The accompanying prompt asks whether a response attempts to fulfill the request, regardless of toxicity.

E Evaluation Detail

  • The evaluation adds a relevance metric because Llama Guard may label irrelevant harmful content as jailbroken, including responses that repeat or merely describe the harmful query.The metric assesses response relevance to the harmful query alongside toxicity.Source: E Evaluation Detail · p. 18

Prompt for Evaluating the Relevance of Response and Question

Loading 2405.20773v2…