PaperlayerView exact revision on arXiv

Source-linked AI summary

JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks

Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, Chaowei Xiao

arXiv:2404.03027v4cs.CRcs.AIcs.CL

TL;DR

The paper addresses whether jailbreak techniques that work on LLMs can also compromise MLLMs, an important gap as multimodal models advance. It introduces JailBreakV-28K, combining transferred textual attacks with image-based attacks, and finds that textual jailbreaks transfer effectively across MLLMs and do not depend on the image input. The results motivate alignment work covering both textual and visual inputs.

  • Problem

    The paper investigates whether techniques that jailbreak LLMs can be transferred to MLLMs as models become harder to secure and align with human values.

  • Method

    The paper introduces JailBreakV-28K, using RedTeam-2K queries to construct text-based LLM-transfer attacks and image-based jailbreak inputs for evaluating MLLM robustness.

  • Results

    Textual jailbreak prompts transfer to MLLMs across foundational models, remain effective regardless of image input, and produce high ASR in evaluation.

  • Takeaways & Limitations

    MLLM alignment requires attention to vulnerabilities arising from both textual and visual inputs.

  • Takeaways & Limitations

    Because the benchmark uses queries already highly successful against LLMs and effective against MLLMs, access requires prior permission to mitigate misuse.

Abstract

from arXiv · show

With the rapid advancements in Multimodal Large Language Models (MLLMs), securing these models against malicious inputs while aligning them with human values has emerged as a critical challenge. In this paper, we investigate an important and unexplored question of whether techniques that successfully jailbreak Large Language Models (LLMs) can be equally effective in jailbreaking MLLMs. To explore this issue, we introduce JailBreakV-28K, a pioneering benchmark designed to assess the transferability of LLM jailbreak techniques to MLLMs, thereby evaluating the robustness of MLLMs against diverse jailbreak attacks. Utilizing a dataset of 2, 000 malicious queries that is also proposed in this paper, we generate 20, 000 text-based jailbreak prompts using advanced jailbreak attacks on LLMs, alongside 8, 000 image-based jailbreak inputs from recent MLLMs jailbreak attacks, our comprehensive dataset includes 28, 000 test cases across a spectrum of adversarial scenarios. Our evaluation of 10 open-source MLLMs reveals a notably high Attack Success Rate (ASR) for attacks transferred from LLMs, highlighting a critical vulnerability in MLLMs that stems from their text-processing capabilities. Our findings underscore the urgent need for future research to address alignment vulnerabilities in MLLMs from both textual and visual inputs.

1 Introduction

The paper asks whether LLM jailbreak techniques transfer to MLLMs and introduces JailBreakV-28K to evaluate this question. Experiments indicate that textual jailbreaks can compromise MLLMs, motivating alignment efforts across both textual and visual inputs.

  • MLLM advances have increased the challenge of securing models and aligning them with human values.
  • Existing MLLM alignment research primarily examines image-based jailbreak methods, leaving transferability from LLM jailbreaks comparatively unexplored.
  • JailBreakV-28K combines 2,000 malicious queries, 20,000 text-based LLM-transfer attacks, and 8,000 image-based jailbreak inputs.
  • Evaluation of 10 open-source MLLMs demonstrates that LLM jailbreak attacks transfer to MLLMs and achieve significantly high ASR.
  • Textual jailbreak prompts remain effective regardless of whether the image input is blank, noisy, or a random natural image.
  • The authors argue that MLLM alignment must address vulnerabilities from both textual and visual inputs.

2 Related Works

Prior MLLM jailbreak research emphasizes image-based attacks, while JailBreakV-28K also evaluates whether LLM jailbreak attacks transfer to MLLMs. The benchmark broadens coverage across attack types, scenarios, and malicious policies.

  • Jailbreak attacks can induce LLMs and MLLMs to produce harmful answers that violate human values.
  • MLLM jailbreak studies include image perturbation attacks and harmful content inserted into images through typography or text-to-image tools.
  • JailBreakV-28K investigates both recent image-based jailbreak attacks and the transfer of existing LLM jailbreak attacks to MLLMs.
  • Unlike prior work focused on image inputs, the benchmark evaluates text jailbreak transfer using 28K text-image pairs across 16 scenarios and malicious policies.

3.1 Overview of JailBreakV-28K

The benchmark is built from RedTeam-2K, a balanced collection of harmful queries, and expands it into diverse text-image jailbreak pairs for evaluating MLLM robustness and attack transferability.

  • RedTeam-2K contains 2,000 harmful queries spanning 16 safety policies and draws from eight distinct sources.
  • RedTeam-2K presents a uniform safety-policy distribution to maintain balance, with about 48.5% of data collected by the authors.
  • JailBreakV-28K contains 28,000 jailbreak text-image pairs, including 20,000 text-based LLM-transfer attacks and 8,000 image-based MLLM attacks.

3.2 RedTeam-2K: A Comprehensive Malicious Query Dataset

RedTeam-2K is a curated dataset of 2,000 harmful queries assembled from multiple sources and refined to improve coverage, quality, and diversity. Its construction combines policy-based organization, manual and GPT-assisted editing, and similarity-constrained generation.

  • Dataset construction: RedTeam-2K combines five identified jailbreak-related datasets with safety-policy decomposition based on OpenAI and LLaMA-2 usage policies.The source datasets include LLM Jailbreak Study, AdvBench, BeaverTails, Question Set, and hh-rlhf.
  • Dataset construction: The data-cleaning stage manually selects and categorizes harmful queries, while GPT reformats poorly structured BeaverTails questions.The process targets AdvBench, hh-rlhf, and BeaverTails, with GPT rewriting applied specifically to BeaverTails.
  • Query generation: GPT-based generation enriches sparse safety policies with harmful queries using statements from OpenAI and LLaMA-2 usage policies.The prompting strategy is designed to generate varied harmful queries rather than relying only on prior examples.
  • Query generation: Cosine-similarity filtering with all-mpnet-base-v2 limits repetitive GPT-generated queries and iteratively expands the dataset until query growth becomes negligible.Valid generated queries are marked “GPT Generate,” and the full construction pipeline is depicted in Figure 3.
  • Dataset comparison: RedTeam-2K is presented as broader and more diverse than existing malicious-request datasets, which are described as limited in scale, syntactic variety, and sample diversity.Table 1 compares dataset size and diversity-related characteristics, including semantic similarity, safety-policy coverage, and syntactic diversity.

3.3 JailBreakV-28K: Attacking MLLMs with LLMs’ Jailbreak Prompts

JailBreakV-28K is built by generating jailbreak prompts from LLMs, selecting the strongest prompts, and combining them with multiple image types. It also includes image-based MLLM attacks to evaluate jailbreak robustness across textual and visual inputs.

  • Benchmark construction: The benchmark first generates jailbreak prompts from LLMs using diverse attack methods and then integrates selected prompts with image data.The stated goal is to evaluate transferability from LLM jailbreak attacks to MLLMs.
  • Text-based attacks: The implemented attacks include real-world prompt templates, GCG suffix generation, and cognitive-overload strategies such as veiled expression.GCG is classified as a template-style attack because it adds prefixes or suffixes without changing the harmful query itself.
  • Prompt selection: 5,000 strong unique text-based prompts are selected by ranking jailbreak prompts according to the number of models they successfully jailbreak.Responses are evaluated with Llama Guard, which classifies whether each response is harmful.
  • Multimodal inputs: The selected prompts are paired with blank, random-noise, ImageNet-2K natural, and Stable-Diffusion-generated images.Synthesized images are generated from jailbreak-prompt keywords to remain relevant to the text topic.
  • Image-based attacks: FigStep and Query-Relevant attacks produce 8,000 image-based MLLM jailbreak inputs for the benchmark.Query-Relevant uses SD and Typography, SD, and Typography image-generation variants.

4 Experiments

Experiments evaluate JailBreakV-28K on open-source MLLMs using Attack Success Rate (ASR), showing substantial vulnerability to transferred text-based jailbreak attacks. Results also identify safety-policy and input-modality patterns in MLLM robustness.

  • Overall Results: 44% average ASR across the whole benchmark and 50.5% for LLM transfer attacks show that JailBreakV-28K strongly challenges MLLM security.The benchmark combines text-based and image-based attacks, with vulnerability varying across models and attack types.
  • Safety-Policy Analysis: 57.9% average ASR for Malware and 53.1% for Economic Health indicate the highest vulnerability under these safety policies.Most evaluated MLLMs show their highest ASR for these two policy categories.
  • Transferability Analysis: 68.7% average ASR on MLLM text encoders shows that LLM transfer attacks remain effective without images.The same attacks achieved 64.4% average ASR on the eight source LLMs and remained especially effective for Template and Logic attacks.
  • Input-Modality Analysis: An average coefficient of variation of 9.0 across four image types indicates that text-based jailbreak effectiveness is primarily determined by the attack text.Blank, noise, and natural-image inputs do not substantially alter the observed jailbreak effect.

5 Conclusion

The paper finds that LLM jailbreak techniques transfer to MLLMs, with text-based attacks remaining effective across image inputs. These results motivate safety alignment research addressing both textual and visual inputs.

  • MLLMs inherit vulnerability from their LLM counterparts through transferable jailbreak attacks.
  • Text-based jailbreak attacks are more effective than image-based jailbreak attacks and remain effective regardless of the image input.
  • The findings support focusing MLLM safety alignment on both textual and visual inputs.
  • Image input contributes minimally to Attack Success Rate (ASR) for text-based jailbreak attacks in MLLMs.

7 Ethics Statement

The ethics statement frames the benchmark as a tool for understanding and mitigating MLLM vulnerabilities while recognizing that its jailbreak queries could prompt malicious outputs. Access is therefore restricted to research or educational purposes with prior permission.

  • The benchmark is intended to support more robust, safe, and aligned language models over time.
  • The authors anticipate that jailbreak research will not cause immediate harm and aim to inspire more effective defense strategies.
  • Its jailbreak queries have demonstrated high success rates against LLMs and strong efficacy against MLLMs.
  • Because the queries could prompt MLLMs to generate malicious outputs, benchmark access requires prior permission for research or educational purposes.

A Appendix

The appendix documents data-generation and safety-policy design, reports jailbreak-generation performance, and presents examples of transferred and image-based MLLM jailbreak attacks. Its figures illustrate the diversity of prompts, policy boundaries, attack methods, image types, and successful cases.

  • Data and policy design: Prompt engineering imposes multi-syntax sentence requirements to ensure syntactic diversity in generated queries.
  • Data and policy design: The appendix includes a safe policy definition and configured Llama-Guard unsafe-category prompts.
  • Data and policy design: RedTeam-2K is designed for large scale while maintaining a scientific distribution of safety policies and clear boundaries between policies.
  • Attack evaluation: Template and Logic generally achieve better LLM-transfer jailbreak performance on MLLMs than Persuade.
  • Attack examples: Examples cover LLM-transfer attacks using random-noise, blank, and stable-diffusion images, alongside FigStep and Query-Relevant image-based attacks.
  • Attack evaluation: The appendix reports high Attack Success Rate (ASR) for most selected jailbreak prompts generated from LLMs.
Loading 2404.03027v4…