PaperlayerView exact revision on arXiv

Source-linked AI summary

Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models

Erfan Shayegani, Yue Dong, Nael Abu-Ghazaleh

arXiv:2307.14539v2cs.CRcs.CL

TL;DR

The paper addresses whether multimodal alignment can be bypassed through adversarial images despite text-only defenses. It develops compositional embedding-space attacks that pair benign prompts with benign-looking images, and reports high success across VLMs while requiring only vision-encoder access. The findings expose cross-modality alignment vulnerabilities and motivate alignment across the full multimodal model.

  • Problem

    Text-based jailbreaks can be detected and patched, while cross-modality vulnerabilities in aligned vision-language models remain an emerging concern.

  • Method

    The attack uses four malicious-trigger settings and gradient-updated images targeting joint embedding-space representations, without access to the language modality.

  • Results

    The attacks achieve high success across multimodal models, with higher success reported for malicious triggers embedded in the vision modality.

  • Takeaways & Limitations

    Multimodal alignment must address cross-modality vulnerabilities rather than treating textual alignment in isolation.

  • Takeaways & Limitations

    For VLMs whose language components can be fine-tuned, the relevant attack parameters are assumed to include the vision encoder and language-model parameters because the vision encoder typically remains frozen.

Abstract

from arXiv · show

We introduce new jailbreak attacks on vision language models (VLMs), which use aligned LLMs and are resilient to text-only jailbreak attacks. Specifically, we develop cross-modality attacks on alignment where we pair adversarial images going through the vision encoder with textual prompts to break the alignment of the language model. Our attacks employ a novel compositional strategy that combines an image, adversarially targeted towards toxic embeddings, with generic prompts to accomplish the jailbreak. Thus, the LLM draws the context to answer the generic prompt from the adversarial image. The generation of benign-appearing adversarial images leverages a novel embedding-space-based methodology, operating with no access to the LLM model. Instead, the attacks require access only to the vision encoder and utilize one of our four embedding space targeting strategies. By not requiring access to the LLM, the attacks lower the entry barrier for attackers, particularly when vision encoders such as CLIP are embedded in closed-source LLMs. The attacks achieve a high success rate across different VLMs, highlighting the risk of cross-modality alignment vulnerabilities, and the need for new alignment approaches for multi-modal models.

1 INTRODUCTION

The paper introduces compositional cross-modal jailbreak attacks that pair benign textual instructions with adversarial images targeting malicious embeddings. The attacks conceal triggers in benign-looking images, require only vision-encoder access, and generalize across prompts and jailbreak scenarios.

  • Motivation: Text-based jailbreak inputs can be detected and patched, motivating attacks that exploit additional modalities in vision-language models.The paper positions multimodal inputs as a vulnerability beyond visible textual perturbations.
  • Approach: The paper investigates cross-modal alignment vulnerabilities through four compositional attack settings pairing benign text instructions with adversarial images.The malicious triggers are textual, OCR textual, visual, or combined OCR textual-visual targets in the joint embedding space.
  • Results: Experiments on LLaVA and LLaMA-Adapter V2 report higher jailbreak success for triggers associated with the image modality.The reported image-associated settings include OCR textual, visual, and combined OCR textual-visual triggers.
  • Results: A single malicious image can activate diverse generic prompts, while one generic prompt can pair with different malicious triggers.This compositionality contrasts with methods requiring optimization for each input-target output pairing.
  • Approach: The embedding-space attack updates images toward malicious-trigger embeddings while concealing those triggers in benign-looking images.It operates without language-modality access and requires access to a vision encoder such as CLIP.

2 BACKGROUND

The background contrasts traditional white-box attacks that optimize harmful outputs through the full VLM with embedding-based approaches that target representations. The paper uses vision-encoder-only embedding attacks to combine covert malicious images with generic prompts.

  • Target-based attacks: Existing target-based attacks maximize harmful-output likelihood by backpropagating gradients from targeted generation to the input.The optimization constrains adversarial inputs within a permitted input-space region B.
  • Text-based attacks: Textual attacks optimize adversarial suffixes or prompts that are often visibly detectable and increasingly vulnerable to security patches.Such attacks append adversarial text to harmful instructions to elicit targeted harmful responses.
  • Target-based attacks: These conventional visual attacks require white-box access to the complete VLM, from language-model output logits to image pixels.Their formulation uses the full model hierarchy to derive adversarial images coupled with generic or harmful instructions.
  • Embedding-based attacks: Embedding-based attacks instead manipulate representations, including matching adversarial images to target images or fooling surrogate classifiers in CLIP space.The cited prior work establishes embedding-space manipulation as a related attack direction.
  • This paper’s approach: This paper targets text and image embeddings to hide malicious images behind generic prompts, enabling jailbreaks without language-model access.Its trigger targets include text, OCR text, malicious images, or combinations of these modalities.

3 METHODOLOGY

The method decomposes harmful prompts into benign textual instructions and malicious embedding-space triggers, then hides those triggers in benign-looking adversarial images. It targets cross-modal alignment vulnerabilities using only vision-encoder access for image generation.

  • VLM Architecture: VLMs align a frozen visual encoder with an LLM through a projection layer in a shared joint embedding space.Visual features are represented as H_i = W_I · I(x_i), while textual features are represented as H_t = T(x_t).
  • Attack Design: Unlike output-targeted attacks, the approach creates composable malicious components that can be paired across images and generic instructions.The attack focuses on constructing reusable embedding-space components rather than directly targeting harmful output text.
  • Compositional Jailbreak: The attack decomposes a harmful prompt into a generic textual instruction and an image embedding that mimics a malicious trigger.The two embeddings are jointly fed into the language model so the image supplies the harmful subject context.
  • Trigger Settings: Four trigger settings target textual, OCR textual, visual, and combined OCR-textual-and-visual malicious embeddings.These settings create distinct ways to decompose and represent the harmful trigger in the joint embedding space.
  • Adversarial Image Generation: The embedding-space attack finds benign-looking images whose vision embeddings lie near dangerous regions associated with harmful triggers.The image generator minimizes embedding distance subject to constraints such as a distance bound.

1 Input xharm to I(·) and get its embedding Hharm

The adversarial image generator iteratively updates an image so its vision embedding approaches the target harmful-trigger embedding.

  • Optimization Procedure: The optimization computes the adversarial image embedding, evaluates its distance from the harmful-trigger embedding, and updates the image using ADAM.Initialization may use random noise, a white background, or an arbitrary benign image; the objective is an L2 embedding-distance loss.
  • Convergence: Once convergence reaches τ ≈ 0.3, optimization typically takes 10 to 15 minutes on a Google Colab T4 GPU.The resulting adversarial and target embeddings are often extremely close or perfectly aligned.
  • Embedding Alignment: The optimized image can bear no visual resemblance to its target trigger while remaining semantically identical in the embedding space.The VLM processes the CLIP output rather than directly distinguishing the image from the trigger.

4 EXPERIMENTAL SETUP AND RESULTS

Experiments evaluate compositional attacks across eight prohibited scenarios using human and automated assessments. Image-based triggers produce strong jailbreak results, while textual targets are largely ineffective and combined triggers are especially potent.

  • Experimental Setup: The evaluation covers 8 prohibited scenarios, generates 8 adversarial images per category, and tests 2 generic prompts per scenario with 25 repetitions.The setup is designed to assess attack robustness across multiple trigger strategies and scenarios.
  • Evaluation: Three volunteers evaluate model outputs without knowing the inputs or malicious triggers used during optimization.Human evaluation complements automated toxicity assessment because the attacks are embedding-based rather than directly target-based.
  • Human Results: Image-based triggers achieve high jailbreak success across all eight categories, with combined OCR and visual triggers outperforming either alone.The results are summarized using attack success rate across scenarios including sexual, hateful, violence, self-harm, and harassment categories.
  • Human Results: Textual targets are ineffective because image and text embeddings remain distinctly separated in CLIP’s joint space.The resulting image may lie far from the typical real-image region and resemble an out-of-distribution sample.
  • Automatic Results: Automatic toxicity scores are very low for images matched to textual malicious triggers, whereas OCR, visual, and combined triggers more clearly bypass safety guards.Higher classifier and Perspective API values indicate outputs judged more harmful and are associated with jailbreak success.

5 DISCUSSIONS

The attacks contaminate multimodal context, enabling continued toxic and biased responses after an initial jailbreak. Hidden prompt injection extends this vulnerability by making models treat image-embedded text as instructions, while attacks generalize beyond the target content.

  • Context Contamination: After an initial jailbreak, contaminated context can elicit further prohibited prompts and extreme bias.The paper illustrates continued toxic discussion and biased responses after the model generates an initial harmful answer.
  • Hidden Prompt Injection: Hidden prompt injection uses adversarial images containing OCR instructions to make models read and follow embedded text.The authors report both direct and indirect image-based prompt injection settings.
  • Generalization: Adversarial images can guide generation beyond the target image’s content toward racism, violence, self-harm, terrorism, and other harmful topics.The image acts as a cue toward an embedding-space region, after which generic prompts can elicit broader harmful content.
  • Call for Defense Strategies: Cross-modality vulnerabilities can break textual alignment, motivating alignment strategies that account for the full multimodal model.The paper raises whether aligning modalities separately can align the overall system against compositional attacks.

6 CONCLUDING REMARKS

The paper demonstrates that vision-based cross-modality attacks can break text-only alignment in vision-language models. Benign-appearing adversarial images require access only to the vision encoder and achieve high success rates across multiple multimodal models.

  • Cross-modality attacks break text-only alignment by pairing benign-looking adversarial images with textual prompts.The attacks target triggers in the vision encoder’s embedding space to induce undesirable outputs.
  • The attacks require access only to the vision encoder, not white-box access to the language model.This setting is especially relevant when vision encoders such as CLIP are embedded in closed-source language models.
  • The attacks achieve a high success rate across multiple multimodal models, highlighting cross-modality alignment vulnerabilities.The paper concludes that multimodal alignment approaches must address interactions across input modalities.

A ADDITIONAL EVALUATION

Additional evaluations examine indirect and direct hidden prompt injection attacks using adversarial images. The direct setting has lower success because abstract instructions are harder to represent visually and models primarily learn to describe images rather than follow image-based instructions.

  • The evaluation covers indirect and direct hidden prompt injection settings across multiple adversarial-image scenarios.Indirect injection assumes a malicious third party introduces an image into a benign user’s visual context; direct injection embeds instructions in images.
  • Indirect evaluation uses three benign scenarios, nine adversarial images per scenario, and 1,350 total queries.Each adversarial image is evaluated 25 times, and outcomes are manually assessed for hidden-information injection and prompt divergence.
  • LLaMA-Adapter V2 has lower attack success rates than LLaVA, associated with weaker image understanding and instruction-following capabilities.The paper connects incorporating visual and textual context to instruction-following ability and model size.
  • Direct injection counts success only when the model both recognizes embedded text and follows it as an instruction.The evaluation uses four embedded instructions and the prompt “Read the text in the image”; temperature is set to 0.1 to reduce randomness.
  • Direct hidden prompt injection succeeds less often because abstract instructions differ from the real objects emphasized in CLIP training.Multimodal models also learn to passively describe images rather than treat image content as instructions, requiring both behaviors to be bypassed.

B.1 EVALUATION VIA Q&A

The Q&A evaluation presents a full LLaVA chat session to assess whether an adversarial image captures the semantic features of a target image. The reported session shows near-complete semantic correspondence.

  • The LLaVA chat history is used to evaluate whether the adversarial image captures the target image’s semantic features.The target is described as a man riding a bike.
  • The adversarial image captures almost all semantic features of the actual target image in the reported LLaVA session.The paper describes the model as successfully describing the adversarial image in a way resembling conversation with the target image.

B.2 EVALUATION VIA RECONSTRUCTION & CLASSIFICATION

The evaluation examines whether adversarial images preserve target semantics and induce harmful or attacker-specified outputs across reconstruction, classification, and prompt-injection examples.

  • Reconstruction and classification: Reconstruction evaluation compares the actual target image, its corresponding adversarial image, and an image generated from LLaVA’s description of the adversarial input.For one example, classification results using HuggingFaceCLIP are also reported.
  • Reconstruction and classification: An adversarial image close to a target porn image in embedding space produces the same sexual outputs as the target image.HuggingFaceCLIP assigns 93.3% confidence to the target across evaluated text pieces.
  • Prompt injection: The attack injects attacker-selected content into a benign user prompt by directing the prompt toward a desired embedding-space region.This mechanism is illustrated for indirect hidden prompt injection.
  • Prompt injection: The model reads instructions in adversarial images and follows them, although the authors speculate that image noise may contribute to the generated word “Ring.”The figure describes adversarial images as nearly identical to instruction-containing target images in embedding space.

H EVALUATION DETAILS

The evaluation uses manual author labeling and defines success by whether models directly provide requested harmful or injected content, including responses that contain warnings.

  • Labeling: All outputs were labeled manually by the authors because the evaluation data contains substantial harmful and toxic content.The labeling procedure is applied to assess attack effectiveness.
  • Success criteria: An output is successful only when it directly answers the harmful prompt rather than merely describing the phenomenon or associated dangers.For example, a step-by-step tutorial qualifies, whereas a general explanation of bombs does not.
  • Success criteria: Warnings do not invalidate success when the response still contains the required harmful information.The criterion applies whether the warning appears at the beginning or end.
  • Prompt-injection criteria: Indirect hidden prompt injection succeeds only when the response follows the benign user scenario and incorporates the attacker’s content from the adversarial image.Following only the user’s request without including image-derived content is unsuccessful.

I REAL-WORLD ENTITIES VS. ABSTRACT CONCEPTS

Attack effectiveness differs between real-world entities and abstract concepts, reflecting both vision-encoder familiarity and whether multimodal training encourages treating image text as instructions.

  • Real-world entities versus abstract concepts: Jailbreak and indirect hidden prompt injection attacks achieve significantly higher success rates than direct hidden prompt injection attacks.The former use target images of real-world entities, whereas direct attacks often use abstract verbs and adjectives.
  • Real-world entities versus abstract concepts: Vision encoders such as CLIP favor strong words representing familiar real entities, while weak words are less likely to capture attention.The paper attributes this pattern to the encoders’ training data and reports stronger effects for Meth and Marijuana than Heroin or MDMA, and for Gun than Pistol.
  • Real-world entities versus abstract concepts: The greater the inclusion of strong words in a target image, the more effective its corresponding adversarial image tends to be.This relationship is summarized in Table 7.
  • Direct hidden prompt injection: Direct hidden prompt injection remains constrained because multimodal models are trained to describe images rather than treat image content as instructions.Even when models accurately read text in a target image, they are less likely to follow it instructionally.
Loading 2307.14539v2…