PaperlayerView exact revision on arXiv

Source-linked AI summary

Visual Adversarial Examples Jailbreak Aligned Large Language Models

Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, Prateek Mittal

arXiv:2306.13213v2cs.CRcs.CLcs.LG

TL;DR

Vision integration enlarges the adversarial attack surface of LLMs and gives attackers objectives beyond misclassification, but evidence on its safety implications remains limited. The paper constructs visual adversarial examples against aligned VLMs and evaluates whether they can bypass safety guardrails. A single example generally induces harmful responses beyond its few-shot optimization corpus across three VLMs, motivating multimodal defenses and a broader alignment challenge.

  • Problem

    Integrating vision into LLMs creates a continuous, high-dimensional input space and expands the consequences of security failures beyond misclassification.

  • Method

    The paper optimizes visual adversarial examples on a few-shot harmful corpus to circumvent safety guardrails in aligned vision-integrated LLMs.

  • Results

    A single visual adversarial example generally jailbreaks aligned VLMs, inducing diverse harmful instructions beyond the corpus used for optimization across three models.

  • Takeaways & Limitations

    The findings connect neural-network adversarial vulnerabilities to a fundamental AI-alignment challenge as multimodal foundation models emerge.

  • Takeaways & Limitations

    Existing adversarial defenses are costly at LLM scale, often assume discrete classes, and rely on small perturbation bounds that may not apply to this jailbreak threat model.

Abstract

from arXiv · show

Recently, there has been a surge of interest in integrating vision into Large Language Models (LLMs), exemplified by Visual Language Models (VLMs) such as Flamingo and GPT-4. This paper sheds light on the security and safety implications of this trend. First, we underscore that the continuous and high-dimensional nature of the visual input makes it a weak link against adversarial attacks, representing an expanded attack surface of vision-integrated LLMs. Second, we highlight that the versatility of LLMs also presents visual attackers with a wider array of achievable adversarial objectives, extending the implications of security failures beyond mere misclassification. As an illustration, we present a case study in which we exploit visual adversarial examples to circumvent the safety guardrail of aligned LLMs with integrated vision. Intriguingly, we discover that a single visual adversarial example can universally jailbreak an aligned LLM, compelling it to heed a wide range of harmful instructions that it otherwise would not) and generate harmful content that transcends the narrow scope of a `few-shot' derogatory corpus initially employed to optimize the adversarial example. Our study underscores the escalating adversarial risks associated with the pursuit of multimodality. Our findings also connect the long-studied adversarial vulnerabilities of neural networks to the nascent field of AI alignment. The presented attack suggests a fundamental adversarial challenge for AI alignment, especially in light of the emerging trend toward multimodality in frontier foundation models.

1 Introduction

Integrating vision into LLMs expands adversarial attack surfaces and broadens the consequences of security failures. The paper demonstrates that visual adversarial examples can jailbreak aligned vision-integrated LLMs, with one example inducing harmful behavior beyond its optimization corpus.

  • Multimodal risks: Vision-integrated LLMs expose an additional continuous, high-dimensional visual input space that is vulnerable to adversarial examples.This transition expands vulnerability surfaces and increases the burden of defense compared with purely textual settings.
  • Multimodal risks: LLM versatility lets visual attackers target toxicity, jailbreaking, misuse, and function creep rather than only classifier misclassification.The paper frames this as a shift toward considering the full use-case spectrum of LLMs.
  • Case study: The attack optimizes an image on 66 derogatory sentences targeting three identities to maximize their conditional generation probability.The visual adversarial example is used to circumvent safety guardrails in aligned LLMs with integrated vision.
  • Case study: A single optimized visual adversarial example generally compels aligned models to follow harmful instructions beyond the few-shot corpus, including murder instructions never explicitly optimized.The effect was observed across MiniGPT-4, InstructBLIP, and LLaVA, with black-box transferability also validated.
  • Implications: The findings connect persistent adversarial-example vulnerabilities in neural networks to a fundamental challenge for AI alignment under multimodality.The authors emphasize that this challenge is especially relevant to frontier foundation models.

2 Related Work

The related work situates this study at the intersection of LLM alignment, visual adversarial examples, jailbreaking, and red teaming. It distinguishes the paper’s universal jailbreak objective from earlier attacks targeting predetermined harmful outputs.

  • LLMs and VLMs: LLMs are autoregressive models trained on web-scale data that exhibit capabilities such as task-agnostic in-context learning and chain-of-thought reasoning.The paper focuses on GPT-like models that predict the next token.
  • LLMs and VLMs: VLMs integrate vision and language modules to process interlaced image-text inputs and generate free-form textual outputs.Visual encoders map inputs into a text-embedding space for subsequent reasoning and inference.
  • Alignment: Instruction tuning and RLHF are the main alignment techniques discussed, training models toward helpful, truthful, harmless, or preference-consistent behavior.Aligned models can refuse harmful instructions, which motivates attempts to bypass those constraints.
  • Jailbreaking: LLM jailbreaking means circumventing alignment guardrails so models produce content otherwise prohibited by their guidelines.The concept became prominent with aligned LLMs whose outputs are subject to explicit content constraints.
  • Adversarial examples: Visual adversarial examples exploit the continuity and high dimensionality of visual space, where quasi-imperceptible perturbations can fool accurate models.This work applies that adversarial-example tradition to behavior control in vision-integrated LLMs.
  • Red teaming: Red teaming broadly covers systematic adversarial testing for harmfulness and vulnerabilities, extending beyond the narrower study of jailbreaking.The paper places its attack within this broader evaluation context.
  • Related attacks: Earlier adversarial attacks elicited specific predetermined harmful content without examining safety-aligned models, whereas this study targets universal jailbreaks.The distinction makes the paper’s use of “jailbreaking” relevant to aligned systems.

3 Adversarial Examples as Jailbreakers

The attack treats an adversarial input as a universal prefix that increases an aligned VLM’s likelihood of harmful behavior. It optimizes against a small harmful corpus, then pairs the resulting input with other harmful instructions at inference time.

  • Threat model: The threat model appends a harmful instruction to an adversarial example, aiming to force an aligned LLM to generate otherwise prohibited content.The attacker seeks usability beyond executing one predetermined harmful instruction.
  • Our attack: The attack begins with a few-shot harmful corpus and maximizes the model’s generation probability for that corpus conditioned on the adversarial input.The optimization is presented as a simple input-space attack rather than model retraining.
  • Our attack: The search constraint B defines the allowed input space for adversarial examples, while inference pairs the optimized input with a separate harmful instruction.The joint input is written as [xadv,xharm].
  • Few-shot harmful corpus: Only 66 derogatory sentences targeting two anonymized identities and humanity are sufficient to bootstrap highly universal adversarial examples.The corpus is used as a few-shot seed rather than an exhaustive set of harmful behaviors.
  • Principle behind the approach: The approach is motivated by prompt tuning: optimizing an adversarial example in input space is treated as technically analogous to tuning prompts for a frozen LLM.The paper links both approaches to exploiting few-shot learning capabilities.
  • Visual attack: For visual attacks, the objective is end-to-end differentiable, enabling direct gradient backpropagation to the image and implementation with PGD.The experiments include unconstrained attacks and constrained attacks initialized from a benign image under an ℓ∞ perturbation bound.
  • Text attack counterpart: A text counterpart replaces adversarial image embeddings with equal-length adversarial text-token embeddings optimized with the same loss.This provides a comparison study for the visual cross-modal attack.

4 Evaluating Our Attacks

The study evaluates visual adversarial attacks against aligned vision-language models, finding that they induce harmful outputs across scenarios, models, and toxic attributes while transferring across models and remaining partly defensible.

  • Models and evaluation: The attacks are evaluated on MiniGPT-4, InstructBLIP, and LLaVA, including both white-box effectiveness and black-box transferability.The models use Vicuna- or LLaMA-2-based aligned backbones with integrated vision.
  • Human evaluation on harmful scenarios: Visual adversarial examples increase susceptibility to harmful instructions across identity attack, disinformation, violence/crime, and X-risk scenarios.The attacks generalize beyond the identity-attack and X-risk corpus used for optimization, also facilitating disinformation and violence/crime.
  • Human evaluation on harmful scenarios: The strongest attack raises identity-attack generation probability from 26.2% to 78.5% and increases violence and X-risk outputs by up to 37.2% and 53.3%, respectively.The reported harmful outputs span minority-group attacks, conspiracy theories, misleading medical advice, violent-action guides, extremist recruitment, and general malevolence.
  • Automated toxicity evaluation: On the challenging RealToxicityPrompts subset, visual adversarial examples increase toxic continuations across multiple toxicity attributes, not only identity attacks.The evaluation uses 1225 prompts and both Perspective API and Detoxify; the observed attribute increases align with manual inspection.
  • Comparison with text attacks: Text attacks also compromise safety but are weaker than visual attacks, whose continuous optimization achieves lower loss despite 12 times less computational effort.The comparison uses the same adversarial objective; the visual attack includes a tight ε constraint of 16/255.

5 Analyzing Defenses

Defending multimodal LLMs is difficult because existing defenses are costly, mismatched with open-ended outputs, and often limited to online deployment. DiffPure substantially reduces toxic generation, but does not eliminate all threat-model risks.

  • Defense challenges: Adversarial-example defenses remain an open problem, and multimodal models increase the burden of protecting every attack surface.The added visual modality introduces another channel requiring defenses.
  • Defense challenges: Existing adversarial training and robustness certification are prohibitively expensive at LLM scale and largely assume discrete-class outputs.Their small-perturbation assumptions also fail when jailbreak perturbations need not be imperceptible.
  • Input preprocessing: DiffPure adds image noise and uses a diffusion model to project the perturbed image back toward its learned data manifold.The evaluation uses Stable Diffusion v1.5 with three selected noise levels.
  • Input preprocessing: All three tested noise levels purify the visual adversarial examples, substantially lowering toxic-content likelihood toward the benign baseline.Perspective API and Detoxify results align well across the tested settings.
  • Defense limitations: DiffPure may fail against adaptive attacks and cannot protect offline models, while filtering APIs have limited accuracy and inconsistent measurements.Offline attackers can independently deploy and misuse models beyond online safeguards.

6 Discussions

The discussion places the attack within broader multimodal security risks, including transfer, dissemination, physical channels, and downstream systems. It also emphasizes policy adaptation while acknowledging that the evaluation is only a proof of concept.

  • Relation to prior work: Earlier adversarial-example studies targeted predetermined harmful outputs without safety-aligned models, whereas this work examines jailbreaking aligned models.The distinction makes bypassing safety restrictions central to the paper’s framing.
  • Practical implications: Open-source offline models remain vulnerable to direct white-box attacks, and preliminary results indicate black-box transfer among some open-sourced models.Model homogenization may make transfer from open-source to proprietary systems a practical risk.
  • Practical implications: A single universal jailbreaker could spread online and be used without specialized knowledge, while attacks may also affect robotics, APIs, tools, and plug-ins.The paper identifies both spreadability and downstream-system implications.
  • Risks of multimodality: Multimodality can create attack surfaces that are easier to optimize and add physical channels through vision, audio, and other modalities.The discussion extends beyond language-only harms to the physical means by which attacks may begin.
  • Policy implications: Instruction tuning and RLHF may be simple to bypass in accessible multimodal models, so safety recommendations should accommodate changing best practices.The paper argues that each added modality requires additional defensive investment.
  • Limitations: The evaluation uses incomplete datasets, manual assessment without a universal standard, and potentially inaccurate API-based toxicity measurement.Accordingly, the authors characterize the findings as a proof of concept for the examined adversarial risks.

7 Conclusion

The paper identifies escalating adversarial risks in multimodal systems and demonstrates that visual adversarial examples can jailbreak vision-integrated LLMs. It connects these vulnerabilities to a broader challenge for AI alignment.

  • The paper highlights expanded attack surfaces and broader security-failure implications as consequences of pursuing multimodality.
  • Visual adversarial examples can jailbreak large language models that incorporate visual inputs.
  • Adversarial vulnerabilities in neural networks create a tension with the nascent field of AI alignment.

A Additional Details of Our Experiments

The experiments study vision-integrated Vicuna-based models and optimize visual adversarial examples using a curated derogatory corpus. Model-specific input templates determine how image embeddings and user text are presented to the language model.

  • Models and input templates: MiniGPT-4 wraps image embeddings and user inputs in the Vicuna v0 chatbot format.Its template includes a system message, an image placeholder, user input, and an assistant continuation.
  • Models and input templates: In MiniGPT-4, the image placeholder is replaced by 32 image-embedding tokens before the user text is appended.Vicuna then autoregressively generates the assistant reply from the wrapped embeddings.
  • Models and input templates: InstructBLIP inserts image embeddings before text embeddings and directly concatenates them as the joint Vicuna input.
  • Models and input templates: LLaVA follows the LLaMA-2-13B-Chat template, wrapping user input with [INST] and [/INST] and the system message with «SYS» and «/SYS».
  • Attack corpus: The attack corpus contains 66 derogatory sentences: 21 targeting <gender-1>, 28 targeting <race-1>, and 17 targeting the human race.The sentences were generated by probing the unaligned LLaMA-1 model, then manually filtered for redundancy and non-harmful content.

A.3 Computing Infrastructure

The experiments use virtual nodes with one Nvidia A100 GPU and eight Intel Ice Lake CPU cores. Optimization runs for 5000 iterations with visual step size α = 1/255 and batches of eight samples.

  • Infrastructure: Each attack and evaluation test point runs on a virtual node with one Nvidia A100 80GB GPU and eight 2.8 GHz Intel Ice Lake CPU cores.Each CPU core has 16GB memory; the environment uses Red Hat 8.5.0-18, CUDA 12.1, PyTorch 1.12.1, and Python 3.9.
  • Optimization settings: All attacks use 5000 optimization iterations, where loss values converge slowly and become relatively stable around that point.
  • Optimization settings: Equation 1 optimization samples eight corpus examples per iteration to fit the memory of a single A100 GPU.Smaller batches may be unstable, whereas larger batches may not fit in memory.

B Attacks on InstructBLIP [21] and LLaVA [47]

The evaluation extends to InstructBLIP and LLaVA on the RealToxicityPrompts challenging subset. Tables 5 and 6 report the percentages of outputs exhibiting toxicity attributes under two toxicity-measurement systems.

  • Evaluation scope: The evaluation repeats RealToxicityPrompts testing for InstructBLIP and LLaVA, with results summarized in Tables 5 and 6.
  • Table 5: InstructBLIP: Table 5 reports InstructBLIP output percentages exhibiting toxic attributes, measured by Perspective API and Detoxify Classifier.The leftmost column represents outputs exhibiting any of six attributes.
  • Table 6: LLaVA: Table 6 reports LLaVA output percentages exhibiting toxic attributes, with the leftmost column covering any of six attributes.

C The Transferability of Our Attacks

The study validates black-box transferability across MiniGPT-4, InstructBLIP, and LLaVA, using surrogate-model adversarial examples and RealToxicityPrompts evaluation. Qualitative comparisons contrast clean and adversarial images.

  • Adversarial examples are optimized on a surrogate model and reused to attack a different target model.The evaluation uses a black-box transfer setting.
  • Transferability is evaluated across MiniGPT-4, InstructBLIP, and LLaVA.
  • The evaluation reports percentages of outputs exhibiting toxic attributes under the RealToxicityPrompts protocol.The reported attributes are measured with Perspective API.
  • Figure 4 compares MiniGPT-4 responses with a clean image against responses with an adversarial image.

E More Quantitative Assessments of Jailbreaks

The paper supplements its jailbreak analysis with a semi-automated study of red-teaming dialogs and reports response-flagging rates using moderation and reward-model criteria.

  • The semi-automated study analyzes red-teaming dialogs sorted by their lowest minimum harmlessness ratings.The dialogs come from the dataset provided by Ganguli et al.
  • Table 8 reports how often 1000 Anthropic red-teaming responses were flagged by the OpenAI Content Moderation API.
  • Responses were also flagged when the OpenAssistant reward model assigned a reward below −2.5.
  • The table presents category-level flagged-response proportions and whether any category was flagged for each sample.
Loading 2306.13213v2…