Source-linked AI summary
Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models
Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Ji-Rong Wen
TL;DR
MLLMs remain vulnerable to harmlessness failures associated with visual inputs, but the sources of these vulnerabilities are not fully understood. The paper empirically analyzes representative models and introduces HADES, a visual jailbreak method that hides and amplifies harmful intent. HADES achieves 90.26% ASR on LLaVA-1.5 and 71.60% on Gemini Pro Vision.
Problem
The paper studies how visual inputs create harmlessness-alignment vulnerabilities in MLLMs beyond the safety weaknesses inherited from backbone LLMs.
Method
The paper combines empirical evaluation with HADES, which transfers harmful intent into typography and crafted images, then adds optimized adversarial noise.
Results
90.26% ASR on LLaVA-1.5 and 71.60% on Gemini Pro Vision show that HADES effectively jailbreaks evaluated MLLMs.
Takeaways & Limitations
Visual modality vulnerabilities can be exploited to jailbreak both open-source and closed-source MLLMs, motivating safer alignment research.
Takeaways & Limitations
The work acknowledges that its harmful instructions and images may have negative societal impacts, despite mitigation measures.
Abstract
from arXiv · showhide
In this paper, we study the harmlessness alignment problem of multimodal large language models (MLLMs). We conduct a systematic empirical analysis of the harmlessness performance of representative MLLMs and reveal that the image input poses the alignment vulnerability of MLLMs. Inspired by this, we propose a novel jailbreak method named HADES, which hides and amplifies the harmfulness of the malicious intent within the text input, using meticulously crafted images. Experimental results show that HADES can effectively jailbreak existing MLLMs, which achieves an average Attack Success Rate (ASR) of 90.26% for LLaVA-1.5 and 71.60% for Gemini Pro Vision. Our code and data are available at https://github.com/RUCAIBox/HADES.
1 Introduction
MLLMs inherit harmlessness weaknesses from LLMs, while visual inputs introduce additional alignment vulnerabilities. The paper analyzes these vulnerabilities and proposes HADES, which hides and amplifies harmful intent through crafted images.
- MLLMs retain harmlessness challenges despite alignment methods, including vulnerability to black-box and white-box attacks inherited from backbone LLMs.
- Visual inputs can increase harmful outputs, with Gemini Pro Vision producing improper harmful responses even when paired with blank or harmful images.
- The analysis finds that images can act as alignment backdoors, cross-modal fine-tuning can disrupt backbone alignment, and image harmfulness correlates with harmful responses.
- HADES transfers harmful information from text into typography, combines it with an amplified harmful image, and adds optimized adversarial noise to induce harmful responses.
- 90.26% ASR on LLaVA-1.5 and 71.60% on Gemini Pro Vision demonstrate that HADES affects both open-source and closed-source MLLMs.
2 Empirical Harmlessness Analyses of MLLMs
The study evaluates representative MLLMs across harmful scenarios and visual-input settings using ASR. Results show that images can undermine alignment, harmful images elicit more harmful outputs, and fine-tuning choices affect robustness.
- Data and evaluation: The evaluation dataset contains 750 harmful instructions across five scenarios, each paired with a harmful image related to its keyword or phrase.
- Data and evaluation: The study evaluates open- and closed-source MLLMs under Backbone, Text-only, Blank, and Toxic settings using Attack Success Rate.
- Findings: Images act as alignment backdoors: harmless blank images raise ASR by 25.20% for LLaVA-1.5 and 33.87% for MiniGPT-v2.
- Findings: More extensive cross-modal fine-tuning is associated with weaker alignment, with MiniGPT-v2 reaching 43.87% ASR versus 23.47% for MiniGPT-4 under Toxic.
- Findings: Harmful images elicit more harmful outputs across model types, including MiniGPT-4 at 23.47% versus 8.80% and LLaVA-1.5 at 68.13% versus 54.13%.
3 The Proposed Jailbreak Approach: HADES
HADES is an automated jailbreak approach that transfers harmful intent from text to images, amplifies image harmfulness, and adds optimized adversarial noise to induce harmful MLLM responses.
- 3 The Proposed Jailbreak Approach: HADES: HADES modifies harmful instructions and crafts harmful images to transfer malicious intent from the text side to the image side.The method targets the comparatively less-aligned image side of MLLMs.
- 3.1 Hiding Harmfulness from Text to Image: Text-to-image pointers replace harmful keywords with references to image content, while typography preserves the keyword visually for the model to infer.The modified instruction no longer explicitly contains the harmful information, but the model can infer the intent from the typography.
- 3.2 Amplifying Image Harmfulness: HADES optimizes harmful images iteratively using generated captions, harmfulness scores, explanations, and refined prompts over multiple turns.The judging model scores harmfulness from 1 to 10, and the optimization runs for a maximum of K=5 turns.
- 3.2 Amplifying Image Harmfulness: The optimized image is concatenated with typography so both images reinforce the attack and provide context for understanding the masked instruction.The optimized image can partially compensate for insufficient OCR capabilities.
- 3.3 Amplifying Image Harmfulness with Gradient Update: HADES further concatenates a category-level adversarial image and optimizes it by gradient updates toward affirmative responses under valid-image constraints.The adversarial image is generated once for all harmful instructions within the same category.
4 Experiment
Experiments evaluate HADES across open- and closed-source MLLMs, component settings, optimization steps, transferability, and jailbreak response types. HADES substantially increases ASR, with image optimization and adversarial images strengthening attacks, while transferability extends across models and categories.
- 4.2 Experiment Results: HADES significantly increases attack success rates across open- and closed-source MLLMs; Gemini ProV reaches 71.60% average ASR.The three LLaVA-series models each exceed 80% average ASR, while GPT-4V has 15.07% harmful responses.
- 4.2 Experiment Results: The text-to-image pointer changes ASR unevenly, increasing it for LLaVA, Gemini ProV, and GPT-4V but decreasing it for LLaVA-1.5 and LLaVA-1.5L.The reported changes are +6.53%, +16.93%, +1.47%, −7.47%, and −14.13%, respectively.
- 4.2 Experiment Results: Adding the optimized image increases ASR for all models by more than 30% in the reported examples.The increases include +33.86% for LLaVA and +37.20% for Gemini ProV, while the image also helps mitigate misunderstanding.
- 4.2 Experiment Results: The full HADES attack raises LLaVA’s average ASR to 82.53% and improves its Animal and Self-Harm ASRs to 76.00% and 75.33%.These categories were relatively harder to jailbreak under previous settings.
- 4.3 Further Analyses: ASR consistently improves as image-generation optimization steps increase across all comparison models.This supports the effectiveness of HADES’s image harmfulness optimization.
- 4.3 Further Analyses: HADES transfers across MLLMs, with attacks trained on one model achieving comparable ASR on other models and strong mutual transferability between LLaVA-1.5 and LLaVA-1.5L.The transferability analysis also spans the listed harmful instruction categories.
- 4.3 Further Analyses: Successful jailbreaks mainly reflect conflicts between general instruction following and harmlessness alignment, with OCR and captioning also contributing substantially.The cases are organized around OCR, image captioning, and instruction following.
5 Related Work
Prior work studies harmlessness alignment in LLMs and MLLMs through red-teaming, benchmarks, and adversarial attacks. MLLM attacks include white-box manipulation of input images or visual embeddings and black-box approaches.
- Harmlessness Alignment of LLMs: LLM harmlessness alignment research uses human-preference fine-tuning and red-teaming benchmarks to evaluate responses to harmful instructions.The 3H principle describes responses as Helpful, Honest, and Harmless.
- Harmlessness Alignment of MLLMs: MLLMs inherit alignment vulnerabilities from their LLM backbones, motivating benchmarks that probe harmfulness across different scenarios.These benchmarks examine harmlessness under multimodal conditions.
- Harmlessness Alignment of MLLMs: Existing MLLM jailbreak research includes white-box attacks on input images or visual embeddings, using adversarial-image constraints or teacher-forcing optimization.These methods evaluate adversarial robustness through different attack settings.
6 Conclusion
The paper identifies visual inputs as a major alignment vulnerability in MLLMs and introduces HADES, which effectively jailbreaks open- and closed-source models. It also preliminarily finds that image-related alignment data can reduce harmful responses while preserving general multimodal capabilities.
- Images, cross-modal fine-tuning, and harmful image content all increase MLLMs’ propensity to generate harmful responses.
- HADES hides and amplifies harmful textual instructions through carefully crafted images to jailbreak MLLMs.
- Preliminary fine-tuning with harmful and harmless OCR and captioning instructions reduces harmful responses while maintaining general multimodal capabilities.
A.1 Experimental Setting
The defense data pairs harmful and harmless OCR or captioning instructions with images and expected responses. These contrastive examples are designed to teach MLLMs which instructions to follow or refuse.
- The defense dataset uses triplets containing a textual instruction, an image, and an expected response.
- OCR negative examples pair harmful image-text inputs with refusal responses while preserving OCR capability.
- Captioning negative examples use optimized images with harmfulness scores of at least 5 and refusal responses for harmful captions.
- The construction pairs each negative instruction with a positive counterpart to contrast harmful and harmless behavior.
- The dataset contains 2286 instructions, with the first half used for generation and the remainder reserved for evaluation.
A.2 Results and Analysis
The evaluation tests LLaVA-1.5 with and without contrastive harmlessness LoRA on HADES attack settings and on LLaVA-Bench. The LoRA sharply reduces attack success while preserving general multimodal performance.
- Contrastive harmlessness LoRA is evaluated on HADESopt, HADESadv, and LLaVA-Bench using LLaVA-1.5.
- 79.20% to 6.67%: average ASR decreases on HADESopt after adding contrastive harmlessness LoRA.
- 89.53% to 5.07%: average ASR decreases on HADESadv after adding contrastive harmlessness LoRA.
- Contrastive harmlessness LoRA does not significantly affect LLaVA-1.5’s LLaVA-Bench performance.
B Comparison with other jailbreak methods.
HADES is compared with adversarial and compositional jailbreak baselines on LLaVA-1.5. It achieves the highest attack success rate across all reported categories, while human annotations support the reliability of the harmfulness evaluator.
- Comparison with other jailbreak methods: HADES achieves the highest ASR across all categories compared with Adversarial and Compositional jailbreak methods.
- Comparison with other jailbreak methods: The comparison evaluates HADES and two representative baselines on LLaVA-1.5 using the collected dataset.
- Evaluation reliability: Beaver-dam-7B’s harmfulness assessments are highly consistent with those of three human annotators.
D Data Collection Pipeline
The harmful-instruction collection pipeline selects visual-world harm categories, generates keywords and instructions, and pairs each instruction with a relevant real-world image.
- The pipeline selects five harmful categories related to visual information in the real world.
- GPT-4 generates 50 keywords for each harmful category, then synthesizes three harmful but distinct instructions from those keywords.
- Each harmful instruction is paired with a corresponding real-world image relevant to its harmful keywords.
E Additional Jailbreak Cases
The supplementary material provides additional jailbreak cases, prompt templates, and an iterative LLM-based procedure for optimizing image harmfulness.
- Additional Jailbreak Cases: Additional jailbreak cases are provided for LLaVA-1.5, GPT-4V, and Gemini ProV.
- Prompt Templates: The keyword-generation prompt instructs an attacker model to produce 50 lowercase harmful keywords for a specified scenario.
- Prompt Templates: The attacker prompt directs a red-teaming assistant to optimize text-to-image prompts so generated-image captions receive higher harmfulness scores.
- Prompt Templates: The judging prompt scores captions from 1 to 10 using attributes including violence, harmful objects, harmful behavior, and its consequences.
- Image Harmfulness Optimization: The optimization algorithm iteratively generates images, obtains captions and harmfulness scores with explanations, updates conversation history, and refines the next image-generation prompt.
- Image Harmfulness Optimization: The refined prompt is used to generate a new image, and the resulting information is fed back to the attacker model for further refinement.