Source-linked AI summary
Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency
Shiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen, Caixin Kang, Shouwei Ruan, Jialing Tao, YueFeng Chen, Hui Xue, Xingxing Wei
TL;DR
MLLM jailbreak methods face limited effectiveness against commercial closed-source models, motivating investigation of their comprehension–safety gap for shuffled harmful instructions. SI-Attack uses toxic-judge feedback to optimize shuffled text-image inputs and improves attack performance across three benchmarks, including against commercial models.
Problem
Existing multimodal jailbreak methods often rely on complex optimization or carefully designed prompts and have limited performance against commercial closed-source MLLMs.
Method
SI-Attack applies query-based black-box optimization with toxic-judge feedback to select harmful shuffled image-text instructions.
Results
SI-Attack improves toxic score and attack success rate for open-source and closed-source MLLMs across three benchmarks.
Takeaways & Limitations
The findings indicate that a mismatch between MLLM comprehension and safety can make comprehension capabilities exploitable by attackers.
Takeaways & Limitations
Claude-3.5-Sonnet shows strong robustness on HADES, possibly because HADES contains sensitive words filtered by its outer safety guardrail.
Abstract
from arXiv · showhide
Multimodal Large Language Models (MLLMs) have achieved impressive performance and have been put into practical use in commercial applications, but they still have potential safety mechanism vulnerabilities. Jailbreak attacks are red teaming methods that aim to bypass safety mechanisms and discover MLLMs' potential risks. Existing MLLMs' jailbreak methods often bypass the model's safety mechanism through complex optimization methods or carefully designed image and text prompts. Despite achieving some progress, they have a low attack success rate on commercial closed-source MLLMs. Unlike previous research, we empirically find that there exists a Shuffle Inconsistency between MLLMs' comprehension ability and safety ability for the shuffled harmful instruction. That is, from the perspective of comprehension ability, MLLMs can understand the shuffled harmful text-image instructions well. However, they can be easily bypassed by the shuffled harmful instructions from the perspective of safety ability, leading to harmful responses. Then we innovatively propose a text-image jailbreak attack named SI-Attack. Specifically, to fully utilize the Shuffle Inconsistency and overcome the shuffle randomness, we apply a query-based black-box optimization method to select the most harmful shuffled inputs based on the feedback of the toxic judge model. A series of experiments show that SI-Attack can improve the attack's performance on three benchmarks. In particular, SI-Attack can obviously improve the attack success rate for commercial MLLMs such as GPT-4o or Claude-3.5-Sonnet.
1. Introduction
The paper identifies a gap between MLLMs’ comprehension and safety abilities for shuffled harmful instructions and proposes SI-Attack to exploit it. Experiments report improved jailbreak performance across open-source and closed-source models.
- Existing multimodal jailbreak methods often require careful prompt design or complex optimization, while commercial closed-source models retain additional safety guardrails.
- MLLMs can understand shuffled harmful instructions while their defense mechanisms fail to protect against them.
- SI-Attack uses Shuffle Inconsistency to construct image-text jailbreak inputs.
- A query-based black-box optimization selects harmful shuffled instructions using toxic-judge feedback to reduce shuffle randomness.
- Experiments across three benchmarks show that SI-Attack improves attack success against open-source and commercial closed-source MLLMs with few queries.
2. Related Work
Prior multimodal jailbreak research uses adversarial perturbations, malicious visual content, and harmful typography, but these approaches can be complex and face stronger commercial safeguards.
- Prior methods embed harmful intent through optimized image perturbations or malicious triggers in otherwise benign images.
- Commercial closed-source MLLMs often add outer safety guardrails that detect harmful intent and limit jailbreak performance.
3. Shuffle Inconsistency for Harmful Prompt
Experiments reveal that shuffling harmful text or image inputs preserves enough comprehension to increase toxic responses, exposing a mismatch between MLLM comprehension and safety. Across mutation types, shuffling produces the strongest inconsistency.
- 3. Shuffle Inconsistency for Harmful Prompt: Prior studies motivate testing whether MLLMs retain comprehension of shuffled text and images in harmful-instruction settings.
- 3.1. Text Shuffle Inconsistency: The study evaluates LLaVA-NEXT, InternVL-2, GPT-4o, and Gemini-1.5-Pro using toxic scores judged from model responses.
- 3.1. Text Shuffle Inconsistency: Toxic scores increase for shuffled text across all four evaluated MLLMs, showing that shuffled inputs preserve harmful-intention comprehension while weakening safety protection.
- 3.2. Image Shuffle Inconsistency: Shuffling image patches also increases toxic scores, with reported increases of 1.21 and 0.52 for LLaVA-NEXT and InternVL-2 and 0.62 and 0.84 for GPT-4o and Gemini-1.5-Pro.
- 3.3. Inner Behavior for Shuffle Inconsistency: Hidden-state PCA shows similar processing of original and shuffled inputs in shallow layers but increasingly different reactions in deeper layers.
- 3.4. Shuffle or Other Mutation Operations?: Random replacement, insertion, deletion, synonym replacement, punctuation insertion, masking, grayscale changes, solarization, flipping, and blurring also increase toxic scores.
- 3.4. Shuffle or Other Mutation Operations?: Shuffling produces the highest toxic scores among the tested mutation operations, likely balancing semantic preservation with safety-mechanism disruption.
4. Jailbreak based on Shuffle Inconsistency
SI-Attack exploits shuffle inconsistency by randomly rearranging words and image patches, then uses black-box toxic-score feedback to select harmful shuffled inputs.
- Stopping and output: The procedure repeats shuffling, querying, and scoring until the score reaches the attack threshold or the maximum query iteration is reached.If successful, it returns the harmful shuffled inputs; otherwise, it returns those with the highest toxic score.
- Input shuffling: SI-Attack shuffles harmful text at the word level and harmful images at the patch level before reassembling them as new input pairs.The text prompt is represented as words, while the image is divided into patch blocks.
- Toxicity-guided optimization: The attack uses a target MLLM to generate responses for shuffled inputs and a toxic judge model to score their harmfulness.The judge evaluates the target response against the harmful input and records the resulting score.
- Toxicity-guided optimization: Black-box optimization selects shuffled text-image inputs that maximize the toxic score without requiring white-box access to the target MLLM.This design addresses the closed-source nature of many MLLMs.
5. Experiments
Experiments evaluate SI-Attack across open-source and commercial closed-source MLLMs, using three text-image jailbreak benchmarks and ablations of its shuffling and query-based optimization. SI-Attack improves toxic score and attack success rate over competing attacks, including against commercial safety guardrails.
- Experimental Settings: Experiments evaluate four open-source and four commercial closed-source MLLMs across MM-safetybench, HADES, and SafeBench.The evaluation includes LLaVA-NEXT, MiniGPT-4, InternVL-2, VLGuard, GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro, and Qwen-VL-Max.
- Overall Results: SI-Attack obviously enhances toxic score and ASR compared with QR-Attack, HADES, and Figstep-Attack.The comparisons use the three benchmark settings and ChatGPT-3.5 toxicity evaluation.
- Performance on Open-source MLLMs: 62.68%, 62.44%, 71.01%, and 40.77% ASR are achieved on MM-safetybench for LLaVA-NEXT, MiniGPT-4, InternVL-2, and VLGuard, respectively.These results outperform QR-Attack by 18.69%, 35.24%, 30.71%, and 31.25%, respectively.
- Performance on Closed-source MLLMs: 47.80%, 39.70%, 50.18%, and 35.59% improvements over QR-Attack are reported for GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro, and Qwen-VL-Max.The results indicate that SI-Attack can bypass outer safety guardrails of commercial closed-source models.
- Ablation Study: Shuffling both images and texts achieves the best attack performance, while text-only shuffling outperforms image-only shuffling.This indicates more serious safety vulnerabilities on the text side than on the image side.
- Ablation Study: Query-based optimization improves over randomly shuffled inputs, and 10 iterations provide nearly the best results with better overall effect and efficiency.Further increasing the maximum query iteration to 20 yields limited improvement.
6. Conclusion
The paper concludes that shuffled harmful text-image instructions expose a mismatch between MLLMs’ comprehension and safety abilities. SI-Attack exploits this Shuffle Inconsistency through toxic-judge-guided black-box optimization, improving toxic score and attack success rate across open- and closed-source MLLMs.
- Conclusion: MLLM defense mechanisms show Shuffle Inconsistency: they understand shuffled harmful instructions while failing to defend against them.The paper frames this mismatch as a vulnerability caused by safety capabilities not matching comprehension capabilities.
- Conclusion: SI-Attack uses query-based black-box optimization with toxic-judge feedback to improve the effectiveness of shuffled harmful instructions.The optimization is designed to address shuffle instability while exploiting the observed inconsistency.
- Conclusion: Experiments show obvious improvements in toxic score and attack success rate for open-source and closed-source MLLMs.The conclusion presents these results as insights for safety researchers.
A. Hyper-parameter of Patch Nums
Patch-count experiments examine how image shuffling affects attack behavior. Within a certain range, shuffled images remain understandable while bypassing safety mechanisms, but increasing the number of shuffled patches eventually changes the outcome.
- Hyper-parameter of Patch Nums: Experiments compare 1, 4, 16, and 64 shuffled image patch blocks.The results are reported in Table A.1 using the MM-safetybench 01-Illegal-Activity sub-dataset without typography.
- Hyper-parameter of Patch Nums: Within a certain patch-count range, shuffled images can be understood by the model while bypassing its safety mechanism.The passage describes this behavior as preserving comprehension alongside weakened safety defense.
B. Influence of Text Shuffling Types
The study compares several text-shuffling strategies for SI-Attack. Randomly shuffling all words achieves the best attack results and is selected for the method.
- Influence of Text Shuffling Types: The comparison includes no shuffling, all-word shuffling, noun-and-adjective shuffling, trigram shuffling, within-trigram shuffling, and token-wise BPE shuffling.These alternatives are evaluated in Table B.1.
- Influence of Text Shuffling Types: Randomly shuffling all words obtains the best results among the tested text-shuffling approaches.The authors therefore select this approach in the final method.
C. Performance on Different Scales’ MLLMs
The paper evaluates SI-Attack across InternVL-2 models at 4B, 8B, and 26B scales using MM-safetybench’s 01-Illegal-Activity subset without typography. It also tests an adaptive variant against perplexity-based defenses.
- Different scales’ MLLMs: SI-Attack is evaluated on InternVL-2 models with 4B, 8B, and 26B parameters.The evaluation uses MM-safetybench’s 01-Illegal-Activity subset without harmful typography.
- Different scales’ MLLMs: The scale experiment keeps the final-version settings for shuffling only images and texts.
- Adaptive SI-Attack against PPL Detector: Against perplexity detection, adaptive SI-Attack uses Trigram-based text shuffling and retains competitive attack performance.The method first applies a perplexity detector and optimizes only texts that pass it.
E. More Results on MM-safetybench
On MM-safetybench without harmful typography, SI-Attack improves attack effectiveness over QR Attack across open- and closed-source MLLMs. The evaluation uses toxic score and attack success rate, with ChatGPT-3.5 judging responses.
- Results on MM-safetybench: SI-Attack achieves higher attack success rates than the original jailbreak instructions across four open-source MLLMs.ASRs are 37.98% for LLaVA-NEXT, 54.88% for MiniGPT-4, 48.15% for InternVL-2, and 39.88% for VLGuard, versus 19.77%, 33.81%, 34.82%, and 25.49%.
- Results on MM-safetybench: 35.95%, 32.21%, 32.14%, and 38.69% are the reported SI-Attack attack-success-rate increases for GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro, and Qwen-VL-Max.
- Harmful cases: The supplementary cases cover harmful images without typography, harmful images with typography, and images containing only harmful typography.These cases are shown in Figures F.1, F.2, and F.3, respectively.
- Toxic-score evaluation: The toxic judge assigns the highest score only when a response is toxic and fully responds to the attacker’s intent.The automatic evaluation prompt includes prohibited activities and toxic-score criteria.
- Results on MM-safetybench: Tables E.1 and E.2 report toxic score and attack success rate for QR Attack and SI-Attack across 13 prohibited-scenario subsets and the full dataset.The harmful instructions come from MM-safetybench without typography and are evaluated by ChatGPT-3.5.