Source-linked AI summary
The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defense
Yangyang Guo, Fangkai Jiao, Liqiang Nie, Mohan Kankanhalli
TL;DR
The paper asks why VLLMs can be easy to jailbreak while existing defenses appear highly effective on benchmarks. It analyzes image-related vulnerability and over-prudent abstention, and proposes an LLM detector before VLLM response. Its findings caution that benign-query abstention and disagreement between evaluation methods complicate the apparent safety picture.
Problem
VLLMs show high jailbreak vulnerability even as recent defenses achieve near-saturation benchmark performance, creating a safety paradox that complicates trustworthy assessment.
Source: §1 · p. 1Method
The paper analyzes VLLM jailbreak and defense behavior and proposes LLM-Pipeline, which uses advanced LLM guardrails as a detector before VLLM response generation.
Source: §1 · pp. 1–2Results
The authors identify image inputs as compromising backbone LLM guardrails, over-prudent defenses that abstain on benign queries, and disagreement between rule-based and LLM-based attack evaluations.
Source: §1 · pp. 1–2Takeaways & Limitations
The authors argue that existing LLM guardrails can serve as a potential vision-free jailbreak detector, while the safety paradox warrants further attention.
Source: §7 · p. 8LLM-Pipeline is not intended as a better jailbreak-defense baseline because the authors see minimal to no room for improvement.
Source: §7 · p. 8
Abstract
from arXiv · showhide
The vulnerability of Vision Large Language Models (VLLMs) to jailbreak attacks appears as no surprise. However, recent defense mechanisms against these attacks have reached near-saturation performance on benchmark evaluations, often with minimal effort. This \emph{dual high performance} in both attack and defense raises a fundamental and perplexing paradox. To gain a deep understanding of this issue and thus further help strengthen the trustworthiness of VLLMs, this paper makes three key contributions: i) One tentative explanation for VLLMs being prone to jailbreak attacks--\textbf{inclusion of vision inputs}, as well as its in-depth analysis. ii) The recognition of a largely ignored problem in existing defense mechanisms--\textbf{over-prudence}. The problem causes these defense methods to exhibit unintended abstention, even in the presence of benign inputs, thereby undermining their reliability in faithfully defending against attacks. iii) A simple safety-aware method--\textbf{LLM-Pipeline}. Our method repurposes the more advanced guardrails of LLMs on the shelf, serving as an effective alternative detector prior to VLLM response. Last but not least, we find that the two representative evaluation methods for jailbreak often exhibit chance agreement. This limitation makes it potentially misleading when evaluating attack strategies or defense mechanisms. We believe the findings from this paper offer useful insights to rethink the foundational development of VLLM safety with respect to benchmark datasets, defense strategies, and evaluation methods.
1. Introduction
The paper investigates why VLLMs can be easy to jailbreak while appearing easy to defend, identifying image inputs and over-prudent defenses as central findings. It also proposes an LLM-based detector before VLLM response and highlights disagreement between jailbreak evaluation methods.
1. Introduction
- Jailbreak attacks are designed to circumvent model safeguards and elicit malicious outputs, and VLLMs have shown high attack success rates on curated datasets.The introduction notes typography and generative image techniques as ways to inject high-risk content.Source: §1 · p. 1
- The paper frames near-zero attack success rates for defenses on benchmark attacks as a paradox because attacks themselves have been relatively painless.It presents the work as a comprehensive study and calls for more attention to attack data, defenses, and evaluations.Source: §1 · pp. 1–2
- The authors attribute VLLM jailbreak vulnerability to image inputs compromising the backbone LLM guardrails, rather than catastrophic forgetting or fine-tuning.They describe this as their explanation for the safety paradox.Source: §1 · p. 1
- Existing defenses can abstain even on benign queries, while rule-based and LLM-based evaluations can disagree about whether attacks succeed.The authors report that both issues undermine reliable assessment of defense helpfulness and attack effectiveness.Source: §1 · p. 2
- LLM-Pipeline places an LLM detector before VLLM response generation and is reported to balance safety alignment with helpfulness while reducing over-prudence.The detector judges textual queries, optionally with an image caption, using guardrails of advanced LLMs.Source: §1 · p. 2
2. Preliminary
The preliminary section defines the study's one-instruction, one-image input setting and describes how harmfulness and jailbreak outcomes are assessed. It also introduces the evaluated datasets and reports high attack risk across the benchmarked VLLMs.
2. Preliminary
- The study limits each VLLM input to one textual instruction and one image, matching existing jailbreak attack datasets.Responses may be abstentions or may follow harmful instructions; compositions in quadrants II, III, and IV should be rejected.Source: §2 · p. 2
- Figure 1 ranks the harmfulness of its three depicted input-composition quadrants as II < IV < III.The figure describes safety attributes for combinations of textual instructions and visual images.Source: Figure 1 · p. 2
- The four evaluated jailbreak datasets mostly use LLM-generated instructions and benign or generated images, with quadrants classified as in Figure 1.Table 1 reports dataset statistics, including the number of harmful scenarios (#HS).Source: §2 · p. 2, Table 1 · p. 2
- Rule-based evaluation searches responses for refusal keywords, whereas LLM-based evaluation judges harmfulness from the attack prompt and model response.The study uses Attack Success Rate (ASR), where higher ASR indicates greater risk of successful jailbreak and a more vulnerable model.Source: §2 · p. 2
- Table 2 reports high ASR across six VLLMs and four jailbreak attack datasets.The caption characterizes all models as at high risk of generating harmful responses on these benchmarks.Source: Table 2 · p. 3
- Figure 2 gives examples of QWen2-VL-generated harmful captions in response to benign prompts, including religious hate speech and racially biased history.More contentious political cases are shown in supplementary material.Source: Figure 2 · p. 3
3. Ease of Jailbreak Attack
VLLMs show vulnerability to jailbreak attacks, and the analysis links compromised safety alignment to processing images alongside text. The section also reports harmful captions from benign prompts and finds that harmful images attract greater attention even under benign instructions.
3.1. Evidence
- Even advanced VLLMs show high attack success rates across four jailbreak benchmarks, including Qwen2-VL.The benchmark results indicate that vulnerability is not limited to older VLLMs.Source: §3.1 · p. 3
- A neutral caption prompt can elicit hateful or racially harmful image descriptions from VLLMs.The examples involve benign, general captioning prompts rather than harmful instructions.Source: §3.1 · p. 3
3.2. Rationale: Inclusion of Vision Inputs
- Safety alignment remains largely intact in text-only variants but is significantly compromised when VLLMs process joint vision-text inputs.The analysis challenges catastrophic forgetting or fine-tuning as the primary explanation for the safety loss.Source: §3.2 · p. 3
- For benign instructions, VLLMs attend more to harmful than safe images, while attention to the same image remains nearly unchanged across safe and unsafe instructions.This pattern points to image harmfulness, rather than instruction safety attributes, as the relevant difference in attention.Source: §3.2 · p. 4, Figure 4 · p. 4
4. Ease of Jailbreak Defense
Simple defenses yield strong benchmark results, but their benign-input abstention and the weak agreement between evaluation methods complicate what those results establish. Deliberate abstention can also appear protective under jailbreak evaluation while sharply reducing responses to benign queries.
4.1. Evidence
- Safety-aware fine-tuning and training-free prompt defenses substantially improve results over their base VLLMs, with some models reaching optimal safeguard on FigStep.The paper examines Mixed VLGuard and AdaShield-A as representative methods from these two straightforward approaches.Source: §4.1 · p. 4
4.3. Rationale 2: Evaluation Dilemma
- Rule-based and model-based evaluations show chance-level agreement, so defense performance under one metric can be contradicted by the other.The reported Cohen’s kappa values are predominantly negative or close to zero.Source: Figure 7 · p. 5, §4.3 · p. 5
4.2. Rationale 1: The Over-Prudence Problem
- Although these defenses extrapolate to unseen jailbreak datasets, they also reject benign caption and text-only inputs, indicating over-prudence.The benign-input evaluations cover safe image-plus-caption prompts and safe textual instructions.Source: §4.2 · p. 5
4.3. Rationale 2: Evaluation Dilemma
- Deliberate post-answer abstention drives FigStep ASR toward zero but raises abstention on benign MM-Vet queries, reaching 100% for Intern-VL2.Intern-VL2’s original MM-Vet accuracy declines slightly; less robust models also experience a modest performance decline.Source: Figure 8 · p. 6, §4.3 · p. 6
5. Jailbreak Detector and Beyond
The section tests vision-free LLM detectors and a detect-then-respond pipeline as ways to address VLLM jailbreaks. It reports competitive detection results and a helpfulness–harmlessness trade-off, while noting limits when attacks require image-text understanding.
5. Jailbreak Detector and Beyond
- The proposed vision-free detectors repurpose LLM guardrails because recent jailbreak datasets mostly contain harmful textual instructions, and recent LLMs have strengthened guardrails.The two variants are Naive, which takes the original instruction, and Scenario, which adds a scenario-aware system protection prompt.Source: §5 · p. 6
5.1. Detector Results on Jailbreak Datasets
- Llama3.1-8B reaches an ASR of 0.7 on VLSafe, while Llama3.1-70B with Scenario reaches an ASR of 2.8 on FigStep; joint image-text cases are exceptions.The results are reported for vision-free detectors; Table 5 covers VLSafe and FigStep, with Scenario denoting an added system protection prompt.Source: §5.1 · p. 6, Table 5 · p. 7
- Caption re-check generally decreases ASR, especially when captions contain OCR information, but QWen2.5’s ASR increases as it often refuses without standard refusal keywords.The authors identify VLGuard and MM-SafetyBench as datasets requiring joint image-text understanding.Source: §5.1 · p. 6
5.2. Detect-then-Respond: LLM-Pipeline
- LLM-Pipeline achieves a better helpfulness–harmlessness trade-off than the compared defenses, while vanilla LLaVA-1.5 performs best on Safe-Safe but compromises defense effectiveness.Figure 9 plots Safe-Unsafe ASR against Safe-Safe winning rate; the pipeline first screens instructions with an LLM detector, then sends passing queries to a VLLM.Source: Figure 9 · p. 7, §5.2 · p. 7
- Figure 10 shows two controversial VLGuard examples labeled safe by Llama3.1.Source: Figure 10 · p. 7
6. Related Work
The review organizes VLLM jailbreak attacks into adversarial perturbation and prompt-injection approaches, then surveys VLLM and LLM defenses. It also contrasts white-box and black-box LLM attacks and their corresponding defenses.
6.1. VLLM Attack
- VLLM attacks include adversarial image perturbations [40] and prompt injection [32] [35], including image-embedded text and generated harmful images.Prompt-injection examples include FigStep [19] and MM-SafetyBench; adversarial inputs can also universally jailbreak aligned VLLMs [21] [47].Source: §6.1 · p. 7
6.2. VLLM Defense
- VLLM defenses add system-prompt guardrails [19] [64] [65], use multi-stage unsafe-content detection and abstention [46] [20], or fine-tune on harmful and benign instructions.The reviewed methods show promise on benchmarks, while multi-stage methods may compromise inference efficiency.Source: §6.2 · p. 8
6.3. LLM Attack and Defense
- LLM attacks are classified as white-box methods using gradients or logits [27] and adversarial fine-tuning [29], versus black-box prompt manipulation [15].Reviewed LLM defenses include system-prompt safeguards [52], supervised fine-tuning [4] [5] [14], and RLHF [3] [17] [43] [58].Source: §6.3 · p. 8
7. Conclusion and Discussion
The paper studies the VLLM safety paradox, including over-prudent defenses, and proposes repurposing LLM guardrails as a vision-free detector. It cautions that LLM-Pipeline is not a stronger defense baseline and identifies future work on attacks, defenses, and evaluation.
7. Conclusion and Discussion
- The paper studies the safety paradox and over-prudence, and proposes repurposing existing LLM guardrails as a vision-free jailbreak detector.Source: §7 · p. 8
- LLM-Pipeline is not intended as a better jailbreak defense baseline because the authors see minimal to no room for improvement.Suggested directions include more comprehensive attack datasets, robust defenses, and human-aligned jailbreak evaluation.Source: §7 · p. 8
- The literature lacks consensus on harmful-scenario definitions, and publishing defense rationales may help malicious users evade detection, with potential societal harm.Source: §7 · p. 9