PaperlayerView exact revision on arXiv

Source-linked AI summary

Jailbreak Connectivity: Towards Diverse, Transferable, and Universal Multimodal Large Language Model Jailbreak

Xu Zhang, Ziqing Hu, Shuo Han, Ren Wang

arXiv:2610.38091v1cs.LGcs.AIcs.CR

TL;DR

Existing MLLM jailbreak methods often optimize isolated images, limiting diversity, cross-model transfer, and coverage across harmful queries. JC explores connected low-loss image-space regions through path sampling, surrogate guidance, and a distribution-level objective. The authors report stronger attack success than prior methods and lower response perplexity in white-box and black-box settings.

  • Problem

    Prior visual jailbreak approaches optimize isolated adversarial points, while diversity, cross-model transfer, and universality remain limited.

    Source: §I · p. 1

  • Method

    JC explores connected low-loss image-space regions, using path sampling, surrogate-guided optimization, and a harmful-output distribution objective.

    Source: §I · pp. 1–2

  • Results

    JC significantly outperforms prior methods in attack success rate and achieves lower response perplexity than baselines in white-box and black-box settings.

    Source: §V · p. 9

  • Takeaways & Limitations

    The reported connected-region structure exposes persistent, structured vulnerabilities in MLLM safety defenses.

    Source: §I · p. 2

    Perplexity reflects fluency under a fixed evaluation model and does not directly measure harmfulness.

    Source: §V · p. 9

Abstract

from arXiv · show

Multimodal large language models (MLLMs) are vulnerable to visual jailbreak attacks that bypass safety safeguards. Existing methods typically optimize isolated adversarial examples, limiting attack diversity, transferability, and generalization. We propose Jailbreak Connectivity (JC), a framework based on the observation that effective adversarial inputs can be connected through low-loss regions in image space. JC optimizes continuous adversarial paths: path sampling yields diverse jailbreak images, lightweight surrogate classifiers guide cross-model transfer, and a distribution-level objective enables universal attacks across harmful queries. On MM-SafetyBench, JC achieves average attack success rates of 84.01%, 85.77%, and 97.95% on Qwen3-VL-8B-Instruct, InternVL3.5-8B, and Gemma-3-12B-IT, respectively, exceeding the strongest baselines by up to 68.36 percentage points. In black-box evaluations, JC improves attack success rates over FigStep by 24.69 and 27.34 percentage points on GPT-5.6 and Gemini 3.1 Pro, respectively. These findings reveal structured vulnerabilities in MLLM safety alignment and inform more comprehensive security evaluation.

I. INTRODUCTION

Visual inputs expand MLLMs’ attack surface, while existing jailbreak methods have limits in diversity, transferability, and universality. Jailbreak Connectivity (JC) addresses these gaps by exploring connected low-loss regions in image space, from which diverse, transferable, and universal attacks can be developed.

I. INTRODUCTION

  • Existing jailbreak approaches commonly produce limited attack diversity, transfer poorly across models, and rarely target many harmful queries with one image.The paper groups prior methods into prompt-to-image injection, visual perturbation, and proxy-model transfer attacks.Source: §I · p. 1
  • JC replaces pointwise optimization with exploration of connected low-loss adversarial regions in image space.The framework is presented as a unified region-level approach to jailbreak generation.Source: §I · pp. 1–2
  • Path sampling generates diverse jailbreak images, surrogate classifiers guide cross-model transfer, and a harmful-output distribution supports universal attacks.The paper frames these capabilities as consequences of exploiting the same connected region, rather than independent objectives.Source: §I · pp. 1–2
  • The paper reports empirical evidence that successful adversarial examples can connect through low-loss image-space regions rather than existing only as isolated points.It presents this geometric structure as a finding about MLLM jailbreak vulnerability.Source: §I · p. 2
  • The authors report substantial gains over prior methods across open- and closed-source MLLMs and identify persistent structured vulnerabilities in current safety defenses.They motivate the work by noting that MLLMs’ visual modality expands the attack surface and that multimodal alignment remains fragile.Source: §I · pp. 1–2

II. RELATED WORK

Prior MLLM jailbreak research spans visual instruction injection, adversarial perturbations, and proxy-model transfer, while text-only jailbreak work explores transferability and diversity. JC frames the remaining multimodal gaps as consequences of treating jailbreaks as isolated points and instead explores connected low-loss regions.

II. RELATED WORK

  • Textual jailbreak research spans gradient-optimized suffixes [6], [7], black-box fuzzing, dialogue-state attacks [24], prompt rewriting [25], and collaborative iterative refinement, while offering limited insight into multimodal vulnerabilities.Source: §II · p. 2
  • MLLM jailbreaks use harmful instructions embedded in images, [27], visual perturbations [13],, [28],, or proxy-model transfer [17], [30],.The literature still reports limited diversity, degraded cross-architecture transfer, and few attacks that work across harmful queries,,.Source: §II · p. 2

III. JAILBREAK CONNECTIVITY

  • JC identifies the assumption that jailbreaks are isolated adversarial points as overlooking the geometric structure of the adversarial space.Its core insight is that MLLM jailbreak vulnerability forms connected low-loss image-space regions.Source: §III · pp. 2–3
  • JC discovers paths between adversarial endpoints, refines them with lightweight surrogates, and generalizes from one harmful query to a distribution of harmful outputs.These stages are presented as producing diversity, transferability, and universality through the same region.Source: §III · p. 3
  • The study focuses on single-turn prompts and permits either full parameter-and-gradient access or query-only black-box interactions.The manipulated jailbreak image adds a small, imperceptible perturbation to the original image.Source: §III · p. 3

A. Region Discovery: Diverse Jailbreak via Path Sampling

JC treats jailbreak generation as exploration of connected low-loss regions rather than isolated images. It optimizes a quadratic Bézier path between jailbreak endpoints so that sampling along the path can yield diverse, effective images.

A. Region Discovery: Diverse Jailbreak via Path Sampling

  • JC connects two independently obtained jailbreak endpoints with a quadratic Bézier curve whose control point determines the path's direction and curvature.The endpoints are found with PGD from random initialization over 1000 iterations, subject to an imperceptibility constraint.Source: A. Region Discovery: Diverse Jailbreak… · p. 3
  • Optimizing the control point to maintain low jailbreak loss along the path enables multiple stable samples instead of a single isolated optimum.JC samples positions uniformly along the curve and optimizes the sampled image loss by backpropagating gradients to the control point.Source: A. Region Discovery: Diverse Jailbreak… · p. 3
  • The paper frames input-space connectivity as an empirical structure that supports diversity, transferability, and universality in MLLM jailbreak attacks.This differs from prior mode-connectivity work on model parameters or same-task solutions.Source: A. Region Discovery: Diverse Jailbreak… · p. 3
  • 100% (21/21) of optimized-curve samples succeeded, compared with 57.14% (12/21) for linear interpolation; failures clustered in the intermediate region.On Qwen3-VL-8B-Instruct, linear interpolation's loss reached 5.26 near the midpoint, while the optimized curve's maximum loss was 10^-4.Source: A. Region Discovery: Diverse Jailbreak… · pp. 3–4, Figure 2 · p. 4

B. Region Refinement: Transferable Jailbreak via Surrogate Guidance

Directly optimizing transfer across many MLLMs requires repeated forward passes and becomes computationally expensive. JC instead uses lightweight behavior-level surrogates to guide path optimization toward images that both bypass refusals and induce harmful outputs.

B. Region Refinement: Transferable Jailbreak via Surrogate Guidance

  • Directly optimizing jailbreak loss across n MLLMs requires repeated forward passes through large models and becomes computationally expensive as n grows.JC addresses this by modeling aligned-model behavior without including all target MLLMs directly in optimization.Source: B. Region Refinement: Transferable… · p. 4
  • JC models jailbreak success as two sequential decisions—whether the MLLM responds and whether its response contains harmful content—and guides paths with lightweight surrogates.The Safety Classifier predicts whether an image triggers a response; the Jailbreak Success Predictor estimates whether that answer is harmful.Source: B. Region Refinement: Transferable… · p. 4
  • The Safety Classifier is trained on observed response-versus-refusal labels, while the Jailbreak Success Predictor uses attack-outcome labels for harmful-content generation.The paper reports that omitting either component yields a distinct failure: refusal despite potentially harmful imagery, or a response that is safe or irrelevant.Source: B. Region Refinement: Transferable… · p. 4
  • The transfer objective combines surrogate guidance for the remaining n −1 MLLMs with direct jailbreak loss on the base model.The transfer term approximates refusal and harmfulness tendencies; α and β balance base-model effectiveness against transferability and safer predictions.Source: B. Region Refinement: Transferable… · p. 5

C. Region Generalization: Universal Jailbreak via Distributional Objectives

JC extends its attack objective from a single harmful query to a distribution of harmful outputs, with text removed during optimization so the adversarial signal is encoded in the image. This produces a universal path that can target a broad range of harmful questions.

C. Region Generalization: Universal Jailbreak via Distributional Objectives

  • JC replaces a fixed harmful target with a distribution of harmful outputs to extend attacks beyond a query-specific jailbreak.The method removes textual input during optimization, placing the adversarial signal entirely in the visual modality.Source: C. Region Generalization: Universal… · p. 5
  • The universal objective approximates the harmful-output distribution with 100 AdvBench sentences and optimizes a path intended to answer a wide range of harmful questions.The passage says the resulting behavior may also transfer to other MLLMs.Source: C. Region Generalization: Universal… · p. 5
  • The universal transferable-path objective is presented as the general formulation: restricting it to one harmful query yields the transferable formulation, and setting α = 1 yields the diverse formulation.The path remains subject to the stated perturbation constraint for every u in [0, 1].Source: C. Region Generalization: Universal… · p. 5

A. Implementation

The implementation evaluates JC on open-source and commercial MLLMs using MM-SafetyBench and AdvBench, with attack success and response fluency metrics and setting-specific baselines.

  • JC is evaluated on Qwen3-VL-8B-Instruct, InternVL3.5-8B, Gemma-3-12B-IT, GPT-5.6, and Gemini 3.1 Pro.The surrogate classifiers use a CLIP-ViT-Base-Patch32 backbone.Source: A. Implementation · p. 5
  • MM-SafetyBench covers 13 prohibited scenarios, while AdvBench contains 520 harmful behaviors mapped to corresponding MM-SafetyBench scenarios after duplicate removal.The surrogate-classifier training set is disjoint from benchmark evaluation queries.Source: A. Implementation · pp. 5–6
  • Attack Success Rate measures the proportion of harmful queries that induce prohibited responses, with mean and standard deviation reported across five runs.GPT-6 serves as the binary harmfulness judge.Source: A. Implementation · p. 6
  • Response perplexity assesses linguistic fluency using Qwen3-8B token-level negative log-likelihood, with lower PPL indicating higher likelihood under that model.Mean and standard deviation are reported across five runs.Source: A. Implementation · p. 6
  • Baselines vary by setting: Adv Example and Query Image for white-box, FORCE for transfer, FigStep for black-box, and Plain Text for baseline vulnerability.All methods use the same optimization budget.Source: A. Implementation · p. 6

B. Experimental Results

JC achieves high white-box attack success across three MLLMs, while its effectiveness varies by model and scenario; its generated responses also have favorable perplexity results.

1) White-Box Diverse Attacks:

  • 84.01%, 85.77%, and 97.95% are JC’s average ASRs on Qwen3-VL-8B-Instruct, InternVL3.5-8B, and Gemma-3-12B-IT, improving over each model’s strongest baseline by 59.46, 57.94, and 68.36 percentage points.JC reaches 100% ASR on all three models in Economic Harm, Health Consultation, Political Lobbying, and Sex.Source: 1) White-Box Diverse Attacks · p. 6
  • JC’s ASR depends on model and scenario: in Malware Generation it reaches 88.91% on Qwen3-VL and 100% on Gemma-3, but 9.59% on InternVL3.5, where Query Image reaches 42.33%.Qwen3-VL ASR is 41.19% in Government Decision and 57.54% in Fraud.Source: 1) White-Box Diverse Attacks · p. 6
  • 5.07, 4.57, and 5.07 are JC’s average PPL values on Qwen3-VL, InternVL3.5, and Gemma-3, versus 7.16, 6.79, and 6.86 for Plain Text.JC has lower average PPL than both visual baselines on all three models and the lowest PPL in 11 of 13 scenarios per model; Government Decision is a notable exception.Source: 1) White-Box Diverse Attacks · p. 6

2) Transferable Jailbreak Performance:

The transfer evaluation generates adversarial images with Qwen3-VL-8B-Instruct and tests them on InternVL3.5-8B and Gemma-3-12B-IT. JC has stronger average transferability than FORCE, although results vary by scenario and target.

  • The evaluation compares FORCE and JC across 13 safety scenarios, using Qwen3-VL-8B-Instruct as the source model and InternVL3.5-8B and Gemma-3-12B-IT as transfer targets.FORCE is described as using layer-aware regularization and spectral feature rescaling to mitigate model-specific feature reliance.Source: 2) Transferable Jailbreak Performance · p. 6, Table III · p. 7
  • JC provides stronger transferability on average than FORCE, with gains varying by scenario and target model.FORCE performs better in Economic Harm on both transfer targets and Privacy Violence on Gemma-3-12B-IT.Source: 2) Transferable Jailbreak Performance · p. 6, Table III · p. 7
  • On InternVL3.5-8B, JC gains 17.49, 12.87, and 10.25 percentage points in ASR in Hate Speech, Health Consultation, and Privacy Violence, respectively.On Gemma-3-12B-IT, JC gains 14.12 percentage points in Hate Speech and 13.36 percentage points in Financial Advice.Source: 2) Transferable Jailbreak Performance · p. 6

3) Black-Box Diverse Attacks:

JC improves black-box attack success over FigStep on GPT-5.6 and Gemini 3.1 Pro, and its universal image succeeds on many unseen queries. The results also show scenario-dependent performance.

3) Black-Box Diverse Attacks:

  • 24.69 and 27.34 percentage points are JC’s average ASR gains over FigStep on GPT-5.6 and Gemini 3.1 Pro, respectively.JC outperforms FigStep across all 13 safety scenarios on both models; results average five independent runs.Source: 3) Black-Box Diverse Attacks · p. 7, Table IV · p. 8
  • JC reaches 60.68% ASR in Privacy Violence, 59.83% in Financial Advice, and 56.11% in Legal Opinion on GPT-5.6; on Gemini 3.1 Pro, it reaches 67.46% in Economic Harm.Gemini 3.1 Pro results also include 65.58% in Financial Advice and 62.81% in Political Lobbying; effectiveness varies across models and scenarios.Source: 3) Black-Box Diverse Attacks · p. 7
  • 11.41 and 13.11 are JC’s average PPL values on GPT-5.6 and Gemini 3.1 Pro, versus FigStep’s 14.52 and 16.41.JC has lower PPL than FigStep in all 13 scenarios on both models.Source: 3) Black-Box Diverse Attacks · pp. 7–8

4) Universal Attacks:

  • On 40 unseen harmful queries, JC averages 34 successful jailbreaks on Qwen3-VL-8B-Instruct versus 17 for Adv Example; the same universal image succeeds on 22 queries on InternVL3.5-8B.The results average five independent runs.Source: 4) Universal Attacks · p. 8

5) Ablation Study:

The ablation study finds that continuous paths contribute most to attack effectiveness, while surrogate classifiers support cross-model transfer and the full design yields the lowest response perplexity. The conclusion frames MLLM safety vulnerabilities as region-level and cautions that perplexity measures fluency, not harmfulness.

5) Ablation Study:

  • Removing continuous path construction reduces transfer ASR by 45.91, 42.33, and 43.52 percentage points on Qwen3-VL-8B-Instruct, InternVL3.5-8B, and Gemma-3-12B-IT, respectively.The study compares component removals under transferable jailbreak settings, using Qwen3-VL-8B-Instruct as the source and the other two models as transfer targets.Source: 5) Ablation Study · p. 8, Table V · p. 8
  • Removing JSP reduces transfer ASR by 15.16 and 24.52 points, while removing SC reduces it by 30.69 and 22.94 points on InternVL3.5-8B and Gemma-3-12B-IT, respectively.On Qwen3-VL-8B-Instruct, either removal slightly increases source ASR; the authors describe SC and JSP as complementary contributors to transfer.Source: 5) Ablation Study · p. 8
  • The full JC method records the lowest PPL across configurations: 5.84, 3.46, and 5.77 on Qwen3-VL-8B-Instruct, InternVL3.5-8B, and Gemma-3-12B-IT, respectively.The study reports ASR and PPL as mean ± standard deviation over five independent runs; lower PPL is preferable.Source: 5) Ablation Study · p. 8, Table V · p. 8
  • Overall, the path drives attack effectiveness, the two surrogate classifiers contribute to transferability, and all three components jointly yield the lowest response perplexity.The authors interpret the lower perplexity as more fluent responses.Source: 5) Ablation Study · p. 8

V. CONCLUSION

  • The conclusion argues that MLLM safety alignment may fail across structured adversarial regions rather than only at isolated inputs, while perplexity reflects fluency under a fixed model, not harmfulness.It suggests defenses should reason about structured adversarial regions rather than individual examples.Source: §V · p. 9

Related papers

Loading 2610.38091v1…