Source-linked AI summary
OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluation
Xiaojun Jia, Jie Liao, Qi Guo, Teng Ma, Simeng Qin, Ranjie Duan, Tianlin Li, Yihao Huang, Zhitao Zeng, Dongxian Wu, Yiming Li, Wenqi Ren, Xiaochun Cao, Yang Liu
TL;DR
Multimodal jailbreaks can exploit cross-modal inputs to bypass MLLM safety alignment, while existing benchmarks and evaluations leave gaps in coverage and standardization. OmniSafeBench-MM combines a risk taxonomy, attack and defense methods, and multidimensional scoring in a reproducible toolbox. Experiments across 18 MLLMs reveal persistent vulnerabilities and differing safety–helpfulness trade-offs among defenses.
Problem
Existing multimodal jailbreak benchmarks have limited risk and prompt-type coverage, while evaluation methods vary and often rely on attack success rate alone.
Source: §1 · pp. 1–2Method
OmniSafeBench-MM combines a dataset of 9 risk domains and 50 categories, 13 attacks, 15 defenses, and scoring for harmfulness, intent alignment, and response detail.
Source: §1 · p. 2Results
Takeaways & Limitations
The unified platform standardizes data, methodology, and evaluation as a foundation for multimodal model-safety research.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Recent advances in multi-modal large language models (MLLMs) have enabled unified perception-reasoning capabilities, yet these systems remain highly vulnerable to jailbreak attacks that bypass safety alignment and induce harmful behaviors. Existing benchmarks such as JailBreakV-28K, MM-SafetyBench, and HADES provide valuable insights into multi-modal vulnerabilities, but they typically focus on limited attack scenarios, lack standardized defense evaluation, and offer no unified, reproducible toolbox. To address these gaps, we introduce OmniSafeBench-MM, which is a comprehensive toolbox for multi-modal jailbreak attack-defense evaluation. OmniSafeBench-MM integrates 13 representative attack methods, 15 defense strategies, and a diverse dataset spanning 9 major risk domains and 50 fine-grained categories, structured across consultative, imperative, and declarative inquiry types to reflect realistic user intentions. Beyond data coverage, it establishes a three-dimensional evaluation protocol measuring (1) harmfulness, distinguished by a granular, multi-level scale ranging from low-impact individual harm to catastrophic societal threats, (2) intent alignment between responses and queries, and (3) response detail level, enabling nuanced safety-utility analysis. We conduct extensive experiments on 10 open-source and 8 closed-source MLLMs to reveal their vulnerability to multi-modal jailbreak. By unifying data, methodology, and evaluation into an open-source, reproducible platform, OmniSafeBench-MM provides a standardized foundation for future research. The code is released at https://github.com/jiaxiaojunQAQ/OmniSafeBench-MM.
1. Introduction
Multimodal interaction expands MLLMs’ capabilities but also creates routes for concealed malicious inputs to bypass safety alignment and elicit harmful content. OmniSafeBench-MM addresses gaps in risk coverage, prompt-type coverage, and standardized evaluation with a unified benchmark and toolbox.
1. Introduction
- Multimodal inputs can conceal malicious intent and bypass MLLM safety alignment, prompting harmful or policy-violating outputs.Unlike classical adversarial attacks that degrade task accuracy, jailbreaks break safety alignment and induce unsafe content.Source: §1 · p. 1
- Existing multimodal safety datasets cover risk categories incompletely and overlook inquiry types such as consultative, instructive, or deceptive prompts.Attack-generation studies also use differing evaluation and defense methods, while many rely on attack success rate alone.Source: §1 · pp. 1–2
- OmniSafeBench-MM introduces a dataset spanning 9 risk domains and 50 subcategories, with samples organized by consultative, imperative, and declarative inquiry type.It also uses an automated pipeline to generate risk-related text–image pairs.Source: §1 · p. 2
- The toolbox integrates about 13 attacks and 15 defenses in a modular platform, with scoring for harmfulness, query-response intent alignment, and response detail.Off-model defenses intercept inputs or filter outputs; on-model methods seek to improve internal alignment through training or fine-tuning.Source: §1 · p. 2
- Experiments on 18 MLLMs find differing vulnerabilities across modalities and architectures, with defenses trading reduced harmfulness against helpfulness or residual vulnerabilities.The evaluation includes 10 open-source models and 8 closed-source commercial systems.Source: §1 · p. 2
2. Related Work
Prior work explores multimodal jailbreaks through white-box optimization and black-box input strategies, while defenses broadly divide into off-model filtering and on-model alignment methods. The benchmark overview and comparison table situate these strands within a common evaluation context.
2.1. Multi-modal jailbreak attack
- White-box attacks use model gradients or architecture information, while black-box attacks exploit visual cues, distribution shifts, multimodal dispersion, or feedback-driven query refinement.Examples include jointly optimizing visual and textual inputs and embedding typographic cues in images.Source: §2.1 · pp. 2–3
- Figure 1 summarizes a benchmark joining 13 attacks, 15 defenses, and scoring for harmfulness, alignment, and detail; Table 1 compares representative multimodal safety benchmarks quantitatively.The supplied table caption does not report its comparison values.Source: Figure 1 · p. 3, Table 1 · p. 3
2.2. Multi-modal jailbreak defense
- Off-model defenses filter inputs or outputs and are deployable across MLLMs but may add time loss; on-model methods adjust generation or train for safer behavior.The cited approaches include safety-aware reward models, constitutional calibration, safety-preference fine-tuning, and reinforcement learning from human feedback.Source: §2.2 · p. 3
3. The Proposed OmniSafeBench-MM
OmniSafeBench-MM combines dataset generation, representative attacks and defenses, and fine-grained safety evaluation. Its risk assessment separates harm, intent alignment, and response detail, then applies rule-based criteria to judge jailbreak success.
3. The Proposed OmniSafeBench-MM
- The framework integrates dataset generation, implemented attacks, implemented defenses, and fine-grained safety evaluation.Source: §3 · p. 3
3.1. Dataset generation
- The dataset-generation pipeline defines nine major risk categories with 4–8 subcategories, creates consultative, imperative, and declarative prompts, extracts unsafe keywords, then generates matching images with PixArt-XL-2-1024-MS.Source: §3.1 · pp. 3–4
3.2. Implemented attack methods
- Attacks are grouped into white-box single- and cross-modal methods requiring internal access, and black-box structured visual-carrier, out-of-distribution, and hidden-risk methods.Source: §3.2 · p. 4
- Implemented mechanisms include optimized or noise-based visual perturbations, joint image-text optimization, semantic visual carriers, distribution or attention shifts, and splitting or encrypting risky intent across modalities.ImgJP and DeltaJP use ensemble strategies to improve transferability.Source: §3.2 · pp. 4–5
3.3. Implemented defense methods
- Defenses span off-model input preprocessing and output filtering, plus on-model inference intervention and intrinsic safety alignment, including hidden-state or logit steering and VLGuard fine-tuning.Source: §3.3 · pp. 5–6
3.4.1. Motivation: Beyond binary success rates
- The fine-grained framework aims to identify High-Impact, Low-Frequency events that aggregate safety metrics can obscure.Source: §3.4.1 · p. 6
3.4.2. The H-F-A evaluation dimensions
- The H-F-A protocol scores harm severity (H), prompt-response intent alignment (A), and response explicitness and self-containment (D) as separate dimensions.Source: §3.4.2 · p. 6
- A breach requires a response to be both harmful and compliant; detail level then maps breaches to four judgment levels, with scores 3–4 counted as jailbreak success.Source: §3.4.2 · p. 6
4. Benchmark Experiments
The experiments compare 13 jailbreak attacks and 15 defenses across MLLMs, finding attack-specific vulnerabilities and defense trade-offs. Results span white-box and black-box attacks and off-model and on-model defenses.
4.1. Experiment settings
- The evaluation includes 13 white-box and black-box attacks and 15 off-model and on-model defenses.Source: §4.1 · pp. 6–7
4.2. Experiment results
- JPS reached 62.93% ASR, while white-box attacks had Avg-D mostly below 2.6 under the strict S ≥3 success criterion.Low Avg-D was the primary reason for the low ASRs reported across attacks.Source: Table 2 · p. 7, §4.2 · p. 7
- MML and CS-DJ achieved the highest black-box ASRs; MML reached 50.7% on Gemini-2.5 and 52.2% on Qwen3-VL-Plus.CS-DJ reached 32.0% on Gemini-2.5 and 38.1% on Qwen3-VL.Source: §4.2 · p. 7, Table 3 · p. 8
- MLLM-Protector reduced off-model ASR to 14.87% on CS-DJ, 4.60% on FigStep, and 0.27% on MML, while input preprocessing was less reliable on MML.Source: §4.2 · p. 7, Table 4 · p. 8
- VLGuard cut HIMRD ASR from 20.40% to 1.20% but raised MML ASR from 0.93% to 1.73%, showing attack-specific effects.Source: Table 5 · p. 8, §4.2 · p. 8
5. Conclusion
OmniSafeBench-MM combines risk coverage, attack and defense implementations, and three-dimensional safety scoring in a unified benchmark. Experiments across 18 MLLMs reveal persistent vulnerabilities and differing defense trade-offs.
- OmniSafeBench-MM is a unified, reproducible benchmark combining a broad risk taxonomy, diverse attack and defense implementations, and three-dimensional safety scoring.Source: §5 · p. 8
- Experiments across 18 open- and closed-source MLLMs reveal persistent vulnerabilities, particularly under cross-modal dispersion and black-box settings.Source: §5 · p. 9
- The benchmark standardizes data, methodology, and evaluation in one platform as a foundation for multimodal model safety research.Source: §5 · p. 9
Supplementary Material
The supplementary material expands the paper's reported results with additional visualizations, tables, and statistical analyses. It covers model-, category-, style-, metric-, and defense-related behavior.
- The supplementary report covers 16 MLLMs, 8 black-box attack methods, 9 risk categories, 3 prompt styles, and 13 defense methods.Source: Supplementary Material · p. 12
- The appendix presents further visualizations, detailed tables, and statistical analyses to report model behavior under adversarial prompting and defense interventions.Source: Supplementary Material · p. 12
- Additional figures expand findings on model-level vulnerability patterns, category- and style-dependent effects, metric distributions, defense effectiveness, and regression-based interpretability.Source: Supplementary Material · p. 12
A.1.1. Model-Level Robustness Overview
The appendix overview uses a heatmap and corresponding numerical breakdown to summarize attack success across models and methods. The figures also report auxiliary metrics and characterize variation across architectures and attack vectors.
- Figure 4 maps ASR across 16 models and 8 attack methods in a heatmap.Source: §A.1.1 · p. 12
- Table 8 provides the corresponding numerical breakdown with Average-T, H, A, and D metrics.Source: §A.1.1 · p. 12
- The visualizations show divergence in safety alignment between open- and closed-source architectures and variation in attack-vector effectiveness.Source: §A.1.1 · p. 12
A.1.2. Attack-Specific Vulnerability Profiles
The section uses model-level ASR radar plots to distinguish attack-specific vulnerability profiles, including whether attacks target particular model families or generalize broadly.
A.1.2. Attack-Specific Vulnerability Profiles
- Radar plots compare model-level ASR across attacks, revealing whether each attack targets specific model families or generalizes broadly.Each plot summarizes one attack method.Source: §A.1.2 · p. 12
A.1.3. Distribution of Evaluation Metrics
The analyses report variation in evaluation metrics, risk-category vulnerability, attack styles, and defense effectiveness; the prose and Fig. 8 caption give conflicting style rankings.
A.2.1. Risk Category Vulnerability
- Attack success varies by risk category, indicating that model vulnerability is content-dependent.The analysis aggregates ASR across nine risk categories and eight attack methods.Source: §A.2.1 · p. 12, Figure 7 · p. 16
A.2.2. Style-Dependent Effects
- The prose reports higher ASR for instructive prompts, while the Fig. 8 caption reports consistently higher ASR for consultative prompts.The figure compares declarative, consultative, and instructive styles across attacks.Source: §A.2.2 · p. 12, Figure 8 · p. 16
A.3.1. Overall Defense Effectiveness
- Across three representative models and four attack methods, the 13 defenses provide highly nonuniform protection.The figure compares defense effectiveness using ASR, where lower values indicate stronger defense.Source: §A.3.1 · p. 12, Figure 9 · p. 17
A.3.2. Defense-Induced ASR Reduction
- The waterfall plot compares ASR before and after each defense to show their relative contributions to robustness improvement.The supplied passages do not report individual defense reductions.Source: §A.3.2 · p. 12, Figure 10 · p. 18
A.4. Statistical Analysis and Interpretation
- The vulnerability heatmap encodes ASR by color and groups open- and closed-source models, while regression coefficients indicate factors associated with higher or lower attack success probability.Darker red denotes higher ASR; positive coefficients indicate increased vulnerability and negative coefficients suggest robustness-related factors.Source: §A.4 · p. 12, Figure 4 · p. 13, Figure 11 · p. 19
- Table 8 compares open- and closed-source multimodal models on benchmarks using ASR and average T, H, A, and D scores.Source: Table 8 · p. 14
A.1.3. Distribution of Evaluation Metrics
- Evaluation-score boxplots summarize T, H, A, and D across models, with model-averaged scores showing distinct patterns across attack methods.The figure covers harmfulness, alignment, detail, and judge scores across eight attacks.Source: §A.1.3 · p. 12, Figure 6 · p. 15