PaperlayerView exact revision on arXiv

Source-linked AI summary

VLSBench: Unveiling Visual Leakage in Multimodal Safety

Xuhao Hu, Dongrui Liu, Hao Li, Xuanjing Huang, Jing Shao

arXiv:2411.19939v3cs.CRcs.AIcs.CLcs.CV

TL;DR

Existing multimodal safety benchmarks can leak risky image content through textual queries, making text-only refusals sufficient and cross-modality evaluation unreliable. The paper builds VLSBench, a 2.2k-pair benchmark designed to prevent this leakage, and finds textual alignment lags multimodal alignment on this benchmark, though textual SFT is effective on benchmarks with VSIL.

  • Problem

    Existing multimodal safety benchmarks may reveal risky image content in textual queries, allowing text-only refusals and making cross-modality safety evaluation unreliable.

    Source: §1 · p. 1

  • Method

    The authors use an automated pipeline to construct VLSBench with 2.2k image-text pairs that prevent visual leakage into textual queries.

    Source: §1 · p. 2

  • Results

    On VLSBench, textual alignment lags multimodal alignment, whereas textual SFT is sufficiently effective on benchmarks affected by VSIL.

    Source: §1 · p. 2

  • Takeaways & Limitations

    VLSBench provides a setting for reliable cross-modality safety evaluation and motivates dedicated multimodal alignment for safety tasks without VSIL.

    Source: §1 · p. 2, §5 · p. 9

    The evaluation omits some MLLMs, especially costly closed-source APIs, and VLSBench construction relies mostly on expensive GPT models.

    Source: §6 · p. 9

Abstract

from arXiv · show

Safety concerns of Multimodal large language models (MLLMs) have gradually become an important problem in various applications. Surprisingly, previous works indicate a counterintuitive phenomenon that using textual unlearning to align MLLMs achieves comparable safety performances with MLLMs aligned with image text pairs. To explain such a phenomenon, we discover a Visual Safety Information Leakage (VSIL) problem in existing multimodal safety benchmarks, i.e., the potentially risky content in the image has been revealed in the textual query. Thus, MLLMs can easily refuse these sensitive image-text pairs according to textual queries only, leading to unreliable cross-modality safety evaluation of MLLMs. We also conduct a further comparison experiment between textual alignment and multimodal alignment to highlight this drawback. To this end, we construct multimodal Visual Leakless Safety Bench (VLSBench) with 2.2k image-text pairs through an automated data pipeline. Experimental results indicate that VLSBench poses a significant challenge to both open-source and close-source MLLMs, e.g., LLaVA, Qwen2-VL and GPT-4o. Besides, we empirically compare textual and multimodal alignment methods on VLSBench and find that textual alignment is effective enough for multimodal safety scenarios with VSIL, while multimodal alignment is preferable for safety scenarios without VSIL. Code and data are released under https://github.com/AI45Lab/VLSBench

1 Introduction

Existing multimodal safety benchmarks can reveal risky image content in textual queries, allowing text-only refusals and making cross-modality evaluation unreliable. VLSBench removes this leakage, and results show textual alignment works on leaky benchmarks but lags multimodal alignment on VLSBench.

1 Introduction

  • The study addresses the counterintuitive finding that textual unlearning can match multimodal alignment despite the distinct visual information available to MLLMs.Prior work reported textual unlearning outperforming multimodal SFT with reduced data-collection and computational costs.Source: §1 · p. 1
  • VSIL occurs when a textual query describes risky image content, letting MLLMs reject unsafe pairs from text alone and weakening cross-modality safety evaluation.The paper reports textual SFT safety rates exceeding 95% on widely used multimodal safety benchmarks affected by VSIL.Source: §1 · pp. 1–2
  • VLSBench contains 2.2k image-text pairs built through an automated pipeline that pairs images with harmless, VSIL-free queries and filters mismatched or safe pairs.The pipeline generates descriptions and harmful queries, detoxifies the queries, generates images, then uses an MLLM and human verification for filtering.Source: §1 · p. 2
  • Current open- and closed-source MLLMs remain challenged by VLSBench across model sizes from 2B to 78B, and tested alignment methods remain insufficient for optimal safety.The reported models include LLaVA, Qwen2-VL, Llama3.2-Vision, InternVL2.5, and GPT-4o.Source: §1 · p. 2
  • On VLSBench, textual alignment lags multimodal alignment, while simple textual SFT is sufficiently effective on benchmarks with VSIL.The contrast supports VLSBench as a more reliable cross-modality safety evaluation.Source: §1 · p. 2, Figure 1 · p. 2

2 Visual Leakage in Multimodal Safety

The section defines visual safety information leakage (VSIL) as safety-related image content revealed in the textual query, then examines its prevalence and its effect on multimodal safety evaluation. It compares textual and multimodal alignment, reporting different outcomes on benchmarks with and without VSIL.

2.1 Visual Safety Information Leakage (VSIL)

  • VSIL is safety-related image content revealed in the textual query; examples include image information repeated in text across benchmark samples.The examples include prison-location information, a fake identification, typography, and a firearm scenario.Source: §2.1 · p. 3, Figure 2 · p. 3
  • Across four evaluated datasets, R1−R2 is less than 1%, indicating that nearly all harmful image-text pairs are also harmful when judged from the textual query alone.R1 is the harmful rate of image-text pairs; R2 counts pairs harmful both as image-text and as text alone.Source: §2.1 · p. 3, Table 1 · p. 3

2.2 The Drawback of VSIL Problem

  • The comparison experiment finds that simple textual SFT can solve these multimodal safety tasks effectively, consistent with rejection based on textual queries rather than integrated cross-modal understanding.Source: §2.2 · p. 3

2.2.1 Experimental Setup

  • The experiment compares textual and multimodal alignment across LLaVA-v1.5-7B/13B and Qwen2-VL-7B, using safety and general-ability evaluations.Methods include multimodal SFT, DPO, and PPO; textual unlearning, SafeUnlearning, SFT, and DPO; safety is judged with LlamaGuard3-8B.Source: §2.2.1 · pp. 3–4

2.2.2 Experimental Results and Analysis

  • On VSIL-affected datasets, textual SFT has an average gap of only 1% from multimodal alignment, scoring 96.40% on SafeBench and 99.64% on VLSafe.These results are less than 1% different from multimodal DPO, the best multimodal safety baseline.Source: §2.2.2 · p. 4, Table 2 · p. 4
  • Textual SFT and DPO generally match multimodal alignment on general ability, while Textual Unlearning lags and produces repetitive outputs on MME and MMVet.Textual SFT scores 33.68 on MMStar versus 34.46 for multimodal SFT, and outperforms multimodal SFT, DPO, and PPO on MMVet.Source: §2.2.2 · p. 4
  • All alignment methods exceed 90% safety on the evaluated existing datasets, which the authors interpret as evidence that VSIL-affected benchmarks lack sufficient challenge.The authors also report that some methods approach 100% safety, including multimodal SFT on Qwen2-VL-7B.Source: §2.2.2 · p. 5

3 VLSBench

VLSBench is designed to provide faithful cross-modality safety evaluation by preventing visual safety information from leaking into textual queries. It contains 2.2k image-text pairs organized into a hierarchical safety taxonomy and is built through a multi-stage data pipeline.

3 VLSBench

  • VLSBench prevents textual alignment from solving benchmark examples through leaked harmful queries, providing a more faithful cross-modality safety evaluation.The benchmark is constructed to prevent visual leakage from images into textual queries.Source: §3 · p. 5

3.1 VLSBench Overview

  • VLSBench pairs safe and unsafe images with naturally harmless queries whose potential risks may become apparent only when considered with the image.For example, “fast results with this equipment” does not explicitly state harmful keywords or intent on its own.Source: §3.1 · p. 5
  • 2.2k image-text pairs span 6 safety categories and 19 sub-categories in VLSBench’s hierarchical two-level taxonomy.Figure 4 presents detailed statistics across the categories and sub-categories.Source: §3.1 · p. 5, Figure 4 · p. 5

3.2 VLSBench Data Collection

  • The construction pipeline generates harmful queries and image descriptions, mitigates leakage in queries, iteratively generates images, and filters final pairs.The final filtration is intended to retain matched, harmful image-text pairs.Source: §3.2 · p. 5, Figure 5 · p. 6
  • Two parallel routes create source material: GPT-4o generates descriptions and harmful queries from harmful elements, while Qwen2-VL-72B generates image analyses and queries from existing images.The existing image datasets mainly contain real-world images.Source: §3.2 · p. 6
  • GPT-4o detoxifies harmful queries into less harmful, less conspicuous queries and filters revisions that retain leakage or change the original meaning.This stage aims to mitigate safety information leakage from image modality to text.Source: §3.2 · p. 6
  • Image descriptions are paraphrased for text-to-image generation, and Qwen2-VL-72B evaluates generated images for alignment with those descriptions.When an image does not meet the criteria, the prompt is revised and image generation is repeated.Source: §3.2 · p. 6
  • GPT-4o filters mismatched or safe pairs, followed by manual review, before the final VLSBench dataset is completed.The final images combine generated images and images collected from existing datasets.Source: §3.2 · p. 6

4 Benchmark Experiments

VLSBench challenges current MLLMs, and its results distinguish safety alignment on visual-leakage benchmarks from alignment when visual leakage is absent. Further analyses examine prompt changes, image captions, and whether benchmark images alone appear harmful to NSFW detectors.

4.1 Experimental Setup

  • The experiments benchmarked open-source and closed-source MLLMs and compared textual and multimodal alignment baselines, using GPT-4o to judge responses as safe with refusal, safe with warning, or unsafe.The safety rate sums refusal and warning responses.Source: §4.1 · pp. 6–7, Figure 6 · p. 7

4.2 Main Results

  • 78.39% was the best safety rate on VLSBench, from Qwen2-VL-7B with multimodal SFT; open- and closed-source models otherwise showed limited safety.Gemini-1.5-pro reached 49.78%, GPT-4o 21.43%, Llama-3.2-Vision 26.29%, and LLaVA-v1.5 5%.Source: §4.2 · p. 7, Table 3 · p. 7, Table 4 · p. 8
  • Higher safety rates were associated with more direct refusals, while models struggled to provide safety warnings with explanations or alternatives.The authors report that improvements in safety performance always came with increased direct refusals.Source: §4.2 · pp. 7–8
  • Textual alignment lagged multimodal alignment on VLSBench without visual leakage, while textual alignment was sufficient on benchmarks with visual leakage.Textual SFT achieved only 13.99% for LLaVA-v1.5-7B, and the authors report a nearly 10% difference between textual and multimodal SFT.Source: §4.2 · p. 8, Table 4 · p. 8

4.3 More Analysis

  • A safety prompt improved tested models’ safety rates and generally outperformed the MMSafetyBench prompt, with a 3.0% difference for Qwen2-VL-7B.LLaVA-v1.5-7B rose from 6.6% to 44.5%, and LLaVA-Mistral-7B from 11.19% to 56.8%; the proposed prompt also improved safety warning rates relative to MMSafetyBench.Source: §4.3 · p. 8, Figure 7 · p. 8
  • Using image captions instead of images could better improve overall MLLM safety; caption inputs increased VLM refusal rates, while LLM responses showed no consistent refusal or warning trend.The comparison used a 200-sample subset of VLSBench.Source: §4.3 · p. 8, Table 5 · p. 9
  • VLSBench images alone were not reliably harmful to NSFW detectors, supporting the authors’ emphasis on cross-modality information for safety evaluation.The analysis used two widely used NSFW image detectors.Source: §4.3 · p. 9, Table 6 · p. 9

5 Conclusions

The paper identifies visual safety information leakage in current multimodal safety benchmarks and constructs VLSBench to prevent image information from leaking into textual queries. Results show that current MLLMs and safety-aligned models struggle on this benchmark, motivating dedicated multimodal alignment methods for settings without visual leakage.

5 Conclusions

  • The paper identifies visual safety information leakage as a widespread problem in current multimodal safety benchmarks, making safety evaluation textual-biased and unreliable.VLSBench contains 2.2k image-text pairs and is designed to prevent image information from leaking into textual queries.Source: §5 · p. 9
  • Experiments on VLSBench show that current MLLMs and safety-aligned models struggle to achieve high safety performance, motivating dedicated multimodal alignment methods for settings without visual leakage.The benchmark is presented as a reliable safety evaluation for MLLMs.Source: §5 · p. 9

6 Limitations

The evaluation omits some MLLMs, particularly costly closed-source APIs, and VLSBench construction relies mostly on GPT models. The authors also leave effective solutions for VLSBench as future work.

6 Limitations

  • VLSBench evaluation excludes some MLLMs, especially expensive closed-source APIs.The authors say not all MLLMs are included in their evaluation.Source: §6 · p. 9
  • VLSBench construction mostly relies on costly GPT models, and replacing all closed-source APIs with open-source models requires further effort.Qwen2-VL-72B is also used as part of the construction tools.Source: §6 · p. 9
  • The authors leave effective solutions for the challenging VLSBench dataset to future exploration.Source: §6 · p. 9

7 Broader Impact and Ethics Statement

The paper describes safeguards for research-only data use and summarizes prior approaches to multimodal safety alignment and benchmarking. Related work spans training-based and training-free alignment, alongside benchmarks covering varied safety settings.

7 Broader Impact and Ethics Statement

  • The dataset is restricted to research use, with image screening, license compliance, privacy protections, and access limited to authorized researchers following ethical guidelines.The authors report that their IRB determined the work did not constitute human subjects research.Source: §7 · p. 9

A Related Work

C VSIL in Existing Datasets

Examples from existing benchmarks show risky image content repeated in textual queries, illustrating visual safety information leakage. The appendix describes separate text-only and image-text judging prompts, with GPT-4o evaluation results reported in Table 7.

C VSIL in Existing Datasets

  • GPT-4o is used to cross-check main results with separate text-harmfulness and image-text-pair-harmfulness evaluation prompts.The image-text prompt asks whether the pair contains unsafe content or intent.Source: C VSIL in Existing Datasets · p. 13
  • The GPT-4o evaluation results are reported in Table 7, whose R1 and R2 columns distinguish pair harmfulness from harmfulness of both the pair and text alone.Source: C VSIL in Existing Datasets · p. 13
  • Selected examples from JailbreakV, SafeBench, Ch3ef, and Harmbench illustrate visual safety information leakage across existing datasets.The examples are presented in Figure 8.Source: C VSIL in Existing Datasets · p. 13
  • The examples include risky image content repeated in text, such as a throat-cutting scene, methamphetamine, malware, wounds, and a gun.Other examples include typography revealing harmful activity and a CAPTCHA image whose content is repeated in the query.Source: C VSIL in Existing Datasets · pp. 13–14

D Dataset Comparison

VLSBench is presented as a 2,241-sample benchmark designed to reduce visual leakage, with safe texts paired to safe and unsafe images. The paper also reports that multimodal alignment performs better than textual alignment on MMSafetyBench, whose samples often use image-descriptive patterns absent from textual training data.

D Dataset Comparison

  • VLSBench contains 2,241 samples, evaluates 20 MLLMs and nine safety-aligned models, and uses an automated pipeline designed to scale.It pairs safe text with safe and unsafe images and includes real-world safe images alongside stable-diffusion-generated harmful images.Source: D Dataset Comparison · p. 14, Table 8 · p. 16

E.1 Detailed Alignment Methods Setup

  • Multimodal SFT uses VLGuard safety and normal samples, while multimodal RLHF uses SPA-VL preference data ranked by GPT4.The appendix specifies 2k safety and 1k normal samples for VLGuard and 30K SPA-VL data for Qwen2-VL-7B.Source: §E.1 · p. 14
  • Textual SFT uses 2,000 harmful and 1,000 benign SafeRLHF samples, while textual DPO masks image input and textual unlearning follows Chakraborty et al. (2024).Textual safe unlearning combines normal-response, safe-response, and unlearning losses, with α = γ = 0.3.Source: §E.1 · pp. 14, 16

E.2 Detailed Evaluation Setting

  • The evaluation uses JailbreakV's mini-set, SafeBench, VLSafe, and MMSafetyBench, with LlamaGuard3 judging JailbreakV, SafeBench, and VLSafe.JailbreakV's mini-set has about 280 pairs; SafeBench has 500 queries and VLSafe has 1k samples.Source: §E.2 · p. 17

E.3 Evaluaiton on MMSafetyBench

  • Multimodal SFT, DPO, and PPO outperform textual SafeRLHF alignment on MMSafetyBench, where samples mostly follow an “the image shows” pattern.The authors attribute this pattern's absence from SafeRLHF textual training data as a reason for the difference.Source: §E.3 · p. 17

E.4 DPO Comparison Experiments

Textual DPO largely matches multimodal DPO on two safety benchmarks, with some tradeoffs elsewhere. The accompanying sections describe VLSBench’s size, iterative image generation, human checks, and category labeling.

E.4 DPO Comparison Experiments

  • Textual DPO performs similarly to multimodal DPO on VLSafe and SafeBench, but has a slightly lower rate on JailbreakV while retaining above 90% average safety.General ability is comparable, though textual DPO has a slight performance drop; the comparison also supports the concern that models can refuse based on text alone.Source: §E.4 · p. 17, Table 10 · p. 18

F.1 Statistics

  • VLSBench contains 2,241 image-text pairs across 6 harmful categories and 19 sub-categories, with 2,241 unique queries and 1,957 unique images.Its textual queries are described as neutral or completely harmless in the word-cloud analysis.Source: §F.1 · p. 17

F.2 Details on Iterative Image Generation

  • Image generation is iterative: Qwen2-VL-72B verifies prompt alignment and safety information, refines prompts when needed, and stops after at most three rounds.Source: §F.2 · pp. 17–18

F.3 Human Verification

  • Human verification checks that queries are harmless without image-derived safety information and neither contradict nor lack relevance to their images.The authors and three additional data labelers participate in verification.Source: §F.3 · p. 18

F.4 Category Labeling

  • GPT-4o assigns categories using the proposed safety taxonomy and its detailed category descriptions.Source: §F.4 · p. 18

G VLSBench Evaluation Analysis

Open- and closed-source MLLMs struggle on VLSBench, while Claude-3.5-sonnet stands out and safety does not consistently track model size. The section also examines judge suitability and human verification of evaluation methods.

Evaluation Method Analysis

  • LlamaGuard is commonly used for efficiency and reproducibility, and prior benchmarks often expose harmful information in both the query and image.That query-image overlap makes text-only judging a natural fit for those prior samples, unlike VLSBench’s neutral or harmless queries.Source: G VLSBench Evaluation Analysis · pp. 18–20
  • LlamaGuard3 and LlamaGuard3-Vision do not reliably identify risky inputs when queries are neutral or harmless, motivating a judge with stronger vision ability.GPT-4o is used as the judge, with labels for safe with refuse, safe with warning, and unsafe.Source: G VLSBench Evaluation Analysis · p. 20, Figure 12 · p. 20

Human Verifications

Complete Evaluation Results

  • Open- and closed-source MLLMs struggle on VLSBench, but Claude-3.5-sonnet stands out; safety performance does not consistently track model size in the InternVL2.5 series from 2B to 78B.The authors suggest Claude-3.5-sonnet’s performance may relate to its safety pre-processing mechanism.Source: G VLSBench Evaluation Analysis · p. 20

H More Experiments on VLSBench

The additional experiments examine a step-by-step safety prompt, caption-based inputs, and image-only NSFW signals. The appendices also show examples and prompts used to construct and filter VLSBench data.

H.1 Safety Prompt

  • The step-by-step prompt raises safety for LLaVA-v1.5-7B from 6.6% to 44.5% and LLaVA-Mistral-7B from 11.19% to 56.8%, outperforming the MM-SafetyBench prompt on those tested models.The experiment uses a proportional 500-sample subset and includes LLaVA-v1.5-7B, LLaVA-v1.6-mistral-7B, and Qwen2-VL-7B; on Qwen2-VL-7B, the prompt slightly trails MM-SafetyBench but boosts warning rate more.Source: §H.1 · p. 21

H.2 Textual Caption Input

  • For VLMs, adding image captions to questions increases refusal rates; for LLMs, caption input has no consistent refusal or warning trend but improves overall safety.The caption experiment samples 200 VLSBench examples and tests two VLMs and two LLMs.Source: §H.2 · p. 21

H.3 NSFW Image Filters

  • NSFW detectors indicate that VLSBench images alone are not necessarily harmful; the authors identify cross-modal image-text information as the key safety signal.Source: §H.3 · p. 21

I More Examples of VLSBench

  • The appendix presents three VLSBench samples with responses from MLLMs, closed-source APIs, and safety-aligned baselines covering the benchmark’s three safety labels.Source: I More Examples of VLSBench · p. 21

J Data Construction Prompt Templates

Related papers

Loading 2411.19939v3…