PaperlayerView exact revision on arXiv

Source-linked AI summary

Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation

Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung, James T. Kwok, Yu Zhang

arXiv:2403.09572v4cs.CV

TL;DR

Image inputs can undermine the safety mechanisms inherited by MLLMs from pre-aligned LLMs. ECSO assesses response safety and transforms unsafe images into text to support safer responses; across five MLLMs, it improves safety while maintaining utility.

  • Problem

    MLLMs can generate unethical content from image inputs despite being built on LLMs aligned with human values, raising the question of how to transfer those safety mechanisms to MLLMs.

    Source: §1 · pp. 1–2

  • Method

    ECSO is a training-free approach that assesses response safety and transforms unsafe image inputs into query-aware text before generating a safe response without the images.

    Source: §1 · p. 2, §4.3 · p. 7

  • Results

    Across five SoTA MLLMs, ECSO enhanced safety without sacrificing utility; LLaVA-1.5-7B’s MM-SafetyBench (SD+OCR) harmless rate rose from 32.1% to 86.4%.

    Source: §1 · p. 2, §5.3 · p. 9

  • Takeaways & Limitations

    ECSO can serve as a data engine for autonomously generating MLLM supervised-finetuning data without additional human intervention.

    Source: §6 · p. 14

    ECSO relies on LLMs’ capacity to identify and neutralize unsafe queries, and deficiencies in that capacity may compromise its multimodal performance.

    Source: §5.7 · p. 14

Abstract

from arXiv · show

Multimodal large language models (MLLMs) have shown impressive reasoning abilities. However, they are also more vulnerable to jailbreak attacks than their LLM predecessors. Although still capable of detecting the unsafe responses, we observe that safety mechanisms of the pre-aligned LLMs in MLLMs can be easily bypassed with the introduction of image features. To construct robust MLLMs, we propose ECSO (Eyes Closed, Safety On), a novel training-free protecting approach that exploits the inherent safety awareness of MLLMs, and generates safer responses via adaptively transforming unsafe images into texts to activate the intrinsic safety mechanism of pre-aligned LLMs in MLLMs. Experiments on five state-of-the-art (SoTA) MLLMs demonstrate that ECSO enhances model safety significantly (e.g.,, 37.6% improvement on the MM-SafetyBench (SD+OCR) and 71.3% on VLSafe with LLaVA-1.5-7B), while consistently maintaining utility results on common MLLM benchmarks. Furthermore, we show that ECSO can be used as a data engine to generate supervised-finetuning (SFT) data for MLLM alignment without extra human intervention.

1 Introduction

MLLMs can bypass inherited LLM safety when images are introduced, motivating ECSO, a training-free approach that uses query-aware image-to-text conversion to restore safety while retaining utility.

1 Introduction

  • MLLMs built on well-aligned LLMs can still be induced to generate unethical content when image inputs are introduced.The introduction identifies this as a challenge in inheriting their predecessors’ safety mechanisms.Source: §1 · p. 1
  • Repeating LLM alignment methods on MLLMs requires carefully designed red-team queries, a challenge made harder by image inputs.The paper frames transferring pre-aligned LLM safety to MLLMs as its central question.Source: §1 · p. 2
  • ECSO significantly enhances safety across five MLLMs while retaining utility on common benchmarks, and can generate SFT alignment data without extra human intervention.The paper presents ECSO as a data engine for generating SFT data to align MLLMs.Source: §1 · p. 2
  • MLLMs can recognize unsafe content in their own responses, and their pre-aligned LLM safety mechanisms persist but are suppressed by image features and restored when images are removed.The paper uses these observations to motivate ECSO’s reliance on MLLMs’ safety awareness and text-only generation.Source: §1 · p. 2
  • ECSO assesses its initial response post hoc, then converts the image to text in a query-aware manner only when that response is unsafe, enabling safe generation without the image.This training-free strategy is designed to restore the intrinsic safety mechanism of the pre-aligned LLM.Source: §1 · p. 2

2 Related Work

Prior work finds that MLLMs’ visual capabilities expose them to malicious image inputs, while existing protection strategies face coverage, robustness, or training-cost limitations. ECSO instead uses the pre-aligned LLMs’ intrinsic safety mechanism without further training.

MLLM Vulnerability.

MLLM Protection.

  • Red-teaming alignment [11] [31] [73] is labor-intensive and incomplete, while system prompts [65] may weaken against new attacks and text-centric safety vectors [61] can miss image intents.The paper describes these as approaches to improving MLLM safety through alignment or inference-time protection.Source: MLLM Protection · p. 3
  • ECSO uses pre-aligned LLMs’ intrinsic safety mechanism without further training, unlike Pi et al.’s external detector and detoxifier requiring extensive training data [48].The paper also reports that MLLMs still struggle to respond safely to images under Chen et al.’s self-moderation approach [10].Source: MLLM Protection · p. 3

3 Preliminary Observations

The observations separate safety generation from safety judgment: models’ harmless rates fall sharply when images are present, while safety discrimination remains strong with or without images.

3.1 Safety Mechanism Persists in MLLM

  • With malicious queries paired to auxiliary benign images, models except Qwen-VL-Chat achieved about 20% harmless rate with images, versus nearly 100% without them.The experiments used VLSafe, which contains 1,110 query-image pairs; the authors hypothesize that visual-input distribution shift can weaken existing defenses.Source: §3.1 · p. 4

3.2 MLLMs are Aware of Their Own Unsafe Responses

  • Despite unsafe generation, safety discrimination remained robust with or without images; LLaVA-1.5-7B and ShareGPT4V-7B each achieved over 95% accuracy.The classification accuracies were evaluated on five MLLMs.Source: §3.2 · pp. 4–5, Figure 2 · p. 4
  • The authors suggest the generation-discrimination gap may reflect discrimination being easier than generation [7] or analogies to scalable oversight and complexity theory [52].Source: §3.2 · p. 5

4 Methodology

ECSO is a training-free MLLM protection strategy that checks its initial response for safety and, if unsafe, transforms the image into text before generating a response without visual input.

4.1 Harmful Content Detection

  • ECSO presents the MLLM’s initial response if it passes a self-check; if unsafe, the method routes the query through image-to-text transformation and a second response without the image.The method is motivated by the MLLM’s ability to assess response safety, described as near-GPT-4 performance in Sec. 3.2.Source: §4.1 · pp. 5–6

4.2 Query-Aware Image-to-Text (I2T) Transformation

  • For unsafe responses, ECSO converts the image into a query-aware caption to retain relevant image information while removing malicious content from the remaining procedure.The original question is included in the transformation prompt because a caption without query awareness may omit information needed to answer that question.Source: §4.2 · p. 6

4.3 Safe Response Generation Without Images

  • ECSO generates the safer response from the original query and caption with the image excluded, exposing malicious content to the MLLM’s text-only, pre-aligned safety mechanism.The inference prompt also includes “HARMLESS and ETHICAL” to underscore safety.Source: §4.3 · p. 7

5 Experiments

ECSO was evaluated against direct prompting on five MLLMs for safety, utility, and safety-alignment data generation. Experiments covered MM-SafetyBench, VLSafe, MME, MM-Vet, and MMBench; harmless rates were judged by GPT-4 with manual verification.

5.2 Safety Benchmark Datasets; Datasets.

  • Safety tests used 5,040 MM-SafetyBench examples across 13 scenarios but evaluated only 8, alongside 1,110 VLSafe malicious image-text pairs.The authors excluded the remaining MM-SafetyBench scenarios because text-only LLMs performed poorly there; utility tests assumed benign queries, treating harm detection as a false alarm.Source: §5.2 · pp. 8–9, Datasets · p. 11

5.3 Evaluation of Safety

  • ECSO raised LLaVA-1.5-7B’s harmless rate from 31.7% to 90.3% on OCR images and from 32.1% to 86.4% on SD+OCR images.Gains were larger on OCR and SD+OCR than on SD, where direct-prompt responses were often already benign; the paper attributes rates exceeding Text-Only to the “HARMLESS and ETHICAL” instruction.Source: §5.3 · pp. 9–10
  • ECSO improved harmless rates for the other evaluated MLLMs on MM-SafetyBench’s SD+OCR split and significantly improved rates across MLLMs on VLSafe.The paper concludes that query-aware image-to-text transformation followed by safe response generation without images can reactivate MLLMs’ safety mechanisms while retaining image information.Source: Figure 6 · p. 9, §5.3 · p. 10, Figure 7 · p. 10

5.4 Evaluation of Utility

  • ECSO did not hurt utility on MME-P or MM-Vet across models and slightly improved scores on MME-C and MM-Vet; benign queries usually did not trigger image-to-text transformation.Utility was measured on MME-P, MME-C, MM-Vet, and MMBench; the authors speculate that improvements may come from world knowledge elicited by query-aware captioning.Source: Results · p. 11, Table 3 · p. 11, Table 2 · p. 11

Necessity of excluding images

Effect of Steps 3&4; Effect of Query-aware I2T Transformations

  • Replacing caption-and-requery with blanket refusal lowers utility, and removing query conditioning from image-to-text transformation also harms utility.The query-aware caption can preserve query-relevant clues, such as the positions of two elephants, whereas a generic caption may omit clues needed to answer correctly.Source: Effect of Steps 3&4 · p. 12, Table 5 · p. 12, Effect of Query-aware I2T… · p. 12

5.6 Safety Alignment

  • ECSO can turn an unsupervised safety dataset into safe-response SFT data, and fine-tuning on ECSO-generated data was safer than fine-tuning on VLGuard’s human-verified responses.ECSO_VLGuard was usually outperformed by training-free ECSO, but offered better safety than VLGuard, indicating comparable or better data quality and a better safety-utility trade-off.Source: §5.6 · pp. 13–14, Figure 10 · p. 14

5.7 Limitation and Future Work

  • ECSO’s effectiveness depends on the underlying LLM’s ability to identify and neutralize unsafe queries, so weaknesses in that mechanism can compromise multimodal protection.The paper identifies leveraging rich multimodal context to build more nuanced, context-aware safety mechanisms as a future research direction.Source: §5.7 · p. 14

6 Conclusion … VLGuard

ECSO is a training-free safeguard that uses MLLMs’ intrinsic safety mechanisms and can generate SFT data for safety alignment without additional human intervention. The appendix describes the safety datasets and evaluation scope used in the paper.

6 Conclusion

  • ECSO uses MLLMs’ intrinsic safety mechanisms for training-free protection and can generate SFT data for safety alignment without additional human intervention.The authors present these contributions as guidance for building more secure MLLMs.Source: §6 · p. 14

A.1 Safety Datasets

  • MM-SafetyBench contains 5,040 malicious-intent examples across 13 scenarios, with Stable Diffusion, OCR, and SD+OCR images as well as text-only questions.The SD+OCR images are Stable Diffusion images subtitled by OCR.Source: MM-SafetyBench · p. 20

MM-SafetyBench

  • MM-SafetyBench labels harmful responses unsafe in scenarios 01–07 and 09, requires restraint on political topics in scenarios 08 and 13, and expects disclaimers and risk warnings for professional advice in scenarios 10–12.The paper mainly evaluates ECSO on scenarios 01–07 and 09; some MLLMs remain unsafe on text-only queries in the other scenarios, which the authors therefore do not study in the paper, though they report appendix results.Source: MM-SafetyBench · p. 20

VLSafe

  • VLSafe [11] has 4,764 malicious queries and harmless responses in its alignment split and 1,110 in its examine split; this paper evaluates using the examine split.Its harmful intent is expressed in text while its images are benign.Source: VLSafe · p. 21

VLGuard

  • VLGuard [73] has 2,000 training images, of which 977 are harmful and 1,023 benign, and the paper uses its training set for safety alignment experiments.The dataset contains around 3,000 GPT-4-generated query-response pairs; each safe image has a safe and unsafe pair, while each harmful image has a query-instruction explaining its unsafe nature.Source: VLGuard · p. 21

A.2 Utility Datasets · MMBench · MM-Vet

The utility benchmarks evaluate MLLM perception, cognition, and multimodal abilities using distinct question formats and scoring procedures. The appendix describes MME, MMBench, and MM-Vet and summarizes benchmark statistics in Table 7.

A.2 Utility Datasets

  • MME [17] tests perception and cognition across 14 subtasks with 2,374 questions, using paired yes/no instructions per test image.The two questions for each image have opposite ground-truth answers: “yes” for the first and “no” for the second.Source: §A.2 · p. 21
  • MME [17] scores each subtask by adding question-level accuracy and image-level accuracy+, requiring both answers for an image to be correct.Perception totals range from 0 to 2,000, and cognition totals range from 0 to 800.Source: §A.2 · pp. 21–22
  • Table 7 summarizes the utility benchmarks’ statistics and characteristics, with metric ranges specified for MME’s perception and cognition categories.The supplied table passage does not provide the underlying cell values.Source: §A.2 · p. 21, Table 7 · p. 22

MMBench

  • MMBench [40] covers 20 ability dimensions with 2,974 single-choice questions, and its utility score is accuracy across all questions.Each dimension includes more than 75 questions; accuracy ranges from 0% to 100%.Source: MMBench · p. 22
  • MMBench [40] uses Circular Evaluation to counter choice preference by rotating answer choices and requiring correct predictions on every pass.Each question is presented N times, where N is the number of choices.Source: MMBench · p. 22

MM-Vet

  • MM-Vet [67] assesses six core vision-language capabilities through open-ended answers, which are more flexible but more complex to evaluate.The capabilities are recognition, OCR, knowledge, language generation, spatial awareness, and math.Source: MM-Vet · p. 22
  • MM-Vet [67] uses GPT-4 with few-shot evaluation prompts to score predictions from 0 to 1, then averages scores and multiplies by 100.The resulting utility score ranges from 0 to 100.Source: MM-Vet · p. 22

A.3 Datasets Used in Preliminary Study

The preliminary study assessed MLLM safety awareness using 1,000 LLaVA-1.5-7B responses sampled from unsafe and safe datasets and classified as safe or unsafe.

A.3 Datasets Used in Preliminary Study

  • MM-SafetyBench contributed 72, 79, and 85 unsafe responses from Illegal Activity, HateSpeech, and Physical Harm, respectively.Source: §A.3 · p. 22
  • VLSafe’s examine split contributed 264 unsafe responses, while LLaVA instruction-tuning data [37] contributed 500 safe responses.MM-SafetyBench and VLSafe used their evaluation prompts; LLaVA_150k used the VLSafe prompt.Source: §A.3 · pp. 22–23
  • 500 of 1,000 responses were safe and 500 unsafe; GPT-4 classified them, with manual double-checking.All responses were generated by LLaVA-1.5-7B.Source: §A.3 · p. 23

B Implementation Details · B.1 Model Inference · B.2 Finetuning

Inference disables sampling across all models, with InternLM-XComposer as the sole model using beam search. The finetuning section covers training data, configurations, and analysis of ECSO-generated data.

B.1 Model Inference

  • Inference disables sampling for all models; only InternLM-XComposer uses beam search, with #beams = 5.Other experimented models do not use beam search; settings follow each model’s officially provided default configuration.Source: §B.1 · p. 23

B.2 Finetuning

  • The finetuning section presents training data and configurations, and analyzes data generated by ECSO.Source: §B.2 · p. 23

Data Construction.

ECSO-generated benign responses are paired with harmful queries to create safety-alignment data, while unchanged benign VLGuard pairs provide utility data. Mixing the two balances safety and utility.

Data Construction

  • ECSO generates safe responses for queries initially detected as harmful; 232 query-response pairs form D′ for safety alignment, while the remaining benign VLGuard pairs are retained as utility data.The retained pairs are considered benign by the MLLMs and remain unchanged.Source: Data Construction · p. 23
  • Mixing more unchanged benign pairs can restore utility to the original model’s level, but may diminish D′’s safety gains, creating a safety-utility trade-off.The paper attributes the utility recovery to these responses being generated by the model itself and representing its original abilities.Source: Data Construction · p. 23
  • Using 150 utility samples, ECSO_VLGuard roughly matches untrained LLaVA-1.5-7B utility while maintaining good safety, yet ECSO outperforms it.The paper describes this as an “optimal” ECSO_VLGuard model.Source: Data Construction · pp. 23–24
  • For comparison with ECSO-generated responses, the authors sample 232 VLGuard ground-truth examples and mix them with utility data to construct D∗ + D′safe.The resulting safety-utility trade-off is shown in Figure 12 (right).Source: Data Construction · p. 24, Figure 12 · p. 24

– For safety alignment, data generated by ECSO are even better … C Experiments

ECSO-generated safety-alignment data provide a better safety–utility trade-off than the comparison data while producing safety performance comparable to GPT-4-generated responses. The experiments also examine data quality, training choices, and utility preservation.

– For safety alignment, data generated by ECSO are even better

Training Configurations.

  • The safety-alignment experiments finetune LLaVA-1.5-7B with LoRA for one epoch using a global batch size of 128 and learning rate 2 × 10^-4.Training uses rank 128, weight decay 0.1, and four Tesla V100-SXM2-32G GPUs.Source: Training Configurations · p. 25
  • The reproduced mix-llava-VLGuard model has the worst utility among the compared models and a large MME-P and MME-C performance gap from the original, so the experiments omit its subsampled instruction data.Instead, the finetuning experiments use D′safe generated by ECSO.Source: Training Configurations · p. 25

C Experiments

C.1 Results on Other Scenarios of MM-SafetyBench. … Training with model-generated data

Supplemental analyses show that ECSO’s protection depends on the underlying text-only safety mechanism, while additional ablations, finetuning results, and case studies examine its strengths and limitations.

Effects of Safety Instructions

  • The safety instruction improves response safety for Direct and ECSO but cannot resolve MLLM safety issues.The ablation reports harmless rates on VLsafe and MM-SafetyBench for the SD+OCR split, averaged over the scenarios in Table 1.Source: Effects of Safety Instructions · p. 26, Table 9 · p. 26

Different Harm-Detection Strategies

  • Detection based on the model’s response achieves the highest accuracy, which the authors attribute to harmful content being more explicit there than in the query or image.The compared detection inputs include the query and image or caption.Source: Different Harm-Detection Strategies · p. 26, Table 10 · p. 27

Usage of LLMs

  • Using the original LLM in ECSO yields a higher harmless rate, but ECSO does not use this option because it requires loading two LLMs into memory.The authors attribute the difference to finetuning after connection to the vision encoder potentially causing forgetting of text-only safety alignment.Source: Usage of LLMs · pp. 26–27

C.3 Applying ECSO on the Finetuned Model

  • ECSO is complementary with finetuning on LLaVA-1.5-7B evaluated on MM-SafetyBench.Table 12 reports harmless rates for the finetuned model on MM-SafetyBench.Source: Table 12 · p. 27, §C.3 · p. 27

C.4 Case Studies

  • Case studies report ECSO succeeds on FigStep, a bomb-picture substitution attack, and adversarial images where Direct generates unsafe content.For the adversarial-image example, the clean image prompts a refusal, whereas the optimized image elicits unsafe content from Direct and ECSO protects the model.Source: §C.4 · p. 28

D More Discussions

C.1 Results on Other Scenarios of MM-SafetyBench.

  • The reported harmless-rate results reveal intrinsic vulnerabilities of the LLMs for these scenarios, so image presence or ECSO should not be assumed to make MLLMs safer.ECSO relies heavily on the LLM’s safety mechanism. The authors observe relatively higher harmless rates for SD, which contains no text in images, so the model might understand the image in a harmless way.Source: §C.1 · p. 26

Training with model-generated data

Related papers

Loading 2403.09572v4…