Source-linked AI summary
Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation
Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung, James T. Kwok, Yu Zhang
TL;DR
MLLMs can lose the safety behavior inherited from aligned LLMs when processing images, despite retaining the ability to recognize unsafe responses. ECSO is a training-free defense that detects unsafe outputs, converts the corresponding images into query-aware text, and regenerates responses without images. It substantially improves safety while preserving utility across common MLLM benchmarks, though its effectiveness depends on the underlying LLM’s safety capacity.
Problem
MLLMs can be induced to generate unethical content by image inputs even though their underlying LLMs are aligned for safety.
Method
ECSO detects unsafe initial responses, converts their images into query-aware text, and generates safer responses without images using the intrinsic safety mechanism of pre-aligned LLMs.
Results
ECSO significantly improves safety across five state-of-the-art MLLMs while maintaining utility scores on common MLLM benchmarks.
Takeaways & Limitations
ECSO provides a training-free protection strategy that restores safer MLLM behavior while retaining information from image inputs.
Takeaways & Limitations
ECSO relies heavily on the underlying LLM’s capacity to identify and neutralize unsafe queries, so deficiencies in that safety mechanism may compromise performance.
Abstract
from arXiv · showhide
Multimodal large language models (MLLMs) have shown impressive reasoning abilities. However, they are also more vulnerable to jailbreak attacks than their LLM predecessors. Although still capable of detecting the unsafe responses, we observe that safety mechanisms of the pre-aligned LLMs in MLLMs can be easily bypassed with the introduction of image features. To construct robust MLLMs, we propose ECSO (Eyes Closed, Safety On), a novel training-free protecting approach that exploits the inherent safety awareness of MLLMs, and generates safer responses via adaptively transforming unsafe images into texts to activate the intrinsic safety mechanism of pre-aligned LLMs in MLLMs. Experiments on five state-of-the-art (SoTA) MLLMs demonstrate that ECSO enhances model safety significantly (e.g.,, 37.6% improvement on the MM-SafetyBench (SD+OCR) and 71.3% on VLSafe with LLaVA-1.5-7B), while consistently maintaining utility results on common MLLM benchmarks. Furthermore, we show that ECSO can be used as a data engine to generate supervised-finetuning (SFT) data for MLLM alignment without extra human intervention.
1 Introduction
MLLMs inherit strong capabilities from aligned LLMs but can generate unethical content when images are introduced. ECSO addresses this gap with a training-free strategy that uses safety assessment and image-to-text transformation to restore safer generation.
- MLLMs can be induced to generate unethical content with image inputs despite being built on well-aligned LLMs.
- Training-based alignment requires carefully designed red-teaming data and becomes more difficult when image inputs are involved.
- ECSO detects unsafe initial responses, converts their image inputs into query-aware text, and generates responses without images to reactivate intrinsic LLM safety.
- ECSO significantly improves safety across five state-of-the-art MLLMs without sacrificing utility performance.
2 Related Work
Prior work documents severe MLLM vulnerability to malicious visual inputs and explores training-based, prompt-based, and activation-based defenses. These approaches motivate protection methods that avoid exhaustive red-teaming and adapt to evolving attacks.
- Malicious textual queries embedded in images and gradient-generated adversarial images can circumvent MLLM defenses and elicit harmful responses.
- Red-teaming-based alignment is labor-intensive and may not cover all potential attack vectors.
- Existing inference-time defenses include manually crafted system prompts and safety steering vectors for unsafe inputs.
3 Preliminary Observations
The preliminary experiments reveal a split: image inputs weaken safe generation, while MLLMs retain safety awareness and can assess their own responses. ECSO exploits this preserved assessment ability and the restored safety of text-only processing.
- 3.1 Safety Mechanism Persists in MLLM: With images present, most tested MLLMs achieve only about 20% harmless responses, whereas removing images raises all models to nearly 100%.
- 3.2 MLLMs are Aware of Their Own Unsafe Responses: MLLMs retain high accuracy when classifying their own responses as safe or unsafe, even when input images are present.
- 3.2 MLLMs are Aware of Their Own Unsafe Responses: LLaVA-1.5-7B and ShareGPT4V-7B exceed 95% accuracy in assessing response safety.
- The contrast between unsafe generation and robust safety discrimination motivates using self-assessment to mitigate multimodal safety risks.
4 Methodology
ECSO first checks an MLLM’s initial response, then processes only unsafe cases by converting images into query-aware captions and generating a text-only response. Excluding images exposes malicious content to the aligned language model while preserving relevant image information through the query-aware caption.
- 4.1 Harmful Content Detection: ECSO detects whether an initial multimodal response is unsafe and returns safe responses directly while routing unsafe responses through additional protection steps.
- 4.2 Query-Aware Image-to-Text (I2T) Transformation: The query-aware caption is generated from the image and original question to retain relevant information that generic captioning might omit.
- 4.3 Safe Response Generation Without Images: ECSO generates the final answer from the original query and caption with the image excluded, reducing the MLLM to a text-only LLM.
- 4.3 Safe Response Generation Without Images: The image-free generation prompt includes “HARMLESS and ETHICAL” to prioritize safer responses.
5 Experiments
Across five SoTA MLLMs, ECSO substantially improves safety while preserving utility by detecting unsafe responses, converting images into query-aware text, and regenerating responses without images.
- Safety evaluation: ECSO protects five SoTA MLLMs across MM-SafetyBench scenarios and significantly improves harmless rates on VLSafe.The evaluation compares ECSO with direct prompting across multiple MLLMs and datasets.
- Utility evaluation: ECSO maintains utility across MME-P and MM-Vet, with slight improvement on MME-C and MM-Vet across different models.Most benign queries are correctly recognized without triggering image-to-text transformation, limiting unnecessary processing.
- Ablation studies: Removing image features is critical: adding them back causes large harmless-rate decreases on both MM-SafetyBench (SD+OCR) and VLSafe.This ablation supports ECSO’s mechanism of restoring pre-aligned language-model safety by generating responses without images.
- Ablation studies: Query-aware image-to-text transformation is indispensable for utility because captions conditioned on the question preserve information needed to answer correctly.Without query conditioning, captions omit relevant clues and utility declines; the elephant example illustrates this failure mode.
- Safety alignment: ECSO can generate SFT data from an unsupervised safety dataset, and finetuning on this data provides better safety than finetuning on human-verified data in the reported comparison.The generated dataset contains safe responses produced by ECSO, while intermediate outputs are discarded before SFT.
- Limitations: ECSO relies heavily on the underlying LLM’s ability to identify and neutralize unsafe queries, so weaknesses in that safety mechanism can compromise multimodal protection.This dependence is identified as the paper’s limitation and motivates future work on multimodal safety mechanisms.
6 Conclusion
The paper presents ECSO as a training-free safeguard and reports its evaluation resources and scope. Its safety evaluation focuses mainly on scenarios where underlying LLMs can provide safe behavior, while utility is assessed across established multimodal benchmarks.
- ECSO is a training-free safeguarding method that uses MLLMs’ intrinsic safety mechanisms and can generate SFT data without additional human intervention.
- The safety resources include MM-SafetyBench, VLSafe, and VLGuard, while utility evaluation uses MME, MMBench, and MM-Vet.
- Safety datasets: ECSO’s main safety evaluation covers MM-SafetyBench scenarios 01–07 and 09, excluding political, legal, financial, and health scenarios where some MLLMs are unsafe even with text-only queries.
- Utility datasets: MME measures perception and cognition over 14 subtasks, MMBench covers 20 ability dimensions, and MM-Vet evaluates six core vision-language capabilities.
A.3 Datasets Used in Preliminary Study
The preliminary safety-awareness study evaluates model responses from unsafe and safe sources. It combines responses from MM-SafetyBench, VLSafe, and LLaVA’s instruction-tuning data into a balanced collection.
- Dataset sources: VLSafe contributes 264 unsafe responses from its examine split, while LLaVA_150k contributes 500 safe instruction-tuning responses.
- The study collects 1,000 responses: 500 safe and 500 unsafe, all generated by LLaVA-1.5-7B and classified with GPT-4 plus manual checking.
B Implementation Details
Implementation fixes inference randomness and follows each model’s official default configuration. Only InternLM-XComposer uses beam search.
- Sampling is disabled for all models during inference to eliminate randomness in generation.
- InternLM-XComposer uses beam search with 5 beams, whereas the other experimented models do not use beam search.
- The section provides implementation details, training configurations, and analysis of data generated by ECSO.
Data Construction.
The data-construction study examines how ECSO-generated responses and benign utility data affect safety–utility trade-offs. Mixing utility examples can restore utility while retaining safety, but the proposed ECSO remains stronger than the compared training setup.
- Data construction: ECSO constructs D′ by retaining query–response pairs initially judged harmful and replacing their harmful responses with safe ECSO-generated responses.
- Data construction: D′safe contains benign responses that remain unchanged and is critical for maintaining utility during safety alignment.
- Fine-tuning with ECSO-generated data alone significantly degrades utility, whereas increasing benign utility samples restores utility toward the original model.
- Safety–utility trade-off: Figure 12 compares safety and utility when mixing D′ or D∗ with D′safe, using harmless-rate sums on three MM-SafetyBench scenarios and utility-score sums from MME-P and MME-C.
- The ECSO_VLGuard model trained with 150 utility samples achieves roughly the untrained LLaVA-1.5-7B utility while maintaining good safety, yet is outperformed by ECSO.
– For safety alignment, data generated by ECSO are even better
ECSO-generated data provides a stronger safety–utility trade-off than comparison data for supervised alignment, while remaining comparable in response quality to GPT-4-generated data.
- ECSO-generated responses achieve safety performance comparable to GPT-4-generated responses, with both reaching around 225.
- ECSO-generated data offers a better safety-utility trade-off than the comparison data.
- With increasing ECSO-generated data, models retain higher utility than models trained on GPT-4-generated data.
- Safety decreases faster for models trained on GPT-4-generated data than for models trained on ECSO-generated data.
- The experiment fine-tunes LLaVA-1.5-7B with LoRA for one epoch using a global batch size of 128.
C.1 Results on Other Scenarios of MM-SafetyBench.
Additional evaluations show that ECSO protects MLLMs across other safety scenarios and attack forms, although its effectiveness depends on the underlying LLM safety mechanism.
- ECSO relies heavily on the LLM safety mechanism and therefore cannot be expected to improve scenarios where the underlying LLM has intrinsic vulnerabilities.
- The safety instruction improves response safety but cannot address all safety issues in MLLMs.
- ECSO is complementary with fine-tuning on LLaVA-1.5-7B evaluated on MM-SafetyBench.
- ECSO also protects MLLMs against OCR-based FigStep attacks and images that replace sensitive words with pictures.
- Against adversarial images, Direct generates unsafe content whereas ECSO succeeds in protecting the MLLM.
Training with model-generated data
The paper illustrates ECSO’s model-generated captions and qualitative safety outcomes across multiple MLLMs, showing that captioning can preserve query-relevant information while enabling safer responses.
- Qualitative examples cover MM-SafetyBench, VLSafe, FigStep, Internet images, and adversarial images across several MLLMs.
- ECSO-generated captions can retain harmful textual content from images, including instructions about illegal drugs.
- On a bomb-related query, ECSO converts the image into a caption describing the bomb and fuse before generating the response.
- For a hate-speech request, the direct response refuses to produce hate speech.