Safety of Multimodal Large Language Models on Images and Texts

1 Introduction

• LLMs have brought convenience but also safety risks, while alignment techniques have enhanced their security.The authors describe MLLM safety research as early-stage and note the lack of a comprehensive survey.Source: §1 · p. 1
• The introduction frames key questions around image-specific risks, how to measure MLLM safety, and methods for resisting unsafe queries.The authors say evaluation is essential for judging how close MLLMs are to achieving safety.Source: §1 · p. 1
• Visual-modality risks include low-cost adversarial image perturbations, obedience to malicious visual instructions through OCR, and weakened alignment after cross-modal training.The survey reports that aligned LLMs usually reject malicious textual instructions, but may obey corresponding visual instructions.Source: §1 · pp. 1–2
• The survey compares safety-evaluation datasets and metrics, reviews attack and defense approaches, and identifies future research opportunities.It focuses on MLLMs handling 2D images and text unless otherwise specified.Source: §1 · pp. 1–2
• The paper’s roadmap covers MLLM and safety background, evaluation, attacks, defenses, future directions, and a conclusion.The roadmap places evaluation in Section 3, attacks in Section 4, defenses in Section 5, and future directions in Section 6.Source: §1 · p. 2

2 Background

• The survey focuses on general-domain safety because professional domains such as robotics, medicine, law, and finance involve distinct, complicated problems.It excludes tool-augmented LLMs and LLM agents.Source: §2 · p. 2, §2.2 · p. 2

2.1 An Overview of MLLMs

• An MLLM combines an LLM of at least 1B parameters, a vision encoder, and a cross-modal fusion module; common fusion designs include linear projection, learnable queries, and cross-attention.Visual instruction tuning lets the fusion module learn cross-modal information for image and text inputs.Source: §2.1 · p. 2

2.2 Understanding of Safety

• Safety has no consensus definition: toxicity is subjective and covers only a subset of potential harms, while prior work uses differing safety dimensions.The paper notes that safety risks vary across development stages and calls for a more mature, unified understanding.Source: §2.2 · pp. 2–3

3.1 Dataset

• Representative MLLM safety datasets cover privacy, meme-based social abuse, toxic image captions, and benchmarks built from visual prompts or natural images.The survey focuses on recently constructed datasets for MLLMs, rather than detailing benchmarks created only for individual experiments.Source: §3.1 · p. 3

3.2 Metric

• Open-ended MLLM answers make subjective evaluation difficult, while evaluation methods must balance cost and accuracy.The survey classifies approaches as human, rule-based, or model-based automatic evaluation.Source: §3.2 · p. 3

Human Evaluation

• Human evaluation uses annotators to judge unsafe answers and often reports attack success rate as the proportion of samples eliciting unsuitable text.Other reported measures include recognition success rate and defense success rate; one study asks whether tool-abuse responses appear natural and helpful.Source: Human Evaluation · pp. 3–4

3.1 Dataset

• Table 1 compares representative evaluation datasets; its volume marker limits the stated count to the multimodal test set, and ToViLaG tests caption toxicity without ground-truth captions.The table caption identifies these qualifications but does not provide dataset-specific comparisons in the supplied passage.Source: Table 1 · p. 4

Rule-based Evaluation

• Rule-based evaluation checks outputs against target strings or treats safety as classification, while generative tasks may use image-captioning and response-utility metrics.Examples include accuracy and F1 for classification, CIDEr and BLEU-4 for image captioning, and SSIM for image perturbations.Source: Rule-based Evaluation · p. 4

Model-based Automatic Evaluation

• Model-based automatic evaluation uses specialized toxicity models or powerful LLMs, which can support more customized judgments of semantic alignment, refusal, and harmlessness.GPT-4 scoring described in the survey considers relevance, safety, and persuasiveness.Source: Model-based Automatic Evaluation · p. 4

4 Attack

• The survey groups attacks into malicious image and text construction, noting that some works study both.For images, it distinguishes adversarial attacks from visual prompt injection.Source: §4 · p. 4, §4.1 · p. 4

Adversarial Attack

• Adversarial-image studies range from end-to-end perturbation optimization to attacks using visual-encoder access, and include attacks on commercial-model defenses.Reported effects include toxic outputs, privacy leakage, and toxic-content abuse; other work studies third-party attacks against honest users, including tool misuse.Source: Adversarial Attack · p. 5
• Some adversarial attacks pair visual and textual perturbations during optimization, while the textual perturbations are absent at test time in CroPA.Qi et al. report visual-attack computational cost at approximately one-twelfth of text-attack cost; Tu et al. evaluate attacks across GPT-4, GPT-4V, and ten open-source MLLM categories.Source: Adversarial Attack · p. 5

Visual Prompt Injection

• Visual prompt injection embeds malicious raw text in an image, which MLLMs may obey through OCR even when corresponding textual prompts are refused.The reviewed examples include altered identity information, unsafe phrases rendered in images, and split-word visual prompts intended to evade OCR detection.Source: Visual Prompt Injection · pp. 5–6

4.2 Malicious Text Construction

• Malicious text attacks use crafted or optimized prompts, including adversarial prefixes and two-hop templates targeting model defenses.Studies also report that textual adversarial attacks can achieve higher success rates on LLaVA than on its original LLMs, and that fine-tuning on LLaVA-Instruct can degrade LLM safety alignment.Source: §4.2 · p. 6

5 Defense

• The survey presents inference-time and training-time alignment as its two main defense lines.It also points to further potential directions for improving MLLM safety.Source: §5 · p. 6

5.1 Inference-time Alignment

• Inference-time alignment uses prompt engineering or safety steering vectors, without requiring the cost of training a model or module.Examples include detailed system prompts and Self-Moderation, which iteratively revises privacy-leaking responses and asks the model to judge them.Source: §5.1 · p. 6, §5.2 · p. 6

5.2 Training-time Alignment

• Training-time approaches add safety feedback or modules that detect and detoxify unsafe outputs.One approach uses GPT-4-generated Natural Language Feedback in modified conditional reinforcement learning; another adds an unsafe-content detector and output detoxifier.Source: §5.2 · pp. 6–7

6.1 Reliable Safety Evaluation

• Safety benchmarks need broader coverage and finer-grained capability taxonomies, with explicit choices about unsafe elements, data resources, diversity, volume, and quality control.For LLM-based evaluation, carefully designed prompts are needed to define objective rules and metrics.Source: §6.1 · p. 7

6.2 In-Depth Study of Safety Risk

• Attack research often lacks analysis of what enables success, quantitative evaluation, and direct comparison across attacks.The survey also calls for studying how architecture, model parameters, cross-modal training data, and attack resource requirements relate to safety.Source: §6.2 · p. 7

Alignment Techniques

• For visual instruction tuning, the survey identifies unresolved choices about safety-dataset quality, example diversity, sample numbers, and training order.It also says the challenges of applying RLHF to MLLM safety and the construction of preference data remain unclear.Source: Alignment Techniques · p. 7

Balance between Safety and Utility

• Alignment must balance refusing malicious instructions against preserving utility, and the appropriate balance may differ across applications and audiences.The survey cites privacy-protection scoring and a loss balancing response utility with attack success as relevant inspiration.Source: Balance between Safety and Utility · p. 7

• The survey reviews evaluation, attack, and defense work to demonstrate the current development status of MLLM safety.Source: §7 · p. 7
• It introduces an overview of MLLMs and the paper’s understanding of safety before reviewing safety research.Source: §7 · p. 7
• It identifies existing challenges and points to promising future research opportunities.Source: §7 · p. 7