Source-linked AI summary
The Side Effects of Being Smart: Safety Risks in MLLMs' Multi-Image Reasoning
Renmiao Chen, Yida Lu, Shiyao Cui, Xuan Ouyang, Victor Shea-Jay Huang, Shumin Zhang, Chengwei Pan, Han Qiu, Minlie Huang
TL;DR
MLLMs’ stronger multi-image reasoning raises a safety gap because harmful intent may emerge during relational reasoning rather than from explicit content. The paper introduces MIR-SafetyBench and evaluates 19 MLLMs, finding widespread vulnerabilities, superficial safe responses, and lower attention entropy in unsafe generations. The benchmark’s coverage and analysis remain bounded by synthetic data, single-turn evaluation, a fixed model set, and correlational internal-state evidence.
Problem
Existing MLLM safety evaluations largely overlook harmful intent that emerges through reasoning across multiple images and textual instructions.
Method
The paper introduces MIR-SafetyBench, a benchmark with 2,676 multi-image instances spanning 9 relation types, and analyzes model responses and attention entropy.
Results
Evaluations on 19 MLLMs reveal widespread multi-image safety risks, with stronger reasoning often coinciding with higher attack success rates and unsafe generations showing lower attention entropy on average.
Takeaways & Limitations
Multi-image reasoning safety requires distinguishing genuine alignment from safe-looking outputs caused by misunderstanding, generic refusals, or evasive answers.
Takeaways & Limitations
The evaluation uses synthetic data, 19 fixed models, single-turn prompts, classifier-based ASR, and attention-entropy analysis of four models with correlational evidence.
Abstract
from arXiv · showhide
As Multimodal Large Language Models (MLLMs) acquire stronger reasoning capabilities to handle complex, multi-image instructions, this advancement may pose new safety risks. We study this problem by introducing MIR-SafetyBench, the first benchmark focused on multi-image reasoning safety, which consists of 2,676 instances across a taxonomy of 9 multi-image relations. Our extensive evaluations on 19 MLLMs reveal a troubling trend: models with more advanced multi-image reasoning can be more vulnerable on MIR-SafetyBench. Beyond attack success rates, we find that many responses labeled as safe are superficial, often driven by misunderstanding or evasive, non-committal replies. We further observe that unsafe generations exhibit lower attention entropy than safe ones on average. This internal signature suggests a possible risk that models may over-focus on task solving while neglecting safety constraints. Our code and data are available at https://github.com/thu-coai/MIR-SafetyBench.
1 Introduction
Improved multi-image reasoning may expand MLLMs’ safety risks because harmful intent can emerge only through cross-image and instruction-level reasoning. MIR-SafetyBench evaluates this risk and distinguishes genuine safety alignment from superficially safe behavior.
- Existing MLLM safety evaluations emphasize explicit harmful content, while neglecting harm that emerges through reasoning over multiple images and instructions.
- The study focuses on multi-image scenarios where cross-image interactions and user instructions conceal harmful intent.
- As reasoning capability increases, models may progress from misunderstanding harmful requests to inferring intent and generating detailed high-risk procedures.
- MIR-SafetyBench contains 2,676 instances across 9 multi-image relation types and 6 safety risk categories, generated from 600 curated harmful seed questions.
- Evaluations of 19 MLLMs find pervasive risks that can increase with stronger multi-image reasoning.
- The study separates genuine safety alignment from safe outputs caused by limitations, using response behavior and attention entropy to examine unsafe generations.
2 Related Work
MLLM research has advanced from single-image safety tests toward complex multimodal reasoning, but existing work lacks a systematic benchmark for safety risks arising from multi-image relations.
- 2.1 Advances in MLLM Reasoning: Recent MLLM research increasingly targets multi-image reasoning for complex relational and contextual scenarios.
- 2.2 Safety Issues in Advanced MLLMs: Early safety benchmarks injected explicit harmful signals into images, while later work examined automated red-teaming and cross-modality alignment.
- 2.2 Safety Issues in Advanced MLLMs: Recent attacks exploit multi-image contexts through distraction, heuristic, visual-chain, and compositional jailbreaks.
- 2.2 Safety Issues in Advanced MLLMs: MIR-SafetyBench addresses the absence of a systematic benchmark for safety risks in multi-image reasoning.
3 Multi-image Relations Taxonomy
The taxonomy organizes multi-image safety risks into four relation categories and nine subcategories that conceal harmful intent through semantic, logical, temporal, and spatial structure.
- The taxonomy defines four primary categories and nine fine-grained multi-image relation subcategories for conveying harmful intent.
- Temporal Relations: Temporal relations represent continuous progression or omit intermediate steps, prompting models to generate or infer harmful actions.
- Spatial Relations: Spatial relations connect distinct scenes or embed harmful objects and queries within benign or authoritative contexts.
- Semantic Relations: Semantic relations hide or reconstruct harmful content through relevance-based distractors and complementary image tiles.
- Logical Relations: Logical relations exploit analogy, causal inference, and decomposition into innocuous sub-questions that can be synthesized into harmful responses.
- Figure 3 presents the multi-stage pipeline used to construct benchmark instances from these relation types.
4 Benchmark Construction
MIR-SafetyBench converts curated harmful questions into validated multi-image reasoning instances through relation-specific rewriting, image generation, testing, judging, and iterative refinement.
- MIR-SafetyBench comprises 2,676 instances spanning 9 multi-image relations and 6 risk categories, constructed from 600 curated harmful prompts.
- A Revisor transforms each harmful question into an indirect textual prompt, image descriptions, and keywords for a target relation.
- An image generator synthesizes composite images from each description-keyword pair using FLUX.1-dev.
- A Tester evaluates whether each rewritten instance preserves the original intent and elicits a harmful response.
- The HarmBench judge assigns harmful or safe labels, while DeepSeek-R1 checks harmfulness, prompt neutrality, relation adherence, and intent fidelity.
- Instances undergo up to five refinement rounds or stop when all quality checks pass, followed by human spot checks for reliability.
5 Experiments
Experiments on 19 MLLMs evaluate attack success, safe-response quality, matched single- versus multi-image vulnerability, and attention-entropy differences between safe and unsafe generations.
- 5.1 Experimental Setup: 19 representative MLLMs are evaluated using Attack Success Rate (ASR), the percentage of instances eliciting harmful responses.
- 5.2 Main Results on MIR-SafetyBench: 87.63% is the highest overall ASR, while most evaluated models remain susceptible to multi-image relational attacks.
- 5.2 Main Results on MIR-SafetyBench: Reasoning-enhanced and multi-image-optimized models often show higher ASR, although the strongest closed-source models can combine strong reasoning with low ASR.
- 5.2 Main Results on MIR-SafetyBench: Categories requiring abstract, multi-step reasoning, including Decomposition and Causality, generally yield higher ASR than direct pattern-recognition categories such as Semantic Relevance.
- 5.3 Analysis of Model Behaviors: Many safe outputs are superficial: models often misunderstand harmful intent, issue generic refusals, or evade with unhelpful content rather than correctly refusing.
- 5.4 Controlled Comparison with Single-Image: All five examined models become markedly more dangerous under matched multi-image relational prompts than under single-image cases.
- 5.5 Internal analysis via attention entropy: Safe and unsafe responses have similar average lengths, making answer length an unlikely sole explanation for the observed entropy gaps.
- 5.5 Internal analysis via attention entropy: In multi-image settings, unsafe generations have lower attention entropy than safe ones on average, a pattern absent from the single-image setting.
6 Conclusion
The paper introduces MIR-SafetyBench for multi-image reasoning safety and reports extensive risks across 19 MLLMs, including internal attention-entropy differences between unsafe and safe generations.
- MIR-SafetyBench is presented as a benchmark for multi-image reasoning safety with 2,676 instances across 9 relation types and 6 risk categories.
- Experiments on 19 representative MLLMs reveal extensive safety risks in multi-image reasoning and analyze attention entropy as an internal mechanism.
- The benchmark is intended to support reliable multi-image safety evaluation and investigation of vulnerabilities arising from complex scenarios.
Limitations
The benchmark provides broad but non-exhaustive coverage and may inherit biases and noise from its source data and automated construction components. The evaluation and analysis are also limited in scope, while mitigation remains largely unexplored.
- MIR-SafetyBench covers 2,676 synthetic instances across 9 relation types and 6 risk categories, but its coverage is broad rather than exhaustive and inherits source-data and seed-rewriting biases.
- Automated rewriting, testing, image-generation, and harmfulness-classification components may introduce systematic noise into instances and safety labels, while human review only spot-checks samples.
- The evaluation uses 19 popular MLLMs, single-turn prompting, and classifier-based ASR, excluding interactive, multi-turn, and tool-augmented use cases.
- Attention-entropy analysis covers only four representative models and provides correlational rather than causal evidence about reasoning load and safety failures.
- The work is diagnostic rather than prescriptive: it neither adapts models with MIR-SafetyBench nor proposes concrete detection, monitoring, or training-time mitigation mechanisms.
Ethical Considerations
The benchmark targets safety-critical harms while using synthetic data and research safeguards. Its construction combines a six-category harm taxonomy, automated filtering, and human curation of harmful seed questions.
- MIR-SafetyBench addresses hate speech, harassment, violence, self-harm, illegal activities, and privacy, so some prompts and outputs contain harmful content.
- The stated purpose is systematic evaluation and analysis of MLLM safety vulnerabilities, supporting development of more robust defenses rather than encouraging harmful behavior.
- The planned safeguards include careful inspection before release and restricting access to people who follow stringent ethical guidelines.
- Human annotators were informed about disturbing content, participated voluntarily, could stop without penalty, and were encouraged to take breaks or avoid distressing examples.
- The dataset excludes personal identifying information about real individuals and uses synthetically generated images rather than images collected from real users.
- Construction combines automated triage of existing safety-benchmark data with human expert curation of 100 questions per risk category, yielding 600 textual harmful prompts.
C Evaluated Models
The evaluation covers 19 representative MLLMs spanning closed- and open-source systems, varied model roles, scales, and architectures. The listed models include chat, reasoning, and single-image systems.
- The study evaluates 19 representative LLMs on multi-image reasoning safety.
- Closed-Source Models: Closed-source coverage includes GPT-4o, GPT-4o-mini, Gemini-2.5-Pro, and Gemini-2.5-Flash, spanning chat and reasoning models.
- Open-Source Models: Open-source coverage includes single-image, chat, and reasoning models such as LLaVA, InternVL3, MiniCPM-o, Qwen2.5-VL, Kimi-VL, QVQ, Skywork-R1V3, and GLM-4.1V.
- The evaluated models span a wide range of scales and dense or mixture-of-experts architectures.
D Computing Environment and Implementation
The analysis runs models under standardized hardware and safety settings, then measures attention entropy over answer tokens across layers and answer segments. Heatmaps compare entropy between safe and unsafe responses for representative chat and reasoning models.
- Computing Environment: Open-source models ran locally on NVIDIA A800 GPUs with 80GB VRAM, while closed-source models were accessed through APIs.The benchmark construction pipeline used four A800 GPUs.
- Implementation: Multiple images were stitched into one horizontally concatenated composite with 50-pixel white gaps for single-image models.This preprocessing addressed models that cannot directly process multiple image inputs.
- Evaluation Settings: All models used their default safety settings, and only final responses were evaluated for reasoning models producing chain-of-thought.The evaluation did not score intermediate reasoning traces.
- Attention-Entropy Analysis: Attention entropy was computed over answer tokens using self-attention weights across Transformer layers, heads, and key positions.Answer tokens were divided into approximately equal contiguous segments for segment-level analysis.
- Attention-Entropy Analysis: The heatmap compares mean segment entropy for safe and unsafe examples, with positive gaps indicating higher entropy for safe responses.Figure 5 presents this comparison for MiniCPM-o-2.6 and Kimi-VL-A3B-Thinking-2506 in multi-image and single-image settings.
G Statics for answer length.
The entropy analysis filters for long responses and finds that safe and unsafe answers have similar, inconsistently ordered lengths. Thus, answer-length differences alone are unlikely to explain the observed entropy gaps.
- Length Filtering: Responses shorter than 1000 characters were excluded before attention-entropy analysis.This filtering removes trivial short or truncated generations.
- Length Comparison: Across all eight model–setting combinations, safe and unsafe responses differed by at most about 20% in average token length.The comparison covered both single-image and multi-image settings.
- Length Comparison: The direction of length differences was inconsistent, with unsafe answers slightly shorter for Qwen2.5-VL-3B-Instruct and MiniCPM-o-2.6 in multi-image settings.Similar patterns appeared when answer length was measured in characters rather than tokens.
- Interpretation: Answer-length differences alone are unlikely to explain the systematic entropy gaps shown in the heatmaps.The analysis therefore treats the gaps as not solely attributable to response length.