PaperlayerView exact revision on arXiv

Source-linked AI summary

When Seeing Overrides Knowing: Visual Dominance and Deferral-Based Method for Personalized Safety in VLMs

Edward Sun, Yuchen Wu, Zixian Ma, Eric Hanchen Jiang, Yijia Xiao, Xiaoyuan Yi, Ranjay Krishna, Wei Wang, Jindong Wang, Aylin Caliskan

arXiv:2609.04281v1cs.CVcs.AIcs.LG

TL;DR

VLMs may give advice that seems reasonable generally but is unsafe for users whose context is unknown, and existing VLM safety work rarely evaluates this personalized risk. The paper introduces MPS-Bench, analyzes visual dominance, and proposes PRISM; across tested models, direct answers are common and PRISM achieves strong deferral-prediction results.

  • Problem

    Existing VLM safety mechanisms rarely evaluate whether seemingly reasonable responses are safe for particular users when context is missing, and multimodal risk cues may be obscured across modalities.

    Source: §1 · p. 1

  • Method

    The paper introduces MPS-Bench and analyzes visual dominance, then proposes PRISM, an input monitor using bidirectional cross-modal modulation to predict when deferral may be needed.

    Source: §1 · pp. 2–3

  • Results

    Across eight VLMs, direct responses occur in 86–99% of cases and personalized safety reaches no more than 2.6/5; PRISM achieves 0.978 AUC and dominates the safety–utility frontier.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The findings suggest safe multimodal systems should recognize when they lack enough information to answer safely.

    Source: §6 · p. 10

Abstract

from arXiv · show

Vision-language models (VLMs) are increasingly deployed in high-stakes settings, where a response that is reasonable in general may still be unsafe for a particular user whose medical, emotional, or situational context is unknown to the model. We study this problem of personalized safety in multimodal systems and introduce MPS-Bench, a benchmark of 5,181 scenarios from 584 real-world images across 12 high-risk domains, each paired with a hidden user profile. Evaluating eight frontier VLMs, we find that they almost always respond directly (86-99%) rather than seek missing context, and none exceeds 2.6/5 on personalized safety. To understand why these failures arise, we analyze multimodal interactions and identify visual dominance: visual information enters text representations early and suppresses textual risk signals during multimodal fusion. Causal interventions reveal a two-stage mechanism in which visual affect is first transferred into the text stream in early layers and then shapes the final decision through this altered text representation, making late-stage internal remediation unreliable. Motivated by this mechanism, we propose PRISM, a lightweight input monitor that uses bidirectional cross-modal modulation to predict when a query is likely to require deferral. PRISM achieves 0.978 AUC and strictly dominates the safety-utility Pareto frontier across all tested models.

1 Introduction

Personalized safety is underexplored in VLMs, where advice that seems reasonable may be unsafe for a particular user and multimodal signals can obscure risk. The paper introduces MPS-Bench, identifies visual dominance as a failure mechanism, and proposes PRISM for predicting when to defer.

1 Introduction

  • VLM safety work rarely tests whether seemingly reasonable responses are safe for a particular user when context is missing, especially in multimodal settings.Visual and textual risk cues may be hidden, ambiguous, or obscured across modalities.Source: §1 · pp. 1–2
  • MPS-Bench contains 5,181 scenarios across 12 high-risk domains for evaluating personalized safety in VLMs.Scenarios use real-world user-uploaded images and structured user profiles hidden from the model.Source: §1 · p. 2
  • Across eight frontier VLMs, 86–99% of responses are direct, and personalized safety scores do not exceed 2.6/5 when models respond directly.The paper identifies visual dominance: visual information enters text representations early and suppresses textual risk cues during fusion.Source: §1 · p. 2
  • PRISM is a lightweight input monitor using bidirectional cross-modal modulation to predict personalized safety failures and support targeted deferral.It achieves 0.978 AUC and dominates the safety–utility Pareto frontier across all tested models.Source: §1 · pp. 2–3

2 Related Work

Prior safety research has largely targeted generally unsafe or policy-violating content, while personalized safety work has focused on text-based interactions. Related work also documents modality biases in VLMs, but personalized safety failures in multimodal systems remain underexplored.

Modality Bias and Multimodal Interaction in Vision–Language Models

  • VLMs can rely on spurious correlations and show modality biases, including over-reliance on text or ignoring visual input (Deng et al., 2025).Source: §2 · p. 3

Personalized Safety and User-Conditional Alignment

  • Prior personalized-safety studies in language models report risks from ignoring user attributes, but examine text-based interactions (Wu et al., 2025b; Röttger et al., 2024).Source: §2 · p. 3
  • Whether personalized-safety failures also arise in multimodal systems remains largely unexplored, despite established concerns about VLM modality interactions and reliability.Source: §2 · p. 3

3 The MPS-Bench Benchmark for Multimodal Personalized Safety

MPS-Bench evaluates whether VLMs recognize when missing user background could make advice unsafe. It combines high-risk domains, hidden user profiles, and image–text tone variation in scenarios built from real-world images, then scores responses across five safety dimensions.

3 The MPS-Bench Benchmark for Multimodal Personalized Safety

  • MPS-Bench targets cases where an image and query seem benign alone but become risky under hidden user context, testing whether models recognize missing information and respond appropriately.Source: §3 · p. 3

3.1 Benchmark Design

  • The benchmark varies high-risk domains, 12 unobserved user-context attributes, and modality-local positive, neutral, or negative tone labels verified by human annotators.Image–text tone captures affective signals, not latent user risk.Source: §3.1 · pp. 4–5

3.2 Data Collection

  • Using 584 human-uploaded images, the authors generate domain-specific, tone-varied queries paired with high-risk profiles to produce 5,181 scenarios.The same image can seed queries in multiple domains.Source: §3.2 · p. 5

3.3 Evaluation Protocol

  • Models rarely ask for more user context in MPS-Bench scenarios, despite the benchmark's personalized safety risks.The evaluation also compares average safety scores across 12 domains for eight frontier VLMs with and without user context.Source: Figure 3 · p. 4, Figure 4 · p. 5
  • Responses are scored on five 5-point dimensions, including risk calibration, empathy, personalization, grounded reasoning, and de-escalation; the overall score is their unweighted mean.Evaluators can see the full user profile, while models receive only the image and query.Source: §3 · p. 4, §3.3 · p. 5

4 Understanding Multimodal Interactions and Affect on Personalized Safety with MPS-Bench

Across evaluated VLMs, missing user context is rarely recognized, and responses without that context are unsafe on the benchmark. Visual tone has a stronger influence than text tone, with image-driven effects entering text representations early and shaping final decisions.

4.1 VLMs Fail to Recognize and Respond to Missing User Context

  • Direct-response rates range from 86% to 99%, and the pattern holds for reasoning and non-reasoning models.Extended reasoning does not improve recognition of missing user context.Source: §4 · p. 5, §4.1 · p. 6
  • Without user context, no evaluated model exceeds 2.6 on the 5-point safety scale, while providing profiles substantially improves safety scores across models.The comparison suggests models can produce safer, more personalized responses when given sufficient context, but rarely recognize when it is required.Source: §4.1 · p. 6

4.2 Visual Dominance on Personalized Safety

  • Safety scores are more sensitive to image tone than text tone; negative image tone raises scores across text tones and reduces text-tone sensitivity.The authors call this asymmetry Visual Dominance; benign imagery can dampen reactions to risky language.Source: §4.2 · p. 6, Figure 5 · p. 6, Figure 6 · p. 6

4.3 Mechanistic Evidence via Final-Token Edge Patching

  • Patching Text→t_final recovers more behavioral shift than patching Image→t_final, despite image-only changes between paired runs.Cross-modal patching indicates that image influence enters the text stream early and is progressively propagated and amplified, making later interventions unlikely to remove it cleanly.Source: §4.3 · pp. 6–8

5 PRISM: Deferral-Based Personalized Safety via Input Monitoring

PRISM monitors multimodal inputs for personalized safety risk and predicts when a query should be deferred. Its bidirectional cross-modal design is motivated by early visual suppression of textual safety cues, and it outperforms the tested baselines across classification and safety–utility evaluations.

5 PRISM: Deferral-Based Personalized Safety via Input Monitoring

  • PRISM targets failures in which visual information enters text representations early and suppresses textual risk signals, making post-fusion correction unreliable.The module amplifies safety cues across modalities before making a deferral decision.Source: §5 · p. 8

5.1 PRISM Architecture

  • PRISM extracts image and query representations, applies bidirectional cross-modal modulation, and predicts deferral risk; for closed-source models it uses a frozen multimodal encoder.The text representation modulates image features and the image representation modulates text features; a residual connection preserves explicit text risk cues when fusion is unhelpful.Source: §5.1 · p. 8

5.2 Deferral Classification Performance

  • PRISM achieves AUROC ∼0.95–0.98 versus ∼0.67–0.79 for Concat+Linear, ∼0.76–0.82 for VLM-as-judge, and ∼0.46–0.57 for similarity, while dominating the safety–utility frontier.The Pareto comparison concerns unsafe exposure and deferral rate; PRISM also dominates the low-exposure regime while deferring fewer benign queries.Source: Figure 8 · p. 9, §5.2 · p. 9
  • Evaluation uses a balanced held-out set of 1,080 image–question pairs and measures both deferral classification and the safety–utility trade-off.The baselines include similarity, VLM Judge, and Concat+Linear methods.Source: §5.2 · p. 9
  • PRISM variants built on CLIP, EVA-CLIP, SigLIP2, and Qwen3-VL maintain AUROC above roughly 0.95 across backbone encoders.The reported performance is not tied to a single encoder family or representation space.Source: §5.2 · p. 9
  • The ablation’s gains from Similarity to Concat+Linear to PRISM indicate that explicit bidirectional modulation adds value beyond using embeddings alone.The comparison attributes the gap to explicitly modeling and amplifying cross-modal safety signals.Source: §5.2 · p. 9

6 Conclusions

The conclusion characterizes personalized safety as a failure mode in current VLMs and presents PRISM as an input monitor for targeted deferral.

6 Conclusions

  • Current VLMs rarely defer and often answer unsafely when user context is missing.The conclusion describes personalized safety as a fundamental failure mode.Source: §6 · p. 10
  • Visual dominance involves early entry of visual information into text representations and suppression of textual risk signals.The conclusion identifies this as a VLM vulnerability.Source: §6 · p. 10
  • PRISM is a lightweight input monitor that predicts personalized safety failures and enables targeted deferral.The conclusion’s broader supported takeaway is that safe multimodal systems should recognize when they lack enough information to answer safely.Source: §6 · p. 10

Ethics Statement

The ethics statement describes review, privacy protections, and the use of publicly available research datasets in MPS-Bench.

Ethics Statement

  • Data collection and processing followed Reddit’s Content Policy and API Terms of Use and received institutional data ethics committee approval.Reddit-derived content was accessed for research via the PushShift API.Source: Ethics Statement · p. 11
  • The researchers report that they collected or retained no personally identifiable information and anonymized, paraphrased, and filtered the data to mitigate privacy risks.Source: Ethics Statement · p. 11
  • Images and other content came from existing publicly available research datasets released for academic use, redistribution, and adaptation.The statement says their inclusion was consistent with fair use principles in a research context.Source: Ethics Statement · p. 11

A.1.1 MPS-Bench Generation Methods

MPS-Bench combines curated real-world images with generated, grounded queries and user profiles to create realistic but controllable personalized-safety scenarios.

A.1.1 MPS-Bench Generation Methods

  • Because naturally occurring image, query, and user-background triplets are rare and domain-skewed, the benchmark combines curated real images with generated but grounded queries and profiles.The authors describe large-scale collection of realistic paired data as highly impractical.Source: §A.1.1 · p. 16
  • Candidate images are curated from diverse real-world sources and filtered for privacy, quality, and contextual ambiguity.The ambiguity filter retains superficially benign images that admit plausible risks under particular user contexts.Source: §A.1.1 · pp. 16–17
  • Each curated image receives a positive, neutral, or negative effective-tone label used to control query tone and study tone mismatches.The tone labels describe local visual tone, not personalized risk.Source: §A.1.1 · p. 17
  • Scenario construction assigns images to high-risk domains and considers physical, medical, psychological, social, financial, and other risks, including latent user-specific vulnerabilities.Source: §A.1.1 · p. 17
  • Generated first-person queries use few-shot examples and positive, neutral, or negative tones so their surface wording does not trivially reveal personalized risk.The examples come from the small collection of real image, query, and user-background samples.Source: §A.1.1 · p. 17
  • Queries are paired with structured user profiles, and natural-language risk rationales are retained as metadata but not used in the pipelines.The rationales explain why an image–query pair may be unsafe for a user while remaining hidden during inference.Source: §A.1.1 · p. 18

Privacy Filter Prompt

The privacy filter removes images showing uncensored, identifiable faces of private individuals. It allows public figures and non-identifiable people, and defaults to allowing an image when identity or private status is unclear.

Privacy Filter Prompt

  • The prompt defines a privacy violation as an image clearly showing an uncensored, identifiable face of a private individual and removes flagged images.The filter is intended to ensure that only images considered safe and compliant proceed to later processing.Source: Privacy Filter Prompt · p. 18
  • Public figures and people whose faces are not identifiable are allowed through the privacy check.Non-identifiability includes faces that are not visible, recognizable, or are blurred, cropped, obscured, too small, turned away, or masked.Source: Privacy Filter Prompt · p. 18
  • The filter rejects images that appear publicly distributed, including news photos, memes, video screenshots, promotional images, and CCTV footage.Meme templates are excluded even when a face is recognizable.Source: Privacy Filter Prompt · p. 18
  • When it is unclear whether a face is identifiable or the person is private, the prompt defaults to allowing the image.Source: Privacy Filter Prompt · p. 18

Spam Filter Prompt

The spam filter removes images with little meaningful information so later processing can focus on content relevant to personalized safety. It keeps meaningful content that may pose context-dependent risks, even when it appears neutral or harmless.

Spam Filter Prompt

  • The filter distinguishes meaningful personalized-safety content from spam, placeholders, ads, blank screens, and other noise before deeper processing.Its stated purpose is to focus later stages on images that may matter for personalized safety analysis.Source: Spam Filter Prompt · p. 19
  • The filter discards images with little meaningful information, including blank screens, placeholders, advertisements, branding-heavy images, and decorative images without real-world information.The task is to retain content relevant to personalized safety and discard noise or spam.Source: Spam Filter Prompt · p. 19
  • Images with meaningful visual or textual content that could pose context-dependent safety risks should be kept, even if they seem neutral or harmless.If uncertain, the prompt defaults to keeping the image unless it is a black screen, ad, or placeholder.Source: Spam Filter Prompt · p. 19

Quality / Safety Relevance Filter Prompt

This section describes the prompts and evaluation methods used to assess personalized safety across hidden user contexts, response dimensions, tone, and visual hazards. It also examines how visual influence affects decisions and whether an external monitor can defer when user context is needed.

Quality / Safety Relevance Filter Prompt

  • The primary filter keeps images that seem ordinary but could pose substantial personalized risks because of contextual ambiguity or hidden user vulnerabilities.Images without personalized safety risks or with non-substantial content are discarded.Source: Quality / Safety Relevance Filter Prompt · p. 20

Unsafe Quality Filter Prompt (Alternative)

  • The alternative filter keeps only clearly explicit dangers or strongly negative content, discarding neutral, mildly negative, subtle, or ambiguous risks.Its criteria include hazards, hateful content, crises, harmful activities, and safety violations; it supports a separate unsafe/negative subset.Source: Unsafe Quality Filter Prompt… · p. 21

Tone Labeling Prompt

  • A visual tone prompt assigns each image a positive, neutral, or negative label, with confidence and explanation, to guide later questions aligned with or contrasting its affect.The prompt labels the image as a whole using image-only analysis.Source: Tone Labeling Prompt · p. 22

Visual Hazard Extraction Prompt

  • Hazard extraction inventories visible and context-dependent risks across physical, medical, psychological, social, financial, cultural, educational, accessibility, legal, and family domains.It directs curators to consider risks arising from hidden vulnerabilities such as disability, financial pressure, trauma, professional concerns, or legal precarity.Source: Visual Hazard Extraction Prompt · pp. 22–23

Domain Identification Prompt

  • Domain identification maps each image to relevant benchmark categories and returns exact taxonomy keys to steer domain-specific question generation.The listed keys are life, education, relationship, health, social, financial, career, culture, disability, security, emergency, and care.Source: Domain Identification Prompt · pp. 23–24

Question Generation Prompt

  • For each image and selected domain, question generation creates casual questions across positive, neutral, and negative tones, aligned with or contrasting the image’s tone.Questions should vary in focus, remain grounded in the same domain, and not reveal the asker’s personal context; the risk should emerge only when answered without knowing that background.Source: Question Generation Prompt · pp. 24–25
  • The question-generation prompt requests exactly three tone-keyed questions and JSON-only output that can be parsed directly.The listed tone keys are positive, neutral, and negative.Source: Question Generation Prompt · p. 25

User Profile Generation Prompt

  • The profile prompt seeks diverse backgrounds that make generic answers risky through interactions among the question, domain, visual hazards, and hidden circumstances.It directs the generator to make relevant image hazards dangerous for the profile, otherwise ground the risk in the question’s domain, and avoid reusing risk types or scenarios.Source: User Profile Generation Prompt · pp. 26, 28
  • Generated profiles must explain why generic advice for the exact question could harm that specific user, and the output must be a JSON array with specified profile fields.The prompt requires the question and domain to match the inputs and requires risk reasoning to connect the question directly to the profile’s hidden factors.Source: User Profile Generation Prompt · p. 28

Response Generation — Without Background

Response Generation — With Background

Judge Evaluation Prompt

  • The judge prompt evaluates whether responses recognize vulnerabilities, connect them to visual hazards, personalize advice, and offer a safe path forward.Source: Judge Evaluation Prompt · p. 29

EVALUATION STEPS

  • The evaluation rubric checks recognition of general and user-specific hazards, profile-specificity via a replacement test, risk calibration, empathy, personalization, and safety-grounded reasoning.Risk calibration ranges from missing the visual risk or vulnerability to identifying subtle secondary risks implied by the image-profile combination; personalization asks whether advice could be reused for another user.Source: EVALUATION STEPS · pp. 29–30

EVALUATION STEPS; A.2.1 Mechanistic Experiment Definition

  • The evaluation prompt assigns 1–5 scores with strict attention to contextual differences, while the mechanistic experiments formalize activation patching on attention edges and their interventions.The safety-grounded reasoning rubric distinguishes generic advice from explicit user-risk links and nuanced counterfactual reasoning; de-escalation ranges from harmful or blocking responses to immediate, targeted help.Source: EVALUATION STEPS · p. 30

A.1.3 Domain Definitions; A.1.4 User Attribute Definitions

  • MPS-Bench spans 12 high-risk domains and profiles users through both situational context and attributes such as age, health, emotional state, disability, and geography.The design captures interactions between user characteristics and environment, while allowing comparison with context-free settings to isolate the role of missing user information.Source: Table 2 · p. 31, §A.1.3 · p. 32, §A.1.4 · p. 32

A.1.5 Evaluation Metrics Details

  • Responses are assessed on five dimensions: risk calibration, emotional empathy, contextual personalization, safety-grounded reasoning, and de-escalation.The dimensions separate recognizing risk, adapting to the user, connecting user background to multimodal hazards, and guiding the user toward safety.Source: §A.1.5 · pp. 32–33

A.1.6 Justification of Tone Analysis

  • Tone analysis is motivated by implicit safety signals: affect across image and text can reveal vulnerability or conflicting cues that may call for more context before advice.The section argues that cross-modal tone can amplify or suppress risk signals, and that confident or positive responses may be interpreted as endorsements.Source: §A.1.6 · p. 34

A.1.7 Metric Focused Analysis of MPS-Bench Results

  • With user background, all eight VLMs score higher across every dimension; without it, personalization and safety-grounded reasoning are near 1–1.5, while risk calibration is about 2–2.5.The largest generic-to-personalized gaps are in contextual personalization and safety-grounded reasoning; no model reliably infers hidden vulnerabilities from image and conversation cues alone, and even the strongest models rarely score 5 in the personalized setting.Source: Figure 9 · p. 34, §A.1.7 · p. 35

A.1.8 Justification of GPT-5 as Evaluation Judge

  • GPT-5’s evaluations show substantial human agreement (κ = 0.680) and strong agreement with Claude Haiku 4.5 (κ = 0.822) and Gemini 3.1 Pro (κ = 0.831).The authors treat this human agreement and cross-model consistency as evidence supporting GPT-5 as a scalable evaluation judge.Source: §A.1.8 · p. 35

A.1.9 Justification of GPT-5 as Tone Annotator

  • GPT-5’s tone annotations agree with human labels at κ = 0.892, supporting its use for scalable annotation across textual and visual inputs.The validation uses a representative multimodal subset labeled independently by humans.Source: §A.1.9 · pp. 35–36

Sample High-Risk Profile in the Life Domain; Sample High-Risk Profile in the Caregiving Domain

A.2.1 Mechanistic Experiment Definition

  • The mechanistic experiments quantify safety posture as the final-token logit difference between caution and helpful tokens, comparing paired prompts with identical text but different visual tone.A positive score indicates refusal-leaning posture and a negative score indicates compliance; image-edge patching isolates final-token effects, while cross-modal patching alters intermediate text representations that propagate through later layers.Source: §A.2.1 · pp. 40–41
  • Causal interventions showed visual information enters text representations in early-to-mid layers and is read by the final token in late layers as an authentic textual signal.The cross-modal effect peaks in early-to-mid layers, while the text-edge effect spikes in final layers, validating the proposed transfer-then-execution circuit.Source: §A.2.1 · p. 41

A.2.2 Naive Scaling Evaluation

  • The best image-activation scaling setting reached only 2.50/5 safety awareness, and aggressive suppression reduced response quality while providing only marginal safety improvement.At α = 0.0, safety awareness was 2.07 and response quality was 2.37–2.40 versus 2.63 at normal inference; partial suppression yielded 1.77–2.27 safety awareness versus 2.20 at α = 1.0.Source: §A.2.2 · p. 42
  • Visual understanding remained 3.23/5 even at α = 0.0, versus 3.93/5 at α = 1.0, because vision features had already been fused before embedding-layer scaling.The authors also attribute this resilience to residual visual information persisting through cross-attention patterns and image–text positional relationships across deeper layers.Source: §A.2.2 · p. 42
  • Because scaling cannot make the VLM reason about undisclosed user attributes and can degrade capability, the paper motivates PRISM as an external monitor that learns when to request user background.The hidden attributes named include health conditions, financial constraints, and cultural sensitivities.Source: §A.2.2 · p. 42

A.3.1 Baseline Implementation Details

  • The similarity baselines test whether image–text disagreement predicts personalized risk, but the paper characterizes disagreement as an indirect proxy for harm under hidden user attributes.They compute cosine similarity from independently encoded image and text embeddings across CLIP, SigLIP2, EVA-CLIP, and Qwen3-VL.Source: §A.3.1 · pp. 42–43
  • Zero-shot scoring prompts a VLM to estimate personalized risk from an image and query, while self-evaluation asks whether additional user context is needed to answer safely.Source: §A.3.1 · p. 43
  • A concatenation-plus-linear baseline tests whether risk is linearly recoverable from frozen multimodal features without explicit cross-modal interaction.It uses the same four encoders as PRISM so performance differences isolate the classifier rather than the representations.Source: §A.3.1 · p. 44

Related papers

Loading 2609.04281v1…