PaperlayerView exact revision on arXiv

Source-linked AI summary

EviSafe: Evidence-Grounded Safety Evaluation for Vision-Language Models

Xuetong Li, Gaofeng Liu

arXiv:2608.23313v1cs.AI

TL;DR

VLM safety benchmarks often score only final responses, leaving unclear whether safe behavior reflects correct multimodal evidence or superficial refusal. EviSafe introduces an evidence-grounded benchmark and three-probe evaluation, finding substantial misalignment among safety behavior, evidence grounding, and counterfactual sensitivity across evaluated VLMs.

  • Problem

    Existing outcome-level safety evaluations cannot distinguish evidence-grounded behavior from keyword-triggered refusal, missed visual hazards, or over-refusal of benign-sensitive inputs.

  • Method

    EviSafeBench evaluates natural responses, structured evidence reports, and counterfactual responses against gold decisions, evidence annotations, policies, and targeted interventions.

  • Results

    Across eleven VLM runs, final-response safety, evidence grounding, and counterfactual sensitivity remain substantially misaligned.

  • Takeaways & Limitations

    EviSafe supports evaluation beyond refusal counts by diagnosing whether multimodal safety behavior is grounded in evidence and changes appropriately with that evidence.

  • Takeaways & Limitations

    The claims concern reportable and behaviorally checked evidence grounding rather than mechanistic faithfulness, and automatic metrics remain sensitive to judge choice and configuration.

Abstract

from arXiv · show

Vision-language model safety benchmarks typically evaluate only final responses: whether a model refuses, warns, or complies. This outcome-level view cannot tell whether a model is safe for the right multimodal reason. Safelooking behavior may reflect keyword-triggered refusal, missed visual hazards, or over-refusal of benign-sensitive inputs. We introduce EviSafe, an evidence-grounded framework for VLM safety that jointly evaluates natural user-facing behavior, explicit grounding in textual and visual evidence, and behavioral sensitivity to counterfactual changes in safety-critical evidence. EviSafeBench instantiates the framework as a controlled benchmark with 1,181 gold image-text scenarios and 2,452 targeted counterfactual variants across eight safety domains and eight risk-source types. Each scenario includes a gold safety decision, evidence annotations, a safe-response policy, and counterfactual interventions. The three-probe protocol queries models with natural-response, evidencereporting, and counterfactual-response prompts, then scores them using an evidence-aware judge. Across eleven evaluated VLMs, natural severity accuracy ranges from 27.6% to 52.8%, relaxed diagnostic consistency from 6.1% to 29.3%, and unsafe-to-safe counterfactual transition success from 30.4% to 58.4%. These gaps show that the evaluated VLMs are not reliably safe for the right multimodal reason and motivate evaluation beyond refusal counts.

1 Introduction

EviSafe argues that VLM safety cannot be judged reliably from final responses alone because safe-looking behavior may ignore or misread the multimodal evidence that determines the correct policy. It introduces an evidence-grounded framework and benchmark that jointly evaluate natural behavior, evidence reporting, and counterfactual sensitivity.

  • Motivation: VLM safety depends on relations among user text, visual content, intent, and context, not isolated cues.The same sensitive cue can support opposite response policies depending on negation, defensive intent, scene context, or image-text composition.
  • Motivation: Most existing evaluations emphasize harmful compliance, refusal, or final-response harmfulness, limiting diagnosis of multimodal safety failures.These evaluations cover image-conditioned risks and over-refusal but generally reduce safety to outcome-level behavior.
  • Motivation: Outcome scoring can conflate evidence-grounded safety with keyword-triggered refusal, missed visual hazards, over-refusal, and other failures across the safety pipeline.It cannot localize failures in perception, OCR, cross-modal binding, intent inference, policy conditioning, or response generation.
  • Framework: EviSafe evaluates whether models behave safely, report relevant textual and visual evidence, identify risk sources and policies, and adapt when safety-critical evidence changes.Counterfactual interventions provide a behavioral check against unfaithful or post-hoc explanations.
  • Benchmark: EviSafeBench contains 1,181 gold scenarios and 2,452 targeted variants, evaluated through natural-response, evidence-reporting, and counterfactual-response probes.The benchmark spans eight safety domains and includes gold decisions, evidence annotations, response policies, and interventions.
  • Findings: Across eleven VLM runs, final-response safety, evidence grounding, and counterfactual sensitivity remain substantially misaligned.The authors present this as evidence that refusal-style metrics hide failures in multimodal safety reasoning.

2 Related Work

Prior work evaluates language and multimodal safety through harmful-compliance, jailbreak, refusal, robustness, privacy, and over-refusal benchmarks. EviSafe builds on multimodal perception, automated judging, evidence grounding, and counterfactual evaluation while targeting evidence-to-policy behavior.

  • Safety benchmarks: LLM safety benchmarks standardize harmful-compliance, jailbreak-robustness, and refusal evaluations.Multimodal benchmarks extend these settings to image-conditioned risks, typographic prompts, and compositional image-text jailbreaks.
  • General multimodal evaluation: General multimodal benchmarks measure perception, OCR, reasoning, and instruction following, capabilities that affect whether models read and bind safety-relevant evidence.Automated judges support scalable evaluation of open-ended multimodal outputs.
  • Evidence and counterfactual evaluation: Evidence-grounded and counterfactual studies test whether answers depend on intended visual evidence rather than dataset priors or superficial changes.Prior methods include counterfactual settings and scene-graph perturbations for relational and compositional consistency.
  • EviSafe's position: EviSafe adapts evidence-grounded and counterfactual evaluation to multimodal safety by linking safety decisions with text evidence, visual evidence, cross-modal relations, and response policies.This positions the benchmark as a diagnostic extension of existing safety and multimodal evaluation paradigms.

3 EviSafeBench: Core Design

EviSafeBench represents each image-text scenario with a four-way safety decision, risk source, evidence, cross-modal relation, response policy, and counterfactual variants. Its staged construction, targeted interventions, three probes, and evidence-aware judging test both safety outcomes and their multimodal basis.

  • 3.1 Task Definition: Each scenario is x = (I, T), pairing an image with user text, and its gold annotation records the expected decision, risk source, evidence, relation, policy, and counterfactuals.The four decisions are safe_answer, safe_answer_with_caution, unsafe_warn, and unsafe_refuse.
  • 3.1 Task Definition: Unlike outcome-only datasets, each scenario links the safety decision to decision-relevant textual and visual evidence and a corresponding safe-response policy.Gold evidence denotes quality-controlled observable cues, not hidden causal truth or model-internal reasoning.
  • 3.2 Dataset Construction: EviSafeBench is constructed through decision labeling, evidence reporting, and counterfactual construction, followed by validity checks and adjudication.The final release contains 1,181 originals and 2,452 counterfactuals after reducing larger intermediate pools.
  • 3.2 Dataset Construction: The benchmark covers seven unsafe or high-stakes domains plus benign-sensitive over-refusal controls, with eight risk-source types tied to intervention design.Risk sources include text, OCR, objects, scenes, cross-modal relations, benign-sensitive traps, and ambiguous or insufficient evidence.
  • 3.2 Dataset Construction: Counterfactuals intervene on annotated decision evidence: 85.5% modify text, OCR, or localized objects, while 12.9% modify scene or context.Variants test unsafe-to-safe transitions after removing unsafe evidence and stability when irrelevant surface features change.
  • 3.4 Evaluation Protocol: The protocol uses natural-response, structured evidence-report, and counterfactual-response probes, separating user-facing behavior from diagnostic reporting and evidence sensitivity.A full attempted run contains 4,814 victim requests per model, while scenario metrics require all necessary outputs.
  • 3.5 Metrics: An external GPT-5.4 judge compares model outputs with gold annotations to score decisions, risk-source families, evidence fields, counterfactual behavior, and failure modes.Human audits reported 92.5% agreement for natural decisions, 88.0% for risk-source families, and 84.3% for evidence fields.
  • 3.5 Metrics: The metrics separate natural and diagnostic severity accuracy, unsafe compliance, risk-source family accuracy, evidence scores, and diagnostic consistency.Evidence scores average text evidence, visual evidence, cross-modal relation, and safe-response policy using correct, partial, and incorrect grades.

4 Experiments

Across eleven VLMs, outcome safety does not reliably align with evidence grounding, policy selection, or counterfactual adaptation. The diagnostic profile localizes major weaknesses in cross-modal reasoning and response-policy execution despite stronger cue detection.

  • Outcome safety: Only 9.8% of 12,459 judged records are evidence-grounded successes, compared with 29.8% heuristic successes.Among the two success categories, evidence-grounded successes account for 24.7%, showing that correct natural behavior often lacks adequate reportable grounding.
  • Failure localization: Under-refusal is the largest explicit failure category at 23.7%, while policy failure reaches 16.2% and complete failure 12.6%.The 7.9% aggregate over-refusal rate indicates that stronger refusal alone would shift errors rather than repair policy selection.
  • Failure localization: Text and visual evidence scores average 71.4 and 71.1, but cross-modal relation and safe-response policy scores fall to 43.6 and 51.1.The best SDC is 15.2 and the best RDC is 29.3, so marginal evidence recovery rarely becomes a jointly correct safety chain.
  • Probe dependence: Diagnostic recognition is prompt-dependent: Doubao gains 20.1 points under structured reporting, while LLaVA and Qwen3-VL-8B show reverse gaps of −12.4 and −8.3.Natural and diagnostic probes therefore expose complementary behavior rather than interchangeable measurements.
  • Counterfactual adaptation and scaling: Counterfactual robustness requires both policy change and preservation: Claude reaches 58.4 U2S but 62.4 preservation accuracy, whereas Qwen3-VL-235B reaches 81.8 preservation and 80.2 CBA.Scaling comparisons likewise show non-monotonic changes across attribution, diagnosis, natural severity, and counterfactual transition metrics.

5 Discussion, Limitations, and Conclusion

EviSafe separates natural safety behavior, evidence reporting, and counterfactual sensitivity to diagnose failures hidden by refusal-only evaluation. Its results and limitations support more informative auditing while stopping short of deployment certification.

  • Discussion: EviSafeBench distinguishes final-response safety from evidence grounding and counterfactual sensitivity, making previously conflated failure types more specific improvement targets.Low RSFA, ES, DSA–NSA, RDC or SDC, and U2S identify different localization, translation, consistency, and transition failures.
  • Discussion: A EviSafe run records responses, reported evidence, selected risk-source families, and behavior under targeted counterfactuals for model profiling and safety auditing.The framework supports comparisons across deployment trade-offs but explicitly stops short of deployment certification.
  • Discussion: 69.9 ES with 51.1 DSA, alongside 31.0 NSA and 61.1 UCR for Doubao-Seed-1.6-Vision, indicates an evidence-to-policy bottleneck rather than cue blindness alone.The pattern illustrates why evidence recognition and user-facing policy execution should be evaluated separately.
  • Discussion: The study identifies evidence-sensitivity training with paired counterfactuals as future work but does not test a mitigation method.The proposed direction would complement existing instruction, preference, or rule-governed supervision.
  • Limitations: The claims concern reportable and behaviorally checked evidence grounding, not mechanistic faithfulness, and use natural-language visual evidence rather than box-level annotations.Other boundaries include counterfactual image artifacts, judge/configuration effects, model-dependent missingness, descriptive rare-category slices, and limited real-world prevalence estimation.
  • Limitations: The benchmark is intended for defensive safety evaluation, with redacted or schematic public examples and non-operational prompts rather than attack optimization.Human annotators should receive content warnings and skip options.
  • Conclusion: The conclusion is negative under EviSafe’s observable definition: evaluated VLMs are not reliably safe for the right multimodal reason.Comparatively strong final answers can coexist with weak risk-source and evidence consistency, while strong evidence reports can coexist with high unsafe compliance.

B Formal Experiment Protocol Details

The formal protocol standardizes multimodal safety evaluation across local and API-served VLMs, using three probes, external judging, normalization, and explicit metric definitions.

  • B.1 Formal Input Files: The OSS-URL formal suite contains 1,181 scenarios, 2,452 counterfactuals, and 3,633 HTTPS image references matching the canonical gold schema.A stratified 96-scenario pilot with 191 variants tunes prompts and tooling only; its results are not formal results.
  • B.2 Request Generation: Each scenario produces natural_original, evidence_report, and counterfactual_natural requests, yielding 4,814 victim requests and up to 1,181 judge requests per full run.Judge rows are created only when the natural, evidence-report, and counterfactual outputs are all available.
  • B.3 Model Serving and API Access: Seven open-source VLMs run locally through vLLM, while four additional models use OpenAI-compatible APIs with the same request and judging protocol.The API models are Gemini 2.5 Pro, Claude Opus 4.6, Doubao-Seed-1.6-Vision, and Qwen3-VL-235B-A22B-Instruct.
  • B.5 Judge Validation: The external judge compares model text with compact gold annotations under a fixed rubric and returns decisions, risk-source judgments, evidence scores, counterfactual judgments, and failure modes.A 100-output pilot audit checked rubric adherence; a 200-record blinded audit used two annotators and discussion-based disagreement resolution.
  • B.6 Retry and Normalization: Retryable API failures are retried with backoff, while unresolved failures remain in failure files and reduce judged sample sizes.Scenario-level rows count only when required victim outputs can be parsed and scored.
  • B.7 Metric Formulas: Natural severity accuracy and diagnostic severity accuracy are computed over parsed judge rows, while unsafe compliance evaluates safe/unsafe-family behavior on originally unsafe cases.Risk-source family accuracy maps eight risk-source labels into six families.
  • B.7 Metric Formulas: Diagnostic consistency requires correct decisions, risk-source judgments, and evidence fields; relaxed consistency additionally requires known evidence, cross-modal grounding, three nonzero fields, and average evidence score at least 0.5.Unsafe-to-safe transition equals original unsafe recognition multiplied by conditional adaptation, so it is diagnostic rather than a standalone ranking metric.

C Additional Results and Data Construction Details

EviSafeBench is assembled from public multimodal safety, preference, alignment, and benign-sensitive over-refusal datasets, preserving source provenance in normalized records.

  • Data sources: The raw pool combines unsafe and contextual-risk datasets, preference and alignment sources, and MOSSBench controls for benign-sensitive over-refusal.Source records are preserved in normalized JSONL form with source metadata.

C.2 Step 1: Natural-Response Expected Decision

Step 1 converts heterogeneous source records into four intended response decisions through source-label mapping, multimodal relabeling, agreement checking, and targeted adjudication.

  • Decision schema: Raw records are relabeled into safe_answer, safe_answer_with_caution, unsafe_warn, or unsafe_refuse.These labels distinguish direct answering, cautious answering, warning or redirection, and refusal of harmful requests.
  • Labeling procedure: Source labels provide prior evidence, GPT-5.4 labels each image-text pair, and an independent Qwen-VL pass supplies agreement information for conflicts.The Qwen-VL pass is an agreement signal rather than a second evidence generator.
  • Labeling procedure: 302 unresolved records are removed, leaving 9,097 records with deterministic labels.Records with label conflicts or low confidence enter a manual-adjudication pool.

C.3 Step 2: Evidence-Report Annotation

Step 2 annotates evidence and response policy, while Step 3 generates targeted counterfactual specifications; validity checks and the formal suite support consistent evaluation.

  • C.3 Step 2: Evidence-Report Annotation: GPT-5.4 generates structured reports containing text evidence, visual evidence, cross-modal relation, safe-response policy, and one of eight risk-source labels.The report is conditioned on the image, user text, and finalized expected decision.
  • C.3 Step 2: Evidence-Report Annotation: The cross-modal relation states whether text and image are individually safe or unsafe and what changes when they are combined.Safe-response policy remains separate from the relation to prevent evidence description from drifting into policy instruction.
  • C.4 Step 3: Evidence-Targeted Counterfactual Construction: Each evidence report determines one to three counterfactual specifications that alter the minimum evidence needed to test the decision.Specifications record the intervention type, changed evidence, purpose, and new expected decision.
  • C.4 Step 3: Evidence-Targeted Counterfactual Construction: Validity checks cover image access, decision-relevant OCR legibility, schema validity, and at least one usable counterfactual.They retain 8,689 full-pool scenarios and 1,488 candidates from a 1,500-record stratified sample.
  • Formal model suite: The formal model suite contains seven locally served open-source models and four API models evaluated with the same full protocol.Table 4 summarizes the configured suite.
  • Results overview: Local open-source results display distinct profiles across final-answer safety, evidence localization, risk-source attribution, diagnostic consistency, and counterfactual sensitivity.The inverted 1-UCR scale uses darker cells for lower unsafe compliance.

C.5 Gold Pool Selection

The gold pool is selected under coverage and quality constraints, then manually adjudicated before counterfactual images are finalized. The resulting benchmark uses retained scenarios and evidence-targeted variants while limiting reliance on artifact-sensitive whole-scene edits.

  • Selection constraints: Gold selection preserves unsafe cases, all four decision labels, all eight domains, all eight risk-source categories, rare risks, and every manually flagged record.
  • Initial pool: 911 of 1,200 initially selected scenarios were unsafe-label cases, while 289 were safe-label cases.
  • Counterfactual construction: Counterfactual construction produces visual interventions whose transition metrics distinguish unsafe-to-safe shifts from decision-preserving behavior.Table 5 defines U2S as OrigU multiplied by cU2S before rounding; Table 17 describes text-only and visual intervention types.
  • Adjudication: 64 manually flagged records yielded 45 retained records after decision-label correction and 19 removals, reducing the gold pool from 1,200 to 1,181 scenarios.
  • Artifact sensitivity: 85.5% of variants avoid whole-scene semantic editing, but this distribution is not itself a validity rate; scene-level edits remain the main artifact-sensitive subset.The finalized gold split contains 903 visual counterfactuals with generated images.

C.7 Benchmark Release Files

The release separates a canonical gold benchmark for primary experiments from an auxiliary full-candidate pool and documents its image-path and adjudication infrastructure. Supporting tables describe evidence-field results and source and domain composition.

  • Gold release: The canonical gold JSONL contains 1,181 original scenarios and 2,452 counterfactuals for all main experiments.
  • Gold release: A gold image-status copy records generated-image availability for the 903 visual counterfactuals.
  • Auxiliary release: The auxiliary full-candidate JSONL contains 8,670 scenarios and is not treated as fully visualized because non-gold visual counterfactuals may use placeholder paths.
  • Path conventions: All image paths are local project-relative paths, with original images under raw_public/images/ and generated images under generated_images/.A separate OSS-URL view can be generated when external inference platforms require public URLs.
  • Adjudication infrastructure: A local adjudication interface displays each scenario and records edits in an append-only change log, while skipped records are removed from the gold split.The tool also stores a merged JSONL file and progress metadata.

E Additional Notes on Data Safety

The paper separates local research materials from public-release artifacts and uses compact, redacted representations for sensitive multimodal safety data. Tables and schemas organize domains, decisions, risk sources, and counterfactual counts without exposing verbatim harmful records.

  • Release safety: Public reports are limited to aggregate statistics, redacted examples, synthetic illustrations, and non-operational evidence descriptions.Counterfactual descriptions specify changed evidence without procedural details for potentially harmful categories.
  • Schema design: The final gold schema stores evidence as strings at the object or scene level, illustrated by a benign household-safety abstraction rather than a verbatim dataset record.
  • Coverage: The benchmark organizes safety domains using abbreviations for physical, cyber, fraud, hate, sexual, health, high-stakes, and benign-sensitive categories.
  • Documentation: Its documentation separately specifies expected-decision distributions, risk-source taxonomy, final domain distribution, and counterfactual counts.
  • Coverage: The benign-sensitive domain is smaller because the raw source pool contains fewer such records, but it is fully retained for over-refusal analysis.
Loading 2608.23313v1…