PaperlayerView exact revision on arXiv

Source-linked AI summary

GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI

Zhesheng Zhang, Jiahao Lu, Wei Liu, Cong Pan, Jianhua Yang, Yixiang Chen, Hongyuan Yu, Mengqi Zhang, Kailin Lyu, Zhumin Chen, Keji He

arXiv:2608.21928v1cs.AIcs.CLcs.RO

TL;DR

Embodied agents must assess whether an instruction is safe in a particular scene, a pre-execution decision that prior approaches do not directly measure. GuardianBench fixes scenes while contrasting safe and hazardous instructions, revealing instruction-insensitive VLM verdicts and supporting targeted verdict calibration.

  • Problem

    Embodied agents must determine whether an otherwise ordinary instruction becomes physically unsafe in the observed scene, a pre-execution decision not directly measured by textual-alignment or low-level control approaches.

  • Method

    GuardianBench uses standards-grounded same-scene Safe/Unsafe instruction pairs across 3,024 examples, with hazard intensity calibrated by a 5×5 Severity–Likelihood Latent Risk Matrix.

  • Results

    30.1% Safety-class accuracy versus 88.1% Utility-class accuracy across primary models reveals a strong permissive tendency alongside low Pair Accuracy; rationale audits localize errors to missed instruction-relevant cues.

  • Takeaways & Limitations

    GuardianBench provides a controlled measurement of instruction-conditioned pre-execution risk recognition, while VLOS supports targeted verdict calibration on open-weight backbones.

  • Takeaways & Limitations

    The benchmark evaluates pre-execution recognition rather than closed-loop control, temporal risk accumulation, recovery, action feasibility, or low-level actuation, and its main data are generated rather than photographed.

Abstract

from arXiv · show

In embodied AI, safety risk can be latent: a benign instruction and a safe scene become hazardous only when composed. Prior work has advanced embodied safety by varying visual contexts or evaluating execution-time dynamics, but the complementary axis of fixing the scene and varying only the instruction remains underexplored. We introduce GuardianBench, an instruction-contrastive benchmark grounded in international safety standards that isolates this latent contextual risk through 3,024 instruction-scene examples organized as same-scene Safe/Unsafe contrastive pairs across various hazard categories. Benchmarking state-of-the-art vision-language models (VLMs) reveals instruction-insensitive verdicts: models disproportionately approve both instructions under a given scene; across the primary models, average pair accuracy is only 24.1%. Our systematic rationale audit localizes the dominant failure: models fail to bind the instruction-relevant cues that differentiate safe from unsafe compositions. As a post-training case study, Verdict Log-Odds Supervision (VLOS), a lightweight verdict-level objective, substantially improves performance on open-weight backbones. Together, our latent contextual risk task formulation, standards-grounded contrastive benchmark construction, pair-level and rationale-level failure diagnosis, and benchmark-enabled verdict calibration establish GuardianBench as a controlled evaluation suite for exposing and improving safety reasoning over instruction-scene compositions under latent contextual risk.

1 Introduction

GuardianBench frames safety as a compositional property of instruction–scene pairs, then evaluates whether models change their verdict when only the instruction changes. Across 16 VLMs, the benchmark exposes instruction-insensitive decisions and supports verdict-level calibration.

  • Problem: Latent contextual risk arises when an otherwise safe scene becomes hazardous only through its composition with a particular instruction.The same living-room scene supports a safe coffee-table action but an unsafe windowsill action near curtains.
  • Task formulation: GuardianBench fixes the visual scene, swaps only the instruction, and tests whether the safety verdict flips in the correct direction.This controlled design isolates instruction-conditioned reasoning from image-level hazard cues.
  • Evaluation: 16 VLMs frequently assign the same verdict to both instructions under one scene, producing low Pair Accuracy despite strong utility-class accuracy.The evaluation also shows a strong permissive tendency, and REAL50 validation confirms directionally aligned rankings and permissive trends.
  • Benchmark construction: 3,024 expert-verified examples form 1,512 same-scene Safe/Unsafe pairs across 14 hazard categories derived from four international safety standards.Hazard intensity is calibrated with a 5×5 Severity–Likelihood Latent Risk Matrix.
  • Failure diagnosis: Rationale audits localize most errors to missed instruction-relevant cues that distinguish safe from unsafe compositions.The failure is therefore concentrated in binding the instruction to the visual evidence before producing the final verdict.
  • Calibration: VLOS directly supervises Safe/Unsafe verdict log-odds and improves the safety–utility balance on open-weight backbones while preserving structured rationales.The objective is used as a lightweight post-training auxiliary objective without an inference-time module.

2 Related Work

Embodied-safety research covers static hazards, planning, dynamic interaction, and changing visual contexts, but GuardianBench isolates the complementary case of fixed visual evidence with varied instructions. The paper targets this controlled pre-execution safety decision rather than low-level control or primarily textual safety.

  • Scope: Textual-alignment pipelines and low-level constraint-based control do not directly measure the controlled instruction–scene decision required before acting.The paper positions this assessment between semantic safety and physical execution safety.
  • Embodied safety: Prior embodied-safety benchmarks assess task-risk rates, hazard rejection, planning failures, dynamic interaction risks, anomalous scenes, and situational safety under changing visual contexts.These studies probe complementary physical-safety settings rather than the same-scene instruction contrast targeted here.
  • Controlled comparison: GuardianBench fixes the visual scene and contrasts benign versus hazardous instructions to test whether verdicts are instruction-conditioned under identical visual evidence.This reverses the static-image comparison used by MSSBench, which varies visual context for a fixed query.
  • Alignment context: Alignment methods span RLHF, offline contrastive objectives, and on-policy GRPO, while persistent safety–utility tension remains in safety tuning.Overly conservative tuning can produce over-refusal, whereas decoupled-reward approaches add safety critics and remain primarily textual.

3 GUARDIANBENCH

GUARDIANBENCH defines latent contextual risk as a property of instruction–scene composition and operationalizes it with same-scene Safe/Unsafe pairs. Its standards-grounded construction uses documented hazard sources, inverse hazard synthesis, risk thresholding, and expert annotation controls.

  • 3.1 Latent Contextual Risk Task: GUARDIANBENCH defines latent contextual risk as neither the instruction nor scene being hazardous alone, while their composition crosses the intolerable-risk threshold.The formal condition is HT(i)=0, HV(v)=0, and HC(i,v)=1.
  • 3.1 Latent Contextual Risk Task: 3,024 instruction–scene examples form 1,512 same-scene contrastive Safe/Unsafe pairs, preventing image-level cues from determining the label.Each scene is paired with a hazardous instruction and a safe counterpart.
  • 3.2 Standards-grounded Contrastive Construction: Four international safety standards supply 93 Hazard Origins and 14 expert-verified Hazard Categories for standards-grounded coverage.The documented provenance anchors unsafe examples in regulatory definitions and constrains category design to source-grounded safety concepts.
  • 3.2 Standards-grounded Contrastive Construction: Inverse hazard synthesis constructs scenes that are safe in isolation but become hazardous under a sampled instruction and hazard consequence.An expert-guided LLM acts as an inverse planner to infer the hazardous instruction and scene description before rendering.
  • 3.2 Standards-grounded Contrastive Construction: Risk labels threshold R=S·L on a 5×5 Severity–Likelihood matrix, retaining only hazardous instructions above the critical threshold and safe counterparts as Safe.Low-risk hazardous candidates are discarded rather than relabeled Safe, preserving separation between the binary class and borderline hazardous cases.
  • 3.2 Standards-grounded Contrastive Construction: Cohen’s κ=0.8316 on an independently re-annotated subset supports agreement on the derived safety label.Trained experts annotate severity and likelihood for hazardous candidates, while safe counterparts receive no independent risk score.

4 Benchmarking Experiments

GuardianBench evaluates 16 VLMs on same-scene Safe/Unsafe instruction pairs using structured rationales and contrastive metrics. Models show permissive, instruction-insensitive verdicts, while rationale audits identify missed cues as the dominant observable error signature.

  • Experimental Setup: 16 VLMs were evaluated on all 3,024 instructions using a unified three-stage rationale and final Safe/Unsafe verdict tag.Evaluations used greedy decoding, with verdicts parsed from the final safety tag.
  • Evaluation Metrics: Pair Accuracy measures pairs where both opposite verdicts are correct, while Flip Rate counts all pairs receiving different verdicts.The partition distinguishes correct flips, wrong-direction flips, and no flips; Pair Accuracy is the primary contrastive-consistency metric.
  • Verdict-Level Results: 88.1% average Utility versus 30.1% average Safety across 13 primary models reveals a permissive tendency under latent risk.Every primary model had Utility greater than Safety, indicating high benign-request compliance alongside weak unsafe-composition detection.
  • Verdict-Level Results: 24.1% mean Pair Accuracy and 30.0% mean Flip Rate show that models rarely reverse verdicts correctly across same-scene pairs.70.0% of pairs received no flip, while only 5.9% were wrong-direction flips; the dominant no-flip mode was Only-S.
  • External Validation: REAL50 overall accuracy correlated with the main benchmark at Pearson r=0.74, while Safety correlated at r=0.86 across primary models.REAL50 contains 50 real photos, 100 independently authored instructions, and 50 same-scene contrastive pairs.
  • Rationale-Level Error Audit: Cue mismatches dominated missed-hazard errors, and CUE was the largest first-unmet stage in every hazard category at 69–84%.Contamination and health categories raised RUL to 22–28%; the audit measures observable rationale evidence and does not exhaustively identify latent cognitive causes.

5 Post-Training Case Study: Verdict-Level Calibration with VLOS

VLOS adds direct verdict-level supervision to calibrate the Safe/Unsafe boundary while preserving rationale generation. Across Qwen and Ministral, it improves the safety–utility balance over standard GRPO and performs best among compared baselines.

  • Method: VLOS adds binary cross-entropy supervision to Safe/Unsafe log-odds at the verdict position, directly calibrating the verdict boundary.The method uses structured outputs and requires no teacher-forced rationales or additional classifier.
  • Baselines: GRPO reaches 99.0 Safety but 75.2 Utility on Qwen, while its sequence-level reward under-constrains the verdict boundary.On Ministral, GRPO corrects a permissive prior toward safety but leaves a gap, showing opposite failure directions across backbones.
  • Results: VLOS lifts Qwen Utility by +18.8 pp and Pair by +16.5 pp over standard GRPO while keeping Safety above 96.It tops Accuracy, H-mean, and Pair among Qwen baselines.
  • Results: VLOS adds +7.3 pp Safety and +2.6 pp Utility over GRPO on Ministral and tops every reported metric column.The same α=0.05 coefficient works on both backbones despite their opposite unaligned priors.
  • Ablations: Three-seed ablations show that VLOS requires two-sided verdict supervision, while α=0.05 provides the best tested safety–utility balance.Unsafe-only and safe-only variants underperform standard GRPO in H-mean and Pair despite high Safety.

6 Conclusion

GuardianBench exposes instruction-insensitive safety decisions by holding scenes fixed and varying instructions, and supports targeted calibration of the verdict boundary. Rationale audits identify missed instruction-relevant cues as the dominant observed failure signature.

  • Conclusion: GuardianBench fixes the scene and varies only the instruction, revealing VLMs that assign the same safety decision to both members of a contrastive pair.The benchmark contains 1,512 same-scene contrastive pairs grounded in safety standards.
  • Conclusion: Low Pair Accuracy and a strong permissive tendency show that models frequently fail to flip verdicts when only the instruction changes.Rationale audits further identify a dominant missed-cue signature before the final verdict.
  • Conclusion: VLOS demonstrates that GuardianBench labels can support targeted verdict calibration for instruction-conditioned safety reasoning.The conclusion presents this as a path toward improved safety reasoning in embodied systems.

Limitations

The benchmark evaluates pre-execution risk recognition from visual observations and instructions, not full embodied execution. Its generated household scenes enable tight same-scene control but limit external validity and deployment coverage.

  • Scope: GuardianBench asks whether an instruction should be executed from a visual observation, focusing on pre-execution risk recognition.The protocol scores the consequence of executing the instruction in the depicted scene rather than the image’s general danger.
  • Scope: The benchmark does not evaluate closed-loop control, temporal risk accumulation, recovery, action feasibility, or low-level actuation.These omitted capabilities define the nearest boundary of the controlled setting.
  • Data: Main data are generated rather than photographed because same-scene pairs require precise control over scene elements while keeping each image safe in isolation.REAL50 provides only partial external validation.
  • External validity: Larger real-photo, egocentric, and non-household evaluations are needed to characterize robustness under occlusion, viewpoint shift, and sensor noise.Scaling beyond the current 1,512 pairs while preserving same-scene control is identified as future work.
  • Scoring assumptions: Risk scores are conditional on executing the given instruction and must be grounded in visible scene facts rather than image-level danger.The rubric separately scores accident likelihood and severity after verifying the scene and action path.

A.2 Benchmark Composition and Hazard Taxonomy

GuardianBench’s hazard-triggering examples use fixed action and object taxonomies, structured reasoning complexity, and standards-grounded hazard domains. Most reference rationales combine multiple commonsense premises.

  • Corpus composition: The corpus composition is reported for 1,512 hazard-triggering pair members, whose action and object labels follow fixed closed coding rubrics.Residual rates are 0.4% for actions and 1.3% for objects.
  • Corpus composition: Instructions span 12 named action classes and 10 named target-object classes beyond small residual categories.Reasoning complexity counts distinct commonsense premises after binding the instruction to the scene.
  • Hazard taxonomy: 85.1% of reference rationales combine at least two premises, indicating that many examples require multi-step commonsense reasoning.The benchmark represents safety knowledge through both 14 standards-grounded hazard domains and instance-specific mechanisms.
  • Generation: Sample generation uses Gemini-family language models and Gemini-2.5-Flash-Image-Preview for text-to-image synthesis.The passage identifies the model families used for LLM generation and image synthesis.

B Positioning among Embodied Physical-Safety Benchmarks

GuardianBench isolates pre-execution latent contextual risk by fixing the visual scene while varying the instruction. It complements benchmarks focused on execution dynamics, broader danger perception, or varying visual contexts.

  • Benchmark scope: The benchmark landscape involves different trade-offs in control, interaction, modality coverage, and diagnostic granularity.Table B1 positions GuardianBench alongside representative embodied and physical-safety benchmarks.
  • Controlled positioning: The benchmark targets a pre-execution decision: whether an agent should execute or refuse given a visual observation and instruction.Interactive benchmarks remain better suited to execution-time dynamics, mitigation ordering, and closed-loop planning failures.
  • Controlled positioning: GuardianBench fixes the scene and varies the instruction, unlike MSSBench-Embodied, which varies visual context for a fixed query.The same image appears in Safe and Unsafe rows, preventing image-level hazard priors from solving the task alone.
  • Controlled positioning: GuardianBench differs from ASIMOV-2.0 by isolating benign-looking instructions that become unsafe only under the current visual context.It does not provide an explicit constraint list, requiring contextual risk inference before execution.
  • Real-image validation: REAL50 validates transfer beyond generated imagery using 50 real indoor images, 100 instructions, and 50 same-scene Safe/Unsafe pairs.The subset is assembled from ADE20K household photos and uses the same taxonomy and source safety standards as GuardianBench.

C.2 Accuracy Results and Correlation with GUARDIANBENCH

REAL50 provides an easier real-photo evaluation regime while preserving the main benchmark’s model-ranking and permissive-bias patterns. Pair-level decomposition shows that approving both instructions is the dominant failure mode.

  • Accuracy results: 73.0 mean overall accuracy, 64.4 mean Safety, 81.6 mean Utility, and 49.4 mean Pair Accuracy are reported across 16 REAL50 models.REAL50 evaluates 100 instructions across 50 same-scene pairs.
  • Accuracy results: 84, 83, and 82 are the top REAL50 accuracies for QWEN3-VL-235B-THINKING, CLAUDE-OPUS-4.5, and GEMINI-3-FLASH-PREVIEW, respectively.The leaderboard is sorted by overall accuracy.
  • Correlation with GuardianBench: REAL50 yields higher absolute accuracies than the main benchmark, with the largest difference on Safety.The comparison uses REAL50 minus main-benchmark accuracy and Safety deltas.
  • Correlation with GuardianBench: r=0.74 for overall accuracy and r=0.86 for Safety correlate primary-model results between GuardianBench and REAL50.The authors interpret these correlations as evidence that the main benchmark’s relative ranking signal transfers to real photos.
  • Pair-level decomposition: Only-S dominates across most models: models approve both instructions, correctly judging only the Safe counterpart.GPT-5.2 reaches 81.2% Only-S but 15.7% Pair Accuracy; primary-model Qwen3-VL-235B-Thinking has the highest Pair Accuracy at 38.6%.

E Error Analysis: Full Tables and Reliability

The audit decomposes errors by unmet cue, rule, verdict-consistency, and residual checks, while reporting unconditional rationale consistency and hazard-category patterns. Residual errors mainly reflect optimistic premises under ambiguity.

  • Audit rubric: The audit judge evaluates cue_match, rule_match, and decision_consistent using the canonical visual context, reference rationale, instruction, and model completion.The text-only judge does not redecide the benchmark label.
  • Deterministic mapping: CUE precedes RUL, which precedes VRI, so reported stage percentages represent first-unmet shares of total error mass.Table E2 provides the separate unconditional view of verdict–rationale consistency.
  • Reliability: 93.1% of error verdicts follow the risk assessment stated in the model’s own rationale, while inconsistency is 5.3% for missed hazards and 15.3% for over-warning.Across 12 audit-primary models, marginal consistency spans 86.1–98.5%.
  • Residual errors: RES accounts for 186 manually reread rows, including 173 where rationales state the hazard mechanism but resolve an open detail benignly.Typical premises assume safe execution, evaluate only the pre-action arrangement, or place admitted risk below threshold.
  • Hazard-category patterns: CUE is the largest stage in every hazard category at 69–84%, while contamination and health categories raise RUL to 22–28%.The pooled decomposition covers all 14 unified hazard categories.
  • Training setup: The reward design jointly encourages structured reasoning, correct safety labels, and non-degenerate response lengths.Malformed outputs receive formatting penalties, while correct predictions require a valid safety label.

F.5 Default and One-sided VLOS Ablation

VLOS improves verdict calibration when its BCE term supervises both Safe and Unsafe sides, whereas one-sided variants underperform the GRPO baseline. A coefficient sweep also favors nonzero supervision.

  • Default and one-sided ablation: Full VLOS improves H-mean and Pair accuracy over GRPO at essentially unchanged Safety.The comparison averages results over three random seeds on Qwen2.5-VL-7B-Instruct.
  • Default and one-sided ablation: Both one-sided VLOS variants fall below even the GRPO baseline.The result indicates that the verdict-level BCE term is effective only when supervising both sides of the label space.
  • Coefficient sensitivity: All four non-zero α coefficients substantially outperform GRPO on Acc, H-mean, Utility, and Pair, while Safety remains ≥96.7.The sweep uses a single seed and tests sensitivity to the VLOS coefficient.
  • Coefficient sensitivity: α=0.05 achieves the best Utility at 94.1 and Pair at 90.8 in the reported coefficient sweep.α=0 recovers standard GRPO under the same train/test and decoding configuration.
Loading 2608.21928v1…