PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking Jailbreak Guardrails for Embodied Agents

Xunguang Wang, Qingyue Wang, Yuguang Zhou, Zongjie Li, Wenxuan Wang, Shuai Wang

arXiv:2610.06122v1cs.AI

TL;DR

Jailbreak attacks can induce embodied agents to perform physically harmful actions, while existing benchmarks assess the agents rather than their guardrails. This paper compares guardrails on a fixed agent backend and finds a trade-off among defense, usability, and efficiency, with no single guardrail dominating in all settings.

  • Problem

    Existing embodied-safety benchmarks evaluate the embodied models rather than attached guardrails, leaving their system-level defense, usability, and runtime performance unclear.

    Source: §1 · pp. 1–2

  • Method

    The paper evaluates six representative guardrails as modules on a fixed agent backend under jailbreak attacks and safe instructions, measuring defense, usability, and efficiency.

    Source: §1 · p. 2, §6 · p. 10

  • Results

    The evaluation finds a consistent three-way trade-off among defense, usability, and efficiency, and no single guardrail dominates in all settings.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    The analysis offers practical guidance for selecting and designing guardrails based on intervention stage, decision mechanism, and input modality.

    Source: §1 · p. 2, §6 · p. 10

    All experiments run in the AI2-THOR simulator, so no physical robot is operated and real-world harm is not tested.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Embodied agents powered by large language models and vision-language models are increasingly deployed in physical environments, but jailbreak attacks can induce these agents to perform physically harmful actions. A growing number of guardrail methods have been proposed to intercept dangerous behavior before it is executed, yet existing safety benchmarks evaluate the embodied models themselves, leaving it unclear how well these guardrails actually defend an embodied agent in practice. We present the first systematic evaluation of jailbreak guardrails for embodied agents. To compare guardrails under identical conditions, we build a pluggable evaluation framework that treats the embodied agent as a fixed backend and each guardrail as a module that can intervene at the perception, planning, or control stage. We subject six representative guardrails to template-based and automated jailbreak attacks as well as safe instructions, and assess them at the system level along three dimensions: defense effectiveness, measured by the bypass rate and the hazard success rate in the simulator; usability, measured by the false-positive rate and the task completion rate on safe instructions; and efficiency, measured by the latency overhead added at runtime. Experiments on guardrails that span different intervention stages, decision mechanisms, and input modalities reveal a clear trade-off among the three dimensions, and show that no single guardrail dominates in all settings. We further analyze how intervention stage, decision mechanism, and input modality shape safety outcomes, and we offer practical guidance for selecting and designing guardrails for embodied agents.

1 INTRODUCTION

Jailbreaks can drive embodied agents to cause physical harm, but existing safety benchmarks evaluate the agents rather than the guardrails deployed around them. This section introduces a fixed-backend framework for comparing guardrails and reports a trade-off among defense, usability, and efficiency.

1 INTRODUCTION

  • Embodied agents translate natural-language instructions into physical actions, making them useful in physical environments but exposing them to harmful instructions.Potential harms include starting a fire, damaging property, or injuring a person.Source: §1 · p. 1
  • Jailbreaks can bypass current embodied-agent safety mechanisms and cause dangerous actions in simulation.The cited examples include template-based BadRobot and automated RoboPAIR and POEX attacks.Source: §1 · p. 2
  • Existing embodied-safety benchmarks assess the embodied model itself, leaving no systematic way to compare attached guardrails on jailbreak resistance, mistaken blocking, and runtime overhead.The paper identifies these deployment questions as the gap its evaluation addresses.Source: §1 · p. 2
  • The framework compares six guardrails and a no-guardrail baseline on template-based and automated jailbreaks and safe instructions, using one fixed agent pipeline.Guardrails can intervene at the perception, planning, or control stage; evaluation measures defense effectiveness, usability, and efficiency.Source: §1 · p. 2
  • The evaluation finds a three-way trade-off among defense, efficiency, and usability, with no guardrail dominating all three.The reported findings also connect outcomes to input modality, intervention timing, and decision mechanism.Source: §1 · p. 2
  • The paper contributes a taxonomy of guardrails across intervention phase, decision mechanism, input modality, and replanning capability.Its evaluation spans six representative guardrails and five attacks across two categories, alongside safe instructions.Source: §1 · p. 2

2 PRELIMINARIES

Embodied agents receive observations, select actions, and interact with environments over time while aiming to complete tasks safely. A jailbreak attacker seeks to use crafted instructions to make the executed actions realize a harmful goal, under stated limits on access and control.

2.1 EMBODIED AGENTS AND ENVIRONMENTS

  • At each time step, an embodied agent receives an observation, selects an action through its policy, and receives a new observation after the environment transitions.The task is to complete assigned goals while adhering to safety constraints and avoiding unsafe behavior.Source: §2.1 · p. 3

2.2 JAILBREAK ATTACKS

  • A jailbreak succeeds when the agent’s executed action sequence reaches an environment state that fulfills the attacker’s harmful goal, whether or not the agent recognizes the instruction as harmful.The attacker crafts a malicious instruction from an unsafe instruction or adversarial goal and the current environment.Source: §2.2 · p. 3
  • The attacker can craft prompts and may know the agent policy and environment dynamics, but cannot access internal model state or directly modify the policy, dynamics, or environment.The attack acts through instructions or prompts that exploit the agent’s decision-making process.Source: §2.2 · p. 3

3 JAILBREAK GUARDRAILS

This section distinguishes internal safety enhancements from external guardrails, organizes guardrails by design choices, and defines a system-level framework for evaluating them. The framework assesses resistance to harm, runtime overhead, and effects on safe task performance.

3.1 JAILBREAK DEFENSE

  • Internal safety approaches use alignment or prompt-based mitigation, but either require costly fine-tuning or remain vulnerable to carefully optimized adversarial instructions.The paper contrasts these with external guardrails deployed around the embodied agent.Source: §3.1 · p. 3
  • External guardrails inspect information flowing through the agent pipeline and intercept items identified as harmful, without modifying the agent’s policy.Their plug-and-play nature and transparency on benign instructions make them a practical defense paradigm in this work.Source: §3.1 · pp. 3–4

3.2 JAILBREAK GUARDRAIL TAXONOMY

  • The taxonomy classifies guardrails by intervention phase, decision mechanism, input modality, and replanning capability.These dimensions distinguish when checks occur, how safety judgments are made, which inputs they use, and whether interception triggers a new plan.Source: §3.2 · pp. 4–5

3.3 GUARDRAIL EVALUATION FRAMEWORK

  • The framework compares guardrails as pluggable modules on a fixed agent backend, testing unsafe instructions from different attack categories and safe instructions.It assesses defense effectiveness, efficiency, and usability.Source: §3.3 · p. 5
  • Defense effectiveness uses bypass rate and hazard success rate; lower values indicate fewer unsafe episodes pass through and fewer harmful outcomes are realized.The hazard success rate can be lower than the bypass rate because a passed instruction does not always lead to successful execution.Source: §3.3 · p. 5
  • Efficiency is measured as guarded-agent latency divided by unguarded-agent latency, while usability measures false positives and safe-task completion.A latency ratio of 1.0 means zero overhead; usability is evaluated on safe instructions.Source: §3.3 · p. 5

4 EXPERIMENT

The evaluation compares six guardrails and a no-guardrail baseline on safe and unsafe household tasks using a fixed embodied-agent backend. Across two backends, guardrails trade off defense, usability, and efficiency; no method leads on every metric.

4.1 EVALUATION SETUP

  • The study uses 300 safe and 300 unsafe SafeAgentBench household instructions in AI2-THOR, testing five jailbreak attacks against six guardrails and a no-guardrail baseline.Tasks span more than 80 floor plans; GPT-4o and DeepSeek-V3.2 serve as planning models.Source: §4.1 · pp. 5–6

4.2 EVALUATION RESULTS

  • RoboSafe has the lowest average bypass rate (7.24%) and hazard success rate (5.75%); without a guardrail, average hazard success is 47.74%.RoboSafe outperforms second-best MADRA (22.08% / 13.41%) on these averages.Source: §4.2 · p. 6, Table 2 · p. 6
  • On safe tasks, Llama Guard (Perception, T+V) has the lowest false positive rate (1.67%), while AgentSpec has the highest guardrail task completion rate (63.76%).RoboSafe intercepts 91.00% of safe instructions and has the lowest completion rate (11.33%).Source: §4.2 · p. 7, Figure 1 · p. 7
  • EMBGuard (Perception, T) has the lowest latency ratio (0.96); Llama Guard and EMBGuard variants are near 1.0 (0.96–1.15), while RoboSafe is highest at 4.45.The ratio reflects end-to-end wall time, including episodes that terminate after interception.Source: §4.2 · p. 7, Figure 1 · p. 7
  • The leaderboard order is identical for GPT-4o and DeepSeek-V3.2: Llama Guard (Perception, T) ranks first, RoboSafe (Control, T) last, and no guardrail leads all five metrics.RoboSafe pairs the best defense scores with the worst usability and efficiency scores.Source: §4.2 · pp. 7–8, Figure 2 · p. 8

4.3 PRACTICAL INSIGHTS & IMPLICATIONS

  • Adding egocentric images consistently worsens defense for Llama Guard and EMBGuard at perception and planning stages, with average bypass rates rising across all four comparisons.For example, Llama Guard’s planning-stage bypass rate rises from 44.42% to 66.11%.Source: §4.2 · p. 7, §4.3 · p. 8
  • The text+vision variants are gentler on safe instructions, while RoboSafe combines the strongest defense with 91.00% false positives and 11.33% task completion.The authors identify intervention aggressiveness as a central design-calibration goal.Source: §4.3 · p. 8
  • Planning-stage checks wait for a complete plan, control-stage checks occur after execution begins, and guardrail-only latency ranges from 0.00s for AgentSpec to 22.81s for RoboSafe.RoboGuard reaches latency ratios up to 3.81 because rejected plans trigger another replanning round.Source: §4.3 · p. 9

5 RELATED WORK

Related work traces embodied agents from LLM-based planning toward visually grounded models, alongside attacks that increasingly automate jailbreak instruction generation.

LLM-powered Embodied Agents

  • LLMs serve as reasoning cores for embodied agents, progressing from high-level planning to feedback-driven execution, memory, and multi-agent collaboration.Examples include grounding instructions in affordances, programmatic robot APIs, and feedback loops (Huang et al., 2022; Yao et al., 2023; Song et al., 2023).Source: §5 · p. 9
  • Multimodal agents add visual grounding, spanning sensor inputs to vision-language-action models and generalist embodied systems.Examples include Driess et al. (2023), Zitkovich et al. (2023), Octo Model Team et al. (2024), Kim et al. (2024), Li et al. (2024), Gemini Robotics Team (2025), and Bjorck et al. (2025).Source: §5 · p. 10

Jailbreak Attacks for Embodied Agents

  • Jailbreak research moves from hazardous-instruction datasets and handcrafted templates to automated attacks that optimize malicious prompts or action-level instructions.Examples include RoboPAIR, POEX (Lu et al., 2024), Blindfold, and Liu et al. (2024).Source: §5 · p. 10

6 CONCLUSION

The paper presents a systematic evaluation framework and taxonomy for embodied-agent jailbreak guardrails, then reports a three-way trade-off and design insights.

6 CONCLUSION

  • Across six guardrails, five attacks, and two agent models, the evaluation finds a consistent trade-off among defense, usability, and efficiency.The authors identify naive multimodal fusion as diluting the safety signal, earlier intervention as intercepting sooner, and decision mechanism as determining guardrail cost.Source: §6 · p. 10
  • The authors used generative AI to implement parts of the framework and draft or edit manuscript sections, and manually reviewed AI-assisted work.They state that they take responsibility for the final content.Source: AI USE STATEMENT · p. 10
  • All experiments ran in the AI2-THOR simulator, with no physical robot operation, human subjects, personal data, or other sensitive information.The evaluation relied on the publicly available SafeAgentBench dataset.Source: ETHICS STATEMENT · p. 10
  • The authors release framework code, attack implementations, metric scripts, dataset-generation details, experimental settings, and reproduction instructions for every table and figure.The experiments use either publicly available SafeAgentBench data or datasets generated by the released scripts.Source: REPRODUCIBILITY STATEMENT · p. 10

A EVALUATION ON DEEPSEEK-V3.2

The DeepSeek-V3.2 evaluation shows defense differences across attacks, a recurring usability–efficiency trade-off, and weaker defense from text-plus-vision variants.

Defense Effectiveness

  • 41.91% of malicious instructions reach harmful outcomes without a guardrail, while MADRA falls to mid-tier defense and is especially vulnerable to BadRobot-CJ and RoboPAIR.MADRA records 47.91% / 24.60% overall and bypass rates of 61.33% on BadRobot-CJ and 73.88% on RoboPAIR; the authors suggest its deliberative detection is more sensitive to attack realization across backends.Source: A EVALUATION ON DEEPSEEK-V3.2 · pp. 13–14
  • 22.54% average bypass rate and 15.07% hazard success rate make RoboSafe the strongest defense; EMBGuard perception-stage text input records 32.06% / 17.47%.Table 3 reports bypass rate / hazard success rate (%) for guardrails under different jailbreak attacks; lower values indicate better defense.Source: A EVALUATION ON DEEPSEEK-V3.2 · p. 13, Table 3 · p. 14
  • Text+vision raises average bypass rates over text-only for both model-based guardrails at perception and planning stages, with hazard success rates increasing alongside them.For example, EMBGuard perception-stage bypass rises from 32.06% to 65.06%, while its hazard success rate rises from 17.47% to 29.55%.Source: A EVALUATION ON DEEPSEEK-V3.2 · p. 14

Usability

  • 74.92% of safe instructions are intercepted by RoboSafe, whose task completion rate is 20.74%, compared with 61.87% for the no-guardrail agent.Llama Guard at perception is the gentlest guardrail, with false positive rates of 2.68% (T) and 1.67% (T+V).Source: Figure 4 · p. 14, A EVALUATION ON DEEPSEEK-V3.2 · p. 15
  • Planning-stage checks produce more false alarms than perception-stage checks, while adding images lowers model-based false positives and improves completion rates.For text-only Llama Guard, false positives are 18.39% at planning versus 2.68% at perception; EMBGuard planning completion rises from 45.15% to 50.67% with images.Source: A EVALUATION ON DEEPSEEK-V3.2 · p. 15

Efficiency

  • 20.40 is RoboSafe’s latency ratio, while RoboGuard and MADRA reach 15.05 and 4.70; Llama Guard planning with text is the exception at 0.91.The figure defines latency ratio relative to the no-guardrail agent across shared tasks, including blocked episodes; its dashed line marks the baseline.Source: Figure 4 · p. 14, A EVALUATION ON DEEPSEEK-V3.2 · p. 15
Loading 2610.06122v1…