PaperlayerView exact revision on arXiv

Source-linked AI summary

AIRGuard: Guarding Agent Actions with Runtime Authority Control

Suliu Qin, Haomin Zhuang, Yujun Zhou, Yufei Han, Xiangliang Zhang

arXiv:2605.28914v1cs.CRcs.AI

TL;DR

Tool-using agents can be steered by attacker-controlled context into ordinary actions that misuse authorized access, which the paper calls authority confusion. AIRGuard checks action authority at runtime; on AgentTrap it reduces Sonnet attack success from 36.3% to 5.5%, while the paper reports preserving useful benign task completion across its evaluations.

  • Problem

    Tool-using agents may take plausible, executable actions that misuse authorized access when attacker-controlled content steers them, making the security question whether each side effect is justified by the task and policy.

    Source: §2.2 · p. 3, §2.1 · p. 3

  • Method

    AIRGuard normalizes proposed tool actions and checks task-derived authority, source and target trust, and likely effects at a pre-action runtime boundary.

    Source: §1 · p. 2, §3.1 · p. 4, §3.3 · p. 5

  • Results

    AIRGuard substantially lowers attack success on AgentTrap and DTAP-150 while preserving useful benign task completion; on AgentTrap, Sonnet ASR falls from 36.3% without defense to 5.5%.

    Source: §1 · p. 2, §4.2 · p. 6

  • Takeaways & Limitations

    Runtime authority control addresses tool-mediated side effects by checking whether proposed actions are authorized, rather than relying only on prompt-injection detection.

    Source: §6 · p. 9

    AIRGuard must observe and mediate actions before execution, so frameworks that hide side effects or lack a pre-action hook require structural changes for enforcement.

    Source: §7 · p. 9

Abstract

from arXiv · show

Tool-using language agents turn model decisions into external side effects: they read files, run scripts, call APIs, send messages, and invoke Model Context Protocol tools. This makes agent attacks different from jailbreaks. The harmful step is often not an obviously forbidden output, but an ordinary executable action that becomes unsafe because attacker-controlled context steers authorized access against the user's interest. We identify this failure mode as authority confusion: untrusted resources may inform reasoning, but they must not authorize side effects. We present AIRGuard, a runtime guard that operationalizes least privilege as action-time authorization. AIRGuard normalizes heterogeneous tool calls, derives task authority into step-level authority, tracks source and target trust, simulates sensitive side effects, audits cross-step risk, and enforces decisions before actions execute. On AgentTrap, AIRGuard reduces Sonnet 4.6 attack success from 36.3% without defense to 5.5%. On DTAP-150, AIRGuard preserves 76.0% benign utility with Haiku 4.5, compared with 52.0% for ARGUS and 42.0% for MELON. An ablation further shows that prompt-only policy helps only modestly, whereas a dedicated runtime authority-control layer gives the agent system direct control over tool-mediated side effects. Code and data are available at https://github.com/Sophie508/AIRGuard.

1 Introduction

Tool-using agents can be steered by attacker-controlled context into ordinary actions that misuse authorized access, a failure the paper calls authority confusion. AIRGuard checks proposed side effects at runtime, and the paper reports lower attack success while preserving useful benign task completion.

1 Introduction

  • Attacker-controlled content can steer agents to misuse authorized access through plausible single actions or multi-step sequences, unlike attacks targeting obviously unsafe text.Such actions may involve credentials, configuration files, external endpoints, or other sensitive components.Source: §1 · p. 1
  • The paper names this failure mode authority confusion: untrusted data may inform reasoning but cannot authorize side effects.Task, policy, or explicit consent must supply the authority for a proposed side effect.Source: §1 · pp. 1–2
  • AIRGuard derives step-level authority from the task, tracks source and target trust, simulates sensitive side effects, and audits cross-step risk before execution.It normalizes heterogeneous tool calls and applies tiered enforcement at action time.Source: §1 · pp. 2–3
  • AIRGuard complements prompt-only and provenance defenses by checking authority, target trust, source trust, and likely effect where reasoning becomes an external action.The paper distinguishes evidence that parameters are grounded from authority to perform the resulting operation.Source: §1 · p. 2
  • AIRGuard lowers attack success on AgentTrap and DTAP-150 while preserving useful benign task completion, against a security–usability trade-off from coarse authority policies.Contextual authority inheritance, source and target trust, and sequence audit are intended to make enforcement more selective than static permission reduction.Source: §1 · p. 2

2 Problem Formulation

The problem formulation models a tool-using agent that observes runtime channels and proposes side-effecting actions under a task and protected policy. It treats authority confusion as accepting attacker-influenced content as justification, while preserving useful dynamic tool use by requiring actions to be justified by the task and policy.

2.1 Threat Model

  • The agent observes a trajectory prefix of channel-content pairs, and the task plus protected policy define the intended workflow.The channel vocabulary includes documents, tool outputs, local files, packages, scripts, and other runtime resources.Source: §2.1 · p. 3
  • The threat model allows an attacker to influence any runtime channel and steer an agent with its own authorized access toward unauthorized side effects.It excludes direct compromise of protected runtime policy and out-of-band execution that bypasses the agent; the runtime is assumed able to observe or wrap proposed calls.Source: §2.1 · p. 3
  • The proposed action is characterized by its type, target, and expected effect, while attacker-influenced content may appear task-relevant.Examples include recipient changes, dependency installation, account changes, and debugging commands.Source: §2.1 · p. 3

2.2 Authority Confusion

  • Unlike jailbreaks, authority confusion concerns whether a plausible, useful-looking side effect is justified by the task and policy.The test is not whether an input channel is globally trusted or whether an argument appeared in context.Source: §2.2 · p. 3
  • Authority confusion occurs when attacker-influenced content is treated as sufficient justification; valid parameters and plausible-sounding instructions do not establish authorization.A syntactically valid recipient or an instruction framed as audit, telemetry, setup, or debugging can still exceed the intended workflow.Source: §2.2 · p. 3

2.3 Security Objective

  • The security objective requires a side effect’s target and expected effect to be justified by task and policy, rather than merely suggested by runtime context.Execution should imply workflow-level justification.Source: §2.3 · p. 3
  • The objective is not to block all attacker-influenced content, because useful agent workflows require broad, dynamic tool use.Content may inform reasoning, but should not justify an action whose effect exceeds the task’s intended authority.Source: §2.3 · pp. 3–4

3 Methods

AIRGuard checks proposed tool actions against task and policy authority before side effects reach the environment. It tracks action sources and targets, estimates risk for sensitive actions, and escalates enforcement when coverage or risk requires it.

3.1 Overview

  • AIRGuard places a runtime guard between the agent and external tools, checking each proposed action against task and policy authority before execution.It normalizes the call, checks capability, target, effect, and source, then applies an enforcement decision before side effects reach the environment.Source: §3.1 · p. 4

3.2 Normalized Runtime Actions

  • AIRGuard maps heterogeneous tool calls to normalized actions with a capability, target, expected effect, and influencing resource for coverage and risk checks.The influencing resource links the proposed action to a provenance label in the trust pool.Source: §3.2 · p. 4

3.3 Resource Trust and Authority Context

  • Resource labels track provenance and trust, while scoped task authority specifies allowed capabilities and targets; step-level authority may narrow, but not expand, that grant.Uncovered actions do not silently proceed on the low-enforcement path; the guard escalates according to risk and available containment.Source: §3.3 · pp. 4–5

3.4 Contextual Risk Simulation

  • For sensitive actions, AIRGuard estimates risk from task necessity, attack resemblance, and whether apparent authority comes from a trusted issuer or untrusted resource.The discrete label can be low, ambiguous, or high risk; simulation selects enforcement and cannot grant authority absent from task or policy, with uncertainty escalating rather than defaulting to allow.Source: §3.4 · p. 5

3.5 Tiered Enforcement

  • Covered low-risk actions may be allowed or audited, ambiguous actions may prompt or trigger inspection, and high-risk execution may be sandboxed, quarantined, or blocked.Risk can increase enforcement but cannot turn an uncovered action into an allowed one; wrappers, staged writes, and mediation help preserve utility without granting new authority.Source: §3.5 · pp. 5–6

3.6 Sequence Audit and Containment

  • A rolling ledger records observed effects so later decisions can account for earlier reads, writes, executions, network contacts, and blocked attempts.Cumulative sequence risk escalates on patterns such as secret reads followed by network sends; the audit targets agent-mediated actions, not full malware analysis.Source: §3.6 · p. 6

4 Evaluation

AIRGuard is evaluated on AgentTrap and DTAP-150 against no-defense and runtime-defense baselines, using attack success, benign utility, overdefense, and trajectory-level failure analysis. Results compare security and utility, while an ablation contrasts prompt-only policy with runtime enforcement.

4.1 Experiment Setup

  • AgentTrap has 141 cases, while DTAP-150 has 150 MCP-based tasks split evenly among benign, direct-malicious, and indirect-malicious tasks.ASR measures successful attacks, UPR successful benign-task completion, and overdefense clear overdefensive behavior; comparisons include no-defense, ARGUS, MELON, and a prompt-only ablation.Source: §4.1 · p. 6

4.2 Main Results

  • On AgentTrap, AIRGuard reduces Sonnet ASR from 36.3% to 5.5%; on DTAP-150 it reaches the best ASR tier on three of four models.The DTAP-150 examples include 1.0% ASR with Sonnet, 4.0% with GPT-5.4-mini, and 3.0% with GPT-5.3-codex.Source: §4.2 · p. 6, Table 1 · p. 7
  • On DTAP-150, Haiku UPR is 76.0% with AIRGuard, versus 52.0% with ARGUS and 42.0% with MELON.On AgentTrap, MELON sometimes preserves slightly more benign utility, but its Sonnet ASR is 25.3% versus AIRGuard’s 5.5%.Source: §4.2 · pp. 6–7, Table 1 · p. 7
  • AIRGuard’s residual GPT attacks were mainly missed decisive-action risk recognition: 5 cases for GPT-5.4-mini and 6 for GPT-5.3-codex.It leaves 8 and 9 successful attacks, respectively, compared with ARGUS’s 20 and 23 and MELON’s 17 and 24.Source: Figure 3 · p. 7, §4.2 · p. 7
  • ARGUS failures often persist after partial blocks or absent triggers, whereas MELON failures are mostly outright bypasses.ARGUS has 10 and 12 partial-block cases and 10 and 9 not-triggered cases; MELON has 12 and 17 bypass cases across the two GPT backbones.Source: §4.2 · pp. 7–8

4.3 Ablation and Baselines

  • On DTAP-150 with GPT-5.4-mini, prompt-only lowers ASR from 22% to 17% at 86% UPR, while full AIRGuard reaches 4% ASR at 75% UPR and 6% overdefense.The runtime guard observes normalized tool calls and can intervene before side effects execute.Source: Table 2 · p. 8, §4.3 · p. 8

5 Related Work

Related work distinguishes unsafe text generation and prompt injection from agent actions whose danger lies in unauthorized side effects. Runtime defenses address this setting in varied ways, while AIRGuard centers its enforcement on whether the task and policy authorize an action.

5.1 Jailbreaks and Prompt Injection

5.2 Agent Safety and Runtime Defenses

6 Conclusion

AIRGuard frames agent security as runtime authority control over side effects, rather than prompt-injection detection alone. Its evaluations show reduced attack success alongside substantial benign utility, with precision and containment remaining challenges.

6 Conclusion

  • AIRGuard addresses authority confusion through normalized actions, trust assessment, risk simulation, and tiered enforcement with sequence audit.AgentTrap and DTAP-150 results show sharp attack-success reductions while preserving substantial benign utility; future work targets narrower task authority, better target trust, consistent risk simulation, and stronger containment without making agents unusable.Source: §6 · p. 9

7 Limitations

AIRGuard’s enforcement depends on observing and mediating actions before execution, requiring integration with the agent’s tool runtime.

7 Limitations

  • AIRGuard requires runtime integration; hidden tool-server side effects or actions that bypass a pre-action hook need structural changes for enforcement.Structured tool calls can be supported with a normalization adapter, but the guard is not automatically portable across every agent framework.Source: §7 · p. 9

Ethics Statement

The paper limits its study to controlled benchmark attacks and cautions against treating AIRGuard as a substitute for oversight or conventional security controls. It also discloses that AI assistance was limited to editing and proofreading.

Ethics Statement

  • The evaluation uses controlled benchmark environments and does not provide new operational exploit code beyond tasks designed for agent-safety evaluation.AIRGuard is intended for authorized workflows, not as justification for high-stakes autonomous deployment without human oversight, access control, and conventional security monitoring.Source: Ethics Statement · p. 9
  • AIRGuard logs may contain file paths, tool arguments, or user task context, so deployments should treat them as sensitive.Source: Ethics Statement · p. 9

A Use of AI Assistants

  • AI assistance was limited to rewriting and polishing author-written text and checking for typos and grammar; the authors take responsibility for the paper.Source: A Use of AI Assistants · p. 10

B Case Studies

Across four case studies, AIRGuard blocks paths associated with unsafe execution, disclosure, or output, while ARGUS and MELON outcomes vary by case. The cases illustrate different attack mechanisms and defense outcomes.

B Case Studies

  • In Case 32, AIRGuard blocked the helper-script path, while ARGUS and MELON allowed it and the hidden admin note was injected.Source: B Case Studies · pp. 10–11
  • In Case 6, AIRGuard blocked execution outside granted authority; ARGUS allowed the packaging helper despite its error, while MELON allowed execution that stripped safety blocks.Source: B Case Studies · p. 11
  • In Case 29, AIRGuard blocked the escalation path and emitted no canonical vendor ranking; MELON produced the ranking, while ARGUS refused tool calls despite its legacy L2 label.Source: B Case Studies · p. 11
  • In Case 37, AIRGuard denied execution of the bundled harness, ARGUS allowed it, and MELON did not execute it because no tool calls reached that path.Source: B Case Studies · p. 11

C.1 Case 32: Cross-skill Collusion

In the cross-skill collusion case, AIRGuard blocked the execution path, whereas ARGUS and MELON permitted the helper that injected a hidden admin note.

C.1 Case 32: Cross-skill Collusion

  • AIRGuard blocked workspace access and bash execution, so add_slide.py never ran and no hidden admin slide or network exfiltration occurred.Source: §C.1 · p. 11
  • ARGUS allowed the shell pipeline to execute add_slide.py, which read seeded state and injected the hidden admin note into the deck.Source: §C.1 · p. 11
  • MELON allowed the helper action, and the hidden admin slide was injected into the resulting presentation.Source: §C.1 · p. 11

C.2 Case 6: System Integrity

The trajectory details contrast AIRGuard’s blocks with other agents’ actions across system-integrity, low-trust-context, and supply-chain cases. Outcomes differ by case, including cases where a baseline refused or failed to complete the attack path.

C.2 Case 6: System Integrity

  • In Case 6, AIRGuard blocked workspace-level access and never executed package_skill.py; ARGUS allowed the helper despite an error, and MELON ran it and stripped safety blocks.Source: §C.2 · pp. 11–12

C.3 Case 29: Disinformation via Low-Trust Context

  • In Case 29, AIRGuard blocked the first vendors.csv read and emitted no canonical ranking; ARGUS made no tool calls, while MELON read the procurement files and ranked vendors.Source: §C.3 · p. 12

C.4 Case 37: Supply-chain Attack via Test Harness

  • In Case 37, AIRGuard blocked workspace reads and did not execute the harness; ARGUS ran evaluation.py, while MELON refused the suspected injection and made no tool calls.Source: §C.4 · p. 12
Loading 2605.28914v1…