Source-linked AI summary
When Tool Outputs Become Commands: Separating Action Induction from Runtime Authorization in Tool-Augmented LLM Agents
Xiaokun Guo, Zhen Xu, Dongdong Huo, Yanqiu Zhang, Wei Wang, Qinfu Yang, Dongjin Yu, Yu Wang
TL;DR
Tool-augmented agents need untrusted Observations for open-ended tasks, but action-inducing tool outputs can drive effects beyond user intent when induction is confused with authorization. SARA separates these roles through persistent action-origin tracking and audited execution evidence, limiting ASR to no more than 0.63% across four primary settings while preserving competitive utility. Its scope is limited to tool-based IPI under a threat model that trusts the user, schemas, runtime, and executor.
Problem
Open-ended agents need untrusted Observations to instantiate legitimate actions, but those Observations can also induce unauthorized tool execution and create a security–utility trade-off.
Method
SARA uses a context-isolated Action Probe for persistent action-origin provenance and authorizes calls using the user objective plus audited evidence from authorized successful executions.
Results
SARA limits Attack Success Rate (ASR) to no more than 0.63% across the four primary evaluation settings and provides consistent security gains across Agent backbones.
Takeaways & Limitations
Separating action induction from execution authorization preserves legitimate runtime adaptation while preventing action origins from becoming execution authority through historical recurrence.
Takeaways & Limitations
SARA is not a formal security guarantee, and its evaluation is restricted to tool-based IPI under a threat model trusting user inputs, tool schemas, the runtime, and executor.
Abstract
from arXiv · showhide
Tool-augmented LLM agents must rely on untrusted runtime Observations to complete open-ended tasks; however, when tool outputs no longer merely provide data but begin to specify concrete actions, they effectively become ``commands'' that can drive real-world side effects beyond user intent. We argue that this risk arises from conflating action induction with execution authorization. To address this distinction, we propose SARA, which treats action induction and execution authorization as distinct runtime roles and separates action provenance from execution authority. On the Observation side, a context-isolated Action Probe exposes action-inducing semantics and persistently records action-origin provenance across steps as a review signal; on the execution side, actual tool calls are authorized only against the user objective and audited evidence from authorized successful executions, while satisfying goal, execution-chain, and argument-level support. To preserve this separation across multi-step execution, SARA applies No-History-Promotion to prevent historical recurrence from laundering action origins into execution authority. Across AgentDojo and AgentDyn, SARA limits ASR to no more than \(0.63\%\) across four primary evaluation settings while maintaining competitive task utility, and consistently reduces ASR across additional Agent backbones.
1 Introduction
Tool-augmented agents need untrusted Observations to solve open-ended tasks, but those Observations can also induce unauthorized real-world actions. SARA separates action induction from execution authorization while preserving runtime adaptability and competitive utility.
- Motivation: Open-ended tool-using agents can access external resources and execute operations with real side effects.Their environments include operating systems, web browsers, databases, emails, files, and external services.
- Motivation: Attackers can embed operational content in webpages, emails, or documents that influences subsequent tool execution and produces external effects.This extends indirect prompt-injection risk from generated content to environmental consequences.
- Problem: Dynamic tasks create a security–utility trade-off because restricting Observation influence can reduce attacks while weakening legitimate runtime adaptability.Legitimate subsequent actions may not be fully determined in advance.
- Problem: Runtime information may instantiate existing user authority, but an Observation’s operational request may instead attempt to expand that authority.SARA distinguishes these cases rather than excluding all Observation-derived information.
- Approach: SARA uses a context-isolated Action Probe to retain action origins and independently audited execution evidence for authorization at the real tool boundary.Authorization checks goal, execution-chain, and argument-level support without filtering raw Observations.
- Evaluation: SARA substantially reduces unauthorized real tool execution while maintaining competitive task utility and consistent gains across Agent backbones.The paper evaluates the system on AgentDojo and AgentDyn alongside representative defenses and component analyses.
2 Problem Definition and Threat Model
The paper defines indirect prompt injection success at the real tool-execution boundary: Observation influence may form candidate calls, but it does not itself establish permission. The threat model therefore permits runtime task information while requiring non-escalating, persistent authorization under explicit trust assumptions.
- Problem Definition: A candidate tool call may be influenced by an untrusted Observation without thereby receiving normative execution authority.The distinction separates action induction from permission under the user request and prior legitimate execution.
- Attack Success: An indirect prompt injection succeeds only when an Observation-induced unauthorized call reaches the real executor and produces an external effect.Reading attack content, generating a plan, or having a rejected call does not independently count as success.
- Authorization Properties: Runtime authorization must allow Observations to provide objects, identifiers, and environmental facts for existing tasks without expanding the user-granted authority scope.This is the paper’s Authority Non-Escalation property.
- Authorization Properties: Action-inducing provenance must persist across normal execution, contextual propagation, and time rather than disappearing automatically.This Persistent Action Origin requirement prevents provenance from being implicitly cleared.
- Authorization Properties: Historical recurrence may add execution evidence but cannot independently override an existing action origin or promote it into authority.This is the stated Origin Non-Override rule.
- Threat Model: The model trusts the user request, tool schema, SARA runtime, and real executor, while treating webpages, emails, documents, and tool Observations as untrusted.The user request is the authorization root and the tool schema defines accessible interfaces and argument structure.
3 Related Work
Prior work progresses from input filtering and behavioral constraints toward action attribution and runtime authorization. SARA focuses on a finer provenance distinction: external content may induce actions, while legitimate execution must establish separate evidence for later authorization.
- Indirect Prompt Injection: IPI research has expanded from indirect influence through external content to attacks involving real tool semantics and benchmarked Agent workflows.AgentDojo and related work evaluate user tasks together with attack goals.
- Defense Evolution: Existing defenses constrain inputs, executable behavior, execution structure, task consistency, trusted control flow, or action origins.Examples include structured channels, tool-action filtering, execution constraints, and causal attribution.
- Runtime Authorization: Runtime-authorization methods analyze whether candidate calls have sufficient authority at the real execution boundary using dependency or task-level rules.These approaches move beyond input filtering toward execution-time control.
- SARA’s Distinction: SARA distinguishes action-inducing provenance from dynamic information that must pass through trusted execution before supporting authorization.Its novelty is preserving both forms of evidence simultaneously.
- SARA’s Distinction: Historical reappearance and cross-step propagation must not implicitly promote action-inducing provenance into new execution authority.This provenance-control requirement sharpens the separation between what content induced and what execution established.
4 SARA: Separating Action Induction from Runtime Authorization
SARA separates action induction from execution authorization: Observations may instantiate user-authorized tasks, but cannot independently expand execution authority. It combines persistent action-origin review with an authorization root and audited execution evidence at the real tool boundary.
- Design Overview: Action Induction ≠ Execution Authorization: runtime Observations can help instantiate existing tasks without creating new authority.SARA distinguishes runtime instantiation, such as discovering a file identifier, from authority expansion, such as adding an unauthorized recipient.
- Constructing the Authorization Root: SARA constructs authorization root K from the user request as the task-level upper bound on permitted effects, operations, scope, and static arguments.K preserves open-ended flexibility while preventing runtime information from creating new task authority.
- Evidence-Driven Authorization at the Execution Boundary: At the execution boundary, SARA authorizes candidate calls using K, persistent action origins Ft, and audited successful-execution evidence Ht against goal, chain, and argument support.No-History-Promotion prevents later historical reappearance from erasing an existing action-inducing origin.
- Action Induction and Persistent Provenance Retention: The context-isolated Action Probe classifies Observations as STATIC or ACTIONABLE and maps actionable content to tool footprints and argument-origin anchors when possible.It records action-inducing provenance rather than deciding whether content is malicious or authorized.
- Action Induction and Persistent Provenance Retention: ACTIONABLE content moves the trajectory to EXPOSED and retains action origins across later searches, reads, and resolution steps for persistent, differentiated review.EXPOSED is a review obligation, not an attack verdict or global block.
5 Experimental Setup
Experiments on AgentDojo and AgentDyn evaluate SARA’s security, utility, cross-backbone stability, component contributions, and inference cost. SARA consistently suppresses unauthorized execution while preserving task utility, though mechanisms and backbone capabilities shape the security–utility trade-off.
- End-to-End Security and Task Utility: SARA reduces ASR across four primary settings while keeping UA no lower than the corresponding Agent-only baseline.With GPT-4o-mini, ASR falls to 0.06% and 0.17%; with Gemini-2.5-Flash-Lite, it falls to 0.62% and 0.63% on AgentDojo and AgentDyn, respectively.
- End-to-End Security and Task Utility: SARA authorizes calls using action origins, audited execution evidence, and actual argument bindings rather than relying on a single restrictive criterion.This execution-boundary control distinguishes action formation from the authority ultimately granted to a tool call.
- Security Gains and Utility across Backbones: Across 8 backbone–benchmark combinations, SARA substantially decreases ASR, while BU and UA vary more with backbone and workflow dynamism.On AgentDojo, BU changes by no more than 2.53 percentage points; on AgentDyn, UA decreases by 4.44–6.94 percentage points for all four backbones.
- Contributions of Key Runtime Mechanisms: Removing AIT raises ASR to 3.37% and 3.02% with less than 1 percentage point BU change, showing the security role of persistent action-origin tracking.AIT preserves the provenance of earlier Observation-induced actions across steps.
- Contributions of Key Runtime Mechanisms: Removing PSG raises ASR to 2.49% and 2.73% despite higher BU and UA, showing that goal and execution-chain support alone cannot prevent argument-level authority expansion.The veto may conservatively reject legitimate calls with ambiguous authorization boundaries.
- Security–Utility Gains and Additional Inference Cost: SARA uses more inference tokens but forms competitive security–utility operating points, with total inputs of 25.86K on AgentDojo and 43.61K on AgentDyn.These totals correspond to ASR values of 0.06% and 0.17%, respectively; authorization analysis is a primary source of the overhead.
6 Conclusion and Limitations
SARA separates action induction from execution authorization to mitigate unauthorized tool execution while preserving legitimate runtime adaptation. Across AgentDojo and AgentDyn, it limits ASR to no more than 0.63% across four primary evaluation settings, but remains a runtime mechanism rather than a formal security guarantee.
- Conclusion: SARA decouples action induction from execution authorization and tracks Observation-induced action provenance across steps.Authorization is reevaluated at the tool boundary using user authorization and audited execution evidence.
- Conclusion: SARA preserves support for legitimate runtime adaptation while mitigating unauthorized tool execution caused by untrusted Observations.
- Limitations: SARA is a runtime authorization mechanism rather than a formal security guarantee, and its semantic judgments may produce false positives or false negatives.Utility after blocked executions depends on the host Agent’s replanning capabilities and workflow dynamics.
- Limitations: The evaluation is limited to tool-based Indirect Prompt Injection workflows under a threat model assuming trusted user inputs, tool schemas, runtime, and executor.The stated scope excludes direct attacks bypassing the authorization runtime, pure data-dependency vulnerabilities, and unconditional control delegation to external sources.