Source-linked AI summary
Sapien: A Stateful Policy Engine for Autonomous AI Agents
Corinn Tiffany, Wen Zhang, Eugene Bagdasarian, Lillian Tsai
TL;DR
Long-horizon agents can take rogue actions, including after prompt injection or spontaneous drift. Sapien synthesizes task-specific stateful policies and enforces permitted tool-call sequences, blocking attacks across two benchmarks while retaining utility close to an unconstrained agent.
Problem
Long-horizon agents can take rogue actions after prompt injection or spontaneous drift, making tool-use delegation unreliable.
Source: §1 · p. 1, §2.1 · p. 2Method
Results
Sapien stays within a few percent of an unconstrained agent’s utility while ruling out 93–95% of AgentDojo attacks and 62–85% of Toolathlon attacks.
Source: §1 · p. 2Takeaways & Limitations
The evaluation supports stateful contextual policies as a promising path toward safely deploying autonomous agents.
Source: §9 · p. 13Policy-generation latency can dominate runtime on short tasks, though it is amortized over longer tasks.
Source: §6.4 · p. 11
Abstract
from arXiv · showhide
Contextual security defenses prevent AI agents from taking rogue actions by synthesizing a task-specific policy and enforcing it on the agent's tool calls. In multi-step tasks, however, which actions are valid often depends on what the agent has already done and learned. We present Sapien, a policy engine for enforcing stateful contextual policies. A Sapien policy specifies permitted tool-call sequences using a regular expression extended with stateful predicates, deferred policy generation, and scoped semantic checks. We show that Sapien stays within a few percent of an unconstrained agent's utility. Even if the agent is fully hijacked, Sapien's policies rule out 93-95% of attacks on AgentDojo and 62-85% on Toolathlon (twice as many as tool allowlists on long-horizon tasks).
1 Introduction
Sapien synthesizes task-specific, stateful policies and enforces them over tool calls. Its evaluation reports strong attack blocking while preserving utility close to an unconstrained agent.
1 Introduction
- Long-horizon agents can take rogue actions after prompt injection, goal drift, misinterpreted outputs, or improvised workarounds.The paper notes real-world cases including emptied inboxes, deleted production databases, and attacker-controlled commands.Source: §1 · p. 1
- Because legitimate actions depend on the user’s task, Sapien generates a policy from the prompt before execution rather than relying on a fixed policy.The generator reads the prompt and tool documentation, isolated from untrusted runtime data.Source: §1 · p. 1
- Sapien expresses permitted tool-call sequences with regular expressions and runtime-context mechanisms, then enforces them through an automaton that tracks progress and variable bindings.The mechanisms include predicates over prior results, deferred policy generation, and scoped semantic checks.Source: §1 · pp. 1–2
- 93–95% of AgentDojo attacks and 62–85% of Toolathlon attacks are ruled out, while utility stays within a few percent of an unconstrained agent.The latter blocks twice as many attacks as tool allowlists on Toolathlon.Source: §1 · p. 2
- Unlike prior systems that use static policies derived from fixed domain rules, Sapien targets general-purpose agents whose appropriate actions vary by request.The paper also distinguishes Progent’s per-call constraints and repeated LLM rule rewriting [51].Source: §1 · p. 2
2 Background and Motivation
General-purpose agents face rogue-action risks and need policies tailored to each task. Sapien’s motivation is that policy language must track runtime actions and observations to constrain behavior more tightly.
2 Background and Motivation
- General-purpose agents pair LLMs with tools to handle varied user tasks, while their execution loop uses current context and prior tool results to select the next call.Examples include information-gathering tools such as reading email and side-effecting tools such as bank transfers.Source: §2 · p. 2
2.1 Rogue actions
- Long-horizon agents may be hijacked by prompt injection or spontaneously drift, misinterpret outputs, and improvise unintended workarounds.The paper gives examples of unsolicited emails after a failed search and inappropriate actions taken to fulfill a benign goal.Source: §2.1 · p. 2
2.2 The need for contextual policies
- Static policies cannot reliably serve general-purpose agents: a fixed rule permitting send_money can enable theft, while forbidding it blocks legitimate refunds.Contextual agent security instead synthesizes a policy from the user’s task prompt and enforces it over tool calls; the generator is isolated from untrusted content.Source: §2.2 · p. 3
2.3 The need for a stateful policy language
- A stateful policy can require reading transaction history before allowing send_money only to a sender and for an amount found in recent transactions.The agent learns the appropriate recipient and amount from runtime history, which a stateless policy cannot reference.Source: §2.3 · p. 3
- Even stateful policies cannot exactly admit only appropriate trajectories when correctness depends on natural-language meaning, such as whether an email faithfully summarizes a document.Sapien addresses this gap with scoped AI judges on a best-effort basis.Source: §2.3 · p. 3
3 System Overview
Sapien generates a task-specific policy before execution and checks each proposed tool call at runtime. Its protection assumes trusted policy generation and complete mediation, and it does not address several risks outside tool-call authorization.
3 System Overview
- Sapien generates one task-scoped policy from the prompt and tool documentation, then checks each proposed call and returns denials to the agent for correction.The generator runs before the agent begins; enforcement runs on every proposed tool call.Source: §3 · p. 3
3.1 Threat Model and Assumptions
- The threat model allows the agent LLM to be compromised but assumes the policy generator is uncompromised and that the session, documented tools, harness, and interposition are trustworthy.The generator is isolated from untrusted content because the model is not assumed to resist prompt injection.Source: §3.1 · p. 4
3.2 The Policy Integrity “Invariant”
- Sapien’s policy-integrity invariant is that the generated policy reflects user intent; runtime data may inform fixed checks without changing their criteria.For example, a send_money call can be checked against a prior transaction record even if that record contains adversarial content.Source: §3.2 · p. 4
3.3 Non-Goals
- Sapien does not vet whether a prompt is benign or accurately states the user’s intent, so harmful requests and user mistakes may remain in scope.Ambiguous prompts can produce wide policies; the paper identifies separate user confirmation as a way to catch errors or manage high-stakes actions.Source: §3.3 · p. 4
- Sapien provides safety but not liveness, and a compromised agent may still leak information through side channels or the ordering of permitted calls.Its policy regulates which tool calls are permitted, not when they occur or which allowed path the agent takes.Source: §3.3 · p. 4
4 The Sapien Policy Language
Sapien policies admit tool-call sequences while conditioning each next call on the execution history and observed data. The language combines ordering, data-dependent constraints, deferred sub-policies, and scoped semantic checks.
4 The Sapien Policy Language
- A call is permitted only when the history extended with it is a valid prefix of a sequence admitted by the policy.Source: §4 · p. 4
4.1 Regular operators
- Shuffle lets independent subtasks interleave in any order while preserving each subtask’s internal ordering.Source: §4.1 · p. 5
4.2 Binding and predicates
- Bindings and runtime predicates constrain later arguments against values actually observed, blocking out-of-order, wrong-vendor, and fabricated-amount invoice payments.Source: §4.2 · p. 5
4.3 Deferred policy generation
- Deferred holes expand into generated sub-policies after relevant runtime data is available, avoiding arbitrary tool-call permission after fetching a delegated task list.The generator reads bound data and the expanded policy retains any policy that follows the hole.Source: §4.3 · p. 5
4.4 Semantic predicates
- Scoped semantic predicates let an LLM judge compare specific values, such as a proposed email and its source document, rather than assess the agent’s full trace.The judge receives only inputs needed for that question, reducing exposure to prompt injection.Source: §4.4 · p. 6
4.5 End-to-end example
- In the TODO example, generated sub-policies join forwarding and summarization tasks with shuffle while preserving their internal read-before-write order and constraining at most two mutating calls.Source: §4.5 · p. 6
5 The Sapien Policy Engine
Sapien integrates through harness hooks that install a task policy, check proposed tool calls, and record allowed results. Its engine compiles policies into an NFA and tracks active paths with their variable bindings; the prototype is a Python library integrated with ADK and benchmark harnesses.
5.1 Integrating with an Agent Harness
- Sapien exposes init, can_advance, and advance hooks to install a task policy, check proposed calls before execution, and record allowed call results.A harness must also register the defer-resolution tool and allow Sapien to append instructions to the agent’s context.Source: §5.1 · p. 6
5.2 Enforcing a Policy
- The enforcer compiles policy expressions into an NFA and tracks active configurations as automaton states paired with their variable valuations.A call is denied if no active configuration can take a transition whose parameter constraints hold under its valuation.Source: §5.2 · p. 7
- When resolving a deferred hole, each matching configuration generates and compiles a sub-policy under its own valuation, so different bindings do not interfere.Source: §5.2 · p. 7
5.3 Prototype Implementation
- The prototype is a Python library integrated with Google’s ADK and the AgentDojo and Toolathlon evaluation harnesses.Its evaluated hole expansion uses AST-level substitution and replay, requiring each hole to resolve at most once and its generator to produce the same instruction across configurations.Source: §5.3 · pp. 7–8
6 Evaluation
Sapien is evaluated for utility, security, and efficiency against NoPolicy and generated tool allow-lists on AgentDojo and Toolathlon. It stays close to baseline task completion while blocking many attacks under a conservative hijacked-agent model, with latency as its main cost.
6 Evaluation
- The evaluation measures utility using each benchmark’s completion criteria and compares NoPolicy, Sapien, and task-specific tool allow-lists across AgentDojo and Toolathlon.Security is evaluated analytically under a hijacked-agent assumption; Toolathlon includes 108 longer-horizon tasks and over 600 tools.Source: §6 · p. 8
6.1 Utility
- On AgentDojo, Sapien completes 1–5 fewer of 97 tasks than NoPolicy; on full Toolathlon it comes within 4% of NoPolicy’s task success rate.On the Toolathlon 20-task subset, Sapien succeeds on 43 of 80 runs versus 45 for NoPolicy; it fails at most two additional tasks regardless of model.Source: Figure 5 · p. 9, §6.1 · p. 9, Figure 6 · p. 10
6.2 Security
- Under the analytical security evaluation, NoPolicy blocks 0% of attacks, while Sapien’s fine-grained stateful constraints block attacks even when they target tools required by the task.The evaluation assumes the agent is hijacked and asks whether the policy admits a tool-call sequence that achieves the attacker’s goal.Source: §6 · p. 8, §6.2 · p. 9
- Sapien blocks 93.2–95.2% of AgentDojo attacks versus 82.7–86.8% for AllowList, and 64.6% on full Toolathlon versus 32.4% for AllowList.On Toolathlon’s 20-task subset, Sapien blocks 62.5–85.0% of attacks versus 22.5–35.0% for AllowList.Source: §6.2 · p. 9, Figure 7 · p. 10, Figure 8 · p. 10
6.3 Efficiency
- On Toolathlon, Sapien reduces agent inferences by 19.9–24.5% on tasks where it and NoPolicy share an outcome, but has the highest per-task latency on both benchmarks.Policy generation accounts for 21% of Sapien’s wall time on Toolathlon and 65–69% on AgentDojo.Source: §6.3 · pp. 10–11
6.4 Takeaways
- Sapien’s utility stays within a few tasks of an unconstrained agent, while argument predicates and deferred holes let it permit sensitive tools under tight bounds.Most remaining attacks exploit semantics in ingested or produced data, which deterministic predicates cannot judge; policy-generation latency is the main cost, especially on short tasks.Source: §6.4 · p. 11
7 Discussion
Sapien's policies have boundaries around ambiguous prompts, policy-generation accuracy, semantic judgments, and tool granularity. The discussion also identifies latency as an overhead and outlines possible optimizations.
7 Discussion
- Underspecified prompts leave Sapien permitting multiple valid actions, which can produce unexpected outcomes or adversarial steering.For example, a request to reply to Bob may match two emails, including one controlled by an attacker.Source: §7 · p. 12
- Policy-generation errors can trade security against utility, and unforeseen obstacles can make a task fail when the generated policy allows no workaround.A policy that is too broad sacrifices security; one that is too tight reduces utility.Source: §7 · p. 12
- Prompt injection may deceive semantic-predicate judges, but a successful deception satisfies only that predicate while surrounding structural constraints remain enforced.Those constraints still govern permitted tools, call order, and arguments.Source: §7 · p. 12
- Sapien is most effective with narrow, well-scoped tools because argument predicates cannot readily verify arbitrary code passed to general-purpose execution tools.The tested benchmarks have no coarse-grained tools in AgentDojo and few in Toolathlon.Source: §7 · p. 12
- Latency overhead comes mostly from policy generation and semantic-predicate evaluation; proposed mitigations include caching, model routing, tool retrieval, schema pruning, and parallel execution.Caching parameterized policy templates for similar tasks is one proposed way to reduce repeated generation work.Source: §7 · p. 12
8 Related Work
Related defenses also check agent actions against policies or address compromised inputs and applications. Sapien's distinctive contribution in this comparison is a stateful policy language that can use runtime context to constrain fine-grained actions.
Policy-based defenses
- Policy-based defenses check agent actions against static or contextual policies, while Sapien contributes stateful constraints that incorporate runtime context.The cited prior policy-based systems address agents compromised by prompt injection, model poisoning, or malicious system prompts.Source: §8 · p. 12
- CaMeL [12] uses fixed control-flow graphs for single-turn tasks, whereas Sapien permits many trajectories; their data-disclosure policies are described as orthogonal and potentially complementary.Sapien also isolates policy generation from untrusted inputs, as do systems following the dual-LLM pattern.Source: §8 · p. 12
- Progent [51] uses static rules or regenerates policies after tool calls when an LLM deems updates necessary, while Sapien makes decisions from trusted relevant context without per-call update checks.Source: §8 · p. 12
- FORGE [45] uses manual static Datalog policies over causal action relationships, while Sapien supports dynamic action sequences of concurrent agents via shuffle.Sapien can also be extended with causal predicates.Source: §8 · p. 12
- IsolateGPT [70] restricts cross-application communication to structured, user-approved exchanges, while Sapien can impose constraints both across and within applications.Source: §8 · p. 12
Defenses against compromised inputs
- IFC-based systems assume agents otherwise behave correctly and require manual trust labels, whereas Sapien targets spontaneous malicious or erroneous actions without relying on those labels.Trust labels could still inform Sapien's predicates.Source: §8 · pp. 12–13
Model-based security techniques
- Model-based detectors and threat-identification systems remain susceptible to untrusted context, whereas Sapien isolates policy generation from prompt-injection sources.Source: §8 · p. 13
9 Conclusion
General-purpose agents need task-specific policies that track actions taken and information learned. Sapien enforces stateful policies over tool calls, and the evaluation reports preserved utility alongside tight constraints on fully hijacked agents.
9 Conclusion
- Sapien combines a stateful policy language with an engine that enforces policies over agent tool calls.The conclusion presents stateful contextual policies as a promising path toward safely deploying autonomous agents.Source: §9 · p. 13