Source-linked AI summary
Contextual Agent Security: A Policy for Every Purpose
Lillian Tsai, Eugene Bagdasarian
TL;DR
General-purpose agents operate across contexts where static policies may over- or under-permit actions, creating a need for more adaptable security. This paper proposes Conseca, which generates contextual policies from trusted context and enforces them deterministically; early results show better utility than a static restrictive policy and comparable utility to a static permissive policy.
Problem
Static, manually specified policies cannot account for every context, risking over- or under-permissioning as agents gain broad capabilities.
Method
Conseca uses a language model to generate just-in-time, contextual policies from trusted context and deterministically enforces them against proposed actions.
Results
Conseca offers better utility than a static restrictive policy and comparable utility to a static permissive policy in preliminary results.
Takeaways & Limitations
Contextual agent security can adapt policy granularity to broad agent purposes while restricting actions outside the current task policy.
Takeaways & Limitations
Conseca’s guarantees depend on policy quality, while limited trusted context restricts prediction of data-dependent actions.
Abstract
from arXiv · showhide
Judging an action's safety requires knowledge of the context in which the action takes place. To human agents who act in various contexts, this may seem obvious: performing an action such as email deletion may or may not be appropriate depending on the email's content, the goal (e.g., to erase sensitive emails or to clean up trash), and the type of email address (e.g., work or personal). Unlike people, computational systems have often had only limited agency in limited contexts. Thus, manually crafted policies and user confirmation (e.g., smartphone app permissions or network access control lists), while imperfect, have sufficed to restrict harmful actions. However, with the upcoming deployment of generalist agents that support a multitude of tasks (e.g., an automated personal assistant), we argue that we must rethink security designs to adapt to the scale of contexts and capabilities of these systems. As a first step, this paper explores contextual security in the domain of agents and proposes contextual agent security (Conseca), a framework to generate just-in-time, contextual, and human-verifiable security policies.
1 Introduction
General-purpose agents expose security systems to many purposes and contexts that static, manually written policies cannot fully anticipate. Conseca addresses this challenge with dynamically generated, contextual policies whose enforcement is deterministic and auditable.
- Static, manually specified policies can overrestrict or overallow actions because they cannot account for every possible context.
- As a system encounters more contexts, policy granularity must increase to avoid impairing utility or security through over- or under-permissioning.
- Conseca generates just-in-time, contextual, transparent policies specifying which actions are not harmful for a given purpose and context while restricting others.
- A language model dynamically produces fine-grained policies, with human-readable rationales that experts can audit to mitigate generation errors.
- Conseca isolates policy generation from adversary-controlled context, using trusted context to improve action prediction while resisting manipulation.
- The paper contributes a call for scalable security mechanisms, the Conseca framework, and a Linux computer-use prototype.
2 Background and Threat Model
Agents combine planning with execution through external tools, using context from user requests and tool responses to select actions. Untrusted context can therefore manipulate an unrestricted agent into harmful behavior.
- An agent consists of a planner that selects actions and an executor that runs them through external tools.
- Task-relevant context may include user requests, usernames, filenames, file contents, job information, and backup locations.
- Unrestricted agents may perform harmful actions accidentally or after prompt-injection attacks modify context.
- Conseca’s threat model lets adversaries alter part of the context while treating the agent as benevolent but potentially mistaken.
- The agent must sometimes use untrusted context, such as email contents, to fulfill the user’s request.
3 Design
Conseca generates a task- and context-specific policy from the user request and trusted context, then deterministically checks each proposed action. Its guarantees depend on policy quality and limited trusted context.
- Conseca generates a task- and context-specific security policy and evaluates whether each proposed action satisfies it.
- The policy generator uses the request, trusted context, and tool documentation, while the enforcer deterministically approves or denies proposed actions.
- Denied actions return rationales to the planner, whereas allowed actions proceed to execution and may produce further context for subsequent planning.
- Policies declaratively constrain tool APIs and include human-readable rationales for their constraints.
- LLMs provide scalable, context-sensitive policy generation, supported by in-context examples to improve policy quality.
- Conseca ensures that no external tool action executes outside the current policy under its stated trust and integration assumptions.
- The guarantees depend on policy quality and are constrained because limited trusted context can hinder prediction of data-dependent actions.
4 Proof-of-Concept
The proof-of-concept integrates Conseca with a Linux computer-use agent exposing filesystem, file-processing, and email tools. Policies map tool calls to execution constraints and rationales, including argument checks.
- The prototype is a Python Linux agent with filesystem, file-processing, and email tools exposed as bash commands.
- Without Conseca, the planner emits bash commands and the executor runs them until completion or a 100-command limit.
- The API includes set_policy for generating policies and is_allowed for returning an allow decision with a rationale.
- A policy specifies whether an API call may execute, a boolean constraint over its arguments, and a human-readable rationale.
- The example policy permits urgent work-email responses only under sender, recipient, and subject constraints and forbids email deletion for that task.
- The prototype represents argument constraints as regular expressions and assumes positional API parameters with optional arguments last and correctly ordered.
- The agent checks proposed commands and returns denied commands with rationales to guide replanning, stopping after up to 10 continuous failures.
- Adding email and a subset of the POSIX API required less than a day, but new tools require documentation and trusted-context extraction.
5 Case Studies
The prototype evaluates Conseca’s utility and security against static restrictive and permissive policies on subjective, multi-step filesystem and email tasks. Conseca completes 60% of tasks on average, matches permissive-policy utility, and denies contextually inappropriate forwarding.
- Evaluation setup: The evaluation uses 20 subjective, multi-step filesystem and emailing tasks, comparing Conseca with unrestricted, static restrictive, and static permissive policies.Tasks range from backing up important files to deduplicating files and emailing a report.
- Utility: Conseca offers utility comparable to the static permissive policy and higher utility than the static restrictive policy.Figure 3 averages results over five trials.
- Observed failures: Four failed tasks required more than 100 actions, and newsletter generation and failed-login checking were too complex for the basic agent.The failures involved permission checks, attachments, urgent emails, and report generation.
- Utility: Conseca achieves 60% completion on average, while no task completes under the restrictive policy because every task requires writes.The unrestricted agent completes 70% of tasks on average.
- Security: Conseca denies email forwarding for every task except acting on urgent emails, whereas the unrestricted agent forwards emails even when inappropriate.This demonstrates contextual denial while retaining higher utility than the restrictive policy.
6 Related Work
Related work positions Conseca among prompt-injection defenses, contextual integrity, access control, and anomaly detection. Its distinctive focus is generating contextual action policies while isolating policy generation from untrusted inputs.
- Prompt-injection defenses: Conseca isolates policy generation rather than planning, allowing the planner to receive full context while requiring few changes to existing agents.This builds on quarantined, isolated-model designs for resisting untrusted inputs.
- Security analysis: Unlike predefined-predicate security analysers, Conseca generates contextual policies independently of a specific policy language or enforcer.It can nevertheless leverage a security analyser.
- Context-aware AI: Conseca extends contextual-integrity ideas from data flows to enforcing contextually appropriate actions.The goal is preventing harm through action-level enforcement.
- Traditional application security: Conseca supports finer-grained access controls and capability restrictions as agent systems require broader permissions than conventional applications.Mobile permissions illustrate the difficulty of managing permissions across many contexts and tasks.
- Anomaly detection: Unlike anomaly-detection approaches based on manual specifications or labeled behavior models, Conseca creates a current-context behavior model with deterministic enforcement.The model is produced by a pretrained LLM rather than learned from labeled data.
7 Discussion
The discussion identifies open directions and boundaries for Conseca, including user interaction, trajectory-level constraints, generation overhead, and applicability beyond broad natural-language agent contexts.
- Future directions: Conseca currently lacks user overrides, undo logs, and other feedback mechanisms that could improve policy understanding and utility.The authors also suggest automated policy verification and sanitized tool or context outputs.
- Trajectory constraints: Current policies check individual actions, while trajectory constraints could prevent harmful compositions and keep agents from following denied paths.Trajectory reasoning is difficult because policies may need to consider potentially infinite action sequences and prior conditions.
- Efficiency: Policy generation can add seconds of per-task overhead depending on model size.Distillation and caching are proposed ways to reduce this cost, potentially trading off quality.
- Scope: Conseca may not apply to agents with restricted contexts, limited-purpose behavior, or no natural-language inputs, where existing access-control or anomaly-detection methods may suffice.The paper also raises non-LLM policy creation as a way to improve reliability and avoid hallucination.
8 Conclusion
The paper argues that security systems for emerging generalist agents must adapt to many purposes, contexts, and unstructured inputs. Conseca is presented as an initial step toward using context to judge actions safely.
- Conclusion: As agents perform human-like tasks across purposes and unstructured inputs, security systems must move beyond manually or statically encoded contexts.The paper frames contextual agent security as preparation for agents with broader capabilities.
- Conclusion: Further research into contextual agent security could help prepare for agents that use context to choose and judge actions.The conclusion qualifies that not all deployed agents will reach this capability scale.
A Case Study Tasks
Table A evaluates 20 multi-step tasks involving email and/or filesystem tools under Global Permissive, Global Restrictive, and Conseca policies. A checkmark indicates that the agent completed a task in the majority of five trials under the corresponding policy.
- Global Permissive: Under the Global Permissive policy, the agent completes tasks such as writing and emailing a blog post, checking disk space, and generating an alert.These examples combine filesystem inspection or file creation with email communication.
- Global Restrictive: Under the Global Restrictive policy, completed examples include compressing and emailing videos, removing duplicate files with a report, backing up files, and sharing a work document.The listed tasks span file transformation, deletion, backup, and email-based sharing.
- Case-study tasks: Additional evaluated tasks inspect logs for personally identifiable information, crashed processes, system updates, and malicious files, while another performs an incremental backup.The corresponding tasks send reports or confirmations by email when specified.
- Case-study tasks: The task set also includes organizing Documents files, extracting agenda notes from emails, summarizing important emails, and producing or emailing data reports.Several tasks require combining information retrieval, file operations, and email actions.
- Evaluation setup: The evaluation covers multi-step tasks of varying complexity using one or both email and filesystem tools, with majority-of-five-trials completion marked by checkmarks.The tasks include file creation, email, log analysis, backups, organization, and permission-related activities.