Source-linked AI summary
When Context Gets Root: Privilege Escalation in LLM Harnesses
Xingbang He, Yuanwei Chen, Yi Qian, Haiyang Wei, Ligeng Chen, Zenan Fu, Linzhang Wang, Hao Wu, Bing Mao
TL;DR
The paper examines how agent harnesses can violate instruction hierarchy by elevating low-level malicious content during context reconstruction. It introduces instruction privilege escalation and evaluates it across coding-agent harnesses and attack objectives, finding complete objective coverage under full access and automatic permission review in supported harnesses.
Problem
Instruction hierarchy and permission review assume role labels faithfully preserve content provenance, but harnesses may reconstruct contexts that relabel tool content as higher-privilege instructions.
Method
The paper defines tool-to-user and tool-to-system escalation and evaluates attacks using multi-agent delegation, persistent goals, scheduled tasks, and custom subagents.
Results
Across six coding-agent harnesses and 13 attack objectives, attacks achieve every objective on every harness under full access and on all three harnesses providing automatic permission review.
Takeaways & Limitations
Instruction privilege escalation is effective across instruction levels, context-construction mechanisms, and permission regimes.
Takeaways & Limitations
System-effective is a relative category and does not imply that every provider publishes the same strict conflict-resolution policy.
Abstract
from arXiv · showhide
Instruction hierarchy is a model-side defense that assigns instructions different levels of privilege according to their sources. These levels constrain which content may direct model behavior. During agent execution, however, agent harnesses construct context for each model invocation. This construction can elevate low-level content to a higher instruction level and grant it greater model-facing privilege. We introduce instruction privilege escalation. In this attack, an attacker induces an agent to elevate low-level malicious content to a higher instruction level. The elevated content then causes the agent to execute instructions it would not follow at their original level. We evaluate this threat by using multi-agent mechanisms to achieve 13 attack objectives across six coding-agent harnesses. These objectives span confidentiality, integrity, availability, and remote code execution. With unrestricted action execution, the attacks achieve all 13 objectives on all six harnesses. Under automatic permission review, the attacks achieve all 13 objectives on all three harnesses that provide this mode. We further reproduce the vulnerability using harness-provided persistent goals and scheduled tasks. These results demonstrate the generality of instruction privilege escalation.
1 Introduction
Agent harnesses can elevate attacker-controlled tool content into higher-privilege instructions during context reconstruction, undermining instruction hierarchy and permission review. Across six coding-agent harnesses and 13 objectives, the attacks are broadly effective.
- Untrusted files, webpages, emails, and tool results can induce harmful actions including data exfiltration, persistent modification, and remote code execution.
- Instruction hierarchy and automatic permission review both depend on role labels faithfully representing content provenance.Working agents use roles to determine governing instructions, while reviewers infer authorization and intent from interaction history.
- Harness context reconstruction can relabel tool content as user intent or persist it as system-effective policy, causing downstream components to treat harmful instructions as authorized.
- Instruction privilege escalation elevates malicious content from lower to higher instruction levels through agent-side context reconstruction, including tool-to-user and tool-to-system forms.The elevated content gains greater model-facing privilege and can alter working-agent or permission-reviewer decisions.
- The attacks use multi-agent delegation, persistent goals, scheduled tasks, and custom subagents to realize tool-to-user and tool-to-system escalation.
- 97.3% mean success rate after escalation was achieved by tool-to-user attacks, versus 80.3% mean per-attempt attack success rate for tool-to-system attacks.The evaluation covered six coding-agent harnesses and 13 objectives spanning confidentiality, integrity, availability, and remote code execution.
2 Background
LLM APIs represent system, user, assistant, and tool interactions through different protocol-specific message formats. Automatic permission review evaluates proposed actions using the working agent’s context and inferred user intent.
- 2.1 Message Types in Model APIs: System, user, assistant, and tool messages form the model-facing conversation context for subsequent outputs.
- 2.1 Message Types in Model APIs: OpenAI, Anthropic, and Google APIs use different message and tool-output representations.OpenAI distinguishes Chat Completions tool messages from Responses function_call and function_call_output items; Anthropic and Google use their own tool block or function item formats.
- 2.2 Automatic Permission Review: Automatic permission review is a model-mediated policy layer that assesses working-agent commands before execution.
- 2.2 Automatic Permission Review: Auto PR considers the working agent’s context to infer user intent, automatically approving low-risk or clearly aligned actions and blocking unrequested high-risk actions.
- 2.2 Automatic Permission Review: Codex Auto PR reviews commands such as npm test using the user message, assistant messages, previous commands, and current action.
3 Motivating Cases
Delegation changes how attacker-controlled content is represented to a subagent, reversing the working agent’s decision and misleading Auto PR about user authorization. The cases show that harness-side message reconstruction changes both model-side and authorization outcomes.
- A message’s assigned role may not correspond to the participant that produced its content, affecting how the model interprets and acts on it.
- Multi-agent delegation presents delegated content to a subagent as a user message even when the main agent, not the human user, generated it.
- Case 1: Model-side decision: The direct attack was refused after Codex recognized the server’s arbitrary-command interface and treated the instruction as tool-provided content.
- Case 1: Model-side decision: The delegation attack caused the subagent to start the same malicious server after the harness reconstructed the task as a user message.The same model reached opposite decisions when the request was presented under different message types.
- Case 2: Harness-side authorization: Auto PR classified the delegated server action as high risk but authorized it because the reconstructed user message appeared to explicitly approve the action.
- Case 2: Harness-side authorization: Elevating tool results to user messages changes both model-side behavior and harness-side security decisions through context reconstruction.
4 Root Cause Analysis
Instruction hierarchy depends on model-facing role labels to distinguish trusted instructions from untrusted data, but context reconstruction can erase provenance and elevate attacker-derived content. The resulting content can influence both agent behavior and permission decisions as if it carried genuine user authorization.
- Instruction hierarchy gives higher-privilege sources greater influence, while tool output is treated as less reliable.This role assignment helps distinguish authorized instructions from malicious instructions embedded in external data.
- Role labels affect model decisions because downstream models give privileged tags greater influence.Prior role-confusion attacks demonstrate this effect by making tool output or retrieved content appear more privileged.
- Context reconstruction can carry existing content into a new invocation at a higher instruction level, increasing its influence over downstream behavior.Subagent delegation can reconstruct main-agent-generated content as a higher-privilege user message.
- Provenance loss prevents defenses from distinguishing elevated attacker content from genuine user instructions.The working agent follows the elevated instruction, while the permission reviewer may treat reconstructed attacker text as user authorization despite recognizing the action as risky.
5 Instruction Privilege Escalation
The paper defines a unified hierarchy for model-facing instruction levels and formalizes escalation as context construction that presents content above its origin level. It distinguishes tool-to-user and tool-to-system escalation while clarifying that escalation itself does not imply execution or impact.
- 5.1 Unified Instruction Hierarchy: The unified hierarchy maps protocol-specific message types to tool, user, and system-effective levels.Table 1 covers four request protocols, and model outputs receive no instruction level in this mapping.
- 5.1 Unified Instruction Hierarchy: Tool content includes external observations, user content includes model-presented tasks, and system-effective content includes higher-priority prompt surfaces.Examples include files and command output at tool level, user tasks at user level, and system prompts or custom-subagent policies at system-effective level.
- 5.1 Unified Instruction Hierarchy: System-effective is relative: it can constrain an ordinary user task without overriding the harness’s core system prompt.The ordering normalizes model-facing placements and does not imply identical conflict-resolution policies across providers.
- 5.2 Initial Model-Facing Instruction Levels: An item’s origin level is where it first enters the agent, whereas its harness-assigned level is determined during each context construction.Origin depends on concrete presentation rather than filename or storage location, and representative artifacts are grouped by model-facing level in Table 2.
- 5.3 Escalation Definition: Instruction privilege escalation occurs when context construction presents content at a higher level than its origin level.The formalization tracks successive context-construction operations carrying content into model contexts and marks the condition with IPE(c) = 1.
- 5.3 Escalation Definition: Tool-to-user escalation elevates tool-originating content to user-level instructions, while tool-to-system escalation elevates it to system-effective instructions.These forms describe increased instruction privilege and do not themselves require execution or a resulting security effect.
6 Threat Model
The threat model treats the harness as the component that constructs model contexts and may mediate action execution through an automatic reviewer. The attacker controls readable workspace artifacts but not the user, harness, models, policies, or runtime state.
- Victim model: The victim system consists of a harness, a working-agent model, and optionally a reviewer model.The harness governs tools, state, and context construction; the working model proposes actions; the reviewer performs automatic permission review when present.
- Context and actions: At each step, the harness maps agent state into an ordered working context presented to the model.The working agent proposes an action from this context, and proposed actions execute directly when no reviewer is used.
- Permission review: With automatic permission review, the harness builds a review context from working history and the proposed action, and execution requires an allow decision or actual user approval.The reviewer’s decision is denoted d_τ, with execution occurring iff d_τ = allow or the user approves after confirmation.
- Attacker model: The attacker knows the harness design and controls workspace artifacts the agent may read, including repository files and task documents.Attacker-controlled content initially enters through file-reading or search tools and can induce normal harness operations such as delegation, goal creation, or task scheduling.
- Attacker limits: The attacker cannot control the user request, harness, models, developer instructions, permissions, sandbox, approval policy, or runtime beyond authorized agent actions.The attack does not rely on prompt injections, model jailbreaks, genuine authorization, or initial shell access.
- Attack goal: The attack goal is execution of a security-sensitive action the actual user did not authorize.Without review, the working model need only propose the action; with review, the attacker must also obtain d_τ = allow.
7 Attack Overview
The attack induces ordinary harness operations that reconstruct attacker-controlled tool content at higher instruction levels. It uses delegation, custom subagents, persistent goals, scheduled tasks, and related built-in features.
- Attack mechanism: An inducing instruction is attacker-controlled tool-level content that causes a context-construction operation to elevate content to a higher instruction level.The attack begins with a normal workflow action such as delegation, custom-subagent installation, goal storage, or task scheduling.
- Tool-to-user escalation: Tool-to-user escalation primarily uses multi-agent delegation, where delegated malicious content is inserted into a subagent context as a user message.The same transition can occur when persistent goals or scheduled tasks reintroduce carried content as user messages.
- Tool-to-user escalation: Direct induction places an explicit delegation directive in README.md, while indirect induction separates benign task steps from general guidance encouraging delegation.The inducing instruction need not contain the malicious behavior; a separate payload such as setup.py can be forwarded to the subagent.
- Tool-to-user escalation: The delegation attack forwards python3 app/setup.py to a subagent as a user-level instruction, completing tool-to-user escalation.Repository guidance attracts the main agent’s attention and makes delegation appear appropriate before the command crosses instruction levels.
- Tool-to-system escalation: Tool-to-system escalation uses custom-subagent installation to place attacker-controlled content in configuration later loaded at the system-effective level.The attack installs a malicious subagent and modified AGENT.md, then relies on later invocation to redirect benign work.
- Tool-to-system escalation: The custom-subagent attack has installation and invocation stages, with variants that explicitly embed malicious code or conceal it behind an external script reference.The hidden-script variant constrains the model’s prescribed execution path and limits opportunities to inspect the referenced script.
- Additional mechanisms: Persistent goals, scheduled tasks, and skills generalize escalation beyond subagent mechanisms by reconstructing tool-level content into user-level or system-effective contexts.A stored persistent goal can be reinjected as a user-level prompt in later turns.
8 Evaluation
The evaluation tests instruction privilege escalation across six coding-agent harnesses, six models, 13 objectives, and full-access or automatic-review settings. Escalation succeeds broadly, including where baseline tool-level attacks usually fail.
- Evaluation setup: The study evaluates 13 objectives across six coding-agent harnesses and six underlying models, comparing baseline attacks with tool-to-user and tool-to-system escalation.Persistent goals and scheduled tasks are additionally tested to assess generalization beyond multi-agent delegation.
- Baselines: Most tool-level baseline attacks fail, and none succeed under Auto PR.The results attribute failures to models recognizing malicious behavior, refusing execution, requesting confirmation, or identifying forged reasoning as prompt injection.
- Tool-to-user escalation: Tool-to-user escalation achieves all 13 objectives on every harness in full-access mode and still succeeds on all objectives across the three harnesses supporting Auto PR.Across objectives, success rates range from 31.7% to 100%, and once escalation succeeds, subsequent attacks succeed in nearly 100% of cases.
- Tool-to-user escalation: Elevated user-level instructions can cause direct execution and can also persuade a reviewer to approve harmful actions appearing explicitly user-requested.A Codex reviewer approved execution after recognizing that a script downloaded, enabled, and ran an external program.
- Tool-to-system escalation: Tool-to-system escalation achieves high installation rates, with 84.6%–100% under full access for Gemini CLI, Kimi, and OpenCode and 46.2%–100% under Auto PR for Claude Code, Codex, and Qwen Code.The direct system-prompt variant completes 55 of 117 objective–configuration combinations.
- Tool-to-system escalation: The hidden-script system-level variant completes all objectives under both full-access and Auto PR modes.Across six harnesses, full-access success ranges from 65.0% to 100%; under Auto PR it reaches 76.5% on Claude Code, 70.0% on Codex, and 61.9% on Qwen Code.
- Generality: Persistent goals and scheduled tasks are broadly available, and all four tested paths achieve all 13 objectives.This demonstrates generalization across multiple context-reconstruction mechanisms rather than dependence on multi-agent delegation.
9 Related Work
Related work covers instruction hierarchy, direct and indirect prompt injection, and role-confusion attacks. The paper distinguishes its threat by focusing on harness-side manipulation of instruction privilege rather than model-side misinterpretation.
- Instruction hierarchy: Instruction-hierarchy research studies how models resolve conflicts between privileged and lower-privilege instructions and how reliably they follow such hierarchies.IHEval reports degraded performance under cross-level conflicts, while Control Illusion finds explicit role separation often insufficient.
- Prompt injection: Prompt-injection research classifies attacks by how malicious instructions reach the model, including direct user input and indirect retrieved or observed content.Prior work demonstrates goal hijacking, prompt leaking, remote control of integrated applications, harmful actions, private-data exfiltration, and defense bypasses.
- Role confusion: Role-confusion attacks exploit ambiguity in how models infer content sources or message roles from textual structure.Examples include imitating model-generated reasoning or chat-template syntax so malicious content appears to have a different role.
10 Discussion
“System-effective” groups prompt surfaces above ordinary user-level instructions, but privilege within this category is not uniform. Differences arise from message types and prompt position.
- System-effective is an analytical category for prompt surfaces placed above ordinary user-level instructions.
- Privilege can differ because request protocols distinguish multiple privileged message types, such as system and developer messages.
- Privilege can also depend on prompt position, including the relative placement of privileged instructions and user messages.
11 Conclusion
Instruction privilege escalation elevates attacker-controlled content through agent-side context construction. Across six coding-agent harnesses and 13 attack objectives, attacks achieved every objective under full access and on all three harnesses with Auto PR.
- Instruction privilege escalation elevates attacker-controlled content to a higher model-facing instruction level through agent-side context construction.
- Tool-to-user escalation uses multi-agent delegation, persistent goals, and scheduled tasks, while tool-to-system escalation uses custom-subagent installation and loading.
- 13 attack objectives were achieved on all six coding-agent harnesses under full access.
- All 13 attack objectives were achieved on all three harnesses that provide Auto PR.
- The results show effectiveness across instruction levels, context-construction mechanisms, and permission regimes.