PaperlayerView exact revision on arXiv

Source-linked AI summary

Reachability-Based Capability Confinement for LLM Agents under Indirect Prompt Injection

Wujie Xiong, Rabimba Karanjai, Yang Lu, Weidong Shi, Lei Xu

arXiv:2608.30041v1cs.CRcs.AI

TL;DR

Indirect prompt injection can let attacker-controlled skill outputs influence later privileged actions, while existing defenses do not directly adapt future authority after contamination. SkillGuard enforces reachability-based capability restriction in the trusted harness using graph summaries, parameter steerability constraints, and an inline monitor. Across AgentDojo and compositional attacks, it improves security and utility without auxiliary model calls or token overhead, subject to sound abstractions and policies.

  • Problem

    Existing defenses mainly classify untrusted content or authorize proposed operations, leaving how future authority should change after untrusted data enters agent state insufficiently addressed.

  • Method

    SkillGuard models security-relevant transitions with a Skill Impact Graph, constrains parameter control with steerability signatures, and restricts capabilities after contamination without auxiliary language-model inference.

  • Results

    Across four AgentDojo suites and two backends, SkillGuard eliminates Tool Knowledge attack success on Travel, Banking, and Workspace, outperforms baselines on Llama compositional attacks, and remains competitive on Gemini with higher benign utility.

  • Takeaways & Limitations

    Fractional-flow restriction preserves substantially more capabilities than binary restriction at the same attack success rate, while SkillGuard adds no model calls or token overhead.

  • Takeaways & Limitations

    The guarantees depend on scalability of the Skill Impact Graph and utility preservation under capability restrictions.

Abstract

from arXiv · show

Large language model agents place outputs from external skills into their execution context, allowing attacker-controlled data to influence later privileged actions. Existing defenses mainly classify untrusted content or authorize proposed operations. They do not directly address how an agent's future authority should change once untrusted data enters its state. We present SkillGuard, a harness-level enforcement layer that treats this event as contamination and restricts future capabilities to disconnect the resulting state from deployer-defined forbidden states. Given sound skill summaries and policies, SkillGuard represents security-relevant transitions with a Skill Impact Graph, specifies admissible control over skill parameters via steerability signatures, and mediates invocations with an inline reference monitor. Following contamination, it computes weighted capability restrictions using binary, fractional, or fractional-flow strategies without auxiliary language-model inference. We evaluate SkillGuard on four AgentDojo suites with two backend LLMs, Gemini 2.5 Flash and Llama3.3-70B, against an LLM-only No Defense baseline and three defenses at different system layers: Spotlighting, CaMeL, and AttriGuard. We construct a compositional attack benchmark in which each attack combines observations individually insufficient to induce target violation and evaluate the same baselines on it. Under AgentDojo's Tool Knowledge attacks, SkillGuard eliminates attack success on three of four suites for both backends and reduces it to 4.8% and 14.3% on Slack. Against compositional attacks, it outperforms every baseline on Llama and matches the strongest baseline on Gemini at higher benign utility. Fractional-flow restriction preserves substantially more capabilities than binary restriction at the same attack success rate. Across both settings, SkillGuard adds no model calls or token overhead.

1 Introduction

LLM agents can be redirected by attacker-controlled skill outputs after those outputs enter the execution context. SkillGuard instead adapts future authority after contamination by enforcing reachability constraints in the trusted harness.

  • Motivation: Skill outputs can embed instructions that redirect agents into misusing the user’s authority, creating a structural indirect-prompt-injection risk.The risk arises because external outputs enter context alongside the user request, while models do not reliably distinguish instructions from data.
  • Research gap: Existing defenses primarily ask whether a proposed action is authorized, whereas SkillGuard asks which future capabilities must be restricted after untrusted data enters state.This reframes defense as prospective reachability from a contaminated state to deployer-defined forbidden states.
  • Approach: SkillGuard uses a Skill Impact Graph, steerability signatures, and an inline reference monitor to encode and enforce deployment security requirements.The design covers state safety, context integrity, capability non-escalation, and information-flow confidentiality.
  • Results: 0% attack success is achieved on Travel, Banking, and Workspace under ToolKnowledge attacks with fractional-flow restriction for both evaluated backends.Slack remains nonzero, at 4.8% and 14.3% for the two backends.
  • Results: 8.00% attack success and 81.82% utility under compositional attacks are achieved with Llama, while Gemini reaches 14.29% attack success and 92.27% benign utility.With Gemini, SkillGuard’s attack success is within 0.29 percentage points of CaMeL’s and benign utility is 10 percentage points higher.
  • Results: Fractional-flow restriction retains 93.21–94.08% of capabilities versus 76.25–79.29% under binary restriction, with both achieving 0% attack success.SkillGuard adds no model calls or tokens for protection.

2 System and Problem Statement

The system comprises an LLM, registered skills, untrusted external data sources, and a trusted harness that mediates execution. The security problem is preventing indirect prompt injection from producing policy-violating actions through later skill invocations.

  • System model: The LLM generates structured skill requests and arguments but is not assumed to reliably distinguish instructions from data.Untrusted context can therefore influence subsequent reasoning, decisions, and invocations.
  • System model: Registered skills execute according to declared interfaces and transition summaries, yet attacker-influenced arguments may still induce unauthorized actions.The skill set may include third-party skills.
  • Adversary model: External data sources—including web pages, documents, databases, and remote services—are treated as potentially adversarial.Their content may contain malicious instructions or crafted information.
  • Harness: The trusted harness maintains execution state, mediates skill invocations, and incorporates skill results into the model context.It intercepts requests, dispatches permitted calls, retrieves results, and enforces SkillGuard policies.
  • Adversary model: The adversary controls or influences external content and seeks to steer subsequent behavior toward unauthorized invocations or arguments without compromising the harness.The adversary knows exposed skill interfaces.
  • Security goals: SkillGuard defines security through state safety, context integrity, capability non-escalation, and information-flow confidentiality.These properties constrain forbidden states, parameter influence, capability evolution, and sensitive transfers.

3 Detailed Design of SkillGuard

SkillGuard combines graph-based state modeling, parameter-level steerability constraints, inline invocation checks, and post-contamination capability restriction. Its reachability enforcement preserves the stated security properties under the paper’s assumptions.

  • 3.1 Overview of SkillGuard: SkillGuard operates in the trusted harness, intercepting model-generated invocations and enforcing decisions before skill-induced side effects occur.The monitor validates requests and outputs at the dispatch boundary.
  • 3.2 Skill Impact Graph: The Skill Impact Graph models security-relevant agent states and skill-induced transitions, recording capabilities together with trust and restriction data.It is a labeled transition system G = (Q, Σ, δ, q0).
  • 3.3 Steerability Signatures: SkillGuard constrains how untrusted inputs may control skill parameters through typed steerability signatures and permitted value envelopes.Envelopes may be exact, harness-refined, or conservative; soundness requires adversary-inducible values to lie within the envelope.
  • 3.4 Inline Reference Monitor: The inline reference monitor rejects disabled skills, arguments outside active signatures, and invalid outputs before they are committed to enforcement state.It suppresses rejected invocations and outputs without terminating the agent or inserting synthetic actions.
  • 3.5 Capability restriction: After contamination, SkillGuard recomputes restrictions over the prospective graph to disconnect all paths to forbidden states at minimal policy-defined cost.Binary restriction revokes capabilities, while finer-grained strategies tighten steerability envelopes to preserve functionality.
  • 3.6 Security properties: Every admitted continuation preserves state safety, context integrity, capability non-escalation, and information-flow confidentiality.The claim depends on a trusted harness, static registry, correct summaries and signatures, faithful enforcement, and a conservative SIG.

4 Workflow of SkillGuard

SkillGuard mediates skill execution with an inline reference monitor and, after contamination, recomputes restrictions that disconnect the current state from forbidden states. It supports binary removal, fractional tightening, and fractional-flow optimization over the Skill Impact Graph.

  • Initialization: SkillGuard initializes skills, steerability signatures, the Skill Impact Graph, execution state, and deployment-defined capabilities and forbidden states.
  • Restriction strategies: Binary restriction removes capabilities, while fractional restriction tightens selected steerability envelopes; fractional-flow jointly optimizes tightening and flow across the complete graph.Fractional-flow accounts for paths shared across capabilities and can avoid unnecessary restrictions.
  • Invocation mediation: Each invocation is pre-checked for capability availability and parameter validity before execution, and rejected if either condition fails.Removing a capability blocks its skill; tightening a signature blocks arguments outside the tightened envelope.
  • Execution workflow: Valid outputs pass a post-check before the execution event updates the state; contamination then triggers recomputation of the active admissibility policy.
  • Capability restriction: Capability restriction computes a minimum-cost restriction on the current Skill Impact Graph to disconnect a contaminated state from forbidden states.The procedure returns capabilities to remove and tightened signatures for retained capabilities, then rebuilds the graph.
  • Complexity: The restriction computation has complexity O((n + m)^2(kn + m)), with runtime increasing as the reachable transition set and graph size grow.

5 Experiments and Evaluation

SkillGuard provides runtime capability restriction that complements prompt-level and per-action defenses, achieving strong protection across models, domains, and attack categories while preserving utility. Its fractional-flow strategy improves capability retention and utility over binary restriction without additional LLM inference.

  • Internal Validity: 0.00% ASR is achieved by adding capability restrictions after structural checks for both Gemini and Llama on the Travel benchmark.Structural checks alone leave non-zero attack success, while the combined defense eliminates it across all 16 attacks.
  • Internal Validity: 100% TPR under binary restriction contrasts with 81.57%–97.27% TPR under fractional-flow restriction, which lowers FPR to 1.84%–4.09%.Fractional-flow preserves 93.21%–94.08% of capabilities while maintaining essentially the same attack success rate.
  • Efficiency: Fractional-flow improves both benign utility and utility under attack over binary min-cut across evaluated models, with 9.82–10.40 ms per-task runtime overhead.The reported trend remains consistent despite substantially different baseline performance and robustness across the two LLMs.
  • Cross-Domain Evaluation: 0.00% ASR is reported for Banking and Workspace across both models and restriction schemes, while Slack remains the only domain with non-zero ASR.Slack ASR is 4.76% for Gemini and 8.57% under binary restriction or 14.29% under flow-based restriction for Llama.
  • Comparison with Existing Defenses: SkillGuard achieves 0% ASR on Travel, Banking, and Workspace for both backend models, and 4.8% on Gemini-Slack versus 14.3% on Llama-Slack.Compared with existing defenses, SkillGuard maintains competitive utility while achieving lower attack success in the reported settings.
  • Comparison with Existing Defenses: 0% ASR on Llama-Travel coincides with BU increasing from 20.0% to 35.0% and UA increasing from 29.2% to 35.0% versus AttriGuard.On Gemini-Slack, ASR falls from 18.1% to 4.8% while BU remains 42.9%.
  • Efficiency: No additional LLM calls or tokens are incurred across four suites and two backend models because protection operates through graph analysis.The comparison reports additional language-model usage specifically attributable to each defense’s protection mechanism.

6 Related Work

Existing defenses address prompt injection through instruction–data separation, detection, provenance-aware policies, or execution-trace analysis. SkillGuard instead restricts the capabilities remaining after contamination, eliminating paths to forbidden states while preserving benign functionality when possible.

  • Indirect Prompt Injection: Prompt injection arises because external tool outputs enter the agent context and can influence subsequent tool invocations.
  • Defenses Against Prompt Injection: Model-level defenses mark or transform untrusted content, separate instruction and data channels, or train models to resist injected instructions.
  • Defenses Against Prompt Injection: Runtime defenses enforce provenance, data-flow, execution-trace, or least-privilege policies over proposed tool invocations and their arguments.
  • SkillGuard: SkillGuard addresses a complementary reachability problem by restricting post-contamination capabilities rather than relying on trusted–untrusted content separation.

7 Discussion

SkillGuard’s protection is conditional on where attacks occur and how quickly contamination is enforced. Its practical limitations include scalability and utility loss under capability restrictions.

  • Failure modes: SkillGuard protects only attacks mediated by tool calls, targeting modeled dangerous actions, with enforcement occurring before the malicious action.Attacks completed in response text, outside the forbidden set, or before state updates can evade or bypass protection.
  • Scope conditions: SkillGuard’s guarantees depend on a trusted harness, conforming skill summaries, conservative SIG coverage, and correct policies and steerability signatures.The guarantee holds only under these stated assumptions about enforcement and abstraction soundness.
  • Failure modes: Compositional attacks can exploit unsafe parameter values or objectives that individual steerability envelopes do not capture.Preventing these cases may require extending the forbidden set to represent security properties across execution sequences.
  • Limitations and future directions: SIG latency grows noticeably with registry size, especially in Workspace, motivating clustered graph partitioning to reduce unrelated construction and storage overhead.The proposed direction isolates semantically unrelated skills into clusters connected through cross-domain boundary nodes.
  • Limitations and future directions: Task failures often occur when SkillGuard unnecessarily restricts skills needed for completion, motivating utility-aware adaptive restriction policies.The proposed reinforcement-learning direction would adjust restriction costs based on failed executions while preserving security guarantees.

8 Conclusion

The paper reframes indirect prompt-injection defense as prospective restriction of an agent’s authority after untrusted data contaminates execution state. SkillGuard implements this in the trusted harness and performs strongly across single-step and compositional attacks, subject to conservative abstractions and correct policies.

  • Conclusion: SkillGuard adapts an agent’s future authority after untrusted data enter its execution state using a Skill Impact Graph, steerability signatures, and an inline reference monitor.The monitor enforces the resulting policy before skill dispatch.
  • Conclusion: Across four AgentDojo suites and two backend models, SkillGuard eliminates Tool Knowledge attack success on Travel, Banking, and Workspace and substantially reduces it on Slack.It also outperforms all evaluated baselines on Llama compositional attacks and remains competitive on Gemini with higher benign utility.
  • Conclusion: Fractional-flow restriction preserves markedly more capabilities than whole-capability removal at the same attack success rate on Travel.The result supports reachability-based enforcement without indiscriminate revocation.

A.1 Compositional Attack Mechanisms

The benchmark models compositional attacks whose malicious objective emerges from combining individually insufficient fragments across agent execution. It covers fragmented instructions, cross-source information, and multi-step workflow manipulation.

  • A.1 Compositional Attack Mechanisms: The benchmark covers Banking, Slack, Travel, and Workspace to test attacks induced through composition of individually insufficient fragments.It is based on AgentDojo version 1.2.2.
  • A.1 Compositional Attack Mechanisms: Scenarios pair a benign user task with an injection task sharing the same prompt, tools, and environment state, differing only in attack fragments.Fragments are embedded through existing content sources or additional files, depending on the scenario.
  • A.1 Compositional Attack Mechanisms: Three attack forms distribute malicious semantics across tool outputs, combine information from multiple sources, or turn permissible actions into a malicious workflow.These forms capture distinct modes of composition during agent execution.

A.2 Validation of Compositional Attack

Compositional benchmark scenarios are validated by controlled ablations that isolate fragment composition and by ground-truth execution checks. Only scenarios showing composition-specific attacks are retained.

  • A.2 Validation of Compositional Attack: Candidate validity requires unchanged prompts, tools, and initial environments while varying only injected fragments.The intended attack must arise from composition rather than any individual fragment.
  • A.2 Validation of Compositional Attack: For each scenario, the full composition must produce the attack while every single-fragment attack remains absent or substantially weaker.The evaluation compares clean, single-fragment, and full-composition configurations.
  • A.2 Validation of Compositional Attack: 22 of 39 candidate scenarios satisfy the compositionality criterion and are included in the benchmark.Scenarios failing the criterion are excluded.
  • A.2 Validation of Compositional Attack: Ground-truth executions verify that benign tasks avoid forbidden states while injection tasks induce the target security violation.These checks ground labels in feasible environment transitions rather than model-specific behavior.

A.3 Representative Scenarios

Travel T4 illustrates a car-rental redirect scenario in which the user asks the agent to reserve a car from SunSet Rent-A-Car for specific dates.

  • Travel T4 concerns reserving a car from SunSet Rent-A-Car for specific dates.
  • The scenario involves a user request to reserve a car with a named, real company.
  • The requested rental company is SunSet Rent-A-Car.

Fragmented Instruction Injection

The benchmark includes fragmented, cross-source, and multi-step attacks whose harmful outcomes emerge from combining observations across sources or workflow stages rather than from individual fragments alone.

  • Fragmented Instruction Injection: Fragmented instruction injection combines a motivation fragment with a separate company-name fragment.The first suggests a policy change without naming a target; the second names a company without saying it should be selected.
  • Fragmented Instruction Injection: 0% attack success occurs when only the motivation or only the company fragment is provided, whereas combining both enables successful attacks.
  • Cross-source information composition: Cross-source information composition uses semantically complete, independent fragments that jointly create agreement on an attacker-controlled replacement account.The fragments include uncertainty about the utility account and two independent reports of the same attacker IBAN.
  • Cross-source information composition: Each single-fragment ablation yields 0% attack success, indicating dependence on cross-source composition.
  • Multi-step workflow manipulation: The multi-step workflow attack redirects a three-installment payment requiring three schedule transaction calls.The installments are $950 each and have different due dates; the attack combines invoice scheduling with payment-routing information.
  • Multi-step workflow manipulation: Success requires redirecting all three transactions to the attacker’s IBAN, testing security over the complete execution trace.
  • Multi-step workflow manipulation: 0% attack success under fragment ablations contrasts with 100% for the complete composition.This result links the attack to information accumulated across the workflow rather than any single tool invocation.
Loading 2608.30041v1…