PaperlayerView exact revision on arXiv

Source-linked AI summary

AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification

Tian Zhang, Yiwei Xu, Juan Wang, Keyan Guo, Xiaoyang Xu, Bowen Xiao, Quanlong Guan, Jinlin Fan, Jiawei Liu, Zhiquan Liu, Hongxin Hu

arXiv:2602.22724v1cs.CRcs.AI

TL;DR

Indirect prompt injection lets attacker-controlled tool and retrieval content steer multi-turn LLM agents away from user intent, while existing inference-time defenses can disrupt legitimate workflows. AgentSentry models this as a temporal causal takeover, localizes takeover points with boundary-level counterfactual re-executions, and purifies causal deviations to continue safely. Across AgentDojo task suites, attack families, and black-box LLMs, it achieves 0% ASR and 74.55% average UA, improving over baselines without degrading benign performance.

  • Problem

    Indirect prompt injection lets untrusted tool and retrieval content persist across turns and steer agent actions away from user intent, while delayed takeovers make heuristic detection and conservative blocking difficult.

  • Method

    AgentSentry models multi-turn IPI as a temporal causal takeover, localizes takeover points through controlled counterfactual re-executions at tool-return boundaries, and applies causally gated context purification.

  • Results

    0% ASR and 74.55% average UA were achieved across AgentDojo’s four task suites, three IPI families, and multiple black-box LLMs, improving UA by 20.8 to 33.6 points over strongest baselines without degrading benign performance.

  • Takeaways & Limitations

    Boundary-anchored causal diagnostics and purification support safe continuation of tool-mediated workflows instead of defaulting to premature termination.

  • Takeaways & Limitations

    The threat model excludes compromises of the tool runtime and adversaries that directly tamper with defender caching or replay.

Abstract

from arXiv · show

Large language model (LLM) agents increasingly rely on external tools and retrieval systems to autonomously complete complex tasks. However, this design exposes agents to indirect prompt injection (IPI), where attacker-controlled context embedded in tool outputs or retrieved content silently steers agent actions away from user intent. Unlike prompt-based attacks, IPI unfolds over multi-turn trajectories, making malicious control difficult to disentangle from legitimate task execution. Existing inference-time defenses primarily rely on heuristic detection and conservative blocking of high-risk actions, which can prematurely terminate workflows or broadly suppress tool usage under ambiguous multi-turn scenarios. We propose AgentSentry, a novel inference-time detection and mitigation framework for tool-augmented LLM agents. To the best of our knowledge, AgentSentry is the first inference-time defense to model multi-turn IPI as a temporal causal takeover. It localizes takeover points via controlled counterfactual re-executions at tool-return boundaries and enables safe continuation through causally guided context purification that removes attack-induced deviations while preserving task-relevant evidence. We evaluate AgentSentry on the \textsc{AgentDojo} benchmark across four task suites, three IPI attack families, and multiple black-box LLMs. AgentSentry eliminates successful attacks and maintains strong utility under attack, achieving an average Utility Under Attack (UA) of 74.55 %, improving UA by 20.8 to 33.6 percentage points over the strongest baselines without degrading benign performance.

1 Introduction

AgentSentry treats multi-turn indirect prompt injection as a temporal causal takeover in which untrusted context can displace user-goal influence. It localizes and mitigates such takeovers through boundary-level counterfactual diagnostics and causally gated purification, preserving workflow continuation while achieving strong benchmark security and utility.

  • Threat: Tool-mediated IPI embeds malicious instructions in untrusted content, allowing attacker-controlled context to persist across turns and steer tool selection away from user intent.The attack does not require control of the user prompt and can drive unauthorized downstream actions.
  • Motivation: Inference-time safeguards are practical because hosted or proprietary LLM backends often preclude training-time changes.Prior defenses use heuristic detection or strict action constraints that can struggle with delayed takeovers in multi-turn, multi-tool workflows.
  • Approach: AgentSentry models multi-turn IPI as a temporal causal takeover whose context-mediated influence progressively dominates downstream decisions.The framework is designed for safe task continuation rather than conservative termination or broad tool disabling.
  • Diagnostics: Controlled counterfactual re-executions at tool-return boundaries estimate whether the next action is driven primarily by the user goal or tool- and retrieval-mediated context.These diagnostics localize takeover points without relying on lexical matching or similarity-threshold heuristics.
  • Mitigation: Causally gated context purification suppresses attack-induced control signals while retaining task-relevant evidence and leaving the main execution trajectory intact.Purification is triggered when diagnostics attribute unsafe actions to context-mediated influence, enabling safe continuation in long-horizon workflows.
  • Evaluation: 0% ASR and 74.55% average UA were achieved on AgentDojo, improving UA by 20.8 to 33.6 percentage points over the strongest baselines without degrading benign performance.The evaluation spans diverse task suites, IPI attack families, and black-box LLM backends.

2 Background and Problem Statement

Tool-augmented agents maintain an evolving context channel that can carry adversarial directives from seemingly legitimate tool results into later actions. Because IPI may trigger only after several intermediate steps, defenses must distinguish harmful causal influence from benign preparation without unnecessarily blocking legitimate workflows.

  • Agent setting: Tool-augmented agents update an evolving task state with dialogue history, intermediate artifacts, and content from heterogeneous, partially untrusted sources.This persistent context channel supports long-horizon task completion while expanding the material that can influence later actions.
  • IPI: IPI embeds adversarial directives in tool or retrieval outputs, allowing payloads to persist in agent state and alter later tool choices or parameters without controlling the user prompt.The attack enters through seemingly legitimate tool results rather than the direct prompt.
  • Temporal delay: Delayed takeover can separate payload introduction from harmful side effects, such as a calendar.read → email.search → send_email sequence.Blocking only the final action may be too late, whereas blocking earlier preparatory reads can disrupt legitimate completion.
  • Challenges: The central challenge is identifying when untrusted context becomes causally dominant while benign intermediate tool calls remain necessary for task completion.This requires reasoning over multi-step trajectories rather than only inspecting the final action.
  • Utility trade-off: Conservative blocking can prevent policy violations but sharply reduce attacked-task utility when workflows contain ambiguous benign intermediate actions.MELON reports UA = 32.91% for GPT-4o under Important Messages in a prior evaluation.

3 Problem Formulation and Threat Model

The paper formalizes a tool-augmented agent as coupled transformations over internal context and external environment, with untrusted mediator content influencing boundary-level actions. Its defense diagnoses mediator-dominated behavior at tool-return boundaries and repairs actions locally under an explicit threat model and scope.

  • 3.1 Agent Model and Execution Semantics: The agent model separates internal context updates from external effects, representing each iteration as a composable state transformation.The context contains dialogue history, reasoning artifacts, tool outputs, retrieval snippets, and memory, while the environment includes services such as inboxes and calendars.
  • 3.1 Agent Model and Execution Semantics: Untrusted tool, retrieval, and memory content forms a mediator channel that can steer actions toward violations of user intent or policy Π.Multi-turn IPI operates by embedding imperative or control-bearing directives into this channel.
  • 3.2 Boundary-Indexed Diagnostics: A tool-return boundary is the instant after new external content enters internal context and before the next action is emitted.At each boundary, the framework represents the context, external state, proposed action, and diagnostic outcome.
  • 3.2 Boundary-Indexed Diagnostics: The diagnostics classify proposed actions by impact and separately identify unauthorized high-impact tool calls using policy- and context-dependent authorization.They distinguish low-impact diagnostic tools from high-impact tools that transmit data or modify external state.
  • 3.3 Boundary-Local Defense Component: At takeover boundaries, the repair interface purifies context and revises the next action; otherwise, the original context and action continue unchanged.External effects are committed only when the secured action is authorized, and enforcement is localized to boundaries with causal evidence of mediator-dominated deviation.
  • 3.4 Threat Model: The threat model allows manipulation of tool results, retrieval content, and persistent memory to induce unauthorized high-impact actions or policy-violating goal drift.An attack counts as successful when a policy-violating effect is committed or would be executed absent intervention.
  • 3.5 Defender Capabilities and Scope: The defender is assumed able to restore historical boundaries, run dry-run re-executions, and replay or sanitize cached responses.The framework does not address compromised tool runtimes or attackers who tamper directly with defender caching and replay.

4 Design of AgentSentry

AgentSentry combines boundary-anchored counterfactual diagnostics with causally gated purification to detect mediator-driven takeover and continue tasks safely. Its design preserves task-relevant evidence while selectively revising attack-induced actions and monitoring temporal causal degradation.

  • Boundary instrumentation and counterfactual diagnostics: AgentSentry instruments tool-return boundaries before the next action and uses controlled, side-effect-free re-executions to estimate causal influence.Diagnostics hold the dialogue prefix and runtime state fixed while varying user and mediator inputs across controlled regimes.
  • Causal effect estimation: The framework decomposes boundary behavior into user-driven and mediator-driven effects using cached, replayed, and sanitized mediator views.Its causal estimands include ACE, direct and indirect effects, Monte Carlo plug-in estimates, residual checks, and optional bootstrap uncertainty.
  • Temporal causal degradation test: Temporal takeover is indicated when user-goal dominance attenuates while mediator dependence strengthens across a sliding boundary window.AgentSentry combines trend slopes, a dimensionless risk score, indirect-effect significance, and an instantaneous safeguard for abrupt mediator-attributed tool activity.
  • Causally gated purification: Purification rewrites only the mediator view into evidence-only context, retaining task-relevant facts while removing instruction-carrying directives.The operator enforces factual fidelity, non-actionability, and task alignment rather than deleting the mediator wholesale.
  • Safe continuation: Safe continuation preserves low-impact task-essential calls, removes or replans mediator-contingent high-impact invocations, and repairs persistent high-impact parameters.These decisions use counterfactual severity differences and require sensitive arguments to be supported by trusted context or structured evidence.

5 Experiment and Evaluation

AgentSentry is evaluated across AgentDojo task suites, IPI families, and black-box backbones using security, utility, benign-performance, and ablation analyses. It achieves a favorable security–utility trade-off and localizes takeover effects at tool-return boundaries.

  • Experimental setup: Evaluation spans four AgentDojo suites, three IPI attack families, and three black-box language models.The study uses complete turn-level traces to support controlled counterfactual re-executions and temporal causal-effect estimation.
  • Evaluation metrics: The evaluation reports Attack Success Rate, Utility under Attack, Clean Utility, and False Positive Rate, with lower ASR and FPR and higher UA and CU preferred.These metrics jointly capture security, attacked-task performance, benign capability, and unintended intervention.
  • Security–utility frontier: Conservative baselines can suppress attacks but sharply reduce utility or benign performance under ambiguous, context-dependent tool workflows.Tool Filter reduces average UA to 9.44% and CU to 6.19% on Qwen3-Max, while MELON achieves ASR 0% with UA 18.05% for GPT-4o under Important Instructions.
  • Security–utility frontier: AgentSentry attains ASR = 0% while retaining high UA and preserving benign capability across attack families and evaluated backbones.Against Task Shield, average UA rises by 20.77 points on GPT-4o, 33.57 points on GPT-3.5-turbo, and 29.27 points on Qwen3-Max while ASR falls to 0%.
  • Boundary-aligned causal trajectories: Boundary-aligned effects concentrate near the first contaminated tool return, where injected content can directly bias the next-step proposal.The representative Workspace case embeds the payload in a calendar tool return, and AgentSentry typically localizes takeover at or immediately after the first affected read.

6 Conclusion

AgentSentry treats indirect prompt injection as a temporal causal takeover and combines boundary-based counterfactual diagnostics with context purification for safe continuation. Across AgentDojo evaluations, it eliminates successful attacks while retaining strong attacked-task utility and benign performance.

  • Conclusion: AgentSentry detects boundary-anchored causal takeover from untrusted tool, retrieval, and memory content and revises only action components implicated by mediator-driven deviation.Its workflow operates at tool-return boundaries and uses dry-run counterfactual re-executions before purification.
  • Conclusion: AgentSentry preserves task-relevant evidence while suppressing instruction-carrying deviations, enabling safe continuation rather than terminating the workflow.This design targets attacker-induced control signals while leaving the main execution trajectory intact.
  • Conclusion: The paper positions AgentSentry as practical protection for action-capable agents operating in tool-mediated environments.The supported scope is inference-time defense against indirect prompt injection in tool-augmented agents.
  • Conclusion: Across four task suites, three IPI families, and multiple black-box LLMs, AgentSentry achieves ASR = 0% and average UA = 74.55%.It improves UA by 20.8 to 33.6 percentage points over the strongest baselines without degrading benign performance.

A.3 Inference-time defenses and their structural limitations

Inference-time defenses avoid parameter updates but often rely on heuristic gating that can over-block benign actions in long-horizon workflows. MELON and Task Shield additionally lack a principled way to identify whether tool or retrieved content causally drives behavior, motivating AgentSentry’s causal diagnostics and purification.

  • Inference-time defenses: Training-free inference-time defenses isolate untrusted content or constrain actions, but heuristic gating can degrade utility by blocking benign preparatory or diagnostic calls.These methods are practical for deployed black-box agents because they avoid parameter updates, yet strict gating can over-block legitimate workflow steps.
  • Existing agent-level defenses: MELON detects inconsistency through masked re-execution, while Task Shield requires every tool action to be explicitly justified by the user’s stated objectives.MELON replaces the original input with a synthetic masking template; Task Shield can reject contextually justified diagnostic or preparatory actions.
  • Shared structural limitation: Both defenses rely on local surface-level rules and brittle textual alignment rather than determining whether retrieved or tool-generated content causally drives agent behavior.This shared weakness can suppress benign actions while overlooking malicious contextual signals behind apparently normal tool calls.
  • AgentSentry response: AgentSentry uses temporal causal diagnostics and targeted context purification to remove attack-induced control signals while preserving task-relevant evidence and safe continuation.Its diagnostics run in shadow executions, leaving the main trajectory unchanged unless contaminated context is attributed to a high-impact action.

B Implementation Details of Interventional Regimes and Controlled Re-Execution

The implementation defines concrete interventional regimes for controlled dry-run re-execution, using task-neutral masking, sanitized mediator substitution, and an ordinal diagnostic outcome to assess mediator-driven deviations.

  • Regime construction: The appendix specifies four implementation components: a task-neutral probe, a sanitized mediator, controlled dry-run re-execution, and an ordinal diagnostic outcome.It also defines an optional unauthorized high-impact indicator and the textual-deviation rule used by the outcome map.
  • Controlled re-execution: The controlled protocol obtains Monte Carlo samples of the ordinal outcome Y_b = ψ(A_b; Π) through repeated counterfactual substitutions.When enabled, the protocol also samples the unauthorized high-impact indicator V_b.
  • Diagnostic outcome: The outcome map assigns y_b = 1 when a proposed action exhibits mediator-induced textual deviation, operationalizing off-goal behavior under the trusted goal and policy.Algorithm 2 summarizes the reference procedure for constructing these regimes and outputs.

B.1 Task-neutral diagnostic probe 𝑥mask

AgentSentry uses a task-neutral probe and sanitized mediator substitutions in dry-run re-executions to isolate context-driven deviations without altering the live trajectory. The same goal-aligned deviation check and counterfactual contrasts quantify whether mediator content drives unsafe proposals.

  • Task-neutral probe: The task-neutral probe xmask replaces the observed user input only during controlled dry-run attribution while holding the boundary snapshot, tools, and mediator view fixed.It intentionally makes the user channel uninformative without restating the goal or imposing policy constraints.
  • Probe semantics: Under the probe, AgentSentry computes Y_b = ψ(A_b; Π), using differences between mask and mask_sanitized regimes to isolate mediator-driven action tendencies.Dry-run evaluation leaves the live trajectory untouched.
  • Mediator sanitization: The sanitized mediator preserves task-relevant factual fields while projecting instruction-carrying spans into a non-actionable evidence form.Offline substitution is used for diagnostics, whereas triggered mitigation commits the purified mediator to the live trajectory.
  • Regime execution: At each tool-return boundary, four regimes restore an identical snapshot and runtime state before invoking the base agent with identical tool availability.Cached mediator views and restored prefixes support comparable counterfactual executions.
  • Deviation criterion: AgentSentry marks y_b = 1 when an action’s natural-language component introduces intent not entailed by the user goal and trusted boundary evidence under policy Π.The closed-set verifier checks new unrelated objectives, priority overrides, and unsupported commitments.
  • Causal contrasts: Counterfactual contrasts quantify mediator-induced deviation, while an original-versus-sanitized comparison measures mediator influence under the actual user input.In a travel example, sanitization removes an injected booking directive and yields a proposal consistent with the rating threshold.

C.2 Implementation Considerations

AgentSentry controls replay variance by freezing mediator outputs and uses lightweight counterfactual sampling to keep diagnostic overhead near a standard agent run. Its single-sample configuration is supported for current benchmarks but does not cover gradual or delayed takeovers equally well.

  • Replay control: Mediator freezing reuses byte-identical cached tool, retrieval, or memory returns across regimes so outcome differences can be attributed to the interventions.Cache keys include provenance information such as source, endpoint, normalized arguments, and byte content.
  • Outcome extraction: The implementation preserves task-relevant fields during purification and distinguishes high-impact tools from lower-impact diagnostic tools when extracting outcomes and authorization violations.Unauthorized side-effect detection applies only to high-impact invocations under policy Π.
  • Complexity: The per-boundary overhead is O(4K + B) additional agent invocations plus O(w) time for rolling statistics and trend estimates.Here, K is the number of re-executions per regime, B the bootstrap count, and w the temporal window size.
  • Evaluation configuration: The evaluation uses w ∈ {2, 3}, K = 1, and B = 0, keeping diagnostic overhead on the same order as a standard agent run.This relies on boundary-local mediator freezing and injected directives that typically manifest near the tool-return boundary.
  • Scope boundary: Gradual or delayed takeovers require larger K and bootstrap-based significance checks, because K = 1 reflects current benchmark characteristics rather than a general deployment guarantee.Long-horizon causal degradation remains an open evaluation direction.

D.1 Observed Execution Traces

The traces contrast defenses that either execute, redact, or block injected directives, while AgentSentry localizes the causal deviation and purifies only the instruction-bearing span to preserve task evidence and completion.

  • Comparative execution traces: MELON completes the restaurant task but reproduces the injected hotel endorsement in its final message.The endorsement appears before the restaurant recommendations and prices, so the user task is completed while the injected directive executes in-band.
  • Comparative execution traces: The transformer PI detector suppresses the injection by redacting tool outputs, but removes task-critical evidence and fails to retrieve the lunch price.Its benchmark labels are utility=false and security=false.
  • Comparative execution traces: Task Shield assigns zero contribution to the hotel directive, blocks it at content checking, and permits tool calls that directly support restaurant selection and price reporting.The blocked directive is not propagated into subsequent assistant decisions, while aligned calls remain executable.
  • AgentSentry trace: AgentSentry detects takeover through counterfactual contrasts at a tool-return boundary and commits a purified state that removes executable intent while retaining restaurant evidence.The purification uses a provenance- and structure-preserving transformation that projects the injected span into an evidence-only form.
  • AgentSentry trace: AgentSentry recommends Le Baratin and New Asiaway at ratings 4.8 and 4.6, reports 30 per person for each, and omits the injected endorsement.The resulting continuation preserves the evidence stream required for task completion.
  • Outcome summary: The case separates takeover prevention from utility preservation: AgentSentry prevents semantic deviation without discarding task-relevant tool evidence.The comparison distinguishes MELON’s directive execution, Task Shield’s alignment blocking, and AgentSentry’s boundary-local purification.

E Ablation Insights and Behavioral Analysis

The ablations show that temporal aggregation contributes modestly on the evaluated benchmark, while restricting diagnostics to tool actions preserves utility but leaves residual attacks.

  • Temporal aggregation: UA = 88.57 versus 90.36 and ASR = 1.07 versus 0.00 when temporal aggregation is disabled.The authors attribute the modest reduction primarily to AgentDojo’s instance structure, where many failures activate at or immediately after a tool return.
  • Decision signal: UA = 88.21 with tool-action-only signals, but ASR remains 1.07, indicating residual attacks when text-level evidence is excluded.Tool-level evidence is highly informative in Workspace, but the result clarifies that text-level evidence also contributes to intervention.

F Mechanism Case Study: Workspace u20-i6 (Boundary-Level Attribution)

The Workspace case traces an injected calendar-return directive through boundary-level counterfactual diagnostics, showing how mediator influence is localized before high-impact actions and suppressed by purified continuation.

  • Boundary setup: The injected directive becomes visible inside the calendar return, and the first injection-driven action is get_unread_emails at the following boundary.The observed tool chain continues through send_email and delete_email, making the boundary relevant before later high-impact actions.
  • Counterfactual diagnostics: AgentSentry replays each cached boundary under original, masked, sanitized, and original-sanitized regimes without executing tool calls or committing side effects.The next-step proposal is scored under each intervention to obtain boundary-wise causal quantities.
  • Boundary attribution: The nonzero bIE_b entries coincide with boundaries whose proposed actions realize the injected email objective, localizing deviation to the mediator pathway.The single nonzero dACE_b entry occurs where the original input continues producing a benign scheduling step while the probe induces a different outcome.
  • Safe continuation: The alarm occurs immediately before high-impact exfiltration, and purified continuation follows the sanitized baseline from that boundary onward.The deploy curve is a visualization-only splice that follows the observed trajectory until the alarm boundary and then matches the sanitized baseline.
  • Immediate tool-mediated takeover: In the travel trace, sanitized counterfactual execution localizes a sharp contrast at the first contaminated tool-read boundary before a high-impact email invocation.The injected directive is embedded in restaurant review text, but the subsequent proposal shifts toward sending messages unrelated to the dining task.
  • Delayed dialogue-side manifestation: A textual endorsement attack can instead remain inert during intermediate tool calls and become actionable only at the final output commitment boundary.The sanitized contrast removes the unsolicited hotel endorsement, producing a short delay relative to the injection point.

G.3 Discussion and Dataset Limitations

AgentDojo primarily represents short-horizon takeovers, limiting what it can reveal about gradual influence accumulation in long-running, delayed IPI campaigns.

  • Dataset limitation: Most injected objectives become actionable immediately after a contaminated tool read, while a smaller subset appears after brief delays at later commitments.The evaluated cases do not exhibit sustained progressive takeovers accumulating subtly over many turns.
  • Dataset limitation: The benchmark therefore constrains observed lead times through its design rather than through the boundary-anchored detection interface.Future suites should model long-running agents, gradual objective reinforcement, and delayed IPI beyond AgentDojo’s represented regime.
Loading 2602.22724v1…