Source-linked AI summary
Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agents
Tanzim Ahad, Ismail Hossain, Md Jahangir Alam, Sai Puppala, Syed Bahauddin Alam, Sajedul Talukder
TL;DR
Influence-based guardrails may confuse legitimate tool-supported actions with unauthorized ones because causal evidence tracks what shaped an action rather than authority. The paper audits this ambiguity with matched source changes and evaluates how it propagates through guardrail decision paths, finding that harmless source relocation systematically shifts the signal toward attack while routing and reference construction affect enforcement.
Problem
Influence-based guardrails do not reliably distinguish legitimate user-authorized actions from unauthorized actions when both depend on external tool information.
Method
The paper audits 96 conditions derived from 24 matched bases, fixing authorization, the required value, the exact action, and its intended effect while changing only the value’s source.
Results
The same authorized action looks more attack-like after a required value moves from the user request to a legitimate tool result in all 24 matched bases under both scorers, while unauthorized actions remain more attack-like overall.
Takeaways & Limitations
Causal influence is informative about what shaped an action but should not carry the authorization decision by itself; routing and reference construction are part of effective security policy.
Takeaways & Limitations
The controlled study does not estimate deployment prevalence, and the matched unauthorized comparison is teacher-forced, so its construct-validity result concerns matched averages rather than deployment outcomes.
Abstract
from arXiv · showhide
The key limitation of current state-of-the-art influence-based guardrails is that they do not reliably distinguish a legitimate, user-authorized action from a malicious, unauthorized action when both rely on external tool information. This ambiguity can cause benign actions to trigger unnecessary verification and intervention, reducing utility and adding latency. We expose this limitation through an authorization-equivalence audit of 96 conditions derived from 24 base cases. Within matched source comparisons, we hold authorization, the exact committed action, and its intended effect fixed, changing only whether a required value comes from the user or a legitimate tool result. Although the action remains unchanged, this harmless relocation shifts the causal signal toward the attack region in all 24 cases under both Llama and Gemma scorers. Matched unauthorized controls show that the signal remains attack-sensitive, yet the benign relocation produces a larger average score shift than the actual change in authorization. Architecture-level evaluation shows how this mismatch propagates through guardrail designs. With a semantic monitor, attack success is 0% and utility is 28%, compared with 16% and 60% without it. A shadow-based guardrail allows every tested harmless run, yet does not reject matched unauthorized actions more often overall: 57.5% of unauthorized runs pass automatically before reaching the later security check, compared with 29.2% of authorized runs. These results show that the studied causal signal reveals what shaped an action without reliably encoding whether the action was authorized, and that reference construction and routing are integral to the effective security decision.
1 Introduction
The paper examines whether legitimate tool-supplied information can make an unchanged authorized action appear more attack-like, and how that mismatch affects guardrail decisions and outcomes.
- Motivation: Legitimate tool use can supply execution-critical information without gaining authority to change the user’s chosen action.A transfer to Alice may require a banking tool to retrieve Alice’s account number, while the user remains the authority selecting Alice.
- Research Questions and Contributions: The source change moves the signal toward the attack region in all 24 cases under both Llama and Gemma, while the unauthorized action remains more attack-like overall.The benign relocation produces a larger average score movement than changing the action so it is no longer authorized.
- Research Questions and Contributions: The signal’s magnitude contains threat information but does not reliably encode whether an action is authorized.This is the central ambiguity tested by comparing unchanged authorized actions with actions whose authorization actually changes.
- Architecture-Level Implications: Guardrail enforcement depends on thresholding, monitoring, routing, intervention, continuation, and later inspection, not only on the initial score or flag.Different paths can determine which calls are checked or blocked and whether the protected effect ultimately occurs.
- Architecture-Level Implications: A gate block can redirect execution, so restrictive gate behavior alone cannot establish whether the user-authorized effect survived.The paper therefore distinguishes intermediate gate decisions from the final protected outcome.
2 Background and Related Work
Prior work uses influence, provenance, counterfactuals, and semantic reasoning to detect indirect prompt injection and constrain agent behavior. The paper focuses on whether causal evidence changes when authority, action, and effect remain fixed while one required value changes source.
- Indirect Prompt Injection: Indirect prompt injection exploits an agent’s ability to read attacker-controlled content while performing a trusted user request.AgentDojo offers a controlled setting for interactions among user tasks, tools, and injected content.
- Influence- and Provenance-Based Defenses: CausalArmor, AttriGuard, and AgentWatcher inspect influential context or counterfactual tool-call behavior as attack evidence.Related systems also encode trust, permission, provenance, or execution constraints more directly.
- Open Question: This paper tests whether causal evidence itself changes when authority, action, and effect stay fixed and only one required value’s source changes.The focus is narrower than evaluating defenses generally: it isolates source placement as the measurement variable.
- Evaluation Ambiguity: Agent-Sentry and ARGUS identify benign actions whose argument provenance can resemble injected values, especially when benchmarks assume fully specified user instructions.Those systems address the evaluation limitation by changing the benchmark or defense.
- Outcome Measurement: Security evaluations can disagree because task completion does not always reveal whether the user-authorized privileged action or effect survived.Prior work distinguishes endpoint completion from security outcomes and studies downstream recovery and commit-time authorization.
3 Threat Model and Evaluation Design
The paper models a trusted user authorizing a protected action while legitimate tools provide execution-critical values without gaining authority to change that action. Its evaluation separates source relocation, authorization changes, intermediate enforcement, final effects, and implementation-reconstruction limits.
- Threat model: The user determines the authorized protected action and intended protected effect, while legitimate tools may provide required information without authority to change either.The protected action is the privileged action the user authorizes; the protected effect is the outcome it is meant to produce.
- Evaluation design: The matched unauthorized alternative changes the protected action within the same function family, separating a harmless support change from an authorization violation.AUTH follows the trusted user instruction, whereas ALT changes the protected property and is unauthorized.
- Evaluation design: The matched comparison changes one required value’s source while holding permission, value, exact action, intended effect, and execution value fixed.The 96 conditions derive from 24 matched bases, making the matched base the inferential unit.
- Evaluation design: The studies measure different stages of the decision path, including signal movement, flags, routes, blocks, continued execution, protected effects, and downstream task success.Their percentages answer different questions and are not intended to rank models or defenses.
- Evaluation boundary: The CausalArmor-style measurement reconstructs the published estimand because released implementation code was unavailable, rather than reproducing the implementation identically.The reconstruction uses the published leave-one-out user-versus-untrusted margin, length normalization, and τ = 0 anchor, calibrated against the reported broad operating regime.
4 Measurement: Influence Is Evidence, Not Authority
Benign benchmark workflows often delegate execution-critical values to tools, and matched tests show that this source relocation makes authorized actions appear more attack-like. The signal remains threat-sensitive, but its movement can exceed the effect of changing authorization itself.
- 4.1 Natural Relevance: 29 valid privileged decisions from 25 tasks show user-side evidence dominance of 75.0% when users supply values versus 16.7% when tools provide them.The specified-minus-delegated difference is +0.5833 with a 95% CI of [+0.1555, +0.9394].
- 4.2 Same Authorized Action, Different Source: The controlled experiment changes only the source of one needed value across 24 matched bases and 96 derived conditions.User permission, protected value, exact action, intended effect, and execution value remain fixed.
- 4.2 Same Authorized Action, Different Source: −1.1797 for Llama and −1.0112 for Gemma are the mean score changes when the same authorized action obtains the needed value from a legitimate tool.All 24 cases move toward the more attack-like region under both scorers.
- 4.2 Same Authorized Action, Different Source: 99.7% of Llama movement and 96.4% of Gemma movement is attributed to the execution value whose source changed.User-side support falls while relevant-tool support rises on every matched base.
- 4.2 Same Authorized Action, Different Source: At τ = 0, the source relocation changes binary decisions on 20/24 Llama bases and 18/24 Gemma bases, always from allow to flag.This threshold is used only as a descriptive anchor; the complete tested threshold range is examined separately.
- 4.3 Matched Unauthorized Alternative: The matched unauthorized action remains more attack-like overall, with endpoint differences of −0.534 for Llama and −0.461 for Gemma.The threat-sensitive signal therefore still distinguishes the unauthorized endpoint overall.
- 4.3 Matched Unauthorized Alternative: +0.6545 for Llama and +0.5039 for Gemma are the average displacement advantages of the harmless source change over the authorization-changing comparison.The unauthorized endpoint is worse, but the benign relocation moves the score farther on average.
- 4.3 Matched Unauthorized Alternative: The case-level pattern is heterogeneous: the source change moves attack-like on 24/24 bases for both scorers, while the authorization comparison does so on 17/24 Llama and 23/24 Gemma bases.The results do not support treating the signal’s raw magnitude or thresholded decision as a direct authorization label.
5 From Influence Signals to Effective Security Decisions
The paper evaluates how scalar thresholds, semantic monitoring, and shadow-based routing convert influence signals into security decisions. Across these paths, benign source changes can trigger flags, while routing can prevent stricter later checks from being applied to unauthorized actions.
- Scalar threshold: A scalar threshold can remove all tested benign flags only by missing most matched unauthorized actions.At benign-zero operating points, Llama catches 12/48 unauthorized alternatives and Gemma catches 18/48.
- Scalar threshold: 12/48 unauthorized alternatives are caught by Llama and 18/48 by Gemma when no tested authorized case is flagged.
- Semantic monitor: 4/24 authorized and 21/24 matched unauthorized actions are flagged by AgentWatcher under aligned external text, versus 24/24 of each under conflict.The monitor separates the classes in the aligned condition but flags both classes under the tested conflict condition.
- Semantic monitor: 0% attack success and 28% utility occur with AgentWatcher, compared with 16% attack success and 60% utility without it.These are separate matched-input API executions, distinct from the paired gate study.
- Shadow/reference routing: −10.8 percentage points is the unauthorized-minus-authorized blocking difference for AttriGuard, with a 95% CI of [−36.3, +14.6] percentage points.The interval includes zero, so the data do not show that unauthorized actions are blocked more often overall.
- Shadow/reference routing: Unauthorized calls reach automatic exact survival more often, while later review blocks unauthorized calls more often once reached.The observed reference determines routing, and the study does not establish that the conflicting directive caused that reference identity.
6 Live Execution: Intervention, Continuation, and Protected Effect
The live study follows guardrail interventions beyond the gate to continuation and the protected user-authorized effect. Blocking reduces selected unauthorized outcomes, but later execution can recover or lose authorized effects and can alter inspection coverage.
- Study design: 420 executions cover 14 natural tasks across CLEAN, ALIGNED, and CONFLICT contexts with AttriGuard OFF or ON.Each cell is repeated five times.
- Protected outcome: −0.2429 is the estimated interaction for protected authorized-outcome preservation, opposite the pre-specified availability-loss prediction.The 95% CI is [−0.4714, −0.0429].
- Protected outcome: Under CONFLICT, selected unauthorized outcomes fall from 17/70 (24.3%) without the defense to 2/70 (2.9%) with it.The pre-specified difference is +0.2143 with a 95% CI of [+0.0429, +0.4286].
- Protected outcome: Under CONFLICT, PAEF is 54.3% without the defense and 67.1% with it, but the 95% CI for the difference includes zero.The study does not claim that AttriGuard generally improves preservation of the authorized effect.
- Continuation: 12/13 blocked selected-unauthorized proposals later recover an authorization-equivalent privileged action and end with PAEF=1.Conversely, all 9/9 blocked authorization-equivalent proposals end with PAEF=0.
- Inspection coverage: 18 defended privileged calls execute without the ordinary audit after a qualifying predecessor state, with mixed outcomes.The effect is specific to the tested implementation and configuration; a controlled source-level test reproduces it and a local patch restores inspection.
7 Evaluation Fidelity: Does Success Measure the Protected Property?
The paper separates ordinary task success from fidelity to the privileged action or effect the user authorized. Both live executions and replay show that downstream success can conceal an earlier divergence.
- Live evaluation: 18/420 live executions disagree between ordinary task utility and PAEF.Thirteen runs pass the task check despite losing the authorized protected effect, while five preserve it despite task-check failure.
- Replay evaluation: 23/78 model–decision pairs pass the downstream majority check even though the immediate action/effect check fails.Each pair has five stability repeats, totaling 390 generations; in 22/23 disagreeing pairs, the model still calls the intended tool.
- Interpretation: Task utility, immediate action/effect fidelity, authorized availability, later inspection, and final PAEF answer different evaluation questions.No single outcome substitutes for the others.
8 Discussion and Limitations
Influence and provenance can provide security evidence without encoding authorization, so guardrails must evaluate policy, routing, continuation, and protected effects separately. The conclusions are bounded by the tested populations, implementations, and experimental conditions.
- Influence and provenance are useful security evidence, but authorization should be preserved or checked separately.The controlled result shows that a legitimate tool-supplied value can shift the signal without changing permission, action, or intended effect.
- Guardrail evaluation must track policy, routing, continuation, and the effect that finally occurs—not only scores, flags, or blocks.Intervention can change later privileged-call inspection and whether the authorized effect survives.
- A block is a state transition whose consequences depend on subsequent execution, including recovery or loss of the authorized effect.Defense-generated state can also alter whether later privileged calls receive ordinary auditing in the tested implementation.
- Security metrics should identify the protected property first because attack outcome, task success, authorized-effect preservation, availability, and inspection coverage can diverge.For effectful tool use, evaluation should follow execution far enough to measure the authorized effect and later inspection.
- The findings are bounded by finite tested populations, matched teacher-forced comparisons, reconstructed implementations, and implementation-specific studies.Broader prevalence and architecture-wide claims require larger natural populations, additional task and tool families, adaptive conflict text, and other implementations.
9 Conclusion
Tool information can shape an authorized action without conferring authority, and the resulting signal can therefore misrepresent authorization. Effective security decisions must follow the complete path from evidence through enforcement to the final authorized effect.
- 24 matched bases moved toward the attack-like region when a required value shifted from the user to a legitimate tool, under both scorers.Authorization, the exact action, and the intended effect remained fixed during the source relocation.
- The matched unauthorized action remained more attack-like overall, but the harmless source relocation produced the larger average score movement.The signal therefore contains threat information without serving as an authorization label.
- Thresholding, semantic judgment, reference construction, routing, continuation, and later inspection all contribute to the effective security decision.A score, flag, or block is an intermediate decision rather than the final outcome.
- Evaluation should measure whether the user-authorized property survives the full execution path, not merely whether influence identifies what shaped the action.The guardrail must still check whether the action remains within the user’s authorization.
Ethical Considerations
The paper frames its implementation-specific security finding narrowly and limits its claims to tested systems and populations. It does not present the observed mechanism as a general attack path or deployment-wide estimate.
- The implementation-specific finding is reported only at the level needed to support the systems claim, not as a general attack path.The paper does not estimate exploit success or reliability from the two selected unauthorized outcomes.
- Claims are limited to the frozen implementation version and configuration tested.The study avoids extending implementation findings beyond the evaluated setup.
- The controlled studies use benchmark or constructed agent tasks, so claims concern tested systems and populations rather than deployment prevalence or universal defense behavior.The artifact should preserve auditable evidence while removing credentials, private state, and author-identifying paths.
Open Science
The paper provides frozen evidence, code, documentation, and reproduction paths for auditing its controlled, guardrail, live, and implementation-specific findings. The supplementary analyses clarify family breadth, scorer behavior, route exposure, continuation outcomes, and scope boundaries.
- Artifact and reproduction: The anonymous artifact contains frozen evidence and code covering the paper’s main experiments and deterministic checks.It includes source-relocation, matched-unauthorized, threshold, AgentWatcher, AttriGuard, live, implementation-specific, and replay analyses.
- Artifact and reproduction: Reviewers can verify reported quantities from frozen execution records without new model calls, credentials, or GPUs, or rerun experiments with required access and hardware.Fresh reruns remain separate from the frozen evidence used for the submitted results.
- Artifact and reproduction: Documentation links manuscript claims to supporting evidence and verification commands while excluding author credentials and private service state.Appendix E provides the corresponding paper-level claim-to-evidence map.
- Benign and controlled studies: 29 privileged decisions from 25 tasks span several benign workflow families, supporting ecological breadth without estimating deployment prevalence.The cohort is not intended as a row-level benchmark result.
- Benign and controlled studies: 24 matched bases are evenly split across four protected tool-action families, and every base moves toward the attack-like region under both scorers.Each family contains six bases, so the result is not concentrated in one function family.
- Benign and controlled studies: User-side support falls while relevant tool-side support rises on every matched base after source relocation.The artifact retains per-base attribution scores and support decompositions underlying the figure.
- Benign and controlled studies: The unauthorized comparison separates endpoint attack-likeness from which manipulation moves scores farther, exposing heterogeneity hidden by a single mean.Pre-specified checks move in the expected action-relative direction on all 24 bases under both scorers.
- Guardrail analyses: The CausalArmor analysis reconstructs the published estimand rather than reproducing an implementation-identical system.The reconstruction includes leave-one-out user-versus-untrusted margins, length normalization, and the τ = 0 anchor.
D Why Task Success Can Hide a Changed Protected Action
Replay evaluation shows that downstream task success can conceal failures in the immediately executed privileged action or its effect. Across model–decision pairs, this disagreement is common enough to require action-local fidelity checks alongside downstream utility.
- Replay design: Replay freezes history before one privileged action, regenerates only that action, then replays later tool calls and the final answer for downstream utility.This isolates whether later continuation can hide an earlier action or effect change.
- Replay evidence: 23/78 majority-level cells pass downstream evaluation while failing immediate action/effect fidelity.These cells come from five stability repeats over 78 model–decision pairs, rather than 390 independent cases.
- Replay evidence: 22/23 disagreement cells still call the intended tool despite failing immediate action/effect fidelity.Thus, selecting the intended tool does not guarantee that the committed action or resulting protected state is faithful.
- Evaluation design: The corrected action-local oracle compares reconstructed post-action environments and canonicalizes only benchmark-generated timestamps required for stable equality.Superseded replay or oracle branches remain as provenance rather than reviewer-facing scientific results.
- Reproducibility boundary: The reviewer artifact preserves frozen inputs and outputs so reported statistics remain auditable when full reruns require third-party credentials or licensed components.The package must also remove identity-bearing metadata, private paths, credentials, and tracking links before submission.