Readable Before Actionable: Causal Tracing of Indirect Prompt Injection

Introduction

• Indirect prompt injection arises when commands in tool outputs, messages, or retrieved memory acquire instruction authority despite the user's task being to process that content.Whether a command should be executed depends on its role in the task, not wording alone.Source: §1 · p. 1
• The study controls content and format in role-readout probes, compares full-state, role-aligned, and orthogonal patches on separate tool-choice pairs, and evaluates attack interventions separately with fixed model weights.Separating role-aligned patches from full-state swaps helps distinguish their effects from other information transferred from donor states.Source: §1 · p. 1
• In controlled Qwen tasks, role decoding precedes strong tool-choice effects from role-component patches, establishing a readability–actionability gap.Actionability here means sensitivity to the specified edit, and the ordering is across network depth rather than reasoning steps.Source: Figure 1 · p. 2, §1 · p. 2
• In attack trajectories, intervention effects depend on edited states: AgentDojo results are position-sensitive, while longer Qwen trajectories show a larger late-layer single-position rebound and span-wide or repeated edits reduce attack success.The span-wide and repeated edits are compared on the same evaluation set.Source: §1 · p. 2
• Role decoding survives removal of the learned channel subspace, but separate intervention tests favor channel-matched directions over transferred directions.The section treats readout and intervention transfer as separate tests.Source: §1 · p. 2

• TaskTracker, role-perception, and IPI-exposure studies motivate testing whether source role remains decodable when content and format cues are controlled (Abdelnabi et al., 2025; Ye et al., 2026; Dong et al., 2026).Source: §2 · p. 2
• Task-relative authority differs from harmfulness: the same command may be authorized or data, so probe accuracy alone cannot identify where an edit changes tool choice.Prior work distinguishes harmfulness recognition from refusal execution and risk readout from error correction; this study measures readout and intervention effects across depth.Source: §2 · p. 2
• Because patch effects depend on the counterfactual and outcome, the study compares full-state, role-aligned, and orthogonal patches to isolate role-component sensitivity.Prior work notes that full-state swaps transfer more than role information and that interpretability may not predict steering utility (Zhang & Nanda, 2024; Makelov et al., 2024; Grant et al., 2026; Wang et al., 2026b).Source: §2 · pp. 2–3
• AgentDojo evaluates fixed directional edits across attack positions, trajectories, and external sources, complementing controlled tool-decision tests (Debenedetti et al., 2024).This extends evaluations beyond defenses that separate trusted from untrusted inputs or steer internal states (Chen et al., 2025a;b; Shi et al., 2025; Wang et al., 2026a; Lu et al., 2025).Source: §2 · p. 3

Problem setting

• Source role is the span’s authorized status as instruction or data, whereas channel identifies its external source or environment; matched-role comparisons vary authorization, and channel comparisons test transfer.A Slack message may arrive through a tool response, so role and transport mechanism are not mutually exclusive.Source: §3 · p. 3
• The white-box setup fixes model weights and inference-time hooks, with attacker-controlled external content and a defender tasked with avoiding injected commands; Qwen-2.5-7B is primary, with other models and AgentDojo trajectories used for comparisons (Qwen Team, 2025; Llama Team, AI @ Meta, 2024; Gemma Team, Google DeepMind, 2024; Debenedetti et al., 2024).Attacks are non-adaptive unless otherwise stated; controlled prompts vary role, content, format, layer, and token position.Source: §3 · p. 3

Causal tests

• Interventions edit residual-stream states at a layer and token position, then continue the unchanged model; layer sweeps hold position fixed, while position sweeps hold layer fixed.The pre-action token is the final serialized input token before the next action; patching replaces a donor state, while directional editing applies a donor-free update.Source: §3 · p. 3
• Tool-choice effects are measured by the patch-induced donor-versus-recipient logit-margin shift; binary sign flips and six-tool top-1 changes are separate outcomes, and a larger margin need not change the choice.With six tools, the donor must outrank every other candidate to become top choice.Source: §3 · pp. 3–4
• Agent trajectories use attack success rate and benign utility; paired position comparisons reuse trajectories and use bootstrap confidence intervals and two-sided exact McNemar tests, while position and suite evaluations use different trajectory sets.Probe weights are fitted for readout, not used as edit directions; the position tests use 96 trajectories and suite evaluations use 240, so their baseline attack rates differ.Source: §3 · pp. 3–4

• The probe uses 1,440 prompts per model, varying role and plain-versus-quoted format across matched contents, with disjoint content groups and a role–format reversal split.The models were Qwen-2.5-7B and Llama-3.1-8B; the probe reads pre-action residuals after serialization without modifying model state.Source: §4 · p. 4
• The intervention direction is the normalized difference between instruction and data class means at the same layer and position, estimated without evaluation trajectories.It points toward the instruction class and was applied without retraining in the AgentDojo outcome-separation analysis.Source: §4 · pp. 4–5
• AUROC was 0.924 for Qwen and 0.918 for Llama on the strictest role–format reversal split, versus 0.508 for the pooled structural baseline.The structural classifier used measured input features, while hidden-state decoding transferred across content, paraphrase family, and counterfactual format swaps.Source: §4 · p. 4, Table 3 · p. 5
• On 192 AgentDojo trajectories, hijacked–resisted AUROC was 0.817 at L12, 0.727 at L16–L20, and 0.835 at L24 (95% CI [0.772, 0.893]).Separation was near chance at L4–L8; this outcome association motivates an intervention test but does not establish an effect on attacks.Source: §4 · p. 5

5.1 LAYERWISE SENSITIVITY TO STATE PATCHING

• 100% sign-flip rates emerge at L22 for Qwen-1.5B and L18 for Qwen-7B, after little effect through L14.At Qwen-1.5B L18, matched-state patching shifts the logit difference by 25.5, versus −0.87 ± 0.21 for a random direction; non-final-position patches shift it by less than 0.1. At L20, six-tool patches flip 95.4% of target decisions and place the patched-in tool first in 96.8% of cases; Llama-3.1-8B shifts rise from 2.3 at L14 to 14.3 at L16.Source: §5.1 · p. 5, Figure 2 · p. 6

5.2 PATCHING THE ROLE-ALIGNED COMPONENT

• At L18–L22, role-direction patches recover 28.51–38.64 LD and flip 82–98% of decisions, versus 4–7% flips for perpendicular patches.At L8–L16, the parallel component stays below 1 LD and never flips the decision; the role direction was estimated from separate instruction–data pairs. Role is decodable before these controlled patches produce large tool-choice changes.Source: §5.2 · p. 6, Figure 2 · p. 6
• At L20, a hijacked-state swap raises resisted AgentDojo-Slack trajectories to 65.4% ASR, while the reverse swap lowers hijacked trajectories to 15.2%.Random-direction, same-layer random-residual, and non-final-position patches yield 5.6%, 6.1%, and 7.5% ASR, respectively, in the resisted-recipient comparison. Full-state swaps retain all donor–recipient differences and do not isolate source role from other state content.Source: §5.2 · p. 6

6.2 TRANSFER TO HELD-OUT ATTACKS WITHIN A CHANNEL

• On Slack, dSlack lowers ASR from the 34.6% baseline to 21.2%; on Workspace, dWorkspace lowers ASR from 31.2% to 18.8%.Controlled-role, random-direction, and L8 controls remain near the Slack baseline, while benign utility changes by less than two percentage points.Source: Figure 3 · p. 7, §6.2 · p. 8
• Leave-attack-family-out tests retain 70–75% of the all-family ASR reduction; with 32 training trajectories, Slack ASR falls by 9.8 points versus 13.4 with 240.The sample-efficiency sweep keeps evaluation fixed; its error bars measure variation across five training subsets.Source: Figure 3 · p. 7, §6.2 · p. 8
• A Llama-specific Slack direction lowers ASR from 32.5% to 19.5%, while controlled-role and random directions leave it near baseline; Gemma’s matched edit lowers ASR from 82.5% to 34.0%.Llama benign utility changes by 2.2 points; Gemma’s results are endpoints rather than a full depth sweep.Source: Figure 3 · p. 7, §6.2 · p. 8

6.1 TOOL-CALL EFFECTS AND POSITION CONTROLS

• On controlled Qwen-2.5-7B, the L24 edit lowers ASR from 46% to 26% without changing benign-task accuracy, and malicious argument use from 88.2% to 12.4%.Benign argument accuracy changes from 92.5% to 91.2%; the Qwen-1.5B estimate remains within sampling uncertainty.Source: §6.1 · p. 7
• Pre-action and injected-span edits reduce AgentDojo ASR by 50.0–56.3 percentage points, while random-token edits change it by about one point (p = 1.0).Benign utility remains 95.8% on Qwen and 94.8% on Llama; Qwen paired ASR differences are −53.1 points and −56.3 points, both with exact McNemar p ≤8.88×10−16.Source: §6.1 · p. 7

6.3 SINGLE-POSITION EFFECTS IN LONGER TRAJECTORIES

• On 96 matched Qwen trajectories, ASR rises from 31.3% at L20 to 54.2% at L24, especially in step-3+ trajectories; span-wide and repeated edits lower L24 ASR to 32.3% and 27.1%.Step-3+ ASR rises from 38.6% to 84.1%, while step-1 remains near 25%; tool-wrapper attention rises from 0.312 to 0.445, but does not identify the components behind the rebound.Source: §6.3 · p. 8
• On Llama-3.1-8B, the fixed direction does not reduce adaptive-attack ASR (26.3% to 27.4%), and obfuscation weakens the tool-output direction.Source: §6.3 · p. 8

• After channel-subspace removal, role AUROC is 0.958 versus 0.972 raw, and residualization retains 96.1% of the original ASR reduction.The source-role direction and channel-identity subspace have a principal angle of 81.2◦, with squared projection cos2(81.2◦) ≈0.0234.Source: §7 · p. 8, Figure 4 · p. 9
• Residualized role AUROC ranges from 0.932 to 0.958 across tested Qwen-Instruct serializations and is 0.887 for Qwen-Base.These results support role readout beyond the learned channel subspace under the tested conditions.Source: §7 · p. 8
• Matched directions lower ASR in their own channels—tool output 85.2% to 15.4%, Slack 33.3% to 21.8%, and memory 62.4% to 18.5%—while mismatched directions have much smaller effects.The obfuscated condition tests a change in attack form within the tool-output setting, not a fourth source channel.Source: §7 · pp. 8–9, Figure 4 · p. 9
• A memory-matched direction lowers memory ASR from 62.4% to 21.5% on Qwen and from 58.7% to 19.8% on Llama, preserving benign accuracy.It has little effect on tool-output ASR; the larger AgentDojo evaluation in §6 shows the same matched-versus-mismatched contrast.Source: §7 · p. 9
• SVD interventions yield diminishing within-suite ASR reductions as rank increases, while cross-suite rank-1 directions remain near baseline.Source: §7 · p. 9

Discussion and limitations

• A useful probe does not specify an effective edit, which must be tested on a particular state component, position, and downstream outcome.This distinction holds even when model weights remain fixed.Source: §8 · p. 9
• Role-aligned patching shows sensitivity to an edit, but not how an unmodified decision uses that direction or a complete behavioral circuit.The evidence locates behavior-sensitive states, not a universal action-commitment layer.Source: §8 · p. 9
• Adaptive and strongly obfuscated attacks can bypass a fixed direction, and the Base/Instruct comparison covers Qwen rather than post-training methods across model families.The findings concern specified interventions on open-weight decoder-only models.Source: §8 · p. 9

9 CONCLUSION

• Source-role decoding precedes strong tool-choice effects from role-aligned patches in controlled Qwen tests, while behavioral effects vary with position, trajectory length, and direction source.Role decoding survives channel removal, but effective directions transfer poorly across the tested channels; a readable signal is a candidate for intervention, not evidence of behavioral effect at every site or task.Source: §9 · p. 9

AI USE STATEMENT

• The authors report using generative AI for brainstorming, experimental-design feedback, data processing, literature discovery, and editing, while taking responsibility for the paper’s content.They report reviewing all text, code, figures, citations, and claims; programmatic checks recompute headline values from sample outputs.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

• The ethics statement notes dual-use implications and reports open-weight models and benchmark or constructed tasks, with no human subjects or external deployment.The authors also report the threat model, intervention assumptions, and failure modes.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

• The appendix specifies datasets, splits, checkpoints, activation positions, interventions, statistics, and compute, and reports controls and supplementary analyses supporting the main results.Source: REPRODUCIBILITY STATEMENT · p. 10

APPENDIX CONTENTS

• The appendix contents guide groups supplementary analyses under robustness and generalization, and reporting and additional audits.The contents page describes its section titles and page numbers as clickable.Source: APPENDIX CONTENTS · p. 12
• The appendix lists structural-only and hidden-state source-role AUROC analyses, with structural results pooled across balanced Qwen and Llama sets.The table caption says model-level structural values show the same near-chance pattern in each model; it does not provide specific AUROC values here.Source: Table 4 · p. 13
• The appendix also reports role readout after channel removal across Qwen-2.5-7B tuning and serialization conditions.The table caption defines “Residual” as role AUROC after removing the learned channel-identity subspace; it gives no values in the supplied passage.Source: Table 5 · p. 13

A COUNTERFACTUAL CONTROLS AND CROSS-MODEL REPLICATIONS

• The structural-feature classifier stays near chance, while role readout remains above chance after channel removal and Instruct intervention retains at least 95.8% of its ASR reduction.Across Native, Wrapper, and Plain serializations, baseline ASR is 84.4% and matched L20 intervention ASR is 31.3%, 32.3%, and 34.4%, respectively; Qwen-Base’s peak action-logit shift is +32.4 at L22.Source: A COUNTERFACTUAL CONTROLS AND… · p. 13
• On AgentDojo, L24 probe AUROC reaches 0.835, while span-wide and repeated patches lower ASR relative to single-token patching on the matched set.Pre-action states yield the highest probe AUROC, injected-span interventions yield the lowest ASR, and random-position readout is near chance with no detected intervention effect; the single-token L24 rebound is largest on longer trajectories.Source: A COUNTERFACTUAL CONTROLS AND… · p. 13, Table 8 · p. 14

B COMPONENT-WISE PATCHING

• For Qwen-2.5-7B, parallel-component patches yield much higher flip rates than perpendicular-component patches at L18–L22.Table 9 reports component-wise patching results on 100 examples.Source: Table 9 · p. 14, B COMPONENT-WISE PATCHING · p. 14

C PER-ATTACK INTERVENTION RESULTS; D INTERVENTION STRENGTH AND LAYER CONTROLS

• At L24 with λ = 32, Qwen-2.5-7B reduces synthetic-injection ASR by 43% relative, with effects concentrated in imperative payloads and no change in benign accuracy.ASR for “reply with INJECTED” and “output 1492” falls from 83% to 17–33%; verbatim-string attacks fall from 100% to 50%, while structural-format and Pig-Latin payloads remain unchanged at zero baseline ASR. The strength sweep reports a 20-percentage-point ASR fall for Qwen-7B, while Qwen-1.5B’s 6-point estimate remains within sampling uncertainty.Source: C PER-ATTACK INTERVENTION RESULTS · p. 15, Table 10 · p. 15, D INTERVENTION STRENGTH AND LAYER… · p. 15

E THREAT MODEL; F LARGER-SCALE AGENTDOJO EVALUATION

• The threat model assumes white-box inference-time access with fixed weights, and the larger AgentDojo evaluation reuses Qwen’s fixed L20, λ = 32 setting across suites.The leave-attack-family-out protocol estimates each direction without examples from its held-out test family, measuring transfer beyond training attacks; reported paired p-values use bootstrap comparisons.Source: E THREAT MODEL · p. 15, F LARGER-SCALE AGENTDOJO EVALUATION · p. 16, Table 11 · p. 16

G MULTI-TOOL PATCHING

• Multi-tool patching tests two- and six-tool choices, and Llama-3.1-8B’s mean patched calc logit-difference crosses zero at L16.The Llama shift goes from base = −7.95 LD to patched = +6.36 LD at L16; the six-tool task covers weather, calc, email, calendar, search, and file.Source: G MULTI-TOOL PATCHING · p. 16, Table 14 · p. 17, Table 12 · p. 17

H SAMPLING ROBUSTNESS; I MODEL-SCALE ANALYSIS

• The L24 intervention produces similar ASR reductions and small utility changes across greedy and stochastic decoding, while ASR is not monotonic across model scale.Among Qwen-2.5 models with at least 90% benign accuracy, baseline ASR rises to 47.0% at 7B then falls to 42.5% at quantized 14B; Llama-3.1-8B has 27.1% ASR at matched benign accuracy.Source: H SAMPLING ROBUSTNESS · p. 17, Table 15 · p. 18, I MODEL-SCALE ANALYSIS · p. 18

J FAILURE ANALYSIS

• The synthetic-tool-output direction transfers poorly to AgentDojo-Slack and memory poisoning, and conservative phrasing or truncated arguments trigger automatic retries in about 4% of intervened synthetic trajectories.AgentDojo-Slack ASR is 33.8% against a 34.6% baseline; memory poisoning responds to a memory-specific direction.Source: J FAILURE ANALYSIS · p. 19

K OBFUSCATION ROBUSTNESS

• Obfuscation weakens the fixed tool-output direction, while directions estimated on obfuscated examples reduce ASR to 22.4–34.2% on stronger levels.The reported decline in Cohen’s d indicates weaker separation along the fixed direction, not the absence of other linearly decodable attack features.Source: K OBFUSCATION ROBUSTNESS · pp. 19–20, Table 18 · p. 20

L SOURCE-ROLE PROBES IN THE MEMORY CHANNEL

• Memory-probe cross-template accuracy rises from 50.8% for 1.5B and 50.3% for 7B at L0 to a late-layer peak, while the 1.5B tool-output probe reaches 88.3% at L0.The memory probe uses five content-grouped folds and cross-template evaluation; these accuracies differ from the counterfactual AUROCs in §4.Source: L SOURCE-ROLE PROBES IN THE MEMORY… · p. 20, Table 19 · p. 21

M TOOL-OUTPUT VERSUS MEMORY LAYER PROFILES

• On Qwen-2.5-1.5B, the tool-output projection gap grows from +1.1 at L18 to +3.95 at L26, while memory-poisoning Cohen’s d stays near 0.3–0.6 without a sharp inflection.These are different readout metrics and their magnitudes are not directly comparable; poor cross-channel direction transfer is established by the separate intervention test in §7.Source: M TOOL-OUTPUT VERSUS MEMORY LAYER… · p. 20

N BROADER IMPACTS

• The paper flags dual-use risks from reporting intervention details and requires model-, position-, and channel-specific validation for operational use.It describes the intervention as diagnostic and recommends combining model-level controls with structured input handling, inference-time filtering, and revalidation after model, template, or system changes.Source: N BROADER IMPACTS · p. 20

O COMPUTE RESOURCES

• Experiments ran on one research node with eight NVIDIA GPUs—six with 24 GB and two with 80 GB—and Table 21 reports approximate GPU-hours by result type.AgentDojo, multi-tool, memory, and sampling evaluations used the same hardware, with settings recorded in run metadata.Source: O COMPUTE RESOURCES · p. 20, Table 21 · p. 22

P LICENSES AND THIRD-PARTY ASSETS

• The experiments use third-party models, datasets, and tools under their respective licenses, and the TaskTracker dataset was not redistributed.The listed assets include Qwen2.5-Instruct, Meta-Llama-3.1-8B-Instruct, Gemma-2-9B-It, AgentDojo, TransformerLens, and nnsight.Source: P LICENSES AND THIRD-PARTY ASSETS · p. 22

Q STATISTICAL PROCEDURES

• For position tests with n = 96 and c = 0, confidence intervals use the empirical paired-bootstrap distribution, with 2.5th and 97.5th percentiles; larger AgentDojo retains its recorded paired-bootstrap tests.Utility intervals resample benign tasks.Source: Q STATISTICAL PROCEDURES · pp. 22–23

R AGENTDOJO DIRECTION CONSTRUCTION

• AgentDojo directions dslack and dworkspace are estimated from hijacked-versus-resisted trajectories using the L20 residual at the last token of the serialized input.This behavior-indexed contrast differs from drole estimated from counterfactual role pairs; a 240-trajectory training pool is separate from each suite’s 240-trajectory evaluation set, with layer sweeps and controls testing suite specificity.Source: R AGENTDOJO DIRECTION CONSTRUCTION · p. 23

S DIRECTION SAMPLE EFFICIENCY

• With 32 Slack trajectories, the estimated direction reaches cosine 0.852 ± 0.041 and reduces ASR by 9.8 ± 1.5 points, or 73% of the full-data effect.Source: S DIRECTION SAMPLE EFFICIENCY · p. 23

T BIDIRECTIONAL PATCHING ON AGENTDOJO TRAJECTORIES

• In matched AgentDojo-Slack pairs, pre-action swaps have their largest effect at L18–L22, while ASR moves toward the unpatched outcome at L24.For resisted recipients at L20, ASR remains low with a random direction (5.6%), same-layer random residual (6.1%), or non-final-position patch (7.5%).Source: T BIDIRECTIONAL PATCHING ON AGENTDOJO… · p. 23, Figure 9 · p. 24

U TOOL-ARGUMENT INJECTION

• Malicious-argument use falls from 88.2% to 12.4%, while benign accuracy changes from 92.5% to 91.2% and JSON validity from 98.1% to 97.6%.Source: U TOOL-ARGUMENT INJECTION · p. 24

W LOW-RANK INTERVENTION ANALYSIS

• Rank-1 SVD intervention lowers AgentDojo ASR from 34.6% to 21.8%; ranks 2, 4, and 8 reach 20.4%, 19.8%, and 19.5%.Most measured within-suite ASR reduction is obtained at low rank; Table 24 uses an SVD rank-1 vector, whereas Figure 3 uses a mean-difference direction.Source: Table 24 · p. 25, W LOW-RANK INTERVENTION ANALYSIS · p. 25

V TRAIN–TEST SEPARATION FOR DIRECTION ESTIMATION

• Train–test splits hold out synthetic tool-output content and trajectories but only some attack families; AgentDojo and memory directions use disjoint trajectories and leave-family-out splits to assess unseen attacks.The synthetic split supports localization and operating-point selection.Source: V TRAIN–TEST SEPARATION FOR DIRECTION… · p. 25, Table 23 · p. 25

W LOW-RANK INTERVENTION ANALYSIS

• Cross-channel rank-1 transfer is limited: Slack→Workspace yields 30.5% ASR and Workspace→Slack yields 32.2%, both near baseline.Source: W LOW-RANK INTERVENTION ANALYSIS · p. 25