PaperlayerView exact revision on arXiv

Source-linked AI summary

CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering

Mark Russinovich

arXiv:2609.36570v1cs.CRcs.AI

TL;DR

Agents can treat instructions in retrieved content as authoritative, creating indirect prompt-injection risk. CounterSteer fits and causally validates a per-model activation direction, then subtracts it from tool-result tokens during prefill. It sits on the measured security–utility frontier among inference-time defenses on capable models, while parameter manipulation remains a boundary.

  • Problem

    Agents may treat retrieved web pages, emails, or documents as instructions even though the attacker never touches the prompt, making indirect prompt injection a ranked security risk for deployed LLM applications.

    Source: §I · p. 1

  • Method

    A five-step recipe selects and validates a per-model direction, then deploys it as an always-on edit to tool-result tokens during prefill.

    Source: A. The Recipe · p. 4

  • Results

    CounterSteer sits on the measured security–utility frontier among inference-time defenses on capable models; gpt-oss-20b AgentDojo compromise was 0.079 defended versus 0.475 undefended.

    Source: §IX · p. 12, Appendix A · p. 16

  • Takeaways & Limitations

    For open-weights deployments with tool-span tagging, CounterSteer is an always-on layer against instructional takeover; compose it with provenance controls when argument integrity is primary.

    Source: §IX · p. 12

    Parameter manipulation is only partially resisted, and delegated-authority Class B is unprotected by design.

    Source: §VII · p. 11

Abstract

from arXiv · show

Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that suppresses this behavior inside the model. Per model, a five-step recipe fits a residual-stream direction from paired episodes differing only in whether an embedded instruction is followed, and retains it only if it passes pre-specified causal and capability gates. At deployment, the direction is subtracted from every tool-result token during prefill. The edit is always on--there is no detection decision to evade--and requires no fine-tuning, auxiliary model, or added tokens, only white-box serving and tool-result span boundaries. Across five open-weights models (8B-106B, five vendor lineages), held-out attack success falls from 0.21-1.00 undefended to 0.00-0.17 defended, and AgentDojo compromise rate from 0.10-0.49 to 0.006-0.079, at 93-100% typography-normalized benign utility, with larger task-dependent costs when reasoning over steered content. A benchmark-level adaptive attacker reaching 0.67-0.73 undefended is held to roughly a quarter of that on the two most deeply evaluated models. Among the defenses we measured on capable models, those achieving lower compromise rates either lost 22-89% of benign utility or fine-tuned the served weights. White-box gradient attacks through the deployed vector compromise at most 2 of 52 episodes, and none of 2,052 replayed human red-team attacks succeeds. CounterSteer largely neutralizes instructional takeover: a black-box framing search cracks 3 of 18 development samples. Parameter manipulation--attacker-chosen arguments in otherwise legitimate calls--is only partially resisted (13 of 18); the decision becomes linearly readable at argument emission but not at the examined pre-generation sites, and is not removed by the tested prefill- or decode-time steering, motivating argument-provenance controls.

I. INTRODUCTION

Indirect prompt injection occurs when agents treat retrieved content as instructions, even though attackers never touch the prompt. CounterSteer targets this role confusion with a per-model, causally validated activation edit, while the paper distinguishes it from other injection-following cases.

I. INTRODUCTION

  • Retrieved pages, emails, and documents can address agents directly, leading them to treat untrusted data as instructions; indirect prompt injection is ranked a top security risk for deployed LLM applications.The attacker publishes content for the agent to fetch rather than touching its prompt.Source: §I · p. 1
  • CounterSteer fits a per-model direction from paired episodes that differ in whether an embedded instruction is followed, then applies a fixed edit to every tool-result token during prefill.The procedure screens candidates, applies causal and capability gates, and requires only tool-result span boundaries; it uses no auxiliary model, fine-tuning, extra calls, or prompt tokens.Source: §I · pp. 1–2
  • CounterSteer is distinguished from nearby activation defenses by causal validation and always-on application, rather than probe-gated intervention or an attribution stage.The authors report that some directions separate attacks but do not causally steer.Source: §I · p. 1
  • The paper organizes injection-following as a three-part map and scopes CounterSteer to the two role-confusion classes.The map’s “what removed it” describes measured findings, not a completeness claim.Source: §I · pp. 1–2, Table I · p. 2
  • The evaluation includes defense-aware adaptive attacks and traces parameter manipulation’s follow decision as readable at argument emission, but not at examined pre-generation sites.The paper contrasts that finding with a benchmark-level adaptive attacker and white-box evaluation of the deployed vector.Source: §I · pp. 2–3

A. Setting

  • The attacker controls only content returned by tools, and tool hijack and parameter manipulation are the two role-confusion objectives counted as compromise.Contamination into another legitimate call is tracked separately, not counted as compromise.Source: A. Setting · p. 3

B. The Defense-Aware Attacker

The paper evaluates an attacker who knows the defense and its deployed vector, defines compromise and utility with a capability guard, and gives a five-step process for selecting and validating a per-model steering direction.

B. The Defense-Aware Attacker

  • The attacker knows the architecture, weights, fitting samples, steered sites, dose, and deployed vector, and can use query attacks, surrogate refitting, or gradients through the exact defended model.The exact-vector white-box arm tests whether protection depends on secrecy rather than the mechanism.Source: B. The Defense-Aware Attacker · p. 3

C. Metrics and the Capability Guard

  • Compromise is the top-priority metric, followed by benign and attacked utility, while contamination is reported separately.On agent benchmarks, compromise uses the benchmark checker; single-turn attack success uses a deterministic scorer.Source: C. Metrics and the Capability Guard · p. 3
  • A capability guard rejects apparent security from a model that stops acting, limiting defended no-action rates to the unattacked rate where that baseline is nonzero.Where the baseline is zero, the paper reports observed values without a ratio threshold; dose searches also use pre-specified benign-utility floors.Source: C. Metrics and the Capability Guard · p. 3

A. The Recipe

  • The recipe builds paired behavioral contrasts, capturing and pooling pre-MLP residual activations on payload spans while controlling for sample, delegation, and requested action.The primary contrast varies authority framing, voice, requested action, and whether context delegates authority to the payload.Source: §III · p. 4, A. The Recipe · p. 4
  • The candidate direction is the per-layer difference in mean activations between followed and resisted injections, chosen to reflect the direction the data moves rather than a maximally separating probe.The paper contrasts this choice with probe weights [15], [16], [17].Source: A. The Recipe · p. 4
  • Reproducibility across fitting shards and AUC on a withheld authority-framing level rank candidates, but only the later causal and capability gates certify them.The screening references are 0.70 for cosine agreement and 0.65 for AUC; most candidates are set aside.Source: A. The Recipe · p. 4
  • The recipe’s causal gate requires subtracting a candidate direction to reduce injection-following and adding it to increase following, with the capability guard intact.The increase criterion passed on four models; on Gemma-4-31B it was untestable because the undefended rate was saturated at 1.000.Source: A. The Recipe · p. 4
  • The surviving direction is refit on the full contrast set and deployed as a constant, norm-preserving edit to all tool content, with the benign-utility cost treated as the price of defense.The activation capture and steering sites differ; the block output is used for the edit, while the pre-MLP residual is used for capture.Source: A. The Recipe · p. 4

B. Deployment

CounterSteer edits tool-result representations during prefill, using model-specific directions and doses. The evaluation spans five open-weight models and combines held-out corpora, baselines, and pre-specified splits and budgets.

B. Deployment

  • During prefill, a forward hook applies the fitted direction at selected layers to each token position in a tool-result span, using a norm-preserving edit scaled by the dose.System, user, and generated positions are not directly edited; inputs without tagged tool-result spans execute byte-identically.Source: B. Deployment · p. 4
  • The five deployed configurations vary by model in direction, dose, and layers, and the two primary models receive the full corpus battery and deepest adaptive evaluation.The other three models receive one-pass held-out tests on their firing corpora and the agentic grid; doses are not comparable across models.Source: B. Deployment · p. 5, §IV · p. 5

IV. EVALUATION SETUP

  • Generation budgets are model-specific but shared across arms within each table, and attack success is scored deterministically rather than by an LLM judge.Single-turn utility uses a blinded pairwise judge, whereas AgentDojo uses its own checkers.Source: §IV · p. 5
  • The evaluation uses AgentDojo and LLMail-Inject alongside four purpose-built single-turn corpora that isolate instructional takeover and parameter manipulation in JSON records and natural HTML carriers.The webpage corpora use template-disjoint splits and score behavior against an unattacked reference, not an attack canary.Source: §IV · p. 5
  • Baselines include prompt-level defenses, classifier and mechanical filters, two internal interventions, and SecAlign, which fine-tunes served weights.The internal ports are CachePrune and AGRI.Source: §IV · p. 5
  • Held-out evaluation separates samples, attacker-template families, and authority-framing levels, with the deployed refit withholding the framing level under the specified recipe.The released artifact includes pre-specified files fixing arms, splits, doses, and hypotheses before test touches.Source: §IV · p. 5

V. RESULTS

CounterSteer certifies across five models, with canonical AgentDojo compromise rates falling on the reported runs. Results vary by model and attack surface, while surviving risk concentrates in parameter manipulation and task-dependent utility costs.

V. RESULTS

  • 0.475 →0.079 for gpt-oss-20b, 0.489 →0.072 for Qwen3-30B, and 0.222 →0.056 for GLM-4.5-Air are the canonical AgentDojo compromise rates.The corresponding benign utilities are 94.4%, 94.1%, and not reported in this passage; repeated runs are not averaged.Source: §V · p. 6
  • Agentic compromise rates are truncation-censored lower bounds, and 12 of 52 webpage-parameter samples closely match development templates.The near-duplicate samples are reported rather than excluded; the measured impact is nil in the stated model checks.Source: §V · p. 6

A. Main Security–Utility Result

  • Five models pass a pre-specified held-out certification, with single-turn ASR ranges and canonical AgentDojo results summarized in Table II.The table caption specifies one held-out test pass per firing corpus class and benign utility as a percentage of clean performance.Source: A. Main Security–Utility Result · p. 6, Table II · p. 6
  • 0.962 to 0.038 parameter-manipulation success and 0.865 to 0.000 tool-hijack success are reported for Gemma-4-31B on its widest held-out test.The parameter estimate is 2/52, but its document-clustered and sample Wilson confidence-interval uppers exceed the 0.05 bar.Source: A. Main Security–Utility Result · p. 6
  • 0.808 →0.019 webpage parameter-manipulation success (1/52) is GLM-4.5-Air’s strongest reported result, with 94.2/94 strict-bracket utility.The comparison is against the primary models’ identical 52-sample split, but base rates and budgets differ.Source: A. Main Security–Utility Result · p. 6
  • 0.212 →0.038 held-out webpage tool-hijack success (11/52 →2/52) is reported for Llama-3.1-8B at 100% byte-exact benign utility.The passage qualifies this result with a thin undefended base and low task capability; AgentDojo compromise rate also halves at 100% benign utility.Source: A. Main Security–Utility Result · p. 6
  • Surviving risk concentrates in parameter manipulation, while tool hijack reaches 0/52 observed per primary model on webpage media.The trade-off is dose- and model-specific: Qwen3-30B at 12σ buys most of 16σ’s compromise-rate reduction at a fraction of its cost.Source: A. Main Security–Utility Result · pp. 6–7

B. Agentic Evaluation

Across agentic evaluations, CounterSteer reduces compromise while retaining most measured benign utility, though residual risk varies by model and task. Adaptive tests preserve the defense against tool hijack but expose parameter manipulation as a persistent weakness.

B. Agentic Evaluation

  • CounterSteer removes 50–97% of grid compromises, about 74% on both primary models under adaptive attack, with 0–2 defense-introduced compromises per grid.Benign utility ranges from 100% of clean on GLM-4.5-Air and Llama-3.1-8B to 92.9% on Gemma-4-31B; the benchmark changes base rates, not the measured defense ranking.Source: B. Agentic Evaluation · p. 7
  • 0.006–0.079 AgentDojo compromise rates accompany 93–100% typography-normalized benign utility across canonical runs.On the two primary models at a shared budget, canonical blocked/introduced counts are 70/0 and 75/0, with normalized benign utility of 94.4% and 94.1%.Source: B. Agentic Evaluation · p. 7
  • The deployed gpt-oss-20b configuration has 0/141 observed non-delegated compromises, while all its surviving compromises are delegated-authority cases.GLM-4.5-Air shows a related pattern: 7 of its 10 survivors occur on one authority-delegating task.Source: B. Agentic Evaluation · p. 7
  • 0.393 →0.031 AgentDyn compromise rate is reported for gpt-oss-20b, with Qwen3-30B at 0.482 →0.021 and GLM-4.5-Air at 0.156 →0.067.On gpt-oss-20b, the defense retains about 83% of clean-solved tasks; Llama-3.1-8B solves only 2 of 60 tasks, a capability bound.Source: B. Agentic Evaluation · p. 7

C. Adaptive Evaluation

  • 3/18 tool-hijack samples and 13/18 parameter-manipulation samples are cracked by the best-of-48 framing search.The strongest parameter-manipulation wordings use bland correction framings; under exact-literal scoring, both searches nearly vanish at 2 of 864 pooled queries.Source: C. Adaptive Evaluation · p. 7
  • AutoDojo holds CounterSteer to roughly a quarter of undefended success on both primary models, while every prompt-level defense collapses.On gpt-oss-20b, static seeds already reach 0.147 of 0.175 AD@6 success; optimization adds only 0.028.Source: C. Adaptive Evaluation · p. 8
  • 0/12 GCG compromises are observed after recipe-faithful refitting, while exact deployed-vector access yields 2/52 compromises.The replayed successful LLMail-Inject texts yield 0/2052 observed successes on gpt-oss-20b; this is replay rather than live adaptation, and relay attacks remain under-powered or incomplete.Source: C. Adaptive Evaluation · pp. 8–9
  • Each test-split result is a single pre-specified pass with no reruns, after search on development splits.A challenger that dominated development comparisons reversed on its one-pass test evaluation and was rejected.Source: C. Adaptive Evaluation · p. 9

D. Comparison with Other Defenses

On capable models, CounterSteer compares favorably with measured defenses, but the comparisons depend on task utility, evaluation budget, and how benign fidelity is measured. Its direction also outperformed matched random controls on one corpus, though the reported test was not conventionally significant.

D. Comparison with Other Defenses

  • On capable models, CounterSteer sat on the measured security–utility frontier; on gpt-oss-20b it removed 81% of compromises at the best utility under attack.It also had the best Class A record among guard-passing arms.Source: D. Comparison with Other Defenses · p. 9
  • On GLM-4.5-Air, PIGuard left 0/180 observed compromises while destroying 42% of benign utility; on Llama-3.1-8B, PIGuard won with only 13–15 of 56 clean tasks solved.On Qwen3-30B, CounterSteer versus DeBERTa was not a robust tier-1 ordering because confidence intervals overlapped.Source: D. Comparison with Other Defenses · p. 9
  • The regeneration reference read only 0.40–0.64 EQUIVALENT, so single-turn composed-prose fidelity remains bracketed rather than resolved.In strict byte-exact terms, steering preserved structured calls best while paying in composed prose.Source: D. Comparison with Other Defenses · p. 9
  • SecAlign recorded 0/91 and 0/70 observed compromises, versus CounterSteer’s 0.013 and 0.045 on the same samples, but its adaptive framings sat inside its training grid.SecAlign's weakest parameter-corpus utility reading was 56% of the same-model floor.Source: D. Comparison with Other Defenses · pp. 9–10
  • At the retained 768 budget, SecAlign left 9/180 compromises versus CounterSteer’s 12/180, a statistically indistinguishable pair (McNemar p=0.45).Their survivors were entirely delegated-authority compromises; the paired ranking is valid, but absolute rates are censored.Source: D. Comparison with Other Defenses · p. 10
  • On the disjoint-tool corpus, the fitted direction reached ASR 0.026 versus 0.191 for a matched control (n=141), but the permutation test was not conventionally significant.Matched random draws also cost more benign utility on AgentDojo, suggesting much of the benign cost was generic perturbation at this magnitude.Source: D. Comparison with Other Defenses · p. 10
  • Prefill latency was indistinguishable from unsteered (0.181 vs. 0.182 s), while no-span GSM8K, MMLU, and IFEval runs served as identity checks, not utility claims.A probe wrapping each item inside a steered span showed uneven preservation.Source: D. Comparison with Other Defenses · p. 10

VI. MECHANISM: CORRELATES, LEVERS, AND WHERE THE DECISION LIVES

The mechanism evidence distinguishes causal steering directions from correlates and locates parameter-manipulation decisions late in generation. Results vary by model and authority framing, while delegated instructions fall outside the defense’s intended scope.

A. Detection Is Not Defense

  • Two observationally successful directions failed causal validation: steering the prohibition direction left attack success unchanged (0.299 vs. 0.303), and the Qwen3-30B user-role axis did not move ASR.The prohibition direction encoded the decision’s outcome rather than its cause.Source: A. Detection Is Not Defense · p. 10

B. The Role Axis Is a Model-Specific Correlate

  • The role axis was coupled to injection-following on gpt-oss-20b but decoupled on Qwen3-30B; role-axis steering moved ASR only on gpt-oss-20b.On gpt-oss-20b, injected spans read more tool-like in 96% of samples yet userness predicted success at 12/12 layers; on Qwen3-30B, role reading was inverted and non-predictive.Source: B. The Role Axis Is a Model-Specific… · p. 10

C. Parameter Manipulation Decides in the Model’s Reasoning

  • With the tested probes, parameter-manipulation decisions were near chance at examined pre-generation sites (AUC 0.447–0.526) but readable at argument-value emission (0.824–0.903).Payload-span probes also generalized at chance within framing families (AUC 0.527, replicated twice).Source: C. Parameter Manipulation Decides in… · p. 10
  • Capability-safe decode-time steering left held-out parameter-corpus attack success statistically unchanged (4/65 vs. 5/65); combined prefill and decode steering still cracked 11/18 adaptive samples.The prefill-only configuration cracked 13/18.Source: C. Parameter Manipulation Decides in… · pp. 10–11
  • The compromise path first quotes the injected sentence into the model’s reasoning, then adopts it, emitting the hijacked value a median ∼1.3k characters into completion.The examined prompt sites showed no earlier readable trace, and neither payload-scoped constant-direction intervention removed the decision.Source: C. Parameter Manipulation Decides in… · p. 11

D. Authority and Delegation

  • Authority framing produced model-specific curves: gpt-oss-20b escalated with claimed authority, while Gemma-4-31B and GLM-4.5-Air peaked below the ladder’s top.On Gemma-4-31B, explicit supersession suppressed parameter-manipulation firing entirely (0/72 observed).Source: D. Authority and Delegation · p. 11
  • When legitimate context grants the payload authority, the behavioral signature is attenuated or absent; a delegation factor was necessary for the direction to generalize.A deny/mention control showed the signature tracked granted authority, and the paper scopes role-consistent delegated compliance out as Class B.Source: D. Authority and Delegation · p. 11

VII. LIMITATIONS

CounterSteer’s limits include white-box deployment requirements, model- and task-dependent utility costs, and remaining attack classes and settings it does not cover. The related-work comparison distinguishes its always-on causal steering from defenses that gate interventions, modify prompts or weights, or operate at the system level.

VII. LIMITATIONS

  • The recipe requires white-box serving access, limiting its stated deployment scope to open-weights models rather than third-party APIs.Per-model fitting takes ∼1 h to ∼2.3 h on one GPU node, while evaluation dominates cost.Source: §VII · p. 11
  • Benign utility costs vary by model and task; reasoning over steered content reduces multiple-choice discrimination by 10.8 pp and IFEval adherence by 15.2 pp at the deployed gpt-oss-20b dose.GSM8K is unchanged, and some models incur no measured cost; composition-heavy webpage tasks show a steering-specific remainder of ∼7–9 pp beyond shared regeneration drift.Source: §VII · p. 11
  • The defense only steers tool-result spans, leaving untrusted content pasted into a user turn unmodified; extension to other tagged spans was not evaluated.It partially resists parameter manipulation and does not protect Class B delegated authority; the authors frame it as an instructional-takeover component, not a complete IPI solution.Source: §VII · p. 11
  • Evaluation is replay and search rather than a live red team, and open-ended attacks, AutoDojo beyond six iterations, and wider surrogate mismatch remain unmeasured.The two secondary models lack adaptive evaluation; Gemma-4-31B’s single-turn steered span is ∼90% of the prompt, so its agentic grid is the only isolation of the payload-scoped edit.Source: §VII · pp. 11–12

VIII. RELATED WORK

  • Among activation-level defenses, CounterSteer differs from probe-gated interventions by using bidirectional causal validation and steering without a per-request gate; CachePrune and V-Steer require attribution.AGRI is the strongest measured inference-time rival, but its adaptive result is 0.143 versus CounterSteer’s 0.175 on gpt-oss-20b at 78.0% benign utility, with ∼10× context inflation.Source: §VIII · p. 12
  • Detection can identify task drift or injection exposure, but detector-based filters add thresholds that fail open on misses and can over-defend through deletion.Instruction-hierarchy training is described as the training-time analogue.Source: §VIII · p. 12
  • Other defenses move separation to training, prompts, or system controls; prompt-level defenses remove only 13–44% of compromises on capable models, while ROPE guards tool parameters through provenance.The paper does not evaluate composing CounterSteer with ROPE.Source: §VIII · p. 12
  • The paper reproduces role-probe predictiveness on gpt-oss-20b but finds the role axis is not a universal intervention target; its steering operator is standard difference-in-means residual steering.Its claimed contribution is the procedure for finding and validating what to steer, rather than a new steering operator.Source: §VIII · p. 12

IX. CONCLUSION

CounterSteer is an always-on, causally gated defense that suppresses each model’s lever for treating untrusted text as instructions. The authors position it on the measured security–utility frontier for capable models, while documenting its deployment scope, released artifacts, and research-process safeguards.

IX. CONCLUSION

  • Across twenty-six batteries and a benchmark-level adaptive attacker, CounterSteer sits on the measured security–utility frontier among inference-time defenses on capable models.For open-weights deployments with tool-span tagging, the conclusion reports 93–100% of clean benign utility, with lower utility when reasoning over steered content.Source: §IX · p. 12

ETHICAL CONSIDERATIONS

  • The authors report that no third-party systems were attacked, no personal data or human subjects were involved, and agent-generated experiment work was reviewed by a separate agent instructed to refute it.The attack drivers target locally served open-weights models and implement published techniques.Source: ETHICAL CONSIDERATIONS · p. 13, LLM USAGE CONSIDERATIONS · p. 13

OPEN SCIENCE

  • The released artifact includes corpora, fitted directions, deployed configurations, scoring and evaluation tools, baseline implementations, adaptive-attack drivers, and per-sample completions.The SecAlign trained weights are not included, but its training driver, configuration, and data are released; raw activations can be regenerated deterministically.Source: OPEN SCIENCE · p. 13

LLM USAGE CONSIDERATIONS

  • No security measurement depends on LLM judgment: deterministic scorers evaluate attack success and capability, while utility uses a validated, blinded pairwise judge alongside mechanical brackets.Human authors set scope and pre-specification decisions, verified claims against artifacts, and take responsibility for the paper’s content.Source: LLM USAGE CONSIDERATIONS · p. 13

APPENDIX A CORPUS AND SCORING NOTES

The appendix documents corpus construction, scoring safeguards, and qualifications that affect how several measurements should be read. In particular, generation-budget censoring changes the reported AgentDojo comparison, while composition shifts explain part of a development-to-test utility gap.

APPENDIX A CORPUS AND SCORING NOTES

  • The corpora embed two attack classes in real tool-log records, with sample- and template-disjoint splits for webpage and JSON tool-hijack data; public IPI benchmarks do not isolate parameter manipulation.Webpage corpora use natural HTML carriers, while the Gemma certification corpus adds 30 fresh URL-disjoint carriers and new attacker wordings.Source: Appendix A · p. 15
  • Utility uses a blinded pairwise judge with controls and a pre-specified validation gate, reported beside a mechanical strict/lenient bracket; the absolute LLM correctness judge was rejected.A scrub-versus-original test bounds the effect of schema annotations on quoted numbers at approximately zero, with any remaining bias described as conservative.Source: Appendix A · p. 15
  • The gpt-oss capability guard passes every Table XV row; defended no-action rates are at most 0.75× unattacked where the unattacked floor is nonzero, and ≤0.058 where it is zero.The worst zero-floor row is 3/52 on the disjoint-tool corpus; completion review found no refusal of the legitimate task.Source: Appendix A · p. 16
  • At 4096 tokens, gpt-oss AgentDojo compromise rates are 0.475 undefended and 0.079 defended; the undefended rate remains a lower bound because three attacked episodes were still truncated.The earlier 768-token run was truncation-censored in both arms and is superseded; the defended rate of 0.079 is the surviving measurement, not evidence the defense was unaffected.Source: Appendix A · p. 16
  • Qwen3-30B AgentDojo runs use 4096 tokens per turn because its 1–4k-token reasoning channel makes 768-token runs unmeasurable; the clean arm falls from 0.875 to 0.411 at 768.The Qwen held-out test rows pass the capability guard; truncation flags are zero except in the webpage tool-hijack defended arm and the defense-on-clean JSON tool-log arm.Source: Appendix A · p. 16
  • Tier-3 contamination is confined to the JSON tool-log corpus on the two primary models, where defended rates exceed unattacked floors; the authors identify it as a partial-compromise channel.Deployments where argument provenance matters should read this alongside the parameter-manipulation limitation.Source: Appendix A · p. 16
  • The 16σ dose ceiling was measured on development splits without a pre-specified test pass, so its zero-observed development results are reported as a ceiling, not the deployment point.The dose is also capability-guard territory in the agentic setting.Source: Appendix A · p. 16
  • At the identical configuration, JSON tool-log benign utility is 45.1%/91% on development versus 21.1%/69% on test; reweighting the test tool mix raises strict utility from 21% to ∼31%.Message-composition tasks make up 60% of test versus 26% of development, and the test also has more structured arguments per call (3.69 versus 2.92).Source: Appendix A · pp. 16–17

APPENDIX B THE PAIRWISE UTILITY JUDGE AND THE FIDELITY

The appendix introduces a blinded pairwise judge for measuring fidelity of model-composed prose against an unattacked reference, with validation and caveats that make its utility scores conservative. It also compares always-on steering with a cross-run regeneration reference, while warning that this reference does not calibrate same-process steering.

Motivation

  • The judge fills a benchmark gap by measuring fidelity of benign model-composed prose, which outcome checkers and byte-exact scoring do not measure appropriately.The instrument targets content fidelity of summaries and subjects; attack success is deterministic elsewhere in the paper.Source: Appendix B · p. 17

Judge instrument

  • The judge scores each candidate against the same run’s unattacked reference using three blinded votes, labeling pairs EQUIVALENT, DEGRADED, or DIFFERENT-TASK.The headline column counts EQUIVALENT completions that are not compromised; compromise is determined separately by the deterministic scorer.Source: Appendix B · p. 17

Validation

  • Validation found 0.867 agreement with an independent adjudicator, conservative rejection of about 25% of its equivalent pairs, and zero majority-verdict flips in a double run.The instrument passed pre-specified validation before scoring; three production batches with failed corrupted-pair controls were discarded and rerun after control-construction defects were repaired.Source: Appendix B · p. 17

Composition fidelity

  • Table V’s pooled steering score is about 7–9 percentage points below the regeneration reference, a lower bound on the true deficit rather than a direct calibration.The reference uses byte-identical prompts regenerated cross-run without defense; the caption says ratios between the two rows are not taken.Source: Table V · p. 17

Caveats

  • The judge’s utility score is a conservative floor, not a point estimate, because DEGRADED includes borderline emphasis changes and may penalize visible tier-3 contamination.Order-invariance showed 0.260 equivalent reference-first versus 0.271 candidate-first across 8,868 votes, a nonsignificant trend against flattering the defense.Source: Appendix B · p. 17

APPENDIX C ADAPTIVE EVALUATION IN DETAIL

The appendix reports defense-aware adaptive evaluations spanning benchmark-level attacks, query framing, gradient attacks, human-crafted replay, and mechanism probes. Results show strong suppression of instructional takeover alongside a persistent parameter-manipulation weakness and explicit limits on several attack conclusions.

Benchmark-level adaptive attack

  • Table VI reports AutoDojo’s adaptive attack (AD@6), the attacker’s best over seeds and optimized rewrites, macro-averaged over 36 pre-specified suite × objective × vector cells per model against each defense.SecAlign (the gpt-oss DPO model) was evaluated on 35 of 36 cells; its remaining slack cell deterministically exhausted 80 GB of memory, and its two zero suites were exact (0/23 cells).Source: Appendix C · p. 17, Table VI · p. 18

Query-based framing search

  • Best-of-24 framing search cracks 3/18 tool-hijack samples and 13/18 parameter-manipulation samples, while CounterSteer has the strongest measured inference-time performance on the parameter search.Its per-query parameter firing rate is 0.303; published inference-time baselines remain at or near full union crack, while SecAlign reaches 1/18 on trained framing families.Source: Appendix C · pp. 18–19

Gradient attacks

  • Exact-vector white-box GCG observes 2/52 tool-hijack compromises, while surrogate-refit and naive-transfer GCG each observe zero compromises in their completed samples.The surrogate direction has cosine 0.974–0.977 to the deployed vector, but its sample-set sensitivity was measured only at the direction level, not in an attack evaluation.Source: Appendix C · p. 19

Parameter-manipulation gradient attacks

  • The parameter-manipulation gradient result is inconclusive as a robustness test: the pre-registered objective targets a path unused by observed real compromises, and the path-matched follow-up yields no gradient-attributable compromises.At argument emission, parameter obedience is strongly decodable (AUC 0.824 at the choice point), unlike the examined payload-span site (AUC 0.527); decode-time steering trims firing but leaves the query union nearly intact.Source: Appendix C · pp. 19–20

Relay attacks

  • The relay probes leave the tier-1 same-turn question open, while the defense suppresses quoted injected instructions in reasoning from 7/24 to 0/24.The cross-turn relay fails even undefended; the defended same-turn result is not separable from chance at this sample size, and the supported finding is suppression of the quotation step.Source: Appendix C · pp. 20–21

LLMail-Inject replay

  • None of 2,052 replayed successful LLMail-Inject attacks succeeds under the defense, including the 22 texts that beat the challenge’s combined stack.The replay measures human-adaptive attacks optimized against other models, not live adaptation; challenge-specific inline hardening raises transfer when removed from 0.070 to 0.254.Source: Appendix C · p. 21

Transfer anatomy

  • Across replay targets, wire-format role forgery accounts for 90–97% of undefended hits, while detector-evasion forms transfer at about 0.004 and the role-confusion class is reduced to zero observed defended hits.The Qwen3-30B replay also records 117/1537 undefended successes and 0/1537 defended; the direction predates that corpus.Source: Appendix C · p. 21

Selection risk and protocol

  • A development-favored challenger reversed on the one-pass test evaluation, illustrating the risk of selecting a defense on its development attack distribution.Its paired-run readings were worse than the deployed configuration on webpage parameter, tool-hijack JSON, and webpage tool-hijack evaluations; test evaluations are specified as single-pass, with no reruns.Source: Appendix C · pp. 21–22

APPENDIX D GENERAL UTILITY AND MECHANISM ABLATIONS

The appendix separates deployment-relevant utility on real tool traffic from mechanism probes that place ordinary benchmark content inside steered spans. It finds near-identical measured runtime, but reasoning over steered content can incur task-dependent costs and some benign failures reflect typography-sensitive decisions.

Runtime cost

  • On gpt-oss-20b, measured prefill time, greedy decode speed, and peak memory were nearly identical between steered and unsteered arms, within a measurement whose episode variability exceeded the arm difference.Median prefill was 0.181 s steered versus 0.182 s unsteered, decode was 13.52 tok/s in both arms, and peak memory was 39.37 GiB in both.Source: Appendix D · p. 22

Benchmark scope

  • Standard capability benchmarks without tool-result spans run on the mathematically unmodified model, so the appendix treats plain benchmark rows as identity checks rather than utility evaluations.Tool-framed rows instead probe costs when benchmark content is artificially placed inside a steered span.Source: Appendix D · p. 22, Table X · p. 23

Steered-span reasoning

  • When benchmark content is tool-framed at the deployed dose, arithmetic reasoning is unchanged, while multiple-choice discrimination drops 10.8 percentage points and IFEval instruction adherence drops 15.2 points.The authors recommend checking utility at the operating dose when agents must reason discriminatively over retrieved content.Source: Appendix D · pp. 22–23

MMLU mechanism

  • The MMLU slip is a position step: later-listed correct options lose 9–12 points, while first-position answers are unaffected within noise.Two failure classes contribute roughly equally: degraded comparison among later options and budget-exhausting re-quoting loops; question comprehension itself is not identified as the failure mode.Source: Appendix D · p. 23

AgentDojo benign failures

  • Three of eight broken AgentDojo benign tasks involve steering flipping near-tie space typography to U+202F, but only one of the eight failures is suppression.The unsteered model already emits U+202F in 34–36% of episodes, so the typography result is not a directional push.Source: Appendix D · p. 23

Random-direction controls

  • Magnitude-matched random directions show that some benign cost can arise from generic perturbation, while a disjoint-tool matched-cost draw has 30× the fitted direction’s attacked ASR.On AgentDojo, two random controls cost −21.8 and −27.3 points versus −14.5 for the deployed configuration, but those readings come from a retired lower-budget run.Source: Appendix D · p. 23

Direction ablation

  • Benign-axis direction surgery showed no headroom in the two reported draws, leaving dose as the live lever in those analyses.Source: Appendix D · p. 23

APPENDIX E BASELINE IMPLEMENTATIONS AND FULL-GRID DETAIL

The appendix documents how baseline defenses were implemented and validated, then reports full-grid comparisons and important qualification details. It also records the held-out behavior and utility of CachePrune and SecAlign.

Baseline implementation

  • The AGRI port fits probes on the authors’ captures and implements gated reasoning prefill; its static and adaptive results differ across the two tested models.For gpt-oss-20b, static compromise is .415 →.062 at 78.0% benign utility and AD@6 is 0.143; for Qwen3-30B, these are .472 →.100 at 100% benign utility and 0.411.Source: Appendix E · p. 23

Baseline validation

  • CachePrune’s independent Llama-3-8B-Instruct validation reduced ASR from 0.51 to 0.01 on 100 SQuAD rows, with defended F1 at 0.734 versus 0.783 unattacked.The single defended hit was a canary-quoting refusal counted by the substring matcher; the corrected defended rate is 0.00, and rows came essentially from one title-sorted article.Source: Appendix E · p. 24

Baseline test results

  • On held-out test splits, CachePrune compromise rates are 0.700 →0.463 for shipped and 0.647 →0.397 for parameter cases, while SecAlign records 0/91 and 0/70 observed compromises.CachePrune’s benign brackets are 35.0%/88 and 29.4%/87, versus CounterSteer’s 25.0%/80 and 22.7%/83; these utility differences are not significant.Source: Appendix E · p. 24

Baseline implementation

  • SecAlign’s DPO LoRA used 8,640 preference pairs drawn from the direction’s fitting split, while evaluation samples and development injection wordings were held out lexically.The framing-level axis and semantic authority register were not held out, nor was the JSON parameter corpus’s attacker-template set.Source: Appendix E · p. 24

Baseline utility

  • SecAlign’s clean-traffic agreement with the base model is 0.506 for JSON tool-log and 0.585 for JSON parameter, below same-model cross-run floors of 0.658 and 0.723.The passage says utility cost is quotable only at the conservative end because the fine-tuned model’s clean behavior is a different reference.Source: Appendix E · p. 24

Full-grid comparison

  • On the 176 shared verdict-bearing AgentDojo cases, SecAlign wins 5 of 7 disagreements with CounterSteer and CounterSteer wins 2, with exact McNemar p=0.45.The passage characterizes the two defenses as statistically indistinguishable.Source: Appendix E · p. 24
  • At 4096 tokens, SecAlign and CounterSteer have under-attack utility parity (101/180 vs. 102/180), while SecAlign’s clean failures fall from 11 to 6 as the base model’s fall from 10 to 1.At the shared 4096 budget, SecAlign is 90.4% of base clean utility; its 768-budget clean utility was 100.0% versus 98.2% for the base model.Source: Appendix E · p. 25
  • The full-grid comparison reports 8/8/3/4 checker-null cases for undefended, CachePrune, CounterSteer, and SecAlign, and worst-case null assignment changes no ranking.Censoring favors the comparators: the fine-tuned model has 0–9 unterminated turns versus 56–58 in attacked comparator arms; seven of SecAlign’s nine leaks exactly match CounterSteer failures.Source: Appendix E · p. 25

APPENDIX F PER-MODEL AUTHORITY-FRAMING CURVES

Authority-framing curves vary by model: gpt-oss-20b rises monotonically with claimed authority, while Gemma-4-31B and GLM-4.5-Air fire most on firm framings.

Per-model authority-framing curves

  • Gemma-4-31B’s parameter-manipulation firing is 0.674 on firm framings and 0/72 observed under explicit supersession; GLM-4.5-Air peaks at 0.576 on firm and reaches 0.382 on explicit supersession.The passage reports gpt-oss-20b escalating monotonically with claimed authority and describes GLM-4.5-Air as having Gemma-4’s shape, but shallower.Source: Appendix F · p. 25

APPENDIX G SUPPLEMENTARY FIGURES AND TABLES

The supplementary appendix provides provenance, scope notes, model and dataset coverage, and additional dose and framing results. It clarifies which comparisons are within-model and flags important holdout, censoring, and reconstruction boundaries.

Scope and provenance

  • Gemma-4’s attack surface is medium-specific: webpage and HTML carriers fire, while JSON and prose-carrier media do not in the measured table.On single-turn webpage corpora, the steered span is about 90% of the prompt, so these results do not isolate the payload-scoped edit used on AgentDojo tool outputs.Source: Appendix G · p. 25
  • The post-hoc delegated-authority partition contains 36 cases, and the Class-B enrichment is 0.806 versus 0.390 undefended, indicating delegation-associated enrichment regardless of defense.All defended failures in the deployed configuration are delegated (14/14), and the passage notes that the partition was motivated post hoc but applied symmetrically.Source: Appendix G · p. 25

Reproducibility

  • The Gemma model revision ambiguity does not change reported weights, but a reproducer resolving the alternate revision gets a materially different chat template.The two revisions have byte-identical weights; rendering all 576 deployed-fit prompts under each revision produced identical text, input IDs, and span indices.Source: Appendix G · pp. 25–26

Dose curves

  • The dose-curve figures use model-specific σ ranges, and their AgentDojo compromise values are budget-censored lower bounds at the 4096-token per-turn budget.The Qwen 16σ point is additionally censored because its truncation violates the capability guard; the shaded band marks where benign utility delta falls below −19 pp.Source: Figure 6 · p. 26
  • The Qwen3-30B α=16 compromise rate of 0.011 is a lower bound, because transcripts show defended episodes cut mid-compliance or ending on an unexecuted call.The α=10–14 rows share the budget but were not re-audited.Source: Appendix G · p. 26

Supplementary figures and tables

  • Table XI compares four-arm batteries on the 180-case AgentDojo grid, including compromise, benign utility, filter deletion, and capability-guard indicators.Its configurations use 4096 tokens per turn, except GLM-4.5-Air at 8192; each defense row is compared with its own same-process undefended baseline.Source: Table XI · p. 27

Supplementary results

  • Table XIII reports within-model contrasts across five corpus classes and five models, with AgentDojo compromise rates and defense-on utility, rather than cross-model comparisons.Table XIII’s provenance notes identify a 4096-token per-turn AgentDojo grid, except GLM at 8192, and document judge-versus-bracket caveats.Source: Appendix G · p. 25, Table XIII · p. 29

Model coverage

  • The five-model coverage spans open-weight models from 8B to 106B across five vendor lineages, with certification defined as completed, adversarially reviewed evaluation.The table’s defended column reports single-turn ASR or AgentDojo compromise rate, according to the cited row’s denominator.Source: Table XXI · p. 34
Loading 2609.36570v1…