PaperlayerView exact revision on arXiv

Source-linked AI summary

Where You Inject Matters: The Trust Region Effect in LLM Agents

Chengquan Guo, Xiaodong Liu, Weiwei Yang, Mingqian Feng, Jialin Song, Qiyao Ma, Carolyn Buractaon, Christopher White, Bo Li, Jianfeng Gao

arXiv:2610.38867v1cs.AIcs.CRcs.LG

TL;DR

Indirect prompt injection threatens agents when malicious instructions are embedded in external environments, while placement across heterogeneous environments has not been systematically characterized. The paper builds three testbeds and studies placement with fixed content, finding uneven susceptibility and developing an attacker that optimizes content and placement. Placement-aware optimization improves attack effectiveness, and the findings motivate accounting for placement in defenses.

  • Problem

    Prior indirect prompt injection work has focused more on content than on how placement across heterogeneous environments affects attack effectiveness.

    Source: §1 · p. 1

  • Method

    The paper builds three agentic testbeds and systematically varies injection surfaces and positions, then develops a staged attacker that optimizes content and placement.

    Source: §1 · p. 2, §3 · p. 3, §3.3 · p. 5

  • Results

    Agent susceptibility varies across environmental surfaces, including a 2.6× higher average following rate for GPT-5.4 injections in repository documentation than in code or data files.

    Source: §4.2 · p. 7

  • Takeaways & Limitations

    Injection placement is an important dimension of agent security, and the findings motivate defenses that account for placement.

    Source: §5 · p. 10

    The three human-verified testbeds do not cover the full diversity of real-world agent environments.

    Source: E LIMITATIONS · p. 21

Abstract

from arXiv · show

LLM agents are increasingly adopted in long-horizon tasks and interact with diverse external environments. Despite their growing capabilities, these agents remain vulnerable to real-world attacks such as indirect prompt injections, which manipulate agents through malicious instructions embedded in external environments. While existing risk assessment work has primarily focused on the injection content, the effect of where an injection appears has not been systematically characterized. Studying injection placement requires agents to operate over longer-horizon tasks and interact with diverse environmental surfaces. Thus, we build three agentic testbeds spanning tool use, coding, and daily workflow domains with diverse injection surfaces. Holding the injection content fixed, we find that agent susceptibility varies across injection placements. We refer to this phenomenon as the trust region effect: agents implicitly assign different levels of trust to different regions of their environments. For example, an agent powered by GPT-5.4 follows an unsafe injection instruction placed in a repository documentation file (i.e., a README or TODO file) 2.6 times more often on average than the same instruction placed in a code or data file. Building upon this observation, we develop learning-based attackers that generate injection content and select better injection placements. Our approach demonstrates that both where an injection is placed and what content it contains can affect the strength of indirect prompt injection attacks. To the best of our knowledge, this work provides the first systematic characterization for the effectiveness of different injection placements, spanning both coarse-grained environmental surfaces and fine-grained positions. The uneven trust that agents implicitly place across their environments provides important insights for future attack and defense design.

1 INTRODUCTION

Indirect prompt injection is a serious agent-security risk, but prior work has focused more on injection content than on placement across heterogeneous environments. This paper builds three testbeds and studies how placement affects attacks, finding uneven trust across environmental regions and developing an attacker that optimizes both content and placement.

1 INTRODUCTION

  • Holding the payload fixed, the same unsafe instruction is followed in README.md but ignored in a data file.The example uses a GPT-5.4 coding agent and motivates studying placement independently of injection content.Source: Figure 1 · p. 2
  • Prior work studies injection content and restricted placement settings, leaving placement across heterogeneous environments less understood.The paper identifies the challenge of providing long-horizon tasks and a controllable platform for varying both surfaces and within-surface positions.Source: §1 · p. 2
  • The authors report that most failed injections are ignored rather than explicitly rejected, and that trust patterns vary across agents.Even advanced models acting as red-teamers cannot reliably select the most effective placement.Source: §1 · p. 2
  • Across three agentic testbeds, the paper identifies the trust region effect and develops an attacker that optimizes both injection content and placement.The testbeds span three domains, and the placement-aware attacker improves attack success over content-only optimization.Source: §1 · pp. 2–3

2 RELATED WORK

Prior benchmarks and attackers have studied indirect prompt injection across tasks, channels, or content, but have not systematically characterized susceptibility across heterogeneous environmental surfaces and positions. This paper varies placement while holding payload fixed to isolate placement effects.

Indirect prompt injection benchmarks and attackers

  • Prior benchmarks do not systematically characterize susceptibility across environmental surfaces and injection positions, while AutoInject and RL-Hammer optimize content with placement fixed.AgentDojo uses relatively short-horizon tasks with limited exposure to diverse external environments; AgentLAB extends to long-horizon interactions, and DecodingTrust-Agent considers multiple injection channels.Source: §2 · p. 3

The placement of the injection

  • The paper systematically varies injection location while holding payload fixed, isolating placement effects across environmental surfaces and positions.This study does not rely on an LLM to predict promising injection locations.Source: §2 · p. 3
  • Recent placement studies focus on specific environments or within-file locations, including UI elements, a skill file, and webpages.MUZZLE ranks candidate injection vessels, POISE uses a position-aware trigger, and EIA examines positions within a webpage.Source: §2 · p. 3

3 METHODOLOGY

The methodology formalizes indirect prompt injection placement and builds three multi-surface testbeds to study it. It then trains an attacker in stages, first optimizing injection content and then jointly optimizing content and placement.

3.1 PROBLEM FORMULATION AND THREAT MODEL

  • The threat model lets an attacker modify an environmental surface encountered during a benign task, but not the user instruction, system prompt, or model parameters.An injection is represented by content c and placement ℓ=(s, p), where s is a surface and p is a within-surface position.Source: §3.1 · p. 4

3.2 MULTI-SURFACE AGENTIC TESTBEDS

  • Three testbeds span coding, tool use, and daily workflows, covering 36 injection surfaces, 30 benign tasks, and 24 injection tasks.The testbeds are designed to expose agents to multiple heterogeneous surfaces during longer-horizon tasks and to ensure injections are perceived, though exposure remains stochastic.Source: §3.2 · p. 4
  • Placement varies both the encountered environmental surface and the injection’s position—START, MIDDLE, or END—within that surface.Candidate surfaces are identified from surfaces encountered during benign execution, with selection procedures varying by domain.Source: §3.2 · p. 5, Figure 2 · p. 5

3.3.1 STAGE I: CONTENT OPTIMIZATION

  • Stage I warms up the attacker with supervised examples, then optimizes injection content using a fixed, training-derived placement prior.The prior selects the highest-ranked available surface category and fixes the position to END; during execution, it overrides the attacker’s predicted placement.Source: §3.3 · p. 5, §3.3.1 · pp. 5–6
  • Stage I’s reward combines output-format validity, deterministic attack success, benign-task utility, and an LLM judge’s goal-alignment and presentation score.The interaction term VaVb rewards attacks that succeed while preserving the original task; the Stage I attacker initializes Stage II.Source: §3.3.1 · p. 6

3.3.2 STAGE II: JOINT CONTENT AND PLACEMENT OPTIMIZATION

  • The attacker’s Stage II executes its generated placement directly and jointly optimizes injection content and placement using the same composite reward as Stage I.Unlike Stage I, the model-predicted placement is not replaced by a task-specific prior.Source: §3.3.2 · p. 6

4 EXPERIMENTS

The experiments find that injection effectiveness depends on environmental surface and within-surface position, with trust patterns varying across agents. Two-stage placement-aware optimization improves learned policy-selected attacks and transfers to a stronger victim.

4.1 EXPERIMENTAL SETUP

  • The evaluation spans 30 benign tasks, 24 injection tasks, 36 surfaces, and START/MIDDLE/END positions; identical payloads isolate placement, with ASR conditioned on successful delivery.Eight agents were evaluated, and deterministic script-based verifiers assessed attack success and benign utility.Source: §4.1 · pp. 6–7

4.2 HOW DOES INJECTION PLACEMENT AFFECT AGENT BEHAVIOR?

  • 81.3% of GPT-4o delivered-but-failed injections were ignored; conditional ASR varied across surfaces, and END was highest in all domains (+11.7 percentage points over START in tool use).For GPT-5.4, README/TODO injections were followed 2.6× more often than the same instruction in code or data files.Source: §4.2 · p. 7, Table 1 · p. 7, Table 2 · p. 7

4.3 DOES THE TRUST REGION EFFECT HOLD ACROSS DIFFERENT AGENTS?

  • Surface preferences varied by agent; Gemini-3.1-Pro opened README/TODO in under 5% of runs versus 77–88% for code/data, making its documentation ASR unstable.The most effective surfaces also differed among agents in the tool-use and workflow domains.Source: §4.3 · p. 8, Figure 3 · p. 8

4.4 CAN LLMS IDENTIFY EFFECTIVE INJECTION PLACEMENTS?

  • A prompted GPT-5.4 selector did not reliably beat a surface-frequency prior: the prior led tool use at B=1 (0.53 vs. 0.42), with mixed outcomes elsewhere.The strategies were compared under a fixed budget of blind attempts while injection content was held constant.Source: §4.4 · p. 8

4.5 DOES PLACEMENT-AWARE OPTIMIZATION IMPROVE ATTACKS?

  • Two-Stage Optimization exceeded Content-Only (Self-Selected Placement) by 20.7–24.8 percentage points on leave-risk and by 4.6–48.1 points on leave-task.An empirical placement prior remained competitive and sometimes outperformed learned strategies; attack gains generally preserved benign utility.Source: §4.5 · pp. 8–9, Table 5 · p. 9
  • The empirical placement prior remained competitive and sometimes outperformed learned strategies; clean-task utility baselines were 56% on leave-risk and 72% on leave-task.Benign task utility remained within a relatively narrow range across variants.Source: §4.5 · p. 9, Table 5 · p. 9
  • 33.3% ASR on 30 held-out contexts transferred to GPT-5.4, versus 0.0% for three templates at matched locations; utility stayed 88.9%.Because all injections were delivered, the gain reflected learned injection content rather than placement alone.Source: §4.5 · p. 9, Table 6 · p. 9

4.6 ADDITIONAL ANALYSIS

  • Workflow ASR varied little across read depths; shorter-horizon benign tasks were easier to attack, while longer host files showed only a weak, non-significant trend toward lower ASR.The analysis covered all 24 workflow permutations (1,440 tests).Source: §4.6 · p. 9

5 CONCLUSION

The paper characterizes injection placement across three multi-surface agentic testbeds and finds that susceptibility varies by surface and within-surface position. It also reports that a staged attacker optimizing content and placement improves attack effectiveness.

5 CONCLUSION

  • Across three multi-surface testbeds, susceptibility varies by environmental surface, end-of-surface position, and model or agent; delivered but unsuccessful injections are more often ignored than refused.The authors report that a staged attacker optimizing both content and placement improves attack effectiveness.Source: §5 · p. 10

ETHICS STATEMENT

The experiments were conducted in controlled, sandboxed containers without real users or third-party systems. The authors describe dual-use risks and responsible disclosure of research artifacts.

ETHICS STATEMENT

  • Experiments used controlled, sandboxed container environments and involved no real users or third-party systems.Source: ETHICS STATEMENT · p. 10
  • The authors acknowledge potential dual-use risks from research on indirect prompt injection.Source: ETHICS STATEMENT · p. 10
  • The authors say they mitigate those risks through responsible disclosure of research artifacts.Source: ETHICS STATEMENT · p. 10

AI USE STATEMENT

The appendix describes the authors’ AI-use disclosure, composed coding-task instructions, testbed surface categories, and injection-risk construction.

AI USE STATEMENT

  • LLMs assisted with manuscript clarity and readability and code debugging; the authors reviewed outputs and take responsibility for the final content.Source: AI USE STATEMENT · p. 10

A CODING-TASK INSTRUCTION ORGANIZATION

B.1 CODING DOMAIN

  • Coding surfaces group source-code files into code-file pools and other non-documentation files into data-file pools; arena-level README.md and TODO.md files are separate.Source: §B.1 · p. 13, Table 7 · p. 13

B INJECTION SURFACE DETAILS

B.2 TOOL-USE DOMAIN

  • The appendix includes a table of injection surfaces considered in the AgentDojo tool-use domain.Source: Table 8 · p. 14

B.3 WORKFLOW DOMAIN

  • The appendix includes a table of injection surfaces considered in the workflow domain.Source: Table 9 · p. 14

C INJECTION RISKS

D ADDITIONAL ANALYSIS

The additional analysis compares environmental injection surfaces with non-environmental channels in the coding domain. It reports that non-environmental instruction channels are generally stronger at steering agent behavior.

D.1 COMPARISON WITH NON-ENVIRONMENTAL INJECTION CHANNELS

  • Non-environmental instruction channels are generally stronger at steering agent behavior than environmental injection surfaces.The coding-domain comparison includes environmental locations and additional reference channels; its table reports conditional ASR (%) for GPT-5.4.Source: §D.1 · pp. 15–16, Table 13 · p. 16

D.2 BENIGN-TASK READ ORDER

Because tasks may expose surfaces in a fixed order, surface-level ASR differences may partly reflect when an injected surface is encountered. A read-order permutation study finds modest variation across read depths while the surface-level pattern remains.

D.2 BENIGN-TASK READ ORDER

  • Fixed read order may confound surface comparisons because ASR differences can partly reflect when the injected surface is encountered.Each evaluation injects into only one surface at a time.Source: §D.2 · p. 16
  • The study evaluates all 24 read-order permutations of a four-surface workflow task, totaling 1,440 runs across surfaces, risks, and within-surface positions.Each surface appears six times at each read depth.Source: §D.2 · p. 16
  • Conditional ASR varies modestly across read depths, with overlapping 95% confidence intervals, while Slack and GitHub retain higher ASR than Confluence and Jira.Read-depth variation is smaller than variation across environment surfaces.Source: §D.2 · p. 16

D.3 TASK HORIZON

These analyses examine task horizon, file length, learned attack framing, placement search, and susceptibility across agents. They report higher ASR for shorter tasks, a weak file-length association, no consistent advantage for LLM placement selection, and substantial variation across agents.

D.3 TASK HORIZON

  • Conditional ASR increases as task horizon shortens, while documentation surfaces retain higher ASR than code and data files across horizons.The groups contain 12 single-atomic-task tasks, 10 tasks composed of 2–3 atomic tasks, and 10 tasks composed of 4–5; all use seven risks and available surfaces.Source: §D.3 · pp. 16–17, Table 16 · p. 17

D.4 HOST-FILE LENGTH

  • Spearman ρ = −0.34 (p = 0.22) indicates only a weak, non-significant negative association between host-file length and ASR.The analysis pools risks, injection positions, and mid- and long-horizon coding tasks across the 15 repository files.Source: §D.4 · p. 17

D.5 ATTACK TEMPLATE AND QUALITATIVE EXAMPLES OF GENERATED INJECTIONS

  • Learned variants preserve the fixed template’s <INFORMATION> structure, either compressing it or adding a security-team authority cue while keeping the malicious goal verbatim.A two-stage attacker trained only against GPT-4o also generates an injection that succeeds against unseen GPT-5.4, spoofing a user turn and restating the benign task.Source: §D.5 · p. 17, Figure 5 · p. 18, Figure 6 · p. 18

D.6 PLACEMENT SEARCH UNDER A QUERY BUDGET

  • Across domains and placement budgets, GPT-5.4 placement selection does not consistently outperform the simple fixed frequency prior.The held-out comparison varies only the order of placements tried under fixed content; the figure also includes random ordering and an oracle ceiling.Source: Figure 8 · p. 19, §D.6 · p. 19

D.7 OVERALL SUSCEPTIBILITY ACROSS MORE AGENTS

  • Across eight agents and three domains, susceptibility varies widely: GPT-5.6 and Claude models are near-immune, while GPT-5-mini and Gemini-3.1-Pro are measurably more susceptible.GPT-4o was evaluated only in the tool-use arena.Source: §D.7 · p. 20, Table 17 · p. 20

D.8 ADDITIONAL ANALYSIS OF CONTENT AND PLACEMENT EFFECTS

The analysis separates placement effects from content effects and describes the placement prior used by the attacker. With content fixed, prior-ranked placement improves ASR over random placement; learned content further increases ASR at the prior placement.

D.8 ADDITIONAL ANALYSIS OF CONTENT AND PLACEMENT EFFECTS

  • 34.7% vs 25.6% conditional ASR on the leave-risk test set and 77.1% vs 38.8% on leave-task result when prior-ranked replaces random surface placement with content fixed.Within-surface position is fixed at the end, isolating surface selection.Source: Table 18 · p. 20, §D.8 · p. 20
  • 62.7% vs 34.7% ASR on leave-risk and 81.8% vs 77.1% on leave-task result when learned Content-Only content replaces the fixed template at prior placement.This comparison isolates the contribution of learned content.Source: Table 18 · p. 20, §D.8 · p. 20

D.9 PLACEMENT PRIOR RANKING

  • The placement prior ranks surface categories offline by average training-set ASR, then chooses the highest-ranked category available and places the injection at its end.The table reports the resulting tool-use-domain ranking.Source: §D.9 · p. 20, Table 19 · p. 21

E LIMITATIONS

The study identifies limitations in interpreting trust, environmental coverage, and threat-model scope.

E LIMITATIONS

  • The trust region effect is measured as a behavioral pattern, not as a direct measurement of agents’ internal trust; explaining why it occurs would require interpretability analyses.The authors characterize it as an empirical behavioral phenomenon.Source: E LIMITATIONS · p. 21
  • Evaluation uses three human-verified testbeds and does not cover the full diversity of real-world agent environments.Expanding to substantially more environments requires validation of functionality, task correctness, injection delivery, and surface semantics.Source: E LIMITATIONS · p. 21
  • The work focuses primarily on indirect prompt injection, so whether similar trust-region patterns occur under broader agent-security threats remains outside its scope.Source: E LIMITATIONS · p. 21

Related papers

Loading 2610.38867v1…