PaperlayerView exact revision on arXiv

Source-linked AI summary

AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?

Hao Li, Ruoyao Wen, Shanghao Shi, Ning Zhang, Yevgeniy Vorobeychik, Chaowei Xiao

arXiv:2602.03117v3cs.CR

TL;DR

Indirect prompt injections threaten tool-using agents because malicious instructions in third-party data can hijack behavior. This paper introduces AgentDyn, a realistic open-ended benchmark with dynamic planning and helpful instructions, and finds that nearly all existing defenses still fall short of real-world deployment.

  • Problem

    Existing benchmarks inadequately capture dynamic replanning, helpful third-party instructions, and complex tasks needed to assess deployable agent defenses.

  • Method

    AgentDyn is a manually designed end-to-end benchmark with 60 open-ended tasks and 560 injection test cases across three suites.

  • Results

    Nearly all evaluated defenses struggle substantially on AgentDyn despite near-perfect performance on existing benchmarks, indicating they are not ready for real-world deployment.

  • Takeaways & Limitations

    AgentDyn highlights the need for more realistic benchmarks that evaluate both security and utility under practical conditions.

  • Takeaways & Limitations

    AgentDyn cannot fully capture real-world complexity and diversity, and its attacks are relatively weak against the latest frontier models.

Abstract

from arXiv · show

AI agents that autonomously interact with external tools and environments have shown great promise across real-world applications. However, their reliance on external data exposes them to serious indirect prompt injection attacks, where malicious instructions embedded in third-party content hijack agent behaviors. To mitigate this threat, a growing number of defenses have been proposed and evaluated under existing agent security benchmarks. These benchmarks provide structured environments for comparing attacks and defenses, and have become a key driver for defense design and optimization. However, as agents move toward more complex and open-ended real-world deployments, there is a pressing need for benchmarks to become more adaptive and better reflect the dynamic environments faced by real-world agentic systems. In this work, we reveal three fundamental flaws in the current benchmarks and push the frontier along these dimensions: (i) lack of dynamic open-ended tasks, (ii) lack of helpful instructions, and (iii) simplistic user tasks. To bridge this gap, we introduce AgentDyn, a manually designed benchmark featuring 60 challenging open-ended tasks and 560 injection test cases across Shopping, GitHub, and Daily Life. Unlike prior static benchmarks, AgentDyn requires dynamic planning and incorporates helpful third-party instructions. Our evaluation of ten state-of-the-art defenses suggests that almost all existing defenses are either not secure enough or suffer from significant over-defense, revealing that existing defenses are still far from real-world deployment. Our benchmark is available at https://github.com/leolee99/AgentDyn.

1 Introduction

Existing agent-security benchmarks underrepresent the dynamic, helpful, and complex conditions of real-world deployment. AgentDyn targets these gaps, and its evaluation finds that nearly all tested defenses remain unsuitable for practical use.

  • Motivation: Indirect prompt injections embed malicious instructions in third-party data, hijacking agents that interact with poisoned webpages or emails.This creates security risks for agents that autonomously invoke external tools and environments.
  • Benchmark gaps: Current benchmarks omit dynamic replanning, helpful third-party instructions, and sufficiently complex user tasks.Static tasks can often be planned upfront, external instructions are usually treated as malicious, and tasks commonly involve short trajectories, few applications, and small tool sets.
  • AgentDyn: AgentDyn provides 60 open-ended tasks and 560 injection test cases across Shopping, GitHub, and Daily Life, requiring dynamic planning and helpful third-party instructions.The benchmark is designed to evaluate deployability under practical dimensions overlooked by existing benchmarks.
  • Evaluation findings: Several defenses, including Prompt Sandwich, Spotlighting, and PromptGuard2, struggle to provide effective security on open-ended attack scenarios.These defenses perform poorly when evaluated under AgentDyn’s more realistic conditions.
  • Evaluation findings: Planning-dependent defenses suffer severe utility drops in dynamic-planning tasks, while filtering-based defenses can reduce utility to near-zero by rejecting helpful instructions.Tool Filter, CaMeL, and DRIFT rely heavily on initial plans; ProtectAI and PIGuard struggle to distinguish benign instructions from malicious injections.
  • Implication: Nearly all existing defenses remain far from real-world deployment, despite 79% of surveyed community effort focusing on defenses.The findings indicate a mismatch between current defense evaluation and practical requirements.

2 Related Work and Preliminaries

Prior work develops multiple defense families and benchmarks for prompt-injection security, but their evaluations leave a substantial gap from realistic deployment. AgentDyn addresses this gap by testing dynamic, helpful, and more complex task conditions.

  • Defense approaches: Prompt-injection defenses include prompting-based, alignment-based, filtering-based, and system-level strategies.Examples use prompt guidance, safety alignment, auxiliary detection models, or constrained action spaces and execution environments.
  • Security benchmarks: Existing benchmarks include InjecAgent, ASB, and AgentDojo, which have been widely used to evaluate agent-defense strategies.InjecAgent and ASB focus on isolated steps, while AgentDojo provides an end-to-end environment.
  • Benchmark comparison: InjecAgent and ASB are single-step benchmarks with one application and no more than three visible tools per task.Their average trajectory length and application count are both exactly one.
  • Benchmark comparison: AgentDojo offers longer trajectories and greater application involvement, but most tasks still average only three steps.This limits its ability to reflect robustness in real-world, long-context deployments.
  • AgentDyn: AgentDyn expands task difficulty through larger toolsets, longer trajectories, and greater application involvement than prevalent benchmarks.Its design aims to expose a broader range of agent-security-system behavior.

3 AgentDyn: An Open-ended Agent Security Environment

AgentDyn is an end-to-end open-ended sandbox built around realistic task planning, helpful environmental instructions, and complex multi-application workflows. Its suites and test cases operationalize these requirements across diverse applications and dynamic scenarios.

  • Overview and structure: AgentDyn is an open-ended sandbox comprising user tasks, injection tasks, tools, and environments for end-to-end security evaluation.The framework is designed similarly to AgentDojo while targeting more comprehensive real-world deployment conditions.
  • User task design: User tasks require agents to adapt their strategies in real time based on environmental feedback.Dynamic planning prevents the task from being fully determined before tool interaction.
  • User task design: Each task embeds at least one helpful instruction in the critical execution path as a prerequisite for completion.Agents therefore inevitably retrieve helpful environmental guidance during execution.
  • User task design: Tasks increase realism through longer trajectories, larger toolsets, and interactions across multiple applications.These criteria are intended to increase evaluation difficulty and complexity.
  • Task suites and tools: AgentDyn contains Shopping, GitHub, and DailyLife suites covering seven application scenarios with corresponding tools.The suites integrate applications such as email, banking, web, filesystem, and calendar alongside shopping or GitHub workflows.
  • Test case synthesis: The benchmark curates 60 user tasks and 28 injection tasks, producing 560 cross-product security test cases.Every test case requires dynamic planning and includes helpful instructions.
  • Dynamic scenarios: Dynamic scenarios are designed for multiple applications to broaden case coverage while maintaining practical realism.Calendar is the stated exception to the use of multiple dynamic scenarios.

4 Experiments

The experiments evaluate 12 agents and 10 defenses on AgentDyn using security and utility metrics across Shopping, GitHub, and Daily Life. Nearly all defenses that perform strongly on existing benchmarks struggle in AgentDyn’s dynamic, long-horizon setting, through weak security, over-defense, or both.

  • Experiment Setup: AgentDyn evaluates 12 LLM agents and 10 defenses across three evaluation suites using benign utility, utility under attack, and attack success rate.Overall performance is averaged across Shopping, GitHub, and DailyLife.
  • Defenses: The study covers prompting-based, filtering-based, alignment-based, and system-level defenses, including Prompt Sandwiching, Spotlighting, three detectors, Meta SecAlign, and four system defenses.The reproduced approaches use official code or pretrained models, with an instruction added to complete tasks without requesting confirmation.
  • Comparison with AgentDojo: AgentDyn exposes deployment gaps that existing benchmarks can miss, including defenses that appear secure but over-defend or fail under dynamic, threatening injections.The authors characterize AgentDyn as practical for more comprehensive evaluation, while acknowledging its scenarios and complexity remain limited relative to real-world settings.
  • Prompting Defense: Prompt Sandwiching and Spotlighting maintain high utility but only slightly reduce ASR relative to the no-defense baseline.Their limited security effectiveness makes them insufficiently deployable in the evaluated open-ended scenarios.
  • Filtering-based Defense: ProtectAI and PIGuard substantially reduce utility through over-defense, while PromptGuard2 reaches 27.15% ASR on GPT-4o and loses utility under attack.These failures reflect difficulty distinguishing helpful instructions from malicious injections and discarding tool outputs containing detected injections.
  • Alignment-based Defense: Meta SecAlign improves utility and slightly reduces ASR relative to its base model, but retains approximately 9% ASR.The resulting security–performance balance is described as more viable for real-world deployment, though residual vulnerability remains.
  • System-level Defense: System-level defenses struggle with dynamic tasks: Tool Filter over-defends, CaMeL produces zero utility and zero ASR, and Progent loses utility on larger toolsets.Static planning blocks later-required tools, whereas dynamic access-control decisions become inaccurate as toolsets and execution horizons grow.
  • Comparison with AgentDojo: Meta SecAlign achieves only 53.4% utility on AgentDyn, despite approximately 80% utility on AgentDojo, while its ASR rises from 1.9% to 9.0%.All GPT-4o-powered defenses experience a sharp utility drop on AgentDyn compared with the undefended baseline, and the benchmark challenges even advanced defended models.

5 Conclusion

AgentDyn is a manually designed open-ended benchmark for evaluating defense deployability through realistic dynamic tasks, helpful environmental instructions, and complex user workflows. Its evaluation finds that nearly all defenses performing near-perfectly on existing benchmarks struggle substantially on AgentDyn, exposing previously hidden failure modes.

  • Conclusion: AgentDyn incorporates realistic dynamic tasks, helpful environmental instructions, and more complex user tasks to evaluate defense deployability from a utility-oriented perspective.The benchmark is manually designed and open-ended.
  • Conclusion: Nearly all existing defenses that achieve near-perfect performance on established agent security benchmarks struggle substantially on AgentDyn.The evaluation reveals failure modes that are not exposed by existing benchmarks.

A Limitations

AgentDyn is designed to improve realism, but it cannot fully capture real-world complexity and currently includes attacks that are relatively weak against frontier models.

  • AgentDyn cannot fully capture the complexity and diversity of real-world environments despite its realism-oriented design.
  • The benchmark’s attacks are relatively weak against the latest frontier models because attack research lags defense development.
  • AgentDyn prioritizes evaluating practical utility and deployability from a utility-oriented perspective rather than focusing primarily on security.
  • Because the framework is open, future advanced attacks can be incorporated dynamically.

B Realistic Design Policies

AgentDyn is designed to approximate real-world agent environments through realistic data formats, plausible third-party attack locations, interactable entities, and dynamic planning.

  • AgentDyn aims to make task and environment design more realistic and better reflect defense performance in practice.
  • Realistic Environment Structure: The benchmark preserves realistic environmental formats, including inboxes, reviews, webpages, and HTML rather than relying mainly on plain text.
  • Realistic Attack Surface: AgentDyn restricts webpage injections to advertisement banners, a plausible third-party location, instead of allowing attackers to manipulate main-body content.
  • Real-time Interactable Entities: The design includes real-time interactable entities, such as inboxes that can receive OTP emails, unlike static predefined content.

D Dynamic Scenarios in AgentDyn

AgentDyn organizes dynamic scenarios into Shopping, GitHub, and DailyLife suites, with representative examples in Table 4 and detailed listings in Tables 5–7.

  • The dynamic scenarios are grouped into Shopping, GitHub, and DailyLife suites.
  • Table 4 presents representative examples of the dynamic scenarios.
  • Tables 5, 6, and 7 provide detailed scenario listings for Shopping, GitHub, and DailyLife, respectively.

E Analysis of Task Trajectory Length

Trajectory length is associated with declining utility under attack, while attack success rate peaks around six steps, highlighting distinct effects of longer task execution.

  • The analysis examines utility and attack success rate across trajectory lengths under attack conditions.
  • Utility falls from 100% at trajectory length two to 23.6% when length exceeds ten.
  • The utility decline is stable as trajectory length increases and is especially sensitive at ten or fewer steps.
  • Attack success rate follows a roughly unimodal distribution, reaching its maximum around trajectory length six.

F Additional Attack

AgentDyn evaluates additional attack types on GPT-5-mini, GPT-5.1, and Claude-Sonnet-4.5. Template-based attacks are weak, while adaptive attacks improve attack success rates but remain limited.

  • Additional Attack Evaluation: The evaluation covers ignore-previous, completion, and manually designed scenario-based adaptive attacks on three recent language models.The models are GPT-5-mini, GPT-5.1, and Claude-Sonnet-4.5.
  • Attack Results: Template-based attacks, including Important Instructions, Ignore Previous, and Completion Attack, are weak against the latest safety-aligned models.
  • Attack Results: Adaptive attacks improve ASR on all three evaluated models, but the improvement remains limited.
  • Implications: The results identify stronger and more realistic attack development as a complementary research direction rather than a limitation specific to AgentDyn.

G Detailed Results on AgentDyn

AgentDyn reports detailed defense results by base model, defense category, task suite, and overall aggregate. The evaluation separates benign utility, utility under attack, and attack success rate.

  • Evaluation Organization: Results are organized by defense taxonomy and reported both per suite and overall across different base models.
  • Benign Utility: Benign Utility is reported for no defense and prompting-based, alignment-based, filtering-based, and system-level defenses.
  • Utility under Attack: Utility under attack is reported for no defense and prompting-based, alignment-based, filtering-based, and system-level defenses.
  • Attack Success Rate: ASR under attack is reported for no defense and prompting-based, alignment-based, filtering-based, and system-level defenses.
Loading 2602.03117v3…