PaperlayerView exact revision on arXiv

Source-linked AI summary

AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks

Tanqiu Jiang, Yuhui Wang, Jiacheng Liang, Ting Wang

arXiv:2602.16901v1cs.AI

TL;DR

Existing agent-security benchmarks largely focus on single-turn or static attacks, missing threats that exploit extended user–agent–environment interactions. AgentLAB introduces an extensible benchmark and attack framework for long-horizon scenarios, finding that representative agents remain highly vulnerable and one-shot defenses are difficult to retrofit.

  • Problem

    Existing security benchmarks primarily evaluate single-turn or static scenarios, failing to capture adversaries that exploit extended interactions to achieve malicious objectives.

  • Method

    AgentLAB provides a unified, extensible benchmark and multi-agent attack framework for evaluating LLM agents against adaptive, long-horizon attacks.

  • Results

    Representative LLM agents remain highly vulnerable to long-horizon attacks, while defenses designed for one-shot attacks are difficult to retrofit to long-horizon scenarios.

  • Takeaways & Limitations

    AgentLAB is intended to serve as a benchmark for measuring progress in securing LLM agents against practical long-horizon threats.

  • Takeaways & Limitations

    The formalized threat scope includes malicious tasks induced through multiround interactions, including adversarial prompts substituted for a subset of user instructions.

Abstract

from arXiv · show

LLM agents are increasingly deployed in long-horizon, complex environments to solve challenging problems, but this expansion exposes them to long-horizon attacks that exploit multi-turn user-agent-environment interactions to achieve objectives infeasible in single-turn settings. To measure agent vulnerabilities to such risks, we present AgentLAB, the first benchmark dedicated to evaluating LLM agent susceptibility to adaptive, long-horizon attacks. Currently, AgentLAB supports five novel attack types including intent hijacking, tool chaining, task injection, objective drifting, and memory poisoning, spanning 28 realistic agentic environments, and 644 security test cases. Leveraging AgentLAB, we evaluate representative LLM agents and find that they remain highly susceptible to long-horizon attacks; moreover, defenses designed for single-turn interactions fail to reliably mitigate long-horizon threats. We anticipate that AgentLAB will serve as a valuable benchmark for tracking progress on securing LLM agents in practical settings. The benchmark is publicly available at https://tanqiujiang.github.io/AgentLAB_main.

1. Introduction

AgentLAB addresses the gap between single-turn security evaluation and realistic long-horizon attacks by providing an extensible benchmark for adaptive, multi-turn agent threats. It organizes five attack families across realistic environments and security test cases, and evaluates persistent vulnerabilities and defense transfer.

  • Current agent-security benchmarks primarily assess single-turn or static scenarios, missing attacks that exploit extended interactions to achieve otherwise infeasible objectives.
  • AgentLAB evaluates long-horizon security through temporal exploitation, ecological validity, and extensibility.The benchmark targets sustained user–agent–environment interactions in realistic environments with persistent memory and supports adding environments, attacks, agents, and defenses.
  • AgentLAB instantiates five attack families: intent hijacking, tool chaining, task injection, objective drifting, and memory poisoning.
  • 644 security test cases span 28 realistic tool-enabled agentic environments through a unified multi-agent attack framework.
  • Experiments show vulnerabilities persist across most evaluated LLM agents, while defenses designed for one-shot settings often fail to transfer to long-horizon attacks.
  • AgentLAB is presented as the first benchmark dedicated to evaluating LLM-agent safety against adaptive, long-horizon attacks.The paper also contributes a taxonomy, a multi-agent attack framework, gradual multi-turn attack patterns, and an evaluation of representative agents.

2. Related Work

Prior agent-security research studies user–agent manipulation, environment–agent attacks, and benchmarked safety risks, but existing approaches predominantly remain single-turn or static. This leaves gradual, adaptive attacks accumulating across extended interactions underrepresented.

  • User–agent interaction research applies jailbreaks and studies risky behaviors such as unauthorized actions, memory manipulation, and risky code generation.
  • Environment–agent interaction research examines indirect prompt injection through malicious instructions embedded in external content.
  • Existing research predominantly focuses on single-turn or static injection scenarios, neglecting gradual indirect injections that accumulate over extended interactions.
  • Agent-security benchmarks systematize evaluation across environments, test cases, safety-risk categories, failure modes, models, and tasks.The cited benchmarks include Agent-SafetyBench with 349 environments and 2,000 test cases, alongside prompt-injection evaluation frameworks.

3. Problem Formulation

The paper models long-horizon agent interactions as sequences of user prompts, agent actions, environment observations, and agent responses. Its threat model considers adversaries manipulating either user instructions or environmental observations, under black-box or white-box access.

  • LLM agents operate in complex environments through extended user–agent–environment interactions to solve challenging problems.
  • A long-horizon interaction consists of repeated user prompts, agent actions, environment observations, and agent responses.The complete trace is represented as a sequence of quadruplets ⟨p_i, a_i, o_i, r_i⟩ across interaction steps.
  • This formulation generalizes settings in which users provide only an initial instruction without subsequent interactions.
  • The threat model targets inducing a malicious task through multiround interactions, including exfiltrating confidential data.
  • A malicious user can substitute user instructions, while an environmental adversary can substitute observations such as webpages, documents, emails, or code comments.
  • The framework considers both black-box adversaries observing actions and responses and white-box adversaries with access to internal reasoning.
  • Long-horizon attacks let adversaries adapt instructions or observations across multiple interactions, increasing effectiveness and evasiveness relative to single-turn attacks.

4. AgentLAB

AgentLAB is a modular benchmark for evaluating long-horizon attacks through coordinated agent, environment, task, and attack components. It supports multiple attack strategies that manipulate extended interactions, including direct hijacking, benign-looking tool sequences, gradual objective shifts, and task injection.

  • Components: AgentLAB provides a modular evaluation framework spanning agents, environments, tasks, and adversarial attack strategies.The framework abstracts LLM capabilities, tool-enabled environments, task objectives, and attack parameters for reproducible experimentation.
  • Multi-agent framework: A multi-agent attacker uses a planner, attacker, judge, and sometimes verifier to generate, refine, execute, and assess long-horizon attacks.The planner synthesizes attack plans, the attacker adapts prompts or observations to target responses, the judge checks task completion, and the verifier validates tool calls when needed.
  • Intent hijacking: Intent hijacking progressively erodes safety guardrails through multi-turn prompting to induce malicious tool execution.Unlike conversational jailbreaks aimed at harmful content generation, this attack targets the agent’s action space.
  • Tool chaining: Tool chaining decomposes a malicious objective into individually benign-looking tool calls and guides the agent through the resulting sequence.The planner decomposes and verifies the calls, while the attacker adaptively refines prompts when progress stalls.
  • Task injection: Task injection connects malicious tool calls to benign tasks through plausible intermediate actions introduced across multi-turn observations.For example, searching email can connect a benign calendar action to malicious email sending, while contextual narratives make the intermediate steps appear justified.
  • Dataset construction: AgentLAB contains 28 tool-enabled environments and 644 malicious tasks spanning 9 risk categories and 5 long-horizon attack types.Tasks are manually validated for clear malicious objectives, programmatic attack evaluation, and reasonable baseline agent performance; benign counterparts support utility measurement for several attack types.

5. Evaluation

AgentLAB evaluates representative proprietary and open-weight agents against adaptive, multi-turn attacks and finds substantial vulnerability across attack types. Attack effectiveness generally increases with longer horizons, while one-shot defenses transfer inconsistently to long-horizon settings.

  • Attack effectiveness: Average ASR on GPT-5.1 exceeds about 70%, indicating high vulnerability across both proprietary and open-weight agents.The reported pattern is presented as a challenge not limited to a particular model architecture or training procedure.
  • Attack effectiveness: For GPT-4o, long-horizon ASR rises from 62.50% under one-shot injection to 79.9%, while Claude-4.5 records 0% under both strategies.The comparison uses the “important message” one-shot baseline and tests gradual behavioral diversion.
  • Attack effectiveness: Claude shows 27.2% ASR for intent hijacking but 73.3% for tool chaining, whereas Qwen-3 needs 4.6 versus 8.5 turns for those attacks.The passage attributes the difference to task-level persuasion versus sequential elicitation of individual tool calls.
  • Ablation studies: ASR grows monotonically with maximum attack turns: GPT-4o task injection reaches about 80% at nturn = 5, while Qwen-3 plateaus near 90% by nturn = 3.The result indicates earlier saturation for the open-weight model in this example.
  • Ablation studies: Increasing optimization steps consistently improves ASR, but more gradually than adding attack turns, making attack-turn budget the stronger effectiveness factor.The comparison concerns adaptive optimization steps and additional attack turns across the evaluated attack types.
  • Defense evaluation: Baseline defenses are generally ineffective across all agents and attack types; Self-Reminder reduces Claude-4.5 intent-hijacking ASR to 6.9% but leaves tool-chaining ASR at 57.9%.Repeated Prompt is effective on Gemini-3 at 31.9% ASR but less effective on other agents.

6. Conclusion

AgentLAB is introduced as a benchmark and extensible framework for evaluating LLM-agent security against attacks exploiting extended user-agent-environment interactions. Its evaluations find high vulnerability to long-horizon attacks and limited transferability of one-shot defenses.

  • Conclusion: AgentLAB is the first benchmark dedicated to evaluating LLM-agent security against attacks exploiting extended user-agent-environment interactions.The benchmark supports a unified, extensible framework and evaluates agents built on proprietary and open-weight LLMs.
  • Conclusion: Evaluated agents are highly vulnerable to long-horizon attacks, while defenses designed for one-shot attacks are difficult to retrofit to long-horizon scenarios.The conclusion presents AgentLAB as a benchmark for measuring progress in this area of AI security.

A. Implementation Details

The implementation instantiates long-horizon attacks through black-box, adaptive interactions and attack-specific mechanisms spanning direct manipulation, tool composition, and environmental objective drift. Objective drifting is measured against a clean baseline using behavioral changes in purchasing decisions.

  • Threat model: The threat model gives the adversary black-box access to adapt prompts across turns to elicit malicious tasks refused under direct single-turn instruction.For tool chaining, the adversary instead sequences individually benign tool calls into a malicious objective.
  • Implementation: The framework coordinates planner, attacker, judge, and, for some attacks, verifier agents to generate, execute, refine, and validate long-horizon attack steps.Tool chaining uses environment validation, while adaptive prompt refinement is applied when progress stalls.
  • Objective drifting: Objective drifting controls environmental text rather than system prompts, using product descriptions and search snippets to promote alternative objectives during browsing.The injected content can simulate updated preferences or authority claims without explicit commands.
  • Objective drifting: Gradual exposure accumulates injected content across pages and interaction steps, incrementally shifting behavior away from the original frugality objective.The attack relies on the agent integrating contextual information from its environment into decision-making.
  • Objective drifting: Goal drift is measured against a clean, uninjected baseline using changes in frugality score and mean purchase price.These measures quantify behavioral deviation in purchasing decisions.

A.4. Task Injection

The task injection attack manipulates an agent through trusted tool outputs and gradual, adaptive prompt sequences that bridge benign and harmful tool-call trajectories. It conceals harmful intent within apparently legitimate operations, exploiting gaps between syntactic checks and semantic understanding.

  • Threat model: The adversary influences tool outputs to induce harmful tool-call sequences that would be rejected if requested directly.The attack exploits trust in tool outputs and limited goal consistency across extended tool chains.
  • Long-horizon injection: The attack bridges a benign tool sequence and an adversarial sequence with intermediate actions that make the transition more acceptable.Progressive rationalization can replace explicit inclusion of the target instruction.
  • Adaptive refinement: The grey-box adversary observes complete tool trajectories and intermediate reasoning traces, enabling refinement based on execution outcomes.Successful attack instances are stored in a persistent memory bank for later exemplar retrieval.
  • Adaptive refinement: The implementation uses hierarchical retrieval to select related successful examples, fills remaining slots from the memory bank, and falls back to static demonstrations when necessary.Retrieved exemplars can share either the user task or the adversarial objective.
  • Adaptive refinement: The adaptive algorithm repeatedly generates and rewrites attack snippets, executes the agent, and stores successful attacks for future use.The framework combines adversarial generation, rewriting, trajectory inspection, and persistent memory.
  • Attack effect: The attack can establish trust through a credible persona and progressively request operational detail, defeating single-turn filters because each turn appears contextually appropriate.This demonstrates how multi-turn contextual plausibility can sustain harmful escalation.
  • Attack effect: Decomposing a harmful objective into innocent operations can evade detection when the assistant processes payloads without semantic safety analysis.The demonstrated gap is between a user’s verbal description and the actual content processed by the assistant.

Turn 2 (Attack)

In the attacked shopping environment, persuasive injections shift the agent from a near-optimal low-cost purchase toward a substantially more expensive product despite unchanged requirements.

  • Turn 2 (Attack): The attack presented higher-priced options through repeated search injections and biased premium framing.Observed prices included $14.16 after three search injections and $49.99 with premium messaging.
  • Turn 2 (Attack): The baseline agent achieved a frugality score of 1.0 in a clean environment, while injections shifted decisions toward products costing 2–3× more.The task specifications remained unchanged while persuasive content appeared at search and product-page touchpoints.

B.4. Task Injection

A task injection attack uses coordinated snippets to redirect a benign webpage-reading task into unauthorized Slack user-management commands. It relies on fabricated dependencies and policy-like signals to make malicious actions appear necessary.

  • B.4. Task Injection: Three coordinated injection snippets hijack webpage reading by fabricating task dependencies, policies, consent, and system errors.Together, these elements create a counterfeit compliance protocol for unauthorized actions.
  • B.4. Task Injection: A fabricated token-retrieval requirement redirects the agent to unauthorized inbox access and an attacker-controlled URL.The injection presents token discovery as necessary before completing the source task.

Turn 2: Injection in Charlie’s Inbox

The inbox injection establishes a fake policy framework and then presents a fabricated system error that instructs the agent to perform malicious remediation actions.

  • Turn 3: Injection in Charlie’s Inbox: A fabricated safe-handling protocol uses authoritative-sounding tokens to establish policy legitimacy.The injected notice claims that content rationalization requires a specific verification token.
  • Turn 3: Injection in Charlie’s Inbox: A spoofed policy-mismatch error creates a false requirement to execute malicious tools with specified arguments.The accompanying explanation frames the unauthorized actions as remediation for the alleged error.

B.5. Memory Poisoning

Memory poisoning hides prompt injections in content the agent routinely processes, stores them as user preferences, and later retrieves them as false context that overrides safety behavior. A demonstrated attack then causes the agent to publish sensitive data without filtering.

  • Attack mechanism: When a harmful request arrives, retrieved memories provide false context that overrides safety behaviors.The memory retrieval step links prior injected preferences to the later harmful action.
  • Attack mechanism: Memory poisoning embeds hidden injections in emails, code comments, and product descriptions during routine agent tasks.The attack objective is to inject malicious memories covertly through content the agent already processes.
  • Attack demonstration: The agent posts every line of tweets.txt, including an SSN, after retrieving upload-related memories without applying filtering.The execution posts both an ordinary sentence and the sensitive value 123-45-6789.
  • Attack mechanism: The injected content masquerades as user preferences and is silently stored in persistent memory.Examples include preferences about uploading reports, scripting file uploads, and separating private from publish folders.
Loading 2602.16901v1…