Source-linked AI summary
BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agents
Yunhao Feng, Yige Li, Yutao Wu, Yingshui Tan, Yanming Guo, Yifan Ding, Kun Zhai, Xingjun Ma, Yu-Gang Jiang
TL;DR
Existing backdoor studies evaluate isolated attack vectors, leaving cross-stage propagation in multi-step LLM agent workflows poorly understood. BackdoorAgent provides a stage-aware framework and benchmark for analyzing planning, memory, and tool attacks across four agent applications. Experiments show that single-stage triggers can persist across multiple steps and propagate through intermediate states, with substantial persistence rates across all three attack stages.
Problem
Existing evaluations are fragmented across isolated modules and protocols, leaving cross-stage backdoor propagation in multi-step agent workflows poorly understood.
Method
BackdoorAgent instruments agent execution across planning, memory, and tool stages and benchmarks four representative applications in language-only and multimodal settings.
Results
Triggers implanted at one stage persist across multiple steps and propagate through intermediate states, with GPT-based persistence rates of 43.58% for planning, 77.97% for memory, and 60.28% for tool-stage attacks.
Takeaways & Limitations
Backdoors are workflow-level phenomena: memory attacks are most persistent, tool and environment attacks dominate closed-loop agents, and clean task performance does not imply robustness.
Abstract
from arXiv · showhide
Large language model (LLM) agents execute tasks through multi-step workflows that combine planning, memory, and tool use. While this design enables autonomy, it also expands the attack surface for backdoor threats. Backdoor triggers injected into specific stages of an agent workflow can persist through multiple intermediate states and adversely influence downstream outputs. However, existing studies remain fragmented and typically analyze individual attack vectors in isolation, leaving the cross-stage interaction and propagation of backdoor triggers poorly understood from an agent-centric perspective. To fill this gap, we propose \textbf{BackdoorAgent}, a modular and stage-aware framework that provides a unified, agent-centric view of backdoor threats in LLM agents. BackdoorAgent structures the attack surface into three functional stages of agentic workflows, including \textbf{planning attacks}, \textbf{memory attacks}, and \textbf{tool-use attacks}, and instruments agent execution to enable systematic analysis of trigger activation and propagation across different stages. Building on this framework, we construct a standardized benchmark spanning four representative agent applications: \textbf{Agent QA}, \textbf{Agent Code}, \textbf{Agent Web}, and \textbf{Agent Drive}, covering both language-only and multimodal settings. Our empirical analysis shows that \textit{triggers implanted at a single stage can persist across multiple steps and propagate through intermediate states.} For instance, when using a GPT-based backbone, we observe trigger persistence in 43.58\% of planning attacks, 77.97\% of memory attacks, and 60.28\% of tool-stage attacks, highlighting the vulnerabilities of the agentic workflow itself to backdoor threats. To facilitate reproducibility and future research, our code and benchmark are publicly available at GitHub.
1 Introduction
LLM agents’ multi-stage workflows expand backdoor attack surfaces because poisoned artifacts can persist and propagate across planning, memory, and tool stages. BackdoorAgent unifies analysis through a stage-aware framework and benchmark spanning four agent applications, with experiments showing substantial trigger persistence.
- Stateful agent architectures expose attack surfaces beyond the backbone model through poisoned memories, manipulated planning traces, and adversarial environmental observations.
- Existing evaluations often isolate attack vectors, overlooking how reused plans, retrieved documents, and tool outputs enable triggers to propagate across stages and persist over time.
- BackdoorAgent decomposes agent backdoor threats into planning, memory, and tool stages for fine-grained, agent-centric analysis of cross-stage propagation.
- The benchmark covers Agent QA, Agent Code, Agent Web, and Agent Drive across language-only and multimodal agent workflows.
- 43.58% of planning attacks, 77.97% of memory attacks, and 60.28% of tool-stage attacks showed trigger persistence with a GPT-based backbone.
2 Related Work
Prior backdoor research on LLMs, RAG, and agents is fragmented across isolated modules, implementations, threat assumptions, and protocols. This fragmentation limits direct comparison and obscures vulnerability behavior across multi-stage agent workflows.
- LLM and RAG backdoor studies commonly use single-step or single-module assumptions that do not capture temporal propagation and feedback dynamics in agents.
- Existing agent evaluations use disparate implementations, threat assumptions, and protocols, limiting direct comparison across approaches.
- These evaluation practices obscure how vulnerabilities manifest across different components of an agent workflow.
3 Preliminaries
The paper models agents as recurrent workflows in which planning, memory, and tool stages produce artifacts that update observable context or internal state. Because these artifacts persist, a trigger injected into one stage can influence later steps and other stages.
- Agent Formulation: Intermediate artifacts from planning, memory access, and tool interaction govern recurrent agent workflows and can persist to influence future decisions.
- Agent Formulation: Each workflow stage consumes the query, observable context, and internal state, then produces an artifact that may be written back into context or state.
- Agent Formulation: Context updates append retrieved snippets and tool responses, while state updates can cache results, log decisions, or update memory indices.
- Backdoor Attacks in Agent Workflows: A backdoor remains dormant during normal execution and activates when a specific trigger is present in a planning, memory, or tool artifact.
- Backdoor Attacks in Agent Workflows: Once injected, triggers enter future context or state through update rules, enabling persistence across steps and cross-stage influence.
- BackdoorAgent Framework: BackdoorAgent instruments multi-step agents to analyze trigger injection and propagation across stages using standardized execution, logging, and evaluation protocols.
4 BackdoorAgent Framework
BackdoorAgent instruments multi-step agents with stage-aligned hooks and trajectory logging to analyze backdoor injection and propagation across planning, memory, and tool-use components. Its benchmark covers representative language-only and multimodal agent applications with standardized evaluation settings.
- Framework design: BackdoorAgent instruments agent workflows to analyze how triggers are injected and propagated across planning, memory, and tool-use stages.The framework exposes explicit component hooks and records structured execution trajectories for systematic analysis.
- Execution and evaluation: The runtime executes configured agents for a fixed step budget or until termination while recording structured trajectories for reproducible diagnosis.Trajectory logging supports identifying trigger injection, activation, and downstream influence across components.
- Backdoor injection: Attacks replace one component locally while leaving the other planning, memory, and tool components unchanged.The injected stage can alter plans, retrieval results, or tool feedback, which are then reused as intermediate artifacts.
- Backdoor injection: BackdoorAgent attaches hooks at component interfaces rather than enforcing a fixed stage order, accommodating agents that invoke memory or tools multiple times per step.Relevant interfaces include plan generation, retrieval calls, and tool execution or return.
- Benchmark: The benchmark spans four representative applications, including retrieval-grounded QA, iterative tool-grounded code synthesis, web interaction, and driving-oriented workflows.Agent Web is evaluated only on multimodal backbones, while Table 2 covers closed-source backbones and Table 3 reports multimodal Agent Web evaluation.
5 Experiments
BackdoorAgent evaluates seven attacks across four agent workflows, measuring clean performance, attack success, and attacked-task accuracy. The experiments show channel- and workflow-dependent vulnerabilities, including persistent memory attacks, sequential amplification, and stealthy behavioral compromise despite preserved accuracy.
- Experimental Setup: BackdoorAgent evaluates four agent workflows and seven attacks spanning planning, memory, and tool/environment injection channels.The evaluation covers Agent QA, Agent Code, Agent Drive, and Agent Web, using both closed- and open-source backbones.
- Evaluation Metrics: Clean ACC, ASR, and ACC under attack jointly measure benign task success, attacker-specified behavior, and triggered-task success.ACC under attack uses the same task-specific verifier as clean evaluation.
- Results and Analyses: High ASR can coexist with limited ACC degradation, so standard task-performance metrics may miss behavioral compromise.Examples include Agent Code with AgentPoison and Agent QA with AgentPoison, where attacked ACC remains close to clean ACC.
- Results and Analyses: Memory-channel attacks frequently achieve ASR above 90% while preserving usable task accuracy, whereas tool and environment attacks dominate closed workflows such as Agent Drive.The reported pattern is organized more by injection channel than by task category.
- Results and Analyses: Agent Drive exhibits cascading failures because perturbed plans or tool feedback alter subsequent states, compounding small perturbations over long horizons.BadChain exceeds 90 ASR on every open-source Drive backbone, while AdvAgent frequently exceeds 90 in Drive settings.
- Ablation Studies: Across model families, memory attacks have the highest average ASR, planning attacks are lower but stable, and tool attacks vary with task dynamics and environment interaction.The aggregation is computed across tasks and attacks by injection channel.
- Ablation Studies: Memory attacks incur the greatest token overhead as trigger strength increases, while planning attacks are most token-efficient, revealing an effectiveness–efficiency trade-off.Memory overhead is attributed to repeated retrieval and reinsertion; planning attacks mainly manipulate transient reasoning traces.
- Ablation Studies: Token-probability analysis is also applied to multi-step agent executions to compare target and non-target token behavior under clean and triggered conditions.Figure 4 reports average probabilities across agent tasks.
6 Conclusion
The paper presents BackdoorAgent as a framework and benchmark for studying backdoors across multi-step agent workflows. Its findings identify workflow-level persistence, channel-specific dominance, and stealthy harmful behavior despite apparently strong task performance.
- Conclusion: BackdoorAgent is a framework and benchmark for analyzing backdoor vulnerabilities in multi-step LLM agent workflows.The benchmark spans representative agent settings and diverse closed- and open-source backbones.
- Conclusion: Memory-channel attacks are the most persistent, while tool and environment-channel attacks dominate closed-loop agents.The conclusion summarizes channel-specific vulnerability patterns across the benchmark.
- Conclusion: Many attacks preserve high task accuracy while reliably inducing harmful behaviors, exposing a stealthy failure mode in agentic systems.The conclusion motivates continued research on backdoor attacks and defenses in agents.
A Additional Defenses
The paper tests whether probability-based defenses for standalone LLM backdoors transfer to multi-step agents. The preliminary results indicate limited separability because agent backdoor signals can be subtle, delayed, and entangled with normal workflow behavior.
- Additional Defenses: The defense study examines whether existing standalone-LLM backdoor signals remain meaningful in multi-step, tool-augmented agents.The paper analyzes transferability and limitations rather than proposing a new defense mechanism.
- Additional Defenses: Existing defenses typically detect single-pass anomalies such as trigger-token probability spikes, logit shifts, or probing inconsistencies.These signals assume a static input–output mapping.
- Additional Defenses: The study compares output-token probability distributions from a target backbone and a reference model under clean and triggered conditions.The comparison is performed across multi-step agent executions.
- Additional Defenses: The detector achieves AUROC above random guessing but only a modest margin, indicating limited separability across agent tasks and attack types.The result is reported for the aggregated ROC curve.
- Additional Defenses: Standalone-LM defenses do not directly generalize because agent backdoor signatures may be subtle, delayed, or entangled with benign reasoning, retrieval, and tool outputs.The paper argues that temporal propagation, cross-module interactions, and environment feedback require agent-aware analysis.
B.1 Task Overview
The benchmark covers four representative agent applications spanning retrieval QA, code synthesis, multimodal web interaction, and closed-loop driving. Each application executes multi-step workflows that reuse intermediate artifacts and applies task-specific verification.
- Four applications instantiate the planning–memory–tools design space: Agent QA, Agent Code, Agent Web, and Agent Drive.
- Intermediate plans, memories, and tool or environment feedback are written into context or state and reused in later steps.
- Agent QA retrieves evidence and synthesizes answers through persistent memory access, with exact or semantic matching as verification.
- Agent Code alternates planning, code generation, and tool execution, revising programs from execution feedback and testing success.
- Agent Web combines visual and textual webpage observations with browser feedback to perform actions, verifying the final interface state.
- Agent Drive repeatedly plans control actions from observations, updating the environment state while checking safety constraints and task completion.
B.2 Backdoor Design per Injection Channel
BackdoorAgent organizes attacks by the workflow stage they compromise: planning, memory, or tools and environment. Each channel injects triggers into a stage-specific artifact that can influence later agent behavior.
- Attacks are implemented as component-local perturbations injected into planning traces, retrieved memory snippets, or tool and environment feedback.
- Planning-channel backdoors embed triggers in planning artifacts so activated agents follow attacker-specified plan patterns while appearing to solve tasks.
- Memory-channel backdoors poison retrieved content so attacker instructions are repeatedly reintroduced and reinforced across steps.
- Tool and environment-channel backdoors place triggers in outputs or observations, biasing subsequent planning and decisions in feedback-driven settings.
B.3 Attack Objectives (Task-Specific)
The framework assigns task-specific attacker objectives to the same stage-based injection design. These objectives target answer semantics, operations, interface actions, or physical control depending on the application.
- Attacker objectives vary by task even though the injection mechanism is unified.
- Agent QA attacks target semantic manipulation by inducing incorrect responses while preserving fluent answers.
- Agent Code attacks target operational manipulation by inducing destructive or unsafe operations while keeping code generation plausible.
- Agent Web attacks target interface-level misdirection by causing incorrect webpage actions while appearing to complete the task.
- Agent Drive attacks target unsafe control by inducing hazardous behavior through small perturbations that compound over time.
- Each run logs full agent trajectories to support analysis of trigger activation, persistence, and cross-stage propagation.