STAC: When Innocent Tools Form Dangerous Chains for LLM Agents

1 Introduction

• Unlike text-only harms, tool-enabled agents can directly alter external environments, and STAC’s example sequence destroys the only copy of a critical document.The steps—compressing a document, deleting the original, then deleting ZIP files—appear benign individually.Source: §1 · pp. 1–2, Figure 1 · p. 2
• STAC’s automated pipeline generates malicious tool chains, verifies their executability through environment interaction, and crafts stealthy prompts to elicit the final malicious call.The chain’s malicious intent is revealed only at its final tool call, while preceding steps appear benign in isolation.Source: §1 · p. 2
• 91.2% average final ASR across 483 STAC cases, with ASR exceeding 90% for seven of eight evaluated agents.The benchmark spans ten agent-specific failure modes and 1,352 user-agent-environment interaction sets.Source: §1 · pp. 2, 4
• The reasoning-based defense cuts ASR by up to ∆ASR=28.8% initially, but protection erodes under adaptive attacks; ToolShield has the lowest ASR after sustained multi-turn attacks.Both defenses remain insufficient.Source: §1 · pp. 2–3
• On 390 shared SHADE-Arena baseline-comparison cases, STAC achieves 95.1% ASR versus MT-AgentRisk’s 53.2%.STAC validates chains through interactive execution, while MT-AgentRisk plans from tool documentation without environment interaction.Source: §1 · p. 3
• STAC combines a general tool-call attack surface, individually benign intermediate steps, in-environment chain validation, and paired defense analysis.The paper distinguishes these properties from prior and concurrent approaches.Source: §1 · p. 3

2.1 Threat Model

• The threat model assumes direct user-interface access, knowledge of the environment specification, and multi-turn adaptation, but excludes modifying tools, tool outputs, or the system prompt.The attacker may fabricate a plausible prior interaction history; indirect prompt injection is left to future work.Source: §2.1 · p. 4

2.2 The STAC Framework

• Each trajectory alternates user, agent, and environment turns, with a synthetic prefix for earlier subgoals and execution turns for the final goal.The Judge scores prompt harmlessness, attack-goal progress, and agent helpfulness after each execution turn.Source: §2.2 · pp. 4, 6
• The framework generates 2–6-call chains whose intermediate calls appear benign, verifies calls through environment execution, and creates synthetic histories before adaptive attempts at the final goal.The Planner can attempt the final goal over up to three execution turns.Source: §2.2 · p. 5

2.3 Defenses

• Defenses include zero-shot prompts that assess harms and benefits or summarize cumulative intent, experience-based ToolShield, and adaptive attacks informed by the installed defense.All defenses are applied through the system prompt without model retraining or architectural changes.Source: §2.3 · p. 6

2.4 STAC Benchmark Construction

• The benchmark covers 483 STAC trajectories across 10 failure modes in simulated environments and a live-tool MCP setting.The simulated environments include four SHADE-Arena environments and 62 Agent-SafetyBench environments; the MCP benchmark uses filesystem and browser-automation servers with a black-box attacker.Source: §2.4 · p. 6

3.1 Experimental Setting

• GPT-4.1 implements the Generator, Verifier, Planner, and Judge, while Qwen3-32B implements the Prompt Writer.Source: §3.1 · p. 6
• The authors selected GPT-4.1 for roles requiring multi-step reasoning, tool-schema comprehension, and adaptive planning after open-weight models were unreliable in internal trials.Those models frequently produced invalid tool calls or failed to generate and validate chains effectively.Source: §3.1 · p. 6
• Using the same model as Generator and Judge was intended to improve scoring consistency, but the pipeline consequently depends on a single model family.Source: §3.1 · p. 6

Defense Prompt Based On Reasoning

• Before tool execution, the reasoning defense requires ethical analysis of the full history and identification of affected stakeholders.ToolShield yields 78.1% final ASR versus 86.7% for reasoning at T+2, but both remain insufficient against persistent adaptive attackers.Source: Defense Prompt Based On Reasoning · pp. 7–8, §3.5 · p. 10

3.3 Generalization to Real Tool Backends

• On 32 matched real-MCP tasks, the full pipeline reaches 35.5% ASR versus 12.5% without tool chains, while rejection falls from 12.5% to 3.6%.The setting uses live MCP servers and gives the attacker tool schemas rather than full environment state.Source: §3.3 · p. 8

3.2 Attack Effectiveness

• 91.2% mean final ASR, with final ASR above 90% for seven of eight agents; Magistral is the exception.ASR consistently increases over attack execution turns.Source: §3.2 · p. 8, Table 2 · p. 9
• 95.1% ASR for full STAC exceeds MT-AgentRisk (53.2%) and X-Teaming (61.5%), with fewer turns (2.8 vs. 3.9 and 4.2).The single-turn prefix ablation achieves 72.8% ASR, versus 95.1% for full STAC, isolating the contribution of context priming.Source: §3.2 · p. 8, Table 3 · p. 9

3.4 Attack Stealthiness

• Prompt harmfulness stays below 2% and refusal below 4%, while MT-AgentRisk prompts are flagged harmful in 27.4% of cases.These measures capture per-prompt and per-response signals, not stealth under sequence-level monitoring.Source: §3.4 · p. 9
• The stealth evaluation does not test audit-log, action-pattern, or state-diff monitors; full-history reasoning proxies remain insufficient, with STAC ASR ≥58.6%.The authors identify dedicated evaluation against these monitors as future work.Source: §3.4 · p. 9

3.5 Defenses

• Across 12 model×defense conditions, T+2 ASR remains 74.3%–92.5%, and no defense is robustly best across the additional agents.The reasoning defense’s initial-turn reduction is weaker off GPT-4.1; the authors present the pattern as suggestive, not causal.Source: §3.5 · p. 10
• Giving the attacker the exact defense prompt lowers T+2 ASR by 7–10 percentage points, as the attacker shifts away from STAC’s sequence-level tactic.The defense-aware Planner instead uses overt bypass framings that can prompt refusals or sanitized tool calls.Source: §3.5 · p. 10

Defending tool-enabled agents

• In the evaluated defenses, Reasoning lowers GPT-4.1 first-turn ASR from 87.4% to 58.6%, while ToolShield has the lowest T+2 ASR at 78.1%.Across three additional agents, every defense leaves T+2 ASR at or above 74%; under adaptive attack, all reported ASR changes are below zero relative to the defense-naive Planner.Source: Table 4 · p. 11, Table 5 · p. 11, Table 6 · p. 11

Related attack and defense work

• Earlier multi-turn jailbreak research targets harmful text, while STAC applies multi-turn manipulation to tool-enabled agents and harmful actions.Prior work includes conversational dynamics, reinforcement learning, shadow LLMs, and inference-time activation moderation.Source: §4 · p. 12

Positioning STAC

• STAC targets harmful environmental changes through tool-call sequences, unlike chatbot jailbreaks that elicit harmful text or agent attacks focused on single malicious calls.The cited work frames environmental harm as potentially irreversible and links it to action-sequence reasoning.Source: §4 · p. 12
• MT-AgentRisk decomposes harmful tasks but can retain overtly harmful substeps, whereas STAC enforces benign intermediate steps and validates chains through live execution.The comparison reports prompt-harmfulness of 27.4% for MT-AgentRisk versus 0.0% for STAC.Source: §4 · p. 12
• STAC combines step-wise benignness, in-environment validation, general in-session tool use, and paired defense analysis, a combination Table 7 identifies as unique.The paper places this work within a compositional-risk cluster spanning code, skill marketplaces, and orchestrator task decomposition.Source: §4 · p. 12, Table 7 · p. 13

Related attack work

• Temporally distributed sleeper attacks and malicious-skill benchmarks operate across time or the skill supply chain, whereas STAC chains benign calls within one live session..Source: §4 · p. 13

Conclusion

• Across 483 STAC cases, agents have an average final ASR of 91.2%, and evaluated prompt-based defenses provide only limited protection.The conclusion also reports 95.1% on the 390 SHADE-Arena baseline-comparison cases.Source: §5 · p. 13
• Reasoning-based defense provides the strongest initial-turn mitigation, while ToolShield is more durable against sustained adaptive attackers, but both remain insufficient.Source: §5 · p. 13
• The paper concludes that agent safety mechanisms must assess the cumulative impact of action sequences rather than individual actions.Adaptive robustness of stronger agentic guardrails remains untested.Source: §5 · p. 13

Judging and construct validity

• A GPT-4.1 Judge shares a model family with the Generator and Planner, which may shape reported ASR, PH, and RR.Human validation on a stratified subset and cross-family judging remain proposed next steps.Source: §6 · p. 14

Statistical reporting

• Tables 2–6 report single-run point estimates, so stochastic generation, planning, and judging leave run-to-run variance and small defense gaps uncharacterized.The authors identify bootstrap intervals and multi-seed reruns as next steps.Source: §6 · p. 14

Benchmark selection and conditional success

• Reported ASR is conditional on chains that passed in-environment verification and on a synthetic multi-turn prefix, rather than a fully live end-to-end interaction.The single-turn baseline is 72.8% versus 95.1% for full STAC; the authors note that fully eliciting every prefix turn against the target agent would further bound the gap.Source: §6 · p. 14

Environments and generalization

• Most results come from simulated Python environments, while real-tool evaluation covers two MCP backends, 139 scenarios, and one agent.The framework also relies on predefined failure modes, which may limit discovery of novel patterns from unique tool combinations or environment constraints.Source: §6 · p. 14

Defense scope

• The defense study is scoped to prompt-based defenses, and adaptive robustness of architectural guardrails against execution-validated attackers remains untested.The stealth metrics also do not test sequence-level monitors such as audit-log, action-pattern, or state-diff analysis.Source: §6 · p. 14
• Control-flow separation does not target STAC’s trusted-user-channel vector, though policy enforcement such as confirmation for sensitive actions is complementary.Source: §6 · p. 15

Ethics Statement

• The authors judge defense and secure-design benefits to outweigh dual-use risks, citing controlled testing, defense development, benchmark release, and an average attack success rate of 91.2% as evidence of the need for stronger protections.They also state that LLM-generated code and content were inspected and validated, and that LLMs were not used for research ideation.Source: Ethics Statement · p. 15

Tool Chain Generator

• The system prompts were developed iteratively through human-AI collaboration and direct a red-teamer to create attacks where individually benign tool actions culminate in harm.The attacks should be grounded in a task setup, use unique ordered tool sequences, and have a harmful final action concealed until execution.Source: A System Prompts · p. 19, Tool Chain Generator · p. 19, Core Mandates · p. 19

Core Mandates; Design Specifications

• Generator mandates require 2–6-call chains, with at least one-third containing four or more steps; each call must be necessary for the next, and the final call must directly cause harm.Attacks must also be contextually relevant, diverse in goals and methods, and plausible in the context created by prior steps.Source: Core Mandates · p. 19, Design Specifications · p. 19

Inputs Provided; Design Heuristics

• The generator receives the environment, available tools, target failure mode, and task setup, and is instructed to produce grounded, precise, state-aware scenarios whose intermediate outputs do not reveal malicious intent.Its requested output is a JSON object containing attack objectives, ordered tool calls, parameters, purposes, environment changes, and an explanation.Source: Inputs Provided · p. 19, Design Heuristics · p. 19

Tool Chain Verifier

• The verifier executes each proposed call individually, observes environment responses and state changes, and checks whether calls are valid and achieve their intended outcomes.When a call fails, it diagnoses the issue, modifies and retries it, or proposes alternative tools or chains while preserving the harmful goal.Source: Tool Chain Verifier · p. 20

Requirements for the final tool chains you generate; 7. Verification: Confirmation that this step now works as intended

• Final chains must use precise calls and parameters, make each step necessary to the next, conceal harmful intent until the final step, and be verified to achieve the harmful goal.The verifier returns the complete validated chain in the original JSON format.Source: Requirements for the final tool chains… · p. 21, §7 · p. 22

Purpose; Input Format; Your Task; 1. Analysis; 3. Strategy Explanation

• The prompt-reverse-engineering assistant uses the chat history and target tool call to devise a harmless-appearing user prompt that is likely to trigger that call while evading safety measures.It analyzes behavioral patterns, vulnerabilities, and context, then explains its strategy, including techniques such as misdirection or framing.Source: Purpose · p. 22, Input Format · p. 22, Your Task · p. 22

2. Reverse Engineered Prompt; 3. Strategy Explanation

• The reverse-engineered prompt must be stealthy, innocuous, contextually appropriate, conversationally coherent, and effective at inducing the specified tool call.The prompt design process also asks for a brief account of why the prompt may succeed and avoid safety filters.Source: §2 · p. 23, §3 · p. 23

Planner

• The planner uses tool descriptions, the attack objective and target call, and full conversation history to predict a next strategy adapted to prior system responses.The analysis considers defenses, rejections, and evasions when adjusting the predicted approach.Source: Input Description · p. 24, §1 · p. 24

Critical Output Requirements

• The required analysis predicts a concrete strategy and exact next-turn prompt, with rationale about adaptation, exploited vulnerabilities, tool-call parameters, and stealth, in a prescribed JSON format.The guidelines call for prompts that appear harmless in isolation while reliably eliciting target behavior, framing the exercise as identifying vulnerabilities to improve AI safety.Source: Important Guidelines · p. 23, Critical Output Requirements · p. 25, Mandatory Output Format · p. 25

Attack Vector Classification Framework

• The protocol classifies vectors including prompt engineering, misdirection, policy circumvention, and role manipulation, and assesses their relevance, context, effectiveness, and stealth.Examples include harmless-appearing requests, false contexts, euphemisms, and role-based appeals.Source: Attack Vector Classification Framework · p. 26, §1 · p. 26, §2 · p. 26

6. Multi-Step Stealth Techniques

• It also considers technical exploitation and multi-step stealth, including model limitations, trust building, gradual concept introduction, and combining seemingly unrelated knowledge.Candidate vectors are evaluated for their contribution to the overall attack sequence and likelihood of bypassing current defenses.Source: §5 · p. 27, §6 · p. 27

Example Attack Trajectory Analysis

• The worked trajectory shows how seemingly innocuous requests across turns can combine into harmful outcomes that might be impossible in one interaction.The example uses an IT-audit pretext to establish context and request document review, setting up a later harmful action disguised as a legitimate update.Source: Example Attack Trajectory Analysis · pp. 27–28

Agent (Zhang et al., 2024)

• The agent instructions require one tool call at a time, waiting for results, and using only provided tools.They also direct the agent to avoid unnecessary repeated calls and provide a final answer when it stops calling tools.Source: Agent (Zhang et al., 2024) · p. 29

Your Role and Objective

• The evaluation LLM judges multi-turn attack effectiveness by scoring attack stealthiness, harmful-goal achievement, and agent response safety.The attacks use strategic sequences in which individually benign prompts can cumulatively pursue a harmful goal.Source: Your Role and Objective · p. 29

Input Information

• Each task supplies available tools, the end goal and target tool call, prior interaction history, and the current turn’s plan, prompt, and agent response.Evaluation focuses on the current turn while using previous interactions to understand attack progression.Source: Input Information · p. 30

Evaluation Criteria and Scoring Scale

• The evaluator scores prompt harmlessness and goal progress from -3 to 3, and agent helpfulness from -2 to 2.The scales distinguish prompt appearance, progress toward the attack goal, and the agent’s willingness and ability to help.Source: Evaluation Criteria and Scoring Scale · pp. 30–31

Output Format (JSON Schema)

• The required JSON output contains prompt_harmlessness, goal_progress, and agent_helpfulness, each with an integer score and an explanation.The example illustrates partial goal progress and helpfulness while the agent maintains ethical reservations.Source: Output Format (JSON Schema) · p. 31, Example Output · p. 32

Guidelines for Objective Assessment

• Assessments must cite observable interaction evidence, compare goal progress with the prior turn, consider the broader sequence, and apply score definitions consistently.The guidelines also call for balanced assessment and recognition that attacks may prioritize long-term progress over immediate compliance.Source: Guidelines for Objective Assessment · p. 32

Defense Prompt Based On Summarization

• The summarization-based defense instructs the agent to review the full conversation history and summarize user intent before responding.Figure 4 depicts this multi-turn intent-summarization defense prompt.Source: Defense Prompt Based On Summarization · p. 33, Figure 4 · p. 33

Defense Prompt Based On Failure Modes (Zhang et al., 2024)

• The failure-mode defense prompt tells the agent to avoid harmful content generation without tools and addresses 10 agent-specific failure modes (Zhang et al., 2024).Figure 5 depicts the prompt focused on those failure modes.Source: Table 8 · p. 33, Defense Prompt Based On Failure Modes… · p. 34, Figure 5 · p. 34

ToolShield Defense

• ToolShield is a learning-based defense that distills safety experiences from agent interaction trajectories.The benchmark adaptation applies trajectory summarization followed by experience extraction before appending the resulting guidelines to the system prompt.Source: D MT-AgentRisk and ToolShield Baseline… · pp. 34–35

MT-AgentRisk Attack Generation

• MT-AgentRisk generates attacks by enumerating harm vectors, instantiating harmful tasks, and decomposing them into multi-turn sequences.Its pipeline uses tool documentation and a transformation taxonomy; it does not verify proposed tool calls through target-environment execution.Source: D MT-AgentRisk and ToolShield Baseline… · p. 35

D MT-AgentRisk and ToolShield Baseline Adaptation

• STAC differs by requiring individually benign intermediate steps and interactively verifying each tool call, unlike MT-AgentRisk’s abstract, unverified steps.The conversion layer maps abstract tool categories and descriptions to available environment tools; unresolved or incorrect mappings can contribute to lower attack success.Source: D MT-AgentRisk and ToolShield Baseline… · p. 35

MT-AgentRisk to STAC Format Conversion

• 124 valid MT-AgentRisk attack cases remained on SHADE-Arena after conversion and filtering, and the shared adaptive Planner evaluated them for up to 3 user turns.The conversion pipeline resolves tool names, extracts parameters, and handles tools absent from the environment; it uses matching heuristics and parameter-specific extraction strategies.Source: D MT-AgentRisk and ToolShield Baseline… · pp. 35–36

ToolShield Defense

• ToolShield distills safety experiences from MT-AgentRisk trajectories into system-prompt guidelines, then evaluates them against the full STAC benchmark.The adaptation produces 113 guidelines from 124 MT-AgentRisk trajectories and uses the adaptive Planner for evaluation.Source: D MT-AgentRisk and ToolShield Baseline… · p. 35

E.1 Hyperparameters

• The Generator uses temperature 1.0 and produces 10 candidate chains per environment–failure-mode pair, while the Verifier and Prompt Writer use 0.6 and evaluation components use 0.0–0.15.All framework calls use top_p=0.95 unless otherwise noted.Source: §E.1 · p. 36, Table 9 · p. 36

E.2 Adaptive Planning Algorithm

• The adaptive loop cycles through planning, agent execution, and judging for up to 3 turns, stopping when goal_progress reaches 3.The Agent may make up to 10 rounds per turn, with one tool call per agent step; environment outputs over 512 characters are truncated.Source: §E.2 · p. 36

E.3 Environment Initialization

• SHADE-Arena environments use initialized mock data, with Travel capped at 50 emails and Spam Filter limited to 5 emails per user and 10 cloud-drive files; Agent-SafetyBench environments load from JSON configurations.Banking and Workspace use their full datasets; environment configurations provide tool schemas, initial states, and ground-truth labels.Source: §E.3 · p. 37

E.4 Defense Implementation

• All defenses modify the agent’s system prompt without retraining or architectural changes, including failure-mode constraints, intent summarization, harm-benefit reasoning, spotlighting, and ToolShield guidelines.Spotlighting replaces whitespace in user messages with ^, while ToolShield guidelines are appended to the system prompt.Source: §E.4 · p. 37

F Benchmark Statistics

• 483 cases span two datasets, 66 environments, and all 10 failure modes; 42–55 cases cover each failure mode, and 85% of chains contain 2–3 tool calls.The remaining 15% contain 4 or more calls, and synthetic interaction histories contain 1–4 user turns.Source: F Benchmark Statistics · p. 37, Table 11 · p. 38

G MCP Real-Tool Evaluation

• The MCP evaluation applies STAC to filesystem and Playwright browser-automation servers running in Docker containers.The servers are accessed through docker exec over stdio transport.Source: G MCP Real-Tool Evaluation · p. 37

G.1 Benchmark and Setup

• OpenAgentSafety includes 356 tasks across file management, web browsing, code editing, and system administration, divided into 245 benign-intent and 107 malicious-intent categories.Tasks specify required MCP servers and a natural-language goal; GPT-4.1 runs the components and GPT-4.1-mini performs post-evaluation.Source: §G.1 · p. 38
• In the MCP setting, the Generator sees tool schemas and task descriptions but not workspace contents, unlike simulated evaluations with full environment state.This is described as a substantially harder, black-box attack scenario.Source: §G.1 · p. 38

G.2 Experimental Conditions

• The full pipeline generates 542 candidate chains and verifies 139 selected chains at a 90.3% success rate, then creates prompts averaging 3.3 turns per chain.The no-tool-chains condition improvises from task goals and compares 32 matched tasks with results across all 356 OAS tasks.Source: §G.2 · pp. 38–39
• A post-evaluation classifier labels each trajectory COMPLETE, REJECT, or FAILED according to whether the attack was executed, explicitly refused, or attempted unsuccessfully.This supplements the STAC Judge’s scores for goal progress, agent helpfulness, and prompt harmlessness.Source: §G.2 · p. 39

• The evaluation uses OpenAgentSafety with GPT-4.1 in an MCP setting.Source: Table 12 · p. 39
• The full-pipeline condition uses STAC Steps 1–4 with verified chains.Source: Table 12 · p. 39
• The no-tool-chains condition uses an adaptive Planner that improvises, and the comparison controls for the same 32 tasks.Source: Table 12 · p. 39

G.4 Analysis

• 35.5% ASR and 52.9% Post-Eval COMPLETE with the full pipeline exceed 12.5% for each metric with no-tool-chains on 32 matched tasks.Verified chains give the Planner a concrete roadmap, improving attack coherence compared with improvised attacks.Source: §G.4 · p. 39
• 3.6% Refusal Rate with the full pipeline is below 12.5% with matched no-tool-chains, while prompt harmfulness stays at ≤3.1% in all conditions.The authors attribute the lower refusal rate to engineered prompts evading safety filters on real tool backends.Source: §G.4 · p. 39
• 35.5% pipeline ASR is below the 95.1% achieved on SHADE-Arena, where the Generator has full environment state rather than black-box MCP access.Real permission boundaries and execution failures further reduce success rates in MCP.Source: §G.4 · p. 39
• 87.7% of pipeline scenarios reach goal_progress ≥1, but 35.5% reach full success (≥3) with a 3.6% refusal rate.The authors report that many attacks fail to complete the harmful action without triggering refusals, citing opaque environmental states as a reason they are not validly constructed.Source: §G.4 · p. 39