Source-linked AI summary
ChainFuzzer: Greybox Fuzzing for Workflow-Level Multi-Tool Vulnerabilities in LLM Agents
Jiangrong Wu, Zitong Yao, Yuhong Nan, Zibin Zheng
TL;DR
Single-tool testing misses LLM-agent vulnerabilities that emerge when tools pass attacker-influenced data across multi-step workflows. ChainFuzzer extracts risky chains, drives their execution, and validates sink effects under guardrails; it confirms reproducible vulnerabilities across 19/20 evaluated apps, most of which require multi-tool execution.
Problem
Single-tool or single-hop testing misses vulnerabilities that emerge from cross-tool dataflows and provides limited debugging value for tracing attacker-influenced content.
Source: §1 · p. 1Method
ChainFuzzer extracts chains using source-to-sink evidence and tool dependencies, repairs prompts using execution traces, and fuzzes payloads with sink-specific validation.
Source: §3.2 · p. 4Results
365 unique, reproducible vulnerabilities were confirmed across 19/20 apps, including 302 that required multi-tool execution.
Source: §1 · p. 2, §5.1 · p. 7Takeaways & Limitations
The findings provide developers with actionable evidence to diagnose and mitigate compositional vulnerabilities in tool-augmented agent systems.
Source: §8 · p. 10Whether a vulnerability reproduces depends on runtime conditions such as network egress, sandbox permissions, and credential availability.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Tool-augmented LLM agents increasingly rely on multi-step, multi-tool workflows to complete real tasks. This design expands the attack surface, because data produced by one tool can be persisted and later reused as input to another tool, enabling exploitable source-to-sink dataflows that only emerge through tool composition. We study this risk as multi-tool vulnerabilities in LLM agents, and show that existing discovery efforts focused on single-tool or single-hop testing miss these long-horizon behaviors and provide limited debugging value. We present ChainFuzzer, a greybox framework for discovering and reproducing multi-tool vulnerabilities with auditable evidence. ChainFuzzer (i) identifies high-impact operations with strict source-to-sink dataflow evidence and extracts plausible upstream candidate tool chains based on cross-tool dependencies, (ii) uses Trace-guided Prompt Solving (TPS) to synthesize stable prompts that reliably drive the agent to execute target chains, and (iii) performs guardrail-aware fuzzing to reproduce vulnerabilities under LLM guardrails via payload mutation and sink-specific oracles. We evaluate ChainFuzzer on 20 popular open-source LLM agent apps (998 tools). ChainFuzzer extracts 2,388 candidate tool chains and synthesizes 2,213 stable prompts, confirming 365 unique, reproducible vulnerabilities across 19/20 apps (302 require multi-tool execution). Component evaluation shows tool-chain extraction achieves 96.49% edge precision and 91.50% strict chain precision; TPS increases chain reachability from 27.05% to 95.45%; guardrail-aware fuzzing boosts payload-level trigger rate from 18.20% to 88.60%. Overall, ChainFuzzer achieves 3.02 vulnerabilities per 1M tokens, providing a practical foundation for testing and hardening real-world multi-tool agent systems.
1 Introduction
Multi-tool workflows let attacker-influenced content move across tools into high-impact operations, but single-tool testing misses risks that emerge only through composition. ChainFuzzer recovers risky chains, drives their execution, and validates sink effects; evaluation finds reproducible vulnerabilities across nearly all tested apps.
1 Introduction
- Multi-tool vulnerabilities arise when tool composition propagates attacker-influenced content across steps into high-impact operations.Intermediate tool outputs can become later inputs, expanding the attack surface of multi-stage workflows.Source: §1 · p. 1
- Existing discovery efforts focus on single-tool or single-hop testing, which misses compositional risks and offers limited debugging value for tracing data across tools.The authors identify three practical challenges: finding risky chains, reliably driving their execution, and validating sink effects under guardrails.Source: §1 · pp. 1–2
- ChainFuzzer combines strict source-to-sink evidence and cross-tool dependencies to extract chains, Trace-guided Prompt Solving to drive them, and guardrail-aware fuzzing to validate sink effects.Its goal is to produce auditable, reproducible vulnerabilities for developers to verify.Source: §1 · p. 2
- 365 unique, reproducible vulnerabilities were confirmed across 19/20 apps, including 302 that required multi-tool execution.The evaluation also reports 3.02 vulnerabilities per 1M tokens.Source: §1 · p. 2
2 Background and Problem Statement
LLM agents coordinate tools at runtime, passing intermediate outputs directly or through persisted artifacts; these dependencies support complex tasks but can also carry untrusted content into unsafe actions. The paper frames the resulting workflow-level risks as a gap in single-tool testing and scopes its framework to deployed tool compositions.
2.1 LLM Agent Background
- An agent’s LLM chooses tools and constructs arguments at runtime, so the sequence of tool calls can vary across similar requests.Tool interfaces include input schemas, structured outputs, and possible environmental side effects.Source: §2.1 · p. 2
- Multi-tool dependencies pass outputs directly to later tool inputs or persist artifacts in shared carriers for later retrieval or use.These dependencies enable workflows such as downloading content, saving it, and later executing the file, while creating paths for untrusted artifacts to reach downstream actions.Source: §2.1 · p. 3
2.2 Problem Statement
- A web-search-to-download-to-file-write-to-execution workflow can turn untrusted downloaded content into unintended command or code execution.Each step may appear reasonable for the user’s task; the risk emerges from their composition.Source: §2.2 · p. 3, Figure 1 · p. 3
- Workflow-level vulnerabilities arise from multi-step chains that reuse intermediate artifacts, including cases where a malicious user steers a remote agent through individually benign steps.Such compositions expand the attack surface through longer dependency chains and artifact reuse.Source: §2.2 · p. 3
- Single-tool or single-hop discovery underexplores cross-tool dependencies and state changes, while the stated scope excludes attackers who modify agent code or tools or require model weights, developer secrets, or extra system privileges.The framework models untrusted external or shared-carrier inputs and malicious natural-language steering in deployed settings.Source: §2.2 · p. 3
3 System Overview
Workflow-level vulnerability discovery must find realistic risky chains, reliably drive a nondeterministic agent through them, and validate unsafe effects despite guardrails. ChainFuzzer extracts candidate chains, repairs prompts using execution traces, then fuzzes payloads and checks sink effects.
3.1 Challenges and Key Ideas
- Finding risky chains is difficult because heterogeneous tool sets create many combinations, and naive exploration can spend effort on irrelevant or semantically invalid sequences.The challenge is to capture accurate dependencies and identify plausible paths to high-impact actions.Source: §3.1 · p. 3
- LLM agents may diverge from a target chain because prompts change their plans, tool choices, argument bindings, or runtime success; guardrails may also refuse or sanitize malicious payloads.Missing preconditions and artifact or parameter mismatches are among the execution problems described.Source: §3.1 · p. 4
3.2 ChainFuzzer Workflow
- ChainFuzzer identifies sink tools and plausible upstream chains, then uses Trace-guided Prompt Solving to revise prompts until the intended chain is reachable and stable.Trace analysis guides repairs for divergence such as missing preconditions, incorrect bindings, or runtime failures.Source: §3.2 · p. 4
- After a chain is executable, ChainFuzzer injects sink-specific payloads through user inputs or untrusted content, mutates payloads when guardrails block them, and validates effects with sink-specific oracles.The resulting evidence includes a minimal proof-of-concept trace.Source: §3.2 · p. 4
4.1 Sink-Related Tool-Chain Extraction
ChainFuzzer works backward from high-impact sink tools to extract plausible upstream chains, using strict source-to-sink reachability and cross-tool dependency evidence. It combines direct and persisted-artifact dependencies, then filters candidate chains for plausible workflows.
4.1 Sink-Related Tool-Chain Extraction
- The extraction module outputs sink tools and candidate chains ending at each sink, annotated with dependency type and an optional key artifact or field.The chains are recovered by working backward from high-impact tool capabilities.Source: §4.1 · p. 4
4.1.1 Sink tool identification.
- A tool qualifies as a sink tool only when a sink API argument can be influenced by the tool’s entry inputs; tools with fixed-constant sink calls are excluded.Table 1 catalogs candidate sink APIs by sink type, and backward dataflow traces sink arguments through tool code.Source: Table 1 · p. 4, §4.1.1 · p. 5
4.1.2 Backward chain extraction.
- Direct dependencies connect compatible upstream return fields to downstream parameters, while indirect dependencies connect tools through persisted content that later tools read or execute.The dependency rules intentionally over-approximate the possible chain space.Source: §4.1.2 · p. 5
- An LLM filters interface-compatible candidate chains by whether their steps form a coherent workflow for a realistic task.This removes chains with no logical linkage between steps before they proceed to later modules.Source: §4.1.2 · p. 5
4.2 Tool-Chain Prompt Generation
ChainFuzzer generates prompts to drive an agent through a target tool chain, then uses execution traces to identify and repair prompt-level mismatches. It preserves successful prompt prefixes through local edits and accepts a prompt after five consecutive successful runs.
4.2 Tool-Chain Prompt Generation
- Trace-guided prompt solving takes a candidate chain and tool metadata as input and produces an executable prompt with its corresponding trace.Agents may choose different tools, arguments, and outcomes across runs, so prompt revisions target semantic mismatches.Source: §4.2 · p. 5
- The seed prompt is a step-by-step user request that directs the agent to invoke the chain’s tools in order for a plausible task.Source: §4.2 · p. 6
4.2.1 Constraint generation from traces.
- Trace records capture the agent’s thinking, invoked tool, arguments, return value, and execution status for comparison with the target chain.A constraint generator identifies actionable causes of divergence, including missing preconditions, incorrect bindings, format mismatches, permission gates, and planner detours.Source: §4.2.1 · p. 6
4.2.2 Prompt solver.
- The solver revises prompts within a constrained local-edit space that satisfies trace-derived constraints while preserving as much of the already-correct prefix as possible.The formal condition requires the revised prompt to encode constraints such as required steps, bindings, and preconditions.Source: §4.2.2 · p. 6
- Local repairs can add a missing step, strengthen an instruction, clarify a binding, or explicitly create a required resource before a later tool call.Source: §4.2.2 · p. 6
- The repair loop stops when the full chain executes consistently or a fixed budget is exhausted; five consecutive successful runs qualify a prompt as stable.Source: §4.2.2 · p. 6
4.3 Fuzzing-Driven Vulnerability Validation
ChainFuzzer tests whether a reachable tool chain can produce a sink-level unsafe effect and records evidence as an auditable proof of concept. It selects plausible injection points, uses sink-specific payloads and oracles, and mutates blocked payloads under a fixed fuzzing budget.
4.3 Fuzzing-Driven Vulnerability Validation
- Vulnerability validation tests whether a reachable chain can be affected and outputs an auditable proof of concept with a triggering prompt, tool-call trace, and sink-effect evidence.Source: §4.3 · p. 6
4.3.1 Injection point selection.
- Injection-point selection maps untrusted user dialogue or external content to the earliest chain boundary where it can enter and later reach the sink.Source: §4.3.1 · p. 6
- When user content enters through the first tool, ChainFuzzer tests user-source injection; for chains ingesting external content, it tests environment-source injection, and tests both when plausible.Source: §4.3.1 · p. 6
4.3.2 Guardrail-aware fuzzing.
- Payloads are parameterized for the ending sink type, including command strings, code snippets, URLs, query fragments, and template expressions.Source: §4.3.2 · p. 6
- Sink-specific oracles require payload-influenced input to reach the relevant execution API and produce the specified probe effect or target response.The oracles cover command and code execution, SSRF, SQLi, and SSTI.Source: §4.3.2 · pp. 6–7
- If guardrails block a payload, ChainFuzzer mutates it while keeping the prompt unchanged, using fragmentation, encoding, or syntax-preserving format changes until success or budget exhaustion.Source: §4.3.2 · p. 7
5 Evaluation
ChainFuzzer is evaluated on popular multi-tool agent apps, with results spanning vulnerability discovery, component effectiveness, and ablations. It finds that most confirmed vulnerabilities require multi-tool execution, while each component contributes to discovery or reproduction.
5 Evaluation
- The dataset comprises 20 open-source agents selected for at least 2,000 GitHub stars and multi-tool support; GPT-5.1 runs in the agent runtime.The LLM-related part of ChainFuzzer is driven by GPT-4o.Source: §5 · p. 7, Table 2 · p. 7
5.1 RQ1: Overall Result and Overhead
- 302/365 (82.74%) confirmed vulnerabilities require multi-tool execution, across 19/20 apps; ChainFuzzer confirms 365 unique vulnerabilities.The evaluation covers 20 agents and 998 tools, with 2,388 candidate chains and 2,213 prompts that execute their target chains successfully in 5/5 runs.Source: §5.1 · p. 7, Table 3 · p. 8
5.2 RQ2: Individual Module Effectiveness
- Edge precision is 96.49% (714/740 edges), and strict chain precision—requiring every edge in a chain to be correct—is 91.50% (183/200 chains), indicating extracted chains are largely valid multi-tool workflows.Source: §5.2 · p. 8
- 27.05% to 95.45%: TPS improves chain reachability; stable prompt rate rises from 9.50% to 92.50%.Reachability is measured over 10 runs; stability requires success in 5/5 consecutive runs.Source: §5.2 · p. 8, Table 7 · p. 8
- Payload mutation raised overall trigger rate from 18.20% (91/500) with initial payload injection to 88.60% (443/500), with especially large gains for CMDi (10.00%→86.00%, +76%) and CODEi (7.00%→82.00%, +75%).Source: §5.2 · p. 9
5.3 RQ3: Ablation Study
- Removing ChainExtraction drops findings from 365 to 63 (−302) and sink-type coverage from 5 to 3; removing TPS yields 96 (−269) and 4 types; removing Mutation yields 132 (−233) while retaining all 5 types.The ablation interpretation assigns ChainExtraction to sink reachability, TPS to executable and reproducible chains, and Mutation to confirming more vulnerabilities after reachability and validity are established.Source: §5.3 · p. 9, Table 9 · p. 9
6 Discussion
The discussion identifies runtime conditions and LLM capability as limits on vulnerability reproduction and module accuracy. It recommends defenses at tool boundaries and sink invocations, not prompt wording alone.
Limitations
- Runtime conditions—including egress policy, sandbox permissions, and credentials—can determine whether chains execute and sink effects are observable.For example, blocked outbound requests or missing endpoint credentials can prevent an SSRF chain from triggering.Source: §6 · p. 9
- LLM hallucination or limited capability can keep ChainFuzzer modules below 100% accuracy; experiments use GPT-4o as a conservative lower bound.The authors expect stronger models to improve planning stability and reduce hallucinations.Source: §6 · p. 9
Mitigation strategy
- Mitigations include sink-side input constraints, mediated typed artifacts with provenance, and dataflow-aware checks that block attacker-controlled sink inputs.Examples include fixed command templates, allowlisted SSRF destinations, and rejecting tainted SQL or template inputs unless parameterized or escaped.Source: §6 · p. 9
7 Related Work
Prior LLM-agent security work covers model-centric and execution-time threats, while most studies evaluate isolated scenarios rather than systematic multi-tool vulnerability discovery. Traditional software security offers analysis, fuzzing, oracle, and taint-tracking techniques that this work adapts to composed agent workflows.
Vulnerabilities in LLM-based Systems
- LLM-agent security work spans model-centric attacks, prompt injection, malicious tools, downstream effects, and privacy leakage, but most studies test isolated scenarios.The paper identifies the lack of a systematic workflow for discovering and confirming vulnerabilities induced by multi-tool composition at scale.Source: §7 · pp. 9–10
- Recent work [59] studies privacy disclosure through external MCP tool composition; this paper instead targets vulnerabilities in agents’ own task-execution workflows.Source: §7 · p. 10
Vulnerability Discovery in Traditional Software
- Traditional software security combines static and dynamic analysis, fuzzing, oracles, and taint tracking; ChainFuzzer adapts these ideas to composed tool interactions.Source: §7 · p. 10
8 Conclusion
ChainFuzzer finds workflow-level vulnerabilities in multi-tool LLM agents and provides evidence for diagnosis and mitigation. Its evaluation reports findings across agent apps, with most requiring multi-tool execution.
8 Conclusion
- 365 unique vulnerabilities across 19 apps; 82.74% require multi-tool execution, highlighting risks missed by single-tool testing.Source: §8 · p. 10
- ChainFuzzer achieves 3.02 vulnerabilities per 1M tokens under practical end-to-end overhead, providing developers actionable evidence to diagnose and mitigate compositional vulnerabilities.Source: §8 · p. 10
- The paper focuses on systemic workflow-level vulnerabilities that emerge when multiple tools interact through intermediate artifacts such as URLs, files, and database records.Source: §8 · p. 10