ToFu: Exploiting and Fuzzing the LLM-Orchestrator-Tool Trust Boundary

1 Introduction

• A Repeat-After-Me attack can make an orchestrator execute attacker-chosen tool calls and arguments reproduced verbatim by an LLM.The LLM need only repeat supplied text; the orchestration layer interprets the structured output as a tool call.Source: §1 · pp. 1–2
• Tool metadata can mismatch implementations, while existing analyses often target predefined weakness classes, taint flows, or particular attacks.These description-code inconsistencies complicate establishing whether implementations honor the semantics their interfaces advertise.Source: §1 · p. 2
• Crash-based oracles miss violations such as accessing files outside an intended workspace when a tool otherwise completes successfully.Detecting such cases requires understanding what an argument represents and the security constraints governing its use.Source: §1 · p. 2
• ToFu maps tool metadata to CWE-guided, tool-specific checks, then generates schema-valid fuzzing harnesses and semantic oracles.It uses runtime introspection when implementation visibility is available and input sanitizers with outcome-oracle checks for opaque tools.Source: §1 · p. 2
• 253 verified bugs across 343 tools in 118 repositories were identified by ToFu, including path traversal, log injection, exfiltration, and command bypass.The findings include failures that a crash-based fuzzing analyzer cannot locate.Source: §1 · p. 3

2.1 Tool-Enabled LLM Agents and Metadata

• An agent’s orchestrator supplies tool descriptions and argument schemas to the LLM, then dispatches its structured tool calls to implementations.Tool outputs may feed back into the same LLM or another LLM in a chain or multi-agent workflow.Source: §2.1 · p. 3

2.2 Fuzzing Semantic Security Bugs

• Semantic security bugs are syntactically valid executions that violate a security property implied by a tool’s documented semantics.A workspace-local file tool can accept a traversal path, access an outside file, and return normally.Source: §2.2 · p. 3
• Argument meaning determines the needed check: paths need containment checks, URLs may need scheme or destination restrictions, and commands need shell-interpretation checks.Primitive types such as strings do not establish these semantics, so there is no universal sanitizer for every string input.Source: §2.2 · p. 3

3 Related Work

• Prompt injection and tool-selection attacks steer what an agent sends or which tool it chooses, while ToFu begins once an adversarial value reaches the tool.Related work also studies contamination of one tool’s output into another tool’s arguments.Source: §3 · p. 4
• Existing defenses filter injected content, train instruction hierarchy, restrict permissions, or use capability leases and runtime guards.Other work maps MCP systems, studies malicious servers, or rewrites tool descriptions to steer agents away from unsafe calls.Source: §3 · p. 4
• MCP scanners commonly use fixed weakness lists, source patterns, threat catalogs, or runtime signals rather than deriving checks from each argument’s documented semantics.Prior measurements report wide disagreement among scanners.Source: §3 · p. 4
• ToFu derives checks from tool documentation, covering properties such as workspace confinement or destination allow-lists that sink-focused approaches may omit.ToolFuzz targets correctness, while crash-oriented fuzzers can miss tools that return normally despite violating a documented property.Source: §3 · p. 4

4 Threat Model

• ToFu assumes the orchestration layer faithfully exposes the declared interface and dispatches schema-valid calls to the corresponding implementation.The analysis infrastructure is trusted, while the LLM and tool implementation are not.Source: §4 · pp. 4–5
• An attacker may influence prompts, retrieved content, or other tool outputs that propagate into structured arguments; a vulnerability occurs when processing violates a property implied by the interface.Source: §4 · p. 5

5 Motivation and Challenges

• Schema validity does not establish semantic safety; tool implementations should be tested against their interface’s security contract, with exploit delivery established separately.The proposed threat model treats every schema-valid argument as potentially adversarial.Source: §5 · p. 5
• Existing approaches largely encode known weaknesses or attacks, bounding coverage when a security-critical value matches no predefined pattern or sink.Table 1 compares approaches for automatically identifying security weaknesses in LLM tools and MCP servers.Source: §5 · p. 5, Table 1 · p. 5
• Tool metadata conveys argument meaning beyond primitive types, enabling checks to start from each argument’s semantics and challenge its implied security properties.For example, a documented workspace path implies containment, while a remote URL implies scheme and destination constraints.Source: §5 · p. 5
• The resulting challenges are to derive reusable semantic checks, specialize them per tool, build oracles under partial visibility, and keep the sanitizer corpus extensible.The checks must translate documented argument meanings into adversarial inputs and observable violations without requiring manual analysis for every implementation.Source: §5 · pp. 5–6

6 Design

• ToFu derives security properties from tool metadata and uses them to generate seeds and fuzz the tool through its original execution path.The design combines metadata parsing, sanitizer mapping, harness generation, and a fuzz loop.Source: §6 · pp. 6–7

6.1 Metadata Extraction

• The Metadata Parser normalizes tool descriptions and schemas, retaining argument names, types, nested fields, and enum values for subsequent processing.Natural-language descriptions also inform sanitizer selection and seed generation.Source: §6.1 · p. 7

6.2 Semantics-Aware Sanitizers

• Semantics-aware sanitizers pair input applicability and adversarial classes with an expected security property and required observations, organized around reusable CWE weakness classes.CWE supplies reusable security knowledge; each sanitizer must translate a weakness into concrete conditions and observable evidence.Source: §6.2 · p. 7

6.2.1 Runtime Introspection.

• Runtime introspection collects sanitizer-required events, such as accessed paths or contacted destinations, so the oracle can detect violations even when execution returns normally.For containment, the oracle can canonicalize observed paths and test whether they escape the authorized root.Source: §6.2.1 · pp. 7–8

6.2.2 Outcome Sanitization.

• Outcome sanitizers flag inputs that violate a security policy and report a violation when the target fails to reject them or provide a safe fallback.The checks can also use runtime side effects; semantic invariants and output validity predicates help handle nondeterministic or side-effect-free tools.Source: §6.2.2 · p. 8

6.3 Mapping Sanitizers

• The mapper selects sanitizers from argument semantics and specializes outcome rules per tool; the corpus is extensible, but missed selections can cost bugs.A successful return violates a raise_exception rule; fallback rules permit only listed values. ToFu selects from registered sanitizers rather than synthesizing new ones at runtime.Source: §6.3 · pp. 8–9

6.4 Schema-Aware Harness Generation

• The harness mutates selected arguments with sanitizer-specific adversarial values while keeping the invocation schema-valid and populating remaining arguments with schema-conforming values.Seeds can adapt curated payloads and generated documents to the schema; the design assumes the target is runnable in the current environment.Source: §6.4 · pp. 9–10

6.5 Fuzzing Loop and Feedback

• The fuzz loop runs tools sequentially, uses coverage to guide mutations, and evaluates each execution for crashes or sanitizer-reported violations.ToFu records the triggering invocation, expected property, sanitizer, and evidence for detected violations.Source: §6.5 · p. 10

7.1 Dataset

• The runtime corpus contains 273 cleanly executable servers and 3,405 extracted tools; credentials, backends, or startup failures restrict runtime reach.The corpus is drawn from MCPCorpus and restricted to Python and JavaScript repositories.Source: §7.1 · pp. 10–11, Table 2 · p. 11

7.2 Experimental setup

• The main comparison runs each reachable tool for 60 seconds, with 34 servers also receiving 24-hour campaigns; ToFu uses fixed-seed LLM steps.The evaluation compares ToFu with five baselines spanning agent testing, taint analysis, metadata scanning, SAST, and LLM code audit.Source: §7 · p. 10, §7.2 · p. 11, §7.3 · p. 12

7.3 Baselines

• ToFu flags 118 of 273 servers with 253 bugs, while baselines measure different properties and do not support a single leaderboard.ToFu uses the least LLM traffic among the analyzers because fuzzing itself runs without the model.Source: §7.3 · p. 12, Table 4 · p. 12

7.4 RQ1: Sanitizer applicability

• The LLM selects an average of three sanitizers per tool, while 193 executed tools receive none beyond always-on and evidence-triggered checks.Thirty-five of 74 sanitizers and 11 of 21 tracers fired in the dataset.Source: §7.4 · p. 14, Table 7 · p. 14

7.5 RQ2: Semantic bug detection

• 253 verified bugs affected 343 tools across 118 servers, and none required a crash; many violated documented semantics despite successful execution.ToFu initially flagged 508 candidates; manual verification reproduced the reported bugs.Source: Table 3 · p. 11, §7.5 · pp. 14–15

7.6 RQ3: Fuzzing effectiveness

• At a 60-second corpus-wide budget, ToFu found 231 of 253 hand-verified bugs; the remaining 22 appeared only under deep fuzzing, between 111 seconds and 70 minutes.Source: §7.6 · p. 15

7.7 RQ4: Component ablation

• Across three full runs, totals stayed within about 2%, but only 342 bugs appeared in all three, so ablations are best compared by totals and bug classes.Every ablated arm fell outside the full-run range; even the smallest effect was about eight standard deviations below the full mean.Source: §7.7 · pp. 16–17, Table 8 · p. 17
• Removing both LLM stages preserves about two thirds of ToFu’s bugs, while seeds, sanitizer selection, and runtime tracing each contribute distinct benefits.Empty seeds lose more bugs and execute fewer tools; removing selection nearly eliminates path traversal, SSRF, and command injection.Source: §7.7 · p. 17

Multi-Agent Systems

• Supporting multi-agent systems requires fuzzing agent messages and reachable tool calls while preserving state across multiple invocations.ToFu leaves this support to future work.Source: §8 · p. 18

Coverage of Third-Party Components

• Coverage limited to tool code may miss behavior executed inside dependencies such as Chromium or native libraries.Component-specific instrumentation interfaces are needed for arbitrary third-party components, which the authors leave for future work.Source: §8 · p. 18

Stateful Tool Interactions

• Bugs dependent on prior calls may require multi-call fuzzing that links outputs, such as resource identifiers, to later arguments.Determining useful invocation sequences across large tool collections remains open and is left for future work.Source: §8 · p. 18

• ToFu derives tool-specific semantic checks from existing metadata and a CWE-guided sanitizer corpus to generate fuzzing harnesses and semantic oracles.Source: §9 · p. 18
• 253 verified bugs were identified across 3,405 tools from 273 repositories, spanning 118 open-source MCP servers.Source: §9 · p. 18
• The verified findings include arbitrary path traversal, log injection, exfiltration, and command bypass.Source: §9 · p. 18
• Tool metadata can serve as a specification for systematically testing the security of the underlying tool ecosystem.Source: §9 · p. 18

• Figure 4 shows returned images for a local file and a directory listing.Manual replay confirmed that page.goto renders files readable by the server's user and directory URLs as listings.Source: Figure 4 · p. 22, A Case Study · p. 23
• The tool's metadata describes a webpage screenshot, but its server checks only whether the URL parses, including file URLs.Source: A Case Study · p. 22
• file:///etc/passwd triggered forbidden-scheme checks, yet the tool returned an image instead of raising an error.Source: A Case Study · p. 22
• Saved login cookies are applied by default, and the tool can also screenshot services on loopback and private networks.Source: A Case Study · p. 23

B Repeat-After-Me on LangGraph

• The second turn produced a structured launch_missiles call with max_distance = 500, which ToolNode executed before the model refused on the following turn.The refusal text and executed call disagreed, and the arguments reached the tool unchanged from the attacker.Source: B Repeat-After-Me on LangGraph · pp. 23, 25
• On the first turn, the model printed the requested call as text, but LangGraph parsed no tool call and nothing ran.The transcript distinguishes raw model output from parsed tool calls.Source: B Repeat-After-Me on LangGraph · pp. 23, 25
• The system prompt shaped the model’s refusal text, but no check between model output and the tool invocation prevented the later call.The tools were stubs with no side effects; the transcript demonstrates the missing pre-execution check rather than an actual harmful action.Source: B Repeat-After-Me on LangGraph · pp. 23, 25

C Sanitizers

• Sanitizers are grouped by evidence channel, and 73 are offered to the LLM while unexpected_crash remains internal and always active.The corpus groups checks by the channel they read.Source: C Sanitizers · p. 25
• The 74 sanitizers include input-contract checks, execution-health and output checks, and runtime-introspection checks; the latter apply only to Python targets.Input-contract predicates need no instrumentation and apply to Python and JavaScript targets.Source: C Sanitizers · p. 25
• Of the 74 sanitizers, 11 run on every execution, 35 when their evidence appears, 27 only when selected by the LLM, and one is registered but not invoked.Selection also drives seed generation and each sanitizer’s expected-outcome rule.Source: C Sanitizers · p. 25

D Sanitizers and Tracers Exercised

• In the first full run, 35 of 74 sanitizers fired on at least one tool, 12 set a reported bug class, and 11 of 21 tracers recorded a flagged event.Source: D Sanitizers and Tracers Exercised · p. 26
• Four checks set the class of most reported bugs: reflected output, path traversal, log injection, and SSRF, each matching patterns common in MCP servers.Sensitive-data logging fired on the most tools but set no bug class; malformed-JSON and resource-exhaustion checks were also robustness signals, not bug classes.Source: D Sanitizers and Tracers Exercised · p. 26
• Ten tracers recorded no events because their sinks were rarely reached with caller data; these checks remain in the corpus for weaknesses common in general software.The sinks included deserialization, XML external-entity parsing, disabled certificate checks, and nonce handling, and the tracers run only in Python servers.Source: D Sanitizers and Tracers Exercised · p. 26

E Validity of the Static Sink Probes

• Static probes approximate whether a tracer’s instrumented API appears anywhere in scanned repository files, not necessarily inside registered tools; the scan has no measured precision or recall.Files over 900 KB and common generated, dependency, build, and test directories are excluded; matches are recorded at repository granularity.Source: E Validity of the Static Sink Probes · pp. 26–27
• Low-specificity probes such as print(, open(, and re-module use indicate repository surface, not vulnerability prevalence; overlapping probe rates cannot be summed.The coverage figures are used only to argue that a tracer has enough reachable surface to be worth running, not to rank prevalence or form detection-rate denominators.Source: E Validity of the Static Sink Probes · pp. 26–27

F Evaluated Repositories

• The 273 repositories with at least one successful tool call form the reachable denominator and head-to-head evaluation set.Source: F Evaluated Repositories · p. 27

G Impact of Each Bug Class

• The impact rubric specifies the attacker-controlled argument, server behavior, violated property, and cases classified as low impact.Low-impact cases are real behaviors whose harm is limited or that match the tool’s documented behavior.Source: G Impact of Each Bug Class · p. 28

H ToFu Prompts

• ToFu’s initialization prompts select sanitizers and expected-outcome rules from tool metadata, then generate schema-shaped fuzzing seeds tailored to the selected checks.The seed prompt requests 6 to 12 argument objects and may use predefined file generators for custom documents.Source: H ToFu Prompts · pp. 28–30