PaperlayerView exact revision on arXiv

Source-linked AI summary

Spoofing the Toolbox: Optimizing Tool Descriptions to Deceive LLM Agents

Shih-Chieh Dai, Xinyu Sun, Guanhong Tao

arXiv:2610.33182v1cs.CR

TL;DR

MCP agents can be misled into selecting malicious tools, while prior description attacks trade off effectiveness and stealth. TSA iteratively optimizes tool descriptions using surrogate-agent feedback and contrastive examples. It reports strong attack success across benchmarks alongside low human and detector detectability.

  • Problem

    Tool descriptions can be manipulated to mislead agents into selecting malicious tools, while prior attacks have relied on detectable keyword techniques or less effective rule-based methods.

    Source: §1 · p. 1, §4 · p. 4

  • Method

    TSA uses an LLM-powered optimizer with agent feedback and contrastive examples to refine malicious tool descriptions for selection effectiveness and naturalness.

    Source: §1 · p. 2

  • Results

    83.23% average attack success rate (ASR) across MetaTool, MCP-Atlas, and TSABench in single- and multi-tool scenarios, outperforming prior methods.

    Source: §1 · p. 2

  • Takeaways & Limitations

    TSA demonstrates that optimization-based malicious descriptions can achieve high attack success while remaining difficult for humans and existing detectors to identify.

    Source: §1 · p. 2, §7 · p. 9

    The study focuses solely on tool descriptions, creates no harmful malicious tools, and uploads none to MCP platforms.

    Source: ETHICAL STATEMENT · p. 10

Abstract

from arXiv · show

Large language model (LLM) agents are widely used in many applications and often rely on external tools. Tool information, such as function descriptions, helps agents determine which tools best suit a given context. However, attackers can manipulate this information to mislead LLM agents into using malicious tools. Prior attacks mainly rely on keyword-based techniques, which are easily detected, or rule-based methods, which have limited effectiveness. In this work, we introduce TSA, a stealthy tool-spoofing attack that crafts descriptions for malicious tools that are difficult to distinguish from those of existing benign tools. To achieve this, we propose an LLM-powered optimization method that iteratively refines malicious tool descriptions. Specifically, TSA begins with a variant of a benign tool description and updates it based on feedback from a surrogate LLM agent. The optimization method incorporates a feedback function and contrastive in-context learning to guide the search for descriptions that are both stealthy and effective. Experiments on both single- and multi-tool use benchmarks demonstrate that TSA achieves an attack success rate of up to 92.13\%, outperforming four state-of-the-art approaches while remaining highly stealthy. Our user study further shows that even human participants cannot distinguish descriptions generated by TSA from benign ones. Moreover, TSA remains effective against five tool-malware detectors.

1 INTRODUCTION

The introduction frames tool spoofing as a security risk of open MCP ecosystems and presents TSA as an LLM-powered method for making malicious tool descriptions effective yet natural. Across three datasets and single- and multi-tool scenarios, TSA outperforms prior methods while retaining low detectability.

1 INTRODUCTION

  • MCP’s low-barrier tool publishing and lack of visibility into tool execution let malicious tools mimic benign behavior while performing additional harmful actions.The agent observes only the returned message and cannot verify what occurs during invocation.Source: §1 · p. 1
  • Prior prompt-injection attacks use conspicuous imperative phrases, while rule-based GAPMA descriptions are less conspicuous but achieve limited attack performance.Explicit directives can also be exposed to users when clients show tool descriptions for confirmation.Source: §1 · p. 2
  • TSA uses an LLM-powered optimizer with feedback on malicious tool selection and description naturalness, guided by successful and failed contrastive examples.It also adjusts mutation magnitude dynamically and uses past generations to explore when optimization gets stuck.Source: §1 · p. 2
  • 83.23% average attack success rate (ASR) across MetaTool, MCP-Atlas, and TSABench in single- and multi-tool scenarios, outperforming prior methods.TSA reaches up to 92.13% ASR on GPT-5; human evaluators perceived its descriptions as least suspicious, and detector detection rates reached a maximum of 4.25%.Source: §1 · p. 2

2 RELATED WORK

Related work distinguishes prompt-injection, tool-spoofing, and tool-poisoning attacks, alongside LLM-based optimization methods and MCP security research. Existing tool-spoofing methods face effectiveness or detectability limitations, while MCP benchmarks and defenses have largely overlooked this attack surface.

Attacks on Tool Use

  • Tool-spoofing methods manipulate descriptions to bias tool selection, but existing LLM-based approaches have limited ASR or are easily detected.AMA uses black-box LLM-driven search but considers only whether the agent selects a tool, unlike TSA’s finer-grained feedback.Source: §2 · p. 3
  • Prompt-injection attacks embed malicious instructions in tool descriptions, whereas tool-poisoning attacks target execution through tools that appear benign at the interface.The passage also distinguishes tool spoofing as description optimization for manipulating selection.Source: §2 · p. 3

LLMs for Optimization

  • LLMs have served as reward models and black-box optimizers of discrete text, proposing candidates from scored trajectories, evolutionary operators, or natural-language critiques.The same optimization recipe has also been used for LLM-based attacks.Source: §2 · p. 3

MCP Security

  • Existing MCP benchmarks and defenses focus primarily on prompt-injection-style threats, leaving tool-spoofing characteristics largely unaddressed.The passage notes both rule-based and learning-based detection frameworks among existing defenses.Source: §2 · p. 3

3 THREAT MODEL

The threat model seeks to make an attacker-controlled tool preferred over benign alternatives with similar functionality, while keeping its description indistinguishable from a benign one. The attacker optimizes offline without access to the target agent or its runtime tools.

3 THREAT MODEL

  • The attacker aims to make the malicious tool selected over benign alternatives with similar or identical functionality across semantically equivalent user queries.The description is intended to be indistinguishable from that of a benign tool with similar functionality.Source: §3 · p. 3
  • The attacker lacks access to the target agent, its query interface, user queries, and the exact runtime tool set, but can optimize offline using a surrogate agent and representative benign tools.After optimization, the attacker publishes the resulting tool on open MCP platforms.Source: §3 · p. 3
  • Once selected, the agent generally cannot verify tool side effects beyond the returned message, allowing hidden actions by the attacker-controlled implementation.The attacker can update benign tools or distribute malicious MCP servers without explicit user notification.Source: §3 · pp. 3–4

4 ATTACK DESIGN

TSA treats malicious tool-description design as a black-box search balancing selection success against stealth. It initializes descriptions from benign tools, then uses agent feedback and LLM-guided updates to find effective, natural descriptions.

4 ATTACK DESIGN

  • TSA addresses discrete, hard-label search and the trade-off between inducing malicious selection and preserving stealth, challenges that limit prompt-injection and rule-based methods.Prompt-injection methods can bias selection but are easy to detect, while GAPMA-style mutations preserve stealth but reach at most 51.06% ASR.Source: §4 · p. 4

4.1 DESCRIPTION INITIALIZATION

  • The seed pool combines LLM-synthesized descriptions semantically similar to a benign tool with a subset containing imperative selection indicators, plus descriptions without them.The mix aims to avoid making the optimizer assume every successful description needs a strong indicator.Source: §4.1 · p. 5

4.2 GUIDED OPTIMIZATION

  • TSA guides black-box optimization with fine-grained selector feedback, an LLM-powered optimizer, and mutation-step adjustment.It uses a surrogate agent because the attacker observes only whether a tool—and which tool—is selected.Source: §4.2 · p. 5, §4.2.1 · p. 5

4.2.1 FEEDBACK FUNCTION

  • The feedback function distinguishes benign, malicious, wrong-tool, and no-tool outcomes, while perplexity feedback encourages descriptions no less fluent than the benign original.The selector outcome directs attack updates; OptAgent is also told to refine descriptions whose perplexity exceeds the original's.Source: §4.2.1 · pp. 5–6

4.2.2 LLM-POWERED OPTIMIZER

  • OptAgent rewrites candidate descriptions using selection and perplexity feedback; contrastive examples, successful-history guidance, and dynamic mutation adjustment provide additional search direction.Contrastive examples include successes and failures; mutation becomes more aggressive in unproductive regions and smaller after malicious selections.Source: §4.2.2 · pp. 6–7

5 EXPERIMENT SETUP

The experiments evaluate TSA across three benchmarks, single- and multi-tool settings, and multiple LLMs, comparing it with four attack baselines. They report benign accuracy, attack success, and malware detection, with white-box GCG excluded from the threat model.

Datasets

  • TSA is evaluated on MetaTool, MCP-Atlas, and TSABench, with single-tool tests on MetaTool and TSABench and multi-tool tests on MetaTool and MCP-Atlas.Main experiments present 10 to 20 tools to the agent.Source: §5 · p. 7

Baselines

  • The four baselines are <IMPORTANT>, ToolHijacker, GAPMA, and AMA; white-box GCG is excluded because it falls outside the threat model.The authors also cite GCG's unnatural sequences as readily detectable.Source: §5 · p. 7

Metrics

  • Evaluation measures benign accuracy, attack success rate (ASR), and detection rate, respectively tracking correct benign-tool selection, malicious-tool selection, and detector flags.Source: §5 · p. 7

Settings

  • Llama-3.3-70B serves as the default surrogate selector and Qwen3-8B as OptAgent; ASR is evaluated on five open-source and proprietary tool-capable LLMs.The evaluated models are Llama-3.3-70B, Qwen-Coder2.5-32B, Qwen3-32B, Qwen3.8-27B, and GPT-5.Source: §5 · p. 7

6 EVALUATION RESULTS

On known LLMs, TSA outperforms the baselines across the reported benchmarks, and it remains effective with more tools and on unseen models. Additional tests examine competing descriptions, optimization overhead, and an ablation.

Attack Performance on Known LLMs

  • TSA achieves 88.65% ASR on TSABench and 87.94% on MetaTool, and 82.5% on MetaTool and 80.1% on MCP-Atlas in multi-tool use.These results outperform the reported baselines; ToolHijacker reaches 82.97% ASR on MetaTool.Source: §6 · p. 8

20-tool setting

  • 80.14% ASR on both benchmarks in the 20-tool setting; ASR decreases across methods as more tools are presented.TSA continues to outperform the other baselines in this more challenging setting.Source: §6 · p. 8

Transferability to Unseen LLMs

  • 90.78% ASR on QwenCoder-2.5-32B and 92.13% on GPT-5 on TSABench, with up to 90.07% on MetaTool for unseen models.On unseen models, AMA averages 64.72% ASR versus 81.91% for TSA; ToolHijacker's TSABench ASR drops by up to 32.62%.Source: §6 · p. 8

Robustness, overhead, and ablation

  • TSA maintains 75.88% ASR when three benign tools satisfy the query and reaches 95.03% when the attacker deploys three malicious tools.TSA requires 272.67 seconds of one-time optimization per task on average, with no additional online cost; removing DMA reduces ASR by 14.19%.Source: §6 · p. 9

Increased ambiguity

  • With all attack descriptions and the benign description presented together, agents select the TSA description 50.4% of the time.The setting increases ambiguity by presenting multiple descriptions for the same tool.Source: Figure 5 · p. 9, §6 · p. 9

7 CONCLUSION

The conclusion presents TSA as an optimization-based tool-spoofing attack with high attack success rates and strong transferability, and says it is less likely to be detected by humans and existing detection methods.

7 CONCLUSION

  • TSA generates malicious tool descriptions with high ASRs and strong transferability, while being less likely to be detected by humans and existing detection methods.The authors underscore MCP's vulnerability and the need for effective defenses against tool-spoofing attacks.Source: §7 · p. 9

AI USE STATEMENT

The authors disclose using generative AI for writing polish, query synthesis, and part of TSA's implementation, while stating that they reviewed generated content and take responsibility for the final work.

AI USE STATEMENT

  • Generative AI was used to polish writing, synthesize optimization and evaluation queries, and write part of TSA's implementation code.The authors state that AI-generated content and artifacts were reviewed by at least three authors and that they take full responsibility for the final work.Source: AI USE STATEMENT · p. 10

ETHICAL STATEMENT

The ethical statement frames the work as demonstrating MCP security weaknesses to support countermeasure development, not real-world attacks. It describes risks to users and ecosystem stakeholders and states that no malicious tools were provided to individuals or uploaded to an MCP platform.

Stakeholder Analysis and Process Impact

  • Tool-spoofing can threaten user trust by causing agents to invoke malicious tools, potentially leading to data leakage, incorrect actions, or security breaches.Tool developers and platform providers depend on standardized descriptions for safe selection, while agent developers face challenges detecting stealthy tools.Source: ETHICAL STATEMENT · p. 10

Justification for Academic Research

  • The authors say the study demonstrates MCP vulnerabilities to enable researchers to develop countermeasures, rather than exploit the attack in real-world settings.They report informing the MCP Client2 maintainer and developing defense methods.Source: ETHICAL STATEMENT · p. 10

Scope and safeguards

  • The study focuses solely on tool descriptions; the authors report creating no harmful malicious tools and uploading none to an MCP platform.They also state that no malicious tool was provided to any individual.Source: ETHICAL STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

Cline shows tool descriptions in a confirmation prompt before invocation, making overt directives visible to users. The reproducibility statement identifies supplementary artifacts and implementation settings.

REPRODUCIBILITY STATEMENT

  • The authors state that prompts, data, and generated descriptions are available in the supplementary materials.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Optimization used five queries, 10 starting points, eight selection indicators, and up to 300 steps on NVIDIA H200 GPUs.Early-stopping and learning-rate patience were both set to 3.Source: REPRODUCIBILITY STATEMENT · p. 10

A CASE STUDY: TOOL CONFIRMATION IN CLINE

  • Cline displays the server, selected tool, and its description before invocation, allowing users to inspect the prompt before approving a call.The example’s appended imperative directive stands out from the ordinary functional summary, and users could plausibly recognize the manipulation and decline.Source: A CASE STUDY: TOOL CONFIRMATION IN CLINE · p. 15

B LIMITATIONS OF EXISTING METHODS

The section describes limitations in prior tool-spoofing approaches and reports TSA’s transfer, human-evaluation, and detector results alongside experimental details. Across these evaluations, TSA transfers to unseen models and is difficult for both people and detectors to identify.

B LIMITATIONS OF EXISTING METHODS

  • Prior approaches trade off attack performance and stealth: directive-based methods expose manipulation cues, while GAPMA’s rule-based rewriting has limited attack performance.AMA’s limitations include a coarse optimization signal, no explicit stealth objective, and reduced performance when descriptions transfer across tool domains.Source: B LIMITATIONS OF EXISTING METHODS · pp. 15–16

C OPTIMIZATION HYPERPARAMETERS

  • After three consecutive successful selections, early stopping may halt optimization before descriptions become sufficiently stealthy; mutation patience instead increases modification strength after repeated failures.Source: C OPTIMIZATION HYPERPARAMETERS · p. 16

D DATASET; F EXPERIMENT SETTING

  • The evaluation combines MetaTool, MCP-Atlas, and TSABench, spanning tool collections and single- or multi-tool tasks.TSABench contains 144 MCP tools and uses seven queries per tool for optimization and three for final evaluation; utility tests also vary tool-set size.Source: D DATASET · p. 16, F EXPERIMENT SETTING · p. 18

E BASELINES; G REPRODUCING GAPMA

  • Baselines include <IMPORTANT>, GAPMA, AMA, and ToolHijacker, with ToolHijacker’s description formed by concatenating a semantic description and an optimized selection directive.The GAPMA reproduction sampled four tools and replaced the original evaluation model; its reproduced implementation performed better than the original report.Source: E BASELINES · pp. 16–17, G REPRODUCING GAPMA · p. 18

F EXPERIMENT SETTING

  • Perplexity was computed with GPT-2 on the attacker side, requiring no access to the target agent.Open-source models were served with vLLM, and GPT-5 was accessed through the OpenAI API.Source: F EXPERIMENT SETTING · p. 18

H TRANSFERABILITY

  • 80.40% average ASR in the 20-tool setting shows TSA outperformed all baselines on every evaluated unseen model across both datasets.TSA reached 90.07% on GPT-5 with MetaTool; ToolHijacker and AMA averaged 68.35% and 62.85%, respectively.Source: Table 10 · p. 19, H TRANSFERABILITY · p. 19

I HUMAN EVALUATION DETAILS

  • Human raters could not distinguish TSA descriptions from benign originals, unlike <IMPORTANT> and ToolHijacker, which were ranked significantly more suspicious.Source: I HUMAN EVALUATION DETAILS · p. 19

J DETECTION DETAILS

  • 6.36% was TSA’s highest detection rate across both datasets, while nearly all detectors reported 0% for GAPMA and AMA.Hybrid detectors detected <IMPORTANT> at 100%; the LLM Detector detected ToolHijacker at 85.3% on TSABench and 78.7% on MetaTool.Source: J DETECTION DETAILS · p. 20

K RELIABILITY OF TSA

TSA remains effective as tool-set ambiguity increases and incurs optimization costs comparable to its strongest ASR baseline. Its ASR rises with more malicious tools and falls moderately with more ground-truth benign tools.

K RELIABILITY OF TSA

  • TSA consistently maintained high ASR and outperformed ToolHijacker as tool-set size varied from 5 to 15.Both methods’ ASR increased when the set shrank from 10 tools to 5.Source: K RELIABILITY OF TSA · p. 21
  • 95.03% ASR was reached with three malicious tools, while TSA achieved 75.88% with three ground-truth benign tools.The former raises attack success; the latter reflects only a moderate decrease as benign options increase.Source: K RELIABILITY OF TSA · p. 21
  • 50.4% of the time, the agent selected TSA’s description when benign and all malicious descriptions were presented together.This result indicates TSA remained preferred under increased ambiguity.Source: K RELIABILITY OF TSA · p. 22

L OVERHEAD

  • TSA’s per-task optimization took 272.67 seconds versus 248.94 seconds for ToolHijacker, with comparable cost and higher ASR and stealthiness.Online cost was the same across methods because each attack simply attaches its optimized description to the tool.Source: L OVERHEAD · p. 22

M ABLATION STUDY

The ablations examine how TSA’s components affect attack success and optimization stability. The discussion also states scope limitations, future security implications, and the main procedure used to optimize descriptions.

M ABLATION STUDY

  • Removing DMA from the full configuration reduced performance by 14.19%, while adding DMA to the base setting improved ASR by 5%.Adding contrastive examples to the DMA configuration increased ASR by 16.66% over base, though that single-query, fixed-tool configuration overfit.Source: M ABLATION STUDY · pp. 22–23
  • With early-stopping patience p_t set to 7, TSA achieved its best performance; setting p_t to 3 yielded 54% LLM Detector detection despite competitive ASR.Source: M ABLATION STUDY · p. 23
  • Disabling mutation-magnitude control or setting p_l to 1 made optimization unstable, while ASR began decreasing when p_l exceeded 5.Ten starting points produced the best ASR, and more training queries and contrastive examples improved performance.Source: M ABLATION STUDY · p. 23

N DISCUSSION AND FUTURE IMPLICATION

O SYSTEM PROMPTS

P DETAILED ALGORITHM OF TSA

  • TSA batch-evaluates candidate descriptions with a surrogate selector, updates positive and negative example sets, and refines candidates through dynamic mutation adjustment.It terminates after the required consecutive successful outcomes and returns the successful description with the lowest perplexity.Source: P DETAILED ALGORITHM OF TSA · pp. 24–25

Related papers

Loading 2610.33182v1…