PaperlayerView exact revision on arXiv

Source-linked AI summary

ToolTweak: An Attack on Tool Selection in LLM-based Agents

Jonathan Sneh, Ruomei Yan, Jialin Yu, Philip Torr, Yarin Gal, Sunando Sengupta, Eric Sommerlade, Alasdair Paren, Adel Bibi

arXiv:2510.02554v1cs.CRcs.AI

TL;DR

Tool-using agents select from expanding pools of tools based largely on natural-language metadata, leaving selection vulnerable to adversarial optimization. The paper introduces ToolTweak, which iteratively manipulates tool names and descriptions, and reports substantial selection-rate gains with transfer across model types. It also examines usage-distribution shifts and evaluates defenses.

  • Problem

    Agents rely heavily on tool metadata when selecting from growing tool ecosystems, creating a vulnerability with implications for fairness and security.

    Source: §1 · pp. 1–2

  • Method

    ToolTweak iteratively updates tool names and descriptions using selection-rate feedback, and the paper analyzes distributional effects and evaluates defenses.

    Source: §1 · p. 2, §3.3 · p. 4

  • Results

    Selection rates rise from around 20% to as high as 81%, with strong transferability across open-source and proprietary models.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Metadata manipulation can disadvantage equally capable providers and distort tool usage, raising fairness and competition concerns in tool ecosystems.

    Source: §6 · p. 9

    Future work should test larger, dynamic tool databases with retrieval systems and white-box, gradient-based attacks.

    Source: §6 · p. 9

Abstract

from arXiv · show

As LLMs increasingly power agents that interact with external tools, tool use has become an essential mechanism for extending their capabilities. These agents typically select tools from growing databases or marketplaces to solve user tasks, creating implicit competition among tool providers and developers for visibility and usage. In this paper, we show that this selection process harbors a critical vulnerability: by iteratively manipulating tool names and descriptions, adversaries can systematically bias agents toward selecting specific tools, gaining unfair advantage over equally capable alternatives. We present ToolTweak, a lightweight automatic attack that increases selection rates from a baseline of around 20% to as high as 81%, with strong transferability between open-source and closed-source models. Beyond individual tools, we show that such attacks cause distributional shifts in tool usage, revealing risks to fairness, competition, and security in emerging tool ecosystems. To mitigate these risks, we evaluate two defenses: paraphrasing and perplexity filtering, which reduce bias and lead agents to select functionally similar tools more equally. All code will be open-sourced upon acceptance.

1 INTRODUCTION

As agents select tools from growing ecosystems, their choices depend heavily on tool metadata. The paper presents adversarially optimized metadata as a vulnerability and introduces ToolTweak to study its effects and possible mitigation.

1 INTRODUCTION

  • Tool Learning connects agents to external information, APIs, and services, while tool marketplaces create growing pools of options for agents acting on users’ behalf.Earlier retrieval-augmented approaches typically use static databases, limiting their usefulness for some tasks.Source: §1 · p. 1
  • Tool choice depends almost entirely on natural-language metadata, which adversaries can optimize to bias agents and gain unfair marketplace advantages.The paper links this vulnerability to risks for individual agents and broader tool-marketplace usage distributions.Source: §1 · pp. 1–2
  • Prior adversarial research has emphasized jailbreaks, prompt injection, and retrieval poisoning; the paper frames persistent tool-selection bias across tasks and time as a distinct, underexplored attack surface.The introduction argues that small metadata manipulations could scale across agents as protocols and marketplaces proliferate.Source: §1 · p. 2
  • ToolTweak manipulates tool names and descriptions to boost selection, with reported rates rising from around 20% to as high as 81% and transfer across open-source and proprietary models.The paper also analyzes usage-distribution shifts and evaluates defenses, including paraphrasing.Source: §1 · p. 2

2 RELATED WORK

Related work covers tool-learning and retrieval systems, adversarial attacks on LLMs and agents, and emerging work targeting tool selection. The paper distinguishes its focus on persistent selection bias from attacks that target a single response or retrieval stage.

Tool Learning and Retrieval

  • Tool-learning systems use external tools and retrieval; ToolLLM and Gorilla use embedding-based retrievers, while ToolGen integrates tool tokens into the model vocabulary.Tool names, descriptions, and parameter schemas are common natural-language cues for selection.Source: §2 · pp. 2–3

Attacks on LLMs and Agents

  • Prior LLM attacks include gradient-free prompt attacks such as PAIR and BoN, gradient-based GCG, and agent backdoors that inject malicious documents into RAG systems.The cited agent backdoors are described as typically gradient-based trigger attacks (Chen et al., 2024; Chaudhari et al., 2024; Zou et al., 2024).Source: §2 · p. 3

Attacks on Tool Use

  • Tool-use attack work includes induced tool calls (Chaudhari et al., 2024), description suffixes (Faghih et al., 2025), retrieval-based prompt injection (Shi et al., 2025), and MCP description-based rug-pull attacks.Tool-selection attacks must bias behavior consistently across queries, unlike jailbreaks that succeed once defenses are bypassed.Source: §2 · p. 3

3 TOOLTWEAK

ToolTweak models an attacker who can revise a tool’s name and description, then iteratively tests and updates them using selection-rate feedback. The setup fixes the parameter schema and treats attacker knowledge and data access as explicit assumptions.

3.1 PROBLEM SETUP AND FORMALIZATION

  • The formal setup represents a tool by its name, description, and parameters, then optimizes the name and description to maximize call probability over tools and user queries.The parameter schema remains fixed, and model-specific tool-definition and call formats are abstracted through rendering functions.Source: §3.1 · pp. 3–4

3.2 THREAT MODEL AND ASSUMPTIONS

  • The threat model assumes a legitimate marketplace provider can freely change metadata, sees the tool database and usage statistics, and receives sample user queries, but may not know the agent architecture.The attacker cannot modify the parameter schema.Source: §3.2 · p. 4

3.3 ATTACK PROCEDURE

  • Each iteration evaluates the target tool’s selection rate on queries, feeds usage feedback and tool history to an attacker model, and updates its name and description.The default is 10 iterations; prompts encourage plausible descriptions using subjective wording, factual claims, and implicit comparisons, while the process remains gradient-free and lightweight.Source: §3.3 · p. 4

4 ATTACK EXPERIMENTS

The experiments evaluate ToolTweak on ToolBench tasks across six models, finding increased target-tool selection and examining distribution shifts, transferability, selection factors, and ablation conditions.

4.1 DATASET

  • The dataset uses five nearest-neighbor-selected APIs for each of 10 ToolBench tasks and generates 100 user queries per task.Tasks include geocoding, news retrieval, IP geolocation, WHOIS lookup, email validation, sentiment analysis, language detection, QR-code generation, and weather forecasting.Source: §4.1 · p. 5

4.2 BASELINES AND METRICS

  • The study measures OSR, BSR, and TSR over 100 queries per tool, with TSR defined as average BSR across tasks and tools for each model.It treats OSR < BSR as attack success and reports normalized improvement relative to the available headroom; baselines are original descriptions and a fixed persuasive suffix.Source: §4.2 · p. 5

4.3 RESULTS

  • 19.90 to 45.92 for Qwen and 20.16 to 81.62 for DeepSeek: ToolTweak raised tool selection rates from baseline levels.On average, the manual suffix attack performed better, while ToolTweak achieved comparable TSRs except for Grok and Qwen.Source: Figure 2 · p. 6, §4.3 · p. 6
  • For all models, most tools fall below the before-versus-after JSD line, indicating improved selection rates for most target tools after attack.DeepSeek and GPT-OSS points concentrate lower in the plot, indicating greater capacity for improvement and distribution bias.Source: §4.3 · p. 6, Figure 3 · p. 7
  • Tool order, parameter schemas, and names influence selection; identical BFCL tools with numeric suffixes showed drastically different selection rates.Qwen showed a first-listed preference in preliminary tests; Task 3's complicated schemas challenged models, and Gemini struggled more to produce tool calls. The authors hypothesize that numeric ordinality signals ranking.Source: §4.3 · pp. 6–7
  • Transferability depends chiefly on the target agent: DeepSeek averages around 60% selection regardless of attack generator, while Gemini and Llama typically reach 20% to 40%.Newer attacker models transfer better overall, and a model often performs best with an attack it generated itself.Source: §4.3 · p. 7

4.4 ABLATION STUDIES

  • Two iterations substantially raise target selection; up to 10 iterations add expected gains, including up to a 20% increase for DeepSeek.More queries and tools provide modest but consistent benefits; in the no-knowledge setting, the attack reaches around 60% TSR versus around 80% with full knowledge.Source: §4.4 · pp. 7–8

5 DEFENSE EXPERIMENTS

Paraphrasing generally moves tool-selection distributions closer to uniform, though ToolTweak remains more robust than manual suffix attacks. Perplexity alone does not separate attacker-written descriptions from original descriptions, but length combined with perplexity may help.

5 DEFENSE EXPERIMENTS

  • The defense experiments distinguish mitigation by paraphrasing from prevention by filtering tools before they enter the tool database.For paraphrasing, the victim model revises descriptions in an objective style before tool selection is reevaluated.Source: §5 · p. 8, §5.1 · p. 8

5.1 PARAPHRASING

  • Paraphrasing reduces selection rates for every model, but ToolTweak remains more robust to it than manually crafted suffix attacks.An attacker that iterates against the paraphrase defense achieves the highest defended-setting selection rates, but generally does not match the base attack.Source: §5.1 · p. 8
  • 60%–88% of tool distributions move closer to uniform under paraphrasing, compared with 42% for Llama 3.1.The paraphrase defense debiases 60% of Qwen’s distributions and 88% of DeepSeek’s; the initial attack is also less effective against Llama 3.1.Source: §5.1 · p. 8, Figure 4 · p. 9

5.2 PERPLEXITY FILTERING

  • Attacker-generated and original descriptions have largely overlapping perplexity distributions, making a simple perplexity threshold ineffective.Attacked descriptions are typically longer; combining log-length and log-perplexity forms generally distinct clusters that could support a lightweight classifier.Source: §5.2 · p. 9

6 CONCLUSION

ToolTweak biases tool selection across tasks and queries, with potential consequences for fairness and provider revenue. The evaluation leaves larger dynamic retrieval databases and white-box gradient-based attacks for future study.

6 CONCLUSION

  • ToolTweak substantially biases tool selection across diverse tasks and queries, with effects that may disadvantage equally capable providers and shift usage-based revenue.The paper likens this to search engine optimization: persuasive names and descriptions can promote tools apart from their capabilities.Source: §6 · p. 9
  • The evaluation leaves larger dynamic tool databases with retrieval systems and white-box gradient-based attacks for future work.Source: §6 · p. 9

A DATA STANDARDIZATION

The appendix standardizes tool definitions for proprietary model services by constraining names and parameter types. It also includes a heatmap of transferability for tools augmented by attacker LLMs.

A DATA STANDARDIZATION

  • Tool names were shortened to under 64 characters and stripped of disallowed characters, while parameter types were mapped to string, boolean, or number.These changes addressed formatting constraints in proprietary tool-calling APIs and produced JSON Schema-compatible types.Source: A DATA STANDARDIZATION · p. 14

B TRANSFERABLE MAP

  • Figure 5 presents a heatmap of transferability effectiveness for tools augmented by attacker LLMs.Source: Figure 5 · p. 14

C ABLATION

Ablations show ToolTweak remains effective without query or tool knowledge, while more iterations, example queries, and available tools improve selection rates. The attacker and victim models use different temperature settings.

C ABLATION

  • At k = 10, iterations increase DeepSeek’s best selection rate by up to 20 percentage points, although much of the gain occurs after the first revision.Across models except Llama 3.1, the largest gain generally comes at k = 2; a one-shot attack can reduce computational costs, but iteration is needed for the most effective attack.Source: C ABLATION · p. 14
  • TSR rises from approximately 74.4% with 2 example queries to approximately 81.6% with 10.Source: C ABLATION · p. 14
  • TSR increases from approximately 73% with 1 tool to the base attack’s 81.6% when more tools are available.Source: C ABLATION · p. 15
  • ≈60% TSR is achieved without knowledge of the full schema, competitor tools, or user queries, compared with above 80% for the full attack.In this black-box setting, the attacker receives usage statistics as its only signal.Source: C ABLATION · p. 15

D.1 TEMPERATURE

  • Victim-model evaluations use temperature 0, while the attacker uses temperature 0.5 to support exploration without frequent instruction-following and JSON-formatting problems.At attacker temperatures of 0.7 or higher, the authors found models followed instructions less well and produced improperly formatted JSON.Source: §D.1 · p. 15

D.2 REPRODUCIBILITY

The reproducibility notes fix a random victim and explain that temperature zero does not ensure deterministic outputs. The appendices include prompts for the attacker and paraphrase-defense agent, and an example of reward-hacking generation.

D.2 REPRODUCIBILITY

  • The experiments use random victim 42 for supported API providers and for shuffling and sampling by the Python client.Source: §D.2 · p. 15
  • Temperature = 0 was intended to make tool-selection evaluations more deterministic, but inference-provider APIs and mixture-of-experts models can remain non-deterministic.The passage notes sequence-level nondeterminism in mixture-of-experts models.Source: §D.2 · p. 15

E PROMPTS

F MISCELLANEOUS APPENDICES

  • A reward-hacking example shows an LLM copying a preferred tool’s name to increase its selection rate.Source: Figure 9 · p. 17

F.1 GAMING TOOL CALLS EXPERIMENTS

The appendix recreates experiments on BFCL v3 simple to examine tool-selection factors, suffix attacks, and defenses. Augmented descriptions were selected more often without defense, while objective paraphrasing reversed that comparison.

F.1 GAMING TOOL CALLS EXPERIMENTS

  • The recreated experiments use the BFCL v3 simple dataset to examine tool-selection rates, factors affecting them, suffix attacks, and defenses.Source: §F.1 · p. 17
  • Tools with augmented descriptions were called < 6.5 times more than tools with the original description in the undefended case.Source: §F.1 · p. 17
  • After objective paraphrase defense, tools with augmented descriptions were called about 1.18 times less than the original tool.Source: §F.1 · p. 17

F.2 JENSEN—SHANNON DIVERGENCE

This appendix presents Jensen–Shannon divergence comparisons and visualizations of tool-selection bias and attack-description perplexity. The figures describe how to interpret divergence shifts and identify name-based selection bias when presentation order is controlled.

F.2 JENSEN—SHANNON DIVERGENCE

  • 72.6% of Qwen2.5-7B selections and 65.6% of Llama3.2-3B selections favored the tool named Tool 1, averaged across presentation orders.The figure reports averages across five trials with default settings.Source: Figure 10 · p. 18
  • The appendix includes log-perplexity histograms for DeepSeek and Llama3.1 attack descriptions and original descriptions.Source: Figure 11 · p. 18, Figure 13 · p. 19
  • The appendix also plots log-sequence length against log-perplexity for DeepSeek and Llama3.1 attack descriptions and original descriptions.Source: Figure 12 · p. 19, Figure 14 · p. 20
  • DJS values before attack are plotted against after-attack values; points above y = x indicate improved selection rates, and distance from y = 0 indicates greater bias.Source: Figure 15 · p. 20
Loading 2510.02554v1…