PaperlayerView exact revision on arXiv

Source-linked AI summary

MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use

Yue Huang, Jiawen Shi, Yuan Li, Chenrui Fan, Siyuan Wu, Qihui Zhang, Yixin Liu, Pan Zhou, Yao Wan, Neil Zhenqiang Gong, Lichao Sun

arXiv:2310.03128v6cs.SEcs.CL

TL;DR

LLMs need to decide both whether tools are necessary and which tools best satisfy user requests, but existing evaluations have focused mainly on collaborating with specific tools. MetaTool introduces the TOOLE dataset and evaluates awareness and selection across multiple tool-use settings; most tested LLMs show weak awareness and unreliable selection, while detailed tool descriptions are associated with better selection efficiency.

  • Problem

    Existing tool-use research and benchmarks leave limited evidence about LLMs' awareness of when to use tools and their ability to select among tools as intelligent agents.

  • Method

    MetaTool combines the 21,127-query TOOLE dataset with evaluations of tool-usage awareness and four tool-selection tasks spanning diverse single- and multi-tool scenarios.

  • Results

    Most of eight evaluated LLMs lack good tool-usage awareness and exhibit unreliable tool selection, while more detailed tool descriptions correspond to higher CSR.

  • Takeaways & Limitations

    Tool developers should provide detailed tool descriptions and choose an appropriate rewrite model based on the downstream LLM.

  • Takeaways & Limitations

    The evaluation separates Thought and Action prompts to avoid influence from different tool lists.

Abstract

from arXiv · show

Large language models (LLMs) have garnered significant attention due to their impressive natural language processing (NLP) capabilities. Recently, many studies have focused on the tool utilization ability of LLMs. They primarily investigated how LLMs effectively collaborate with given specific tools. However, in scenarios where LLMs serve as intelligent agents, as seen in applications like AutoGPT and MetaGPT, LLMs are expected to engage in intricate decision-making processes that involve deciding whether to employ a tool and selecting the most suitable tool(s) from a collection of available tools to fulfill user requests. Therefore, in this paper, we introduce MetaTool, a benchmark designed to evaluate whether LLMs have tool usage awareness and can correctly choose tools. Specifically, we create a dataset called ToolE within the benchmark. This dataset contains various types of user queries in the form of prompts that trigger LLMs to use tools, including both single-tool and multi-tool scenarios. Subsequently, we set the tasks for both tool usage awareness and tool selection. We define four subtasks from different perspectives in tool selection, including tool selection with similar choices, tool selection in specific scenarios, tool selection with possible reliability issues, and multi-tool selection. We conduct experiments involving eight popular LLMs and find that the majority of them still struggle to effectively select tools, highlighting the existing gaps between LLMs and genuine intelligent agents. However, through the error analysis, we found there is still significant room for improvement. Finally, we conclude with insights for tool developers -- we strongly recommend that tool developers choose an appropriate rewrite model for generating new descriptions based on the downstream LLM the tool will apply to. Our code is in https://github.com/HowieHwong/MetaTool.

1 INTRODUCTION

METATOOL addresses the need to evaluate whether LLMs know when tools are needed and can select appropriate tools, beyond prior work focused mainly on later tool-use stages. It introduces the TOOLE dataset and evaluates awareness and selection across diverse single- and multi-tool scenarios.

  • LLM tool use can provide external information, support user interactions, address uncertain questions, and process multimodal information.
  • Prior benchmarks mainly evaluate tool execution stages, leaving tool-usage awareness and tool selection important gaps for intelligent-agent applications.
  • METATOOL introduces TOOLE, a dataset of 21,127 diverse user queries covering both single-tool and multi-tool use.
  • TOOLE uses multiple prompting methods and tool merging and decomposition to address limited query diversity and overlapping tool functionality.
  • METATOOL evaluates tool-usage awareness and four tool-selection tasks covering similar choices, specific scenarios, reliability issues, and multi-tool selection.
  • Experiments with eight well-known LLMs find weak tool-usage awareness and unreliable tool selection that varies across daily scenarios.

2 METATOOL DESIGN

MetaTool evaluates whether LLMs recognize when external tools are needed and select appropriate tools across single- and multi-tool settings. Its TOOLE dataset and four selection subtasks address query diversity, overlapping functionality, scenario variation, reliability concerns, and complex tool combinations.

  • Dataset generation: TOOLE contains 21.1k diverse tool-related user queries, with each entry pairing a request with its corresponding tool name and description.The dataset-generation process uses tool descriptions as the basis for creating queries that trigger tool use.
  • Dataset generation: Four query-generation techniques create single-tool examples: direct diverse, emotional, keyword, and details generation.These techniques guide ChatGPT/GPT-4 to generate queries from tool descriptions under specified constraints.
  • Dataset generation: Tool merging and decomposition ensure each query has one ground-truth label despite overlapping tool functionality.Similar tools are merged, while multi-purpose tools are decomposed before merging.
  • Dataset generation: Multi-tool queries cover parallel and causal tool use, using pairs drawn from the 15 most popular tools and manually checking whether combinations are reasonable.The queries are generated after overlap resolution because remapping labels is difficult in multi-label settings.
  • Evaluation design: The benchmark evaluates tool usage awareness and tool selection through four selection subtasks covering similar choices, specific scenarios, reliability issues, and multi-tool selection.The awareness task includes positive queries requiring tools and negative queries that LLMs can solve directly; selection tasks test semantic comprehension, adaptability, reliability, and inference.

3 EXPERIMENTS

Experiments evaluate eight LLMs on tool usage awareness and four tool-selection settings using sampled TOOLE queries and CSR alongside standard awareness metrics. Most models remain weak or uneven, especially for reliability, longer tool lists, scenario variation, and multi-tool selection, while error analysis and description rewriting indicate room for improvement.

  • Experimental Setup: Eight models are evaluated on sampled TOOLE queries using standard awareness metrics and Correct Selection Rate (CSR) for tool selection.The setup includes ChatGPT, ChatGLM2, Llama2, Vicuna, Baichuan2, and Koala variants; CSR measures the percentage of correct selection actions.
  • Tool Usage Awareness: Only ChatGPT exceeds 70% on both accuracy and F1 in zero-shot awareness, while Llama2-13b reaches only 11.53% F1.Five-shot prompting improves some models, including Llama2-13b by 42.79% and Vicuna-7b by 42.28%, but most still perform poorly.
  • Tool Selection Results: Similar-tool selection shows large model disparities, with few-shot gains no higher than 7% and Vicuna-7b declining by 10% under five-shot prompting.The zero-shot gap between the best and worst models is nearly 30%, and the five-shot gap between ChatGPT and Llama2-13b exceeds 20%.
  • Tool Selection Results: Reliability remains difficult: although few-shot prompts improve every model, most CSR values stay below 20%.The reported errors include fabricating non-existent tools and avoiding a none response by selecting irrelevant tools.
  • Tool Selection Results: CSR generally decreases as tool-list length grows, while scenario performance is highest for elderly and artists-and-designers tools and lowest for student tools.The results indicate weaknesses in long-text processing and scenario generalization, motivating field-specific model choice for downstream applications.
  • Tool Selection Results: Multi-tool selection varies sharply across models: ChatGPT outperforms ChatGLM2 by nearly 70%, while explicit two-tool instructions raise Vicuna-33b CSR above 90%.Omitting tool selections is common; Vicuna-33b selects only one tool in 48.49% of cases under the reported setting.
  • Error Analysis: Incorrect similar-tool choices are often semantically close, with nearly 50% falling within Top@5 and more than 15% within Top@1.This error pattern suggests that selection mistakes retain useful similarity information and leave substantial room for improvement.
  • Insights for Tool Developer: More detailed tool descriptions correlate with higher CSR, while description rewrites produce different benefits across downstream model groups.Llama2-70b-rewritten descriptions improve Llama2-13b by 7.83% but do not significantly improve Vicuna models.

4 CONCLUSION

METATOOL evaluates LLM tool-usage awareness and tool-selection capabilities using the TOOLE dataset. The authors find that most LLMs lack strong awareness and remain substantially behind genuine intelligent agents in tool selection.

  • METATOOL benchmarks whether LLMs recognize when tools are needed and select appropriate tools.
  • TOOLE provides diverse queries designed to trigger LLM tool use.
  • Most evaluated LLMs show weak tool-usage awareness and a significant tool-selection gap relative to genuine intelligent agents.

A TOOLE DATASET DETAILS

TOOLE addresses overlapping tool functionality through merging and decomposition, then validates the resulting tool organization and dataset quality. Its construction also applies manual filtering and motivation labeling to make tool-use evaluation more controlled.

  • Overlapping functionality is addressed by merging tools with similar functions and decomposing tools that serve multiple purposes.These operations aim to prevent a query from being plausibly assigned to multiple tools.
  • Tool embeddings are compared before and after merging and decomposition using silhouette coefficients to assess functional overlap.The authors seek lower overlap and a more uniform distribution of tool functionalities in embedding space.
  • After the operations, silhouette coefficients are significantly smaller, indicating a more uniform tool distribution and reduced functional overlap.
  • Tool descriptions are manually reviewed, and tools with unclear or low-quality explanations are removed.
  • Queries containing tool names are removed because they can provide an obvious hint during evaluation.
  • Simple calculation queries are removed, while complex calculation queries are retained because LLMs may still perform poorly on them.
  • Tool-use motivations are categorized into real-time or external data, specialized inputs or outputs, domain-task enhancement, and user customization or interaction.Two experts annotate the motivations of tools in TOOLE.

A.5 OTHERS STATISTICS OF TOOLE

TOOLE combines multiple generation strategies with human review to create a diverse dataset of tool-use queries. Its statistics emphasize short, varied user inputs and comparisons with earlier datasets.

  • 29,000 generated entries were reduced to 20,881 curated TOOLE entries after human review.
  • Most TOOLE queries contain 40 words or fewer, matching typical daily question lengths.
  • TOOLE emphasizes diversity resembling real user scenarios, including variation in expression style, mood, and level of detail.
  • The dataset adopts concise user-facing tool descriptions rather than verbose machine-readable descriptions.The latter may focus on prompting instructions for tool inputs and outputs that are not pertinent to the benchmark.
  • The dataset uses direct diverse, emotional, keyword, and details generation techniques to produce queries.

B TEST SET FOR EVALUATION ON THE AWARENESS OF TOOL USAGE

The tool-usage awareness test distinguishes queries that require tools from those LLMs can solve directly. Positive and negative samples are selected through human evaluation and model checking, with ambiguous intermediate cases discarded.

  • The dataset combines positive TOOLE queries with equal-sized negative samples from daily-conversation and common-sense datasets.
  • Positive samples require external tools, while negative samples can be answered directly by LLMs without tool assistance.
  • Queries are categorized as tool-required, LLM-solvable, or intermediate cases that current models cannot consistently solve.
  • Two human experts determine samples for tool-required and LLM-solvable categories through unanimous agreement.
  • For intermediate queries, GPT-4 refusals or failures are treated as positive tool-required cases.
  • Remaining queries are evaluated across eight LLMs, and only those solved well by all models are labeled negative.Queries solved by only some models are discarded.

C.1 MODELS AND TEST SAMPLES

The experiments evaluate eight commonly used LLMs under a fixed temperature and apply answer-matching procedures for tool-awareness and tool-selection outputs.

  • Models and test samples: Eight commonly used LLMs were evaluated with temperature uniformly set to 0.0.The models included ChatGPT, ChatGLM2, Llama2, Vicuna, Baichuan2, and Koala variants.
  • Evaluation procedures: Tool-usage-awareness outputs were normalized by matching responses to yes or no.Predefined rules handled explicit affirmative, negative, and necessity phrases, while uncovered cases were analyzed manually.
  • Evaluation procedures: Tool-selection outputs were evaluated by matching tool names against the ground-truth label or multi-tool criteria.Single-tool cases allowed None or one matched tool, whereas multi-tool cases required at least two matches; some ambiguous cases received manual analysis.

C.3 TASK COMPARISON

The paper compares four tool-selection subtasks using scenario-specific sampling and reports both LLM and human comparison results.

  • Task comparison: Table 12 compares the four tool-selection subtasks to clarify their differences.The comparison is presented as an overview of the task designs.
  • Results reporting: Table 13 reports CSR percentages for zero-shot and five-shot settings across different scenarios.It defines Δ as the CSR change between five-shot and zero-shot conditions.
  • Sampling design: Subtasks 1 and 2 each use five randomly sampled tools with one corresponding query per tool.The same sampling method was applied to both subtasks.
  • Sampling design: Subtask 3 balances None and tool answers at either a 2:3 or 3:2 ratio.This balancing controls the relative frequency of the two answer types.
  • Results reporting: Table 14 compares human performance with the best and average CSR of eight LLMs.ModelMAX denotes the best LLM performance and ModelAVG the average across the eight models.

C.5 HUMAN EVALUATION

The human evaluation uses questionnaires mixing four tool-selection subtasks, while the dataset-generation prompts construct diverse single-tool and multi-tool queries.

  • Human evaluation: Human performance was evaluated through questionnaires containing mixed questions from all four tool-selection subtasks.Participants selected zero to two tools using the query and candidate-tool options.
  • Human evaluation: 240 valid responses were collected from questionnaires containing 10 or 15 questions.The results are reported in Table 14.
  • Human evaluation: Human CSR exceeded both average and maximum LLM CSR in subtasks 1, 2, and 3.The passage highlights a particularly large difference in subtask 3, where human performance reached 96% versus 9% for models.
  • Human evaluation: In subtask 4, human performance exceeded average LLM CSR but remained below the maximum LLM CSR.The comparison concerns the relative performance levels reported for this multiple-choice-style task.
  • Direct diverse generation: Direct diverse generation prompts request ten distinct user inputs based on descriptions of users and ChatGPT.The prompt frames the plugin as helping ChatGPT answer user questions.
  • Details diverse generation: Details diverse generation prompts request five user inputs that would prompt plugin use to enhance responses.The prompt asks for examples grounded in a plugin description and its intended responsiveness.
  • Keyword-based generation: Keyword-based generation first asks for five short tool labels, then uses labels to generate five prospective user sentences.The two prompt formats connect label extraction with example-query generation.
  • Multi-tool query generation: The multi-tool generation prompt asks for five user queries requiring two specified tools simultaneously.It supplies both tool names and descriptions and restricts the output to what users would say.

D.2 PROMPT TEMPLATE OF EXPERIMENTS

The experimental prompts separate tool-usage awareness from single-tool and multi-tool selection, and the failure cases illustrate confidence and capability errors in LLM tool decisions.

  • D.2 PROMPT TEMPLATE OF EXPERIMENTS: The experimental prompt template distinguishes a Thought part for tool-use awareness from an Action part for tool selection.The sections below instantiate these components for different task settings.
  • D.2 PROMPT TEMPLATE OF EXPERIMENTS: The awareness prompt asks whether external tools are necessary for a user query and requires a brief explanation.It provides several reasons for possible tool use before requesting a yes-or-no response.
  • D.2 PROMPT TEMPLATE OF EXPERIMENTS: The single-tool action prompt requires selecting at most one applicable tool or None from a named tool list.The model must also provide a brief explanation of its choice.
  • D.2 PROMPT TEMPLATE OF EXPERIMENTS: Table 15 presents examples of tool-usage awareness on ChatGPT.The table is part of the experimental prompt-template appendix.
  • D.2 PROMPT TEMPLATE OF EXPERIMENTS: The multi-tool task is identified separately from the single-tool action prompt.Its placement in the prompt template marks a distinct multi-tool selection setting.
  • E FAILURE CASE STUDY: Failure cases show that LLMs can be overconfident about capabilities such as reminders and internet searches or lack confidence in tool use.The examples are presented as evidence of confidence-related tool-selection errors.
  • E FAILURE CASE STUDY: Table 16 provides examples of tool-usage awareness on Vicuna.It complements the ChatGPT examples in the failure-case materials.
Loading 2310.03128v6…