PaperlayerView exact revision on arXiv

Source-linked AI summary

ToolTalk: Evaluating Tool-Usage in a Conversational Setting

Nicholas Farn, Richard Shin

arXiv:2311.10775v1cs.CLcs.AIcs.LG

TL;DR

LLMs can converse naturally but remain limited in accessing outside knowledge, computation, and interaction with the world, motivating evaluation of tool-using assistants. ToolTalk provides a conversational benchmark with simulated tools and an evaluation method that accounts for side effects. GPT-3.5 and GPT-4 achieve 26% and 50% conversation-level success, respectively, and the paper identifies recurring error categories.

  • Problem

    LLMs lack access to knowledge outside their training data, have limited mathematical and computational abilities, and cannot otherwise interact with the outside world, motivating evaluation of assistants augmented with tools.

    Source: §1 · p. 1

  • Method

    ToolTalk evaluates multi-step tool use in dialogue with 28 tools across 7 plugins, simulated implementations, and an evaluation method that distinguishes side-effecting actions.

    Source: §1 · pp. 1–2

  • Results

    GPT-3.5 and GPT-4 achieve conversation-level success rates of 26% and 50%, respectively, on ToolTalk.

    Source: §1 · p. 2

  • Takeaways & Limitations

    ToolTalk’s inclusion of action tools reflects that such tools are necessary for an assistant to automate users’ tasks.

    Source: §1 · p. 2

    The evaluation distinguishes action tools because incorrect calls can be more consequential, and not all actions are reversible.

    Source: §3.3 · p. 5

Abstract

from arXiv · show

Large language models (LLMs) have displayed massive improvements in reasoning and decision-making skills and can hold natural conversations with users. Many recent works seek to augment LLM-based assistants with external tools so they can access private or up-to-date information and carry out actions on behalf of users. To better measure the performance of these assistants, this paper introduces ToolTalk, a benchmark consisting of complex user intents requiring multi-step tool usage specified through dialogue. ToolTalk contains 28 tools grouped into 7 plugins, and includes a complete simulated implementation of each tool, allowing for fully automated evaluation of assistants that rely on execution feedback. ToolTalk also emphasizes tools that externally affect the world rather than only tools for referencing or searching information. We evaluate GPT-3.5 and GPT-4 on ToolTalk resulting in success rates of 26% and 50% respectively. Our analysis of the errors reveals three major categories and suggests some future directions for improvement. We release ToolTalk at https://github.com/microsoft/ToolTalk.

1 INTRODUCTION

LLMs can converse fluently but lack access to outside knowledge, have limited mathematical and computational abilities, and cannot otherwise interact with the world. ToolTalk is a conversational benchmark for evaluating assistants’ multi-step tool use, including actions, with GPT-3.5 and GPT-4 results and error analysis.

1 INTRODUCTION

  • ToolTalk contains 78 conversations and 178 turns using 28 tools in 7 categories, with an evaluation methodology tailored to measuring accurate tool use.The benchmark targets realistic, challenging conversations for evaluating tool-using assistants.Source: §1 · p. 1
  • The benchmark includes side-effecting “action tools,” such as sending emails or changing calendar events, alongside tools that only query information.The paper states that action tools are necessary for assistants to automate users’ tasks.Source: §1 · p. 2
  • ToolTalk uses tool invocation recall and incorrect action rate within turns, plus conversation-level success, to account for differences between action and non-action tools.Incorrect action calls may have particularly negative effects, while extraneous non-action calls may still provide useful information.Source: §1 · p. 2
  • GPT-3.5 and GPT-4 achieve conversation-level success rates of 26% and 50%, respectively, and both can hallucinate arguments, misunderstand documentation, or claim task completion without calling tools.The paper presents these results as evidence that conversational tool use remains difficult even for these models.Source: §1 · p. 2
  • The paper contributes a conversational dataset with ground-truth tool-call annotations, multi-turn intents using multiple tools, an evaluation method distinguishing side-effecting tools, and GPT-3.5/GPT-4 error analysis.The reported error analysis identifies hallucinated arguments and misunderstood documentation among the issues.Source: §1 · p. 2

2 DATASET DESIGN

ToolTalk models personal assistants configured with purpose-centered plugins, each containing tools, and supplies simulated implementations and databases. Its conversations combine manually written hard multi-tool scenarios with easier examples involving a single tool call.

2.1 PLUGINS AND TOOLS

  • ToolTalk’s plugins are designed for a personal-assistant setting, with each plugin organized around a purpose and containing multiple related tools.The paper gives calendar management as an example, with tools to create, search, modify, and delete events.Source: §2.1 · p. 2
  • The dataset defines 7 plugins containing 28 tools, and provides each tool with documentation, a simulated implementation, and mock databases for evaluation.Plugins group related functions, such as calendar management; tools are individual functions such as creating an event.Source: §2.1 · pp. 2–3

2.2 CREATING CONVERSATIONS

  • GPT-4 generated roughly 400 candidate scenarios from subsets of three plugins, prompting for scenarios involving at least five tool calls and coverage of a specified tool.The authors sampled candidates across tools, discarded unsuitable scenarios, and wrote the selected conversations by hand.Source: §2.2 · p. 3
  • The 50 hard conversations contain at least three tool calls and pair each user utterance with ground-truth calls, expected results, and assistant responses; 28 easy conversations each end in one tool call.Together, the hard and easy versions comprise 78 conversations; the simulated databases were set up to return the ground-truth values.Source: §2.2 · p. 4

3 EVALUATION METHODOLOGY

ToolTalk evaluates assistants by running them on conversation prefixes, executing their predicted calls, and comparing those calls with ground truth. Its correctness criteria account for tool type and arguments, while conversation success requires complete ground-truth coverage without incorrect actions.

3 EVALUATION METHODOLOGY

  • Evaluation runs an assistant on each conversation prefix ending in a user utterance, executes its predicted calls, and compares them with the ground-truth calls for that prefix.The assistant may use the execution results to decide whether to make further calls or respond.Source: §3 · p. 4

3.1 TOOL CALL CORRECTNESS

  • For action tools, predicted calls are matched to ground truth using tool-specific equivalence functions; free-form text arguments may be judged by embedding similarity above 0.9.The matching rules can treat different argument orderings as equivalent, and optional arguments absent from ground truth are disregarded.Source: §3.1 · p. 4

3.2 CONVERSATION SIMULATION

  • Conversation simulation resets the world, supplies prior turns and the current user utterance, then executes predicted calls one at a time until the assistant replies.The process provides execution feedback to the assistant after each predicted tool call.Source: §3.2 · pp. 4–5

3.3 INCORRECT ACTIONS

  • An incorrect action is an executed action-tool call that matches no ground-truth call; incorrect actions are tracked because such calls can affect the outside world and may not be reversible.The evaluation excludes calls that error, including calls with incorrect argument counts or types.Source: §3.3 · p. 5

3.4 METRICS

  • ToolTalk reports precision, recall, and incorrect action rate, and defines conversation success as matching all ground-truth calls with no incorrect actions.Success rate is the key metric; precision measures efficiency by indicating unnecessary predicted calls.Source: §3.4 · pp. 5–6

4 EXPERIMENTS AND ANALYSIS

The experiments evaluate GPT-3.5 and GPT-4 with all 28 tools available, then analyze their failures and test performance without tool documentation. GPT-4 performs better overall, but both models still struggle with conversational tool use.

4.1 EXPERIMENTS

  • 85.7% and 92.8% success rates on easy ToolTalk, versus 26.0% and 50.0% on hard ToolTalk, for GPT-3.5 and GPT-4 respectively.GPT-4 outperforms GPT-3.5 but has similar incorrect action rates; its precision indicates greater efficiency.Source: §4.1 · p. 6

4.2 ANALYSIS

  • Both models fail through premature tool calls, poor planning, or incorrect or missing arguments, with GPT-3.5 more susceptible.The evaluation supplies documentation for all 28 tools and simulates conversations in ToolTalk’s easy and hard subsets.Source: §4.2 · p. 6, §4.1 · p. 6
  • Premature calls often hallucinate missing argument values; this can directly cause failure for action tools, although both models generally choose the right tools.The authors attribute premature calls to failure to recognize when clarification is needed; faulty planning also leads to omitted or wrong tools.Source: §4.2 · pp. 6–7
  • Planning failures include hallucinating a recipient instead of querying for an email address, or forgetting one of several requested actions.In severe cases, models hallucinate tools or claim task completion without predicting any tool use.Source: §4.2 · p. 7
  • Incorrect arguments can reflect misunderstood documentation or tool outputs, or weak mathematical skills, even when the model selects the right tool.Examples include malformed time values, miscalculated event durations, and passing a newly created reminder to a deletion tool.Source: §4.2 · p. 7
  • GPT-4 produces fewer errors in every category and generally identifies all required tools more often, but shared failures often have similar causes.Error types are determined automatically by comparing predicted and ground-truth tool calls for each turn.Source: §4.2 · p. 7
  • The authors suggest self-reflection or grounding for argument values and minimizing GPT-3.5 tool arguments to improve tool use.They attribute the latter suggestion to GPT-3.5’s greater difficulty recovering from errors and tendency to give up.Source: §4.2 · p. 7

4.3 EXPERIMENT REMOVING DOCUMENTATION

  • Removing tool and parameter descriptions significantly reduces performance except for incorrect action rate; faulty planning accounts for most errors without documentation.Both models may repeat incorrectly formatted arguments after execution errors, then give up and miss remaining ground-truth tool calls.Source: §4.3 · p. 7

5 RELATED WORK

Related work spans tool use in traditional tasks, autonomous agent benchmarks, tool-use datasets, and task-oriented dialogue. ToolTalk is positioned around evaluating multi-step tool use through conversation with automated evaluation.

5 RELATED WORK

  • The paper frames prior evaluation around dialogue, complex multi-step tool use, action tools, and fully automated assessment.Table 5 compares prior work and ToolTalk along these criteria.Source: §5 · p. 8, Table 5 · p. 9
  • Tool augmentation includes applying tools to traditional tasks such as question answering and mathematical reasoning, as well as automation such as robotics or web browsing.The paper notes that traditional tasks may not distinguish internal knowledge from tool use and do not require tools that affect the external world.Source: §5 · p. 8
  • Common agent benchmarks cover varied autonomous environments, but the paper says they lack conversational interaction and generally use a single user utterance.ToolTalk conversations instead include multiple utterances and intermediary tasks.Source: §5 · p. 8
  • Many tool-use datasets require manual evaluation, use unrealistic or simple queries, or treat utterances in isolation rather than as dialogue.The passage identifies Li et al. (2023) as an exception to the isolation pattern.Source: §5 · pp. 8–9
  • Task-oriented dialogue datasets overlap in realistic dialogue but usually omit tools, multi-intent prediction, and execution feedback.TicketTalk includes a simulated movie-booking API, but it lacks execution feedback and permits loosely specified arguments.Source: §5 · p. 9

6 CONCLUSION

ToolTalk evaluates multi-tool orchestration in conversations using simulated tools and a tool-sensitive correctness methodology. GPT-3.5 and GPT-4 error analysis identifies three failure categories, while the authors point to broader dataset coverage and API redesign as future directions.

6 CONCLUSION

  • ToolTalk combines complex conversational orchestration, simulated implementations of all tools, and correctness evaluation sensitive to tool-specific errors and incorrect actions.The benchmark evaluates GPT-3.5 and GPT-4 and analyzes their errors.Source: §6 · p. 9
  • The analysis groups assistant errors into premature tool calls, faulty reasoning, and incorrect invocations of the correct tool.Source: §6 · p. 9
  • The authors identify more conversations, greater plugin diversity, and redesigning existing API interfaces for LLMs as future directions.Source: §6 · p. 10

7 REPRODUCIBILITY

The paper documents model versions and prompt and API details to support reproducibility.

7 REPRODUCIBILITY

  • The paper specifies the GPT-3.5 and GPT-4 API versions, scenario-generation prompt location, and system-prompt and Chat completions API details.Source: §7 · p. 10

A COMPLETE LIST OF TOOLS

ToolTalk's tools cover account management, alarms, calendars, email, messages, reminders, and weather, including both information retrieval and actions.

A COMPLETE LIST OF TOOLS

B SCENARIO PROMPT

The scenario prompt asks for varied, detailed user scenarios that exercise multiple APIs and meet a minimum call count. It specifies API-use constraints and a response format.

B SCENARIO PROMPT

  • Each scenario must be complex and detailed, involve at least five API calls and three different APIs, including one explicitly required API.Source: B SCENARIO PROMPT · p. 13
  • The prompt favors APIs used in tandem, such as using a returned playoffs location to search for hotels.Its example describes five calls and allows an API to be reused within a scenario.Source: B SCENARIO PROMPT · pp. 13–14
  • Across scenarios, API combinations must differ, every supplied API must appear at least once, and only supplied APIs may be used.Source: B SCENARIO PROMPT · p. 14
  • The response format calls for three scenarios, and API use is described in parentheses rather than as explicit call statements.Source: B SCENARIO PROMPT · p. 14

C UNREALISTIC QUERIES

The appendix examples show queries that may create complex tool interactions but are unrealistic because they combine unlikely APIs, assume knowledge of API endpoints, or use unnatural phrasing.

C UNREALISTIC QUERIES

  • The examples include requests that explicitly name SQUAKE API endpoints and demand detailed agency contact fields (Qin et al., 2023b).The SQUAKE request asks to check API health and retrieve projects; another seeks an agency’s postal code, email, name, and phone number.Source: C UNREALISTIC QUERIES · p. 14
  • Other examples pair an awkward arithmetic question with a highly specified jacket-shopping request (Yao et al., 2022a).The shopping query specifies size, color, material, warmth, and a price ceiling.Source: C UNREALISTIC QUERIES · p. 14
  • The examples are useful for generating complex tool interactions or unusual tool combinations, despite their unrealistic phrasing.Source: C UNREALISTIC QUERIES · p. 14

D NUANCES COMPARING PRIOR WORK

Prior benchmarks trade off tool-use depth, task complexity, and evaluation method: some use hard QA that does not require tools, while others pair simple automated tasks with complex manually evaluated ones.

D NUANCES COMPARING PRIOR WORK

  • ReAct and ART use hard QA tasks that traditionally do not require tools, while API-Bank meets the comparison criteria but splits simple automated examples from complex manually evaluated ones.API-Bank’s level 1–2 examples are automated but simple; its level 3 examples are complex but require manual evaluation.Source: D NUANCES COMPARING PRIOR WORK · p. 15
  • ToolBench tasks require difficult-to-use tools but have shallow solution paths; its deeper environments reuse WebShop and TableTop datasets (Yao et al., 2022a).Source: D NUANCES COMPARING PRIOR WORK · p. 15

Related papers

Loading 2311.10775v1…