Source-linked AI summary
ToolScan: A Benchmark for Characterizing Errors in Tool-Use LLMs
Shirley Kokane, Ming Zhu, Tulika Awalgaonkar, Jianguo Zhang, Thai Hoang, Akshara Prabhakar, Zuxin Liu, Tian Lan, Liangwei Yang, Juntao Tan, Rithesh Murthy, Weiran Yao, Zhiwei Liu, Juan Carlos Niebles, Huan Wang, Shelby Heinecke, Caiming Xiong, Silivo Savarese
TL;DR
Many tool-use benchmarks report success rates without explaining failure cases, limiting insight into LLM errors. TOOLSCAN offers a diverse benchmark and framework for identifying seven error patterns; its evaluation finds these patterns across leading models, with API-focused fine-tuning and feedback associated with more reliable function calling.
Problem
Method
TOOLSCAN combines a benchmark spanning 10 environment categories and over 30 tasks with a framework for analyzing seven error patterns and a 150-query human-annotated dataset.
Source: §1 · p. 2Results
Leading LLMs exhibit distinct tool-use failure patterns; API-call-fine-tuned models are generally less prone to hallucinating function and argument names, and insufficient API calls are the most common error.
Source: §8 · p. 6Takeaways & Limitations
The analysis can help developers identify model weaknesses and focus improvements on those areas, while the paper concludes that feedback and output relevance can guide more accurate function-calling behavior.
Source: §9 · p. 9
Abstract
from arXiv · showhide
Evaluating Large Language Models (LLMs) is one of the most critical aspects of building a performant compound AI system. Since the output from LLMs propagate to downstream steps, identifying LLM errors is crucial to system performance. A common task for LLMs in AI systems is tool use. While there are several benchmark environments for evaluating LLMs on this task, they typically only give a success rate without any explanation of the failure cases. To solve this problem, we introduce TOOLSCAN, a new benchmark to identify error patterns in LLM output on tool-use tasks. Our benchmark data set comprises of queries from diverse environments that can be used to test for the presence of seven newly characterized error patterns. Using TOOLSCAN, we show that even the most prominent LLMs exhibit these error patterns in their outputs. Researchers can use these insights from TOOLSCAN to guide their error mitigation strategies.
1 INTRODUCTION
Tool-use benchmarks often measure success without explaining failures, leaving distinct model errors difficult to diagnose. TOOLSCAN addresses this gap with a broad benchmark, seven-pattern analysis framework, and human-annotated query set.
1 INTRODUCTION
- Existing benchmarks cover varied tool-use environments, but most report success rates without explaining failures; some also have limited dataset scope.The introduction notes that TOOLBENCH spans varied tasks, tools, and categories, yet still assesses whether the final outcome was achieved.Source: §1 · p. 1
- TOOLSCAN covers 10 environment categories and over 30 tasks to provide detailed diagnostic feedback on LLM tool-use tasks.Its categories include Tools, Movies, Travel, Sports, Entertainment, Data, Social, Media, Weather, and Video Images.Source: §1 · p. 2
- The benchmark identifies seven recurring tool-use error patterns and applies a unified analysis and feedback framework across agents and environments.The feedback mechanism is tailored to fundamental tool-calling criteria and is intended to help agents refine their actions.Source: §1 · p. 2
- Its 150 human-annotated queries highlight multiple pathways to an objective and are used to identify error patterns in leading LLMs.The paper reports a case study demonstrating TOOLSCAN on several leading models.Source: §1 · p. 2
2 RELATED WORK
Prior benchmarks examine tool interaction, reasoning, and planning, but their evaluation can miss action errors or misjudge progress. TOOLSCAN emphasizes the quality of generated actions and their constraints.
2 RELATED WORK
- GORILLA, AGENTBOARD, TOOLBENCH, TOOLEYES, and MINTBENCH examine tool interaction, reasoning, and planning, with differing strengths in multi-turn or diverse-query settings.GORILLA also offers some error-pattern analysis but has limited task diversity.Source: §2 · p. 2
- Evaluations may overlook models’ existing knowledge and penalize redundant sub-goals when they rigidly emphasize sub-goal order.Such rigidity can inaccurately assess a model’s true task-completion capability.Source: §2 · pp. 2–3
- The query-generation workflow feeds queries and API information from open-source toolsets to an LLM, which generates augmented queries and required API calls.Figure 1 describes this generation flow.Source: Figure 1 · p. 3
- TOOLSCAN instead assesses action quality against query constraints, including hallucinated API or argument names, repetitive calls, and invalid formats.The paper argues that quantifying these issues gives deeper insight into needed model improvements.Source: §2 · p. 3
3 ERROR PATTERNS IN TOOL-USE
The paper characterizes tool-use errors as systematic failures in function-call selection and arguments. Because small call errors can change downstream results, it defines seven error types for analysis.
3 ERROR PATTERNS IN TOOL-USE
- Tool-use errors are understudied relative to general LLM output errors, despite outputs requiring function calls and their necessary arguments.The paper illustrates a web-search call using a query and top-k arguments.Source: §3 · p. 3
- A slight argument change, missing argument, or incorrect tool call can produce drastically different and incorrect downstream results.The authors use this brittleness to motivate systematic characterization of tool-use errors.Source: §3 · p. 3
- The seven types are insufficient API calls (IAC), incorrect argument value (IAV), incorrect argument name (IAN), incorrect argument type (IAT), repeated API calls (RAC), incorrect function name (IFN), and invalid format error (IFE).IAV also includes omitting required arguments; RAC refers to repeating the exact same API call.Source: §3 · p. 3
4 TOOLSCAN DATASET
TOOLSCAN builds complex tool-use queries by augmenting seed queries and filtering the resulting set. Its dataset-generation process combines constraint changes, sentence transformations, and added irrelevant information.
4 TOOLSCAN DATASET
- Table 2 presents examples of error patterns identified by comparing ground-truth labels with model outputs.The caption defines GT as Ground Truth Labels and MO as Model Output.Source: Table 2 · p. 4
- Seed queries for each API environment come from benchmark tests in TOOLBENCH and AGENTBOARD before augmentation.The process is also depicted in Figure 1.Source: §4 · p. 4
- Constraint-based augmentation uses GPT-4 prompt engineering to add multiple arguments and their value options to an original query.The prompts include detailed descriptions of each argument and corresponding choices.Source: §4 · p. 4
- Sentence-transformation augmentation asks GPT-4 to rephrase seed queries while preserving their context and requirements.Source: §4 · p. 4
- Relevant-data augmentation inserts unnecessary information around query keywords while retaining the original context.Source: §4 · p. 4
- The authors generate 600 queries, then filter for feasibility, constraint validity, and diversity, leaving about 150.Infeasible queries, such as those with incorrect function names or arguments, are removed.Source: §4 · p. 4
5 FRAMEWORK
TOOLSCAN models tool interaction as a deterministic agent-environment workflow, where actions follow from instructions, a query, available APIs, and the model policy. A feedback mechanism checks actions before execution and returns observations to guide the next step.
5 FRAMEWORK
- Table 3 compares sampled queries, interactions, and APIs across environments.The accompanying text describes the benchmark environments as deterministic, so agent trajectories depend on the model policy and selected actions.Source: Table 3 · p. 4, §5 · p. 4
- Agents receive an instruction, query, and API list, then use the model policy to choose actions that produce deterministic state transitions.The interaction is framed as a finite-horizon POMDP with a goal, states, actions, transitions, and observations.Source: §5 · pp. 4–5
- The framework checks generated actions for parsing, action-space, argument-name, and argument-type errors before executing valid actions in the environment.The checks provide available actions or argument descriptions and required types when relevant errors are found.Source: §5 · p. 5
- After an error-free action executes, its resulting observation is returned to the model to inform subsequent actions.The workflow's figure describes an agent assessing prior information and a predefined goal to choose its next action.Source: §5 · p. 5, Figure 2 · p. 5
6 METRICS
TOOLSCAN defines error metrics for API-call errors and trajectory-based errors, alongside query success rate. Higher error-metric scores indicate fewer issues for the corresponding pattern.
6 METRICS
- For IFN, IAN, IAT, IFE, and RAC, the metric is the percentage of API calls executed without the respective error.The per-query error count e_i is the number of API calls with that pattern in query q_i.Source: §6 · p. 6
- For IAC and IAV, the metric compares API-call errors with each query’s possible ground-truth trajectories.G_i is the set of possible trajectories for query q_i, and the count is of calls with the error pattern relative to a ground-truth trajectory.Source: §6 · pp. 5–6
- The benchmark calculates success rate per query by checking whether the model completes it successfully.The query set is Q; N denotes the maximum steps permitted for a query, and G_i denotes its ground-truth labels.Source: §6 · pp. 5–6
- Higher scores on the error metrics mean fewer issues in the corresponding error pattern because their calculations are directly proportional to success rate.The stated interpretation applies to the error metrics, not as a separate ranking of models.Source: §6 · p. 6
7 EXPERIMENTS
The experiments assess several performant LLMs and report that models that perform well on TOOLBENCH nevertheless show different failure patterns on TOOLSCAN. The evaluation uses one-shot examples and task-specific prompts with structured-format instructions.
7 EXPERIMENTS
- Models that perform well on TOOLBENCH exhibit different failure patterns when evaluated with TOOLSCAN.The assessed models include GPT-4, GPT-3.5-Turbo, Meta-Llama3-8b, DeepSeek-R1-Distill models, Code-Llama-13B, Vicuna-13B-16k, Mistral and Mixtral models, and xLAM.Source: §7 · p. 6
- The prompt setup uses a one-shot in-context example, task-specific instructions, and precise format instructions for structured, parsable API calls.Chat-optimized versions are used when available; multiple rounds per API call help account for server inconsistencies.Source: §7 · p. 6
8 RESULTS
GPT-4 achieved the highest success rate among evaluated models, while TOOLSCAN identified distinct error patterns across models and tool-use conditions. The results also show that irrelevant constraints, missing feedback, incorrect formatting, and structurally similar API names are associated with lower performance.
8 RESULTS
- API-calling fine-tuned models were generally less prone to hallucinating function and argument names than chat-focused models such as Vicuna.The authors attribute the difference to API-call-specific tuning.Source: §8 · p. 6
- Insufficient API Calls (IAC) was the most common error, often because models stopped after part of a multi-question query or confused similar APIs.Models could ignore constraints and select a function that omitted required information, such as a date.Source: §8 · p. 6
- Smaller untuned models sometimes repeated failing API calls, while argument-value and argument-type errors included ignored required arguments and string–integer confusion.The authors report a correlation between repeated calls and insufficient calls, and note that argument errors vary by model.Source: §8 · pp. 6–7
- GPT-4 achieved the highest success rate among all models evaluated.Table 4 reports success rate alongside percentages of API calls without each error metric.Source: §8 · p. 7, Table 4 · p. 7
8.1.1 IMPACT OF IRRELEVANCE ON MODEL PERFORMANCE
- Introducing irrelevant constraints increased Incorrect Function Name (IFN) errors across the examined settings.The analysis examined success rate and fictitious API-call tendencies across two environments.Source: §8.1.1 · p. 7
8.1.2 IMPACT OF FEEDBACK ON MODEL PERFORMANCE
- With feedback, models improved subsequent interactions; without it, they often repeated errors and struggled to recover from initial mistakes.Feedback addressed API names, argument selection, and expected argument types.Source: §8.1.2 · p. 8, Figure 3 · p. 8
8.1.3 IMPACT OF OUTPUT FORMAT METHOD ON MODEL PERFORMANCE
- Incorrect Format Errors (IFE) were associated with declining performance in the function-calling task, which the authors describe as more complex than the benchmarks in Tam et al. (2024).The task has a larger API space and more varied function calls, argument names, and argument types.Source: §8.1.3 · p. 8, Figure 4 · p. 8
8.1.4 IMPACT OF API COMPLEXITY ON MODEL PERFORMANCE
- Environments with structurally similar API names were associated with lower accuracy, particularly on Insufficient API Calls, as models selected incorrect APIs.The analysis measured API-name similarity using Jaccard similarity after removing tokens shared across names within an environment.Source: §8.1.4 · p. 9, Figure 5 · p. 9
9 CONCLUSION
The conclusion presents TOOLSCAN as a tool-use benchmark focused on error patterns and feedback, and argues that targeted analysis can help developers improve model effectiveness. The appendix passages describe query augmentation, verification, environment selection, and selection of 150 base queries.
9 CONCLUSION
- Task-specific API-call tuning, larger models such as GPT-4, and TOOLSCAN feedback, relevance, and unstructured output are identified as factors associated with improved accuracy or reliability.The conclusion says API-call-tuned models are more reliable and reduce hallucinations compared with chat-oriented models.Source: §9 · p. 9
- Analyzing error metrics can help developers identify model weaknesses and focus improvements; future work will add environments with multiple families of actions.The conclusion says this targeted approach can enhance overall performance and reliability.Source: §9 · p. 9
A.1 PROMPTS FOR QUERY AUGMENTATION
- The appendix introduces the prompts used to generate queries, with separate approaches for constraint-based, sentence-transform, and irrelevance augmentation.The supplied passages identify the prompt sections but provide no further description of the sentence-transform or irrelevance procedures.Source: §A.1 · p. 11, §A.1.1 · p. 11, §A.1.2 · p. 12
A.1.1 CONSTRAINT BASED AUGMENTATION
- Constraint-based augmentation uses environment-specific few-shot examples in its query-generation prompt.The passage describes a sample prompt for the spaceflight environment and says each new environment has its own few-shot examples.Source: §A.1.1 · p. 11
A.1.4 QUERY DATA VERIFICATION
- Generated queries are refined through feasibility and constraint checks, followed by manual review of relevance, diversity, difficulty, and representativeness.An LLM proficient in function calling generates the tool calls used to evaluate and refine queries; the checks ensure API compatibility and executable arguments.Source: §A.1.4 · p. 12
A.2 ENVIRONMENTS SELECTION FOR TOOLSCAN
- The authors curate environment-specific functions and parameter requirements using open-source APIs.Source: §A.2 · p. 12
B.1 QUERY SELECTION FOR TOOLSCAN
- 150 feasible base queries are randomly selected, with duplicate or closely related augmented queries excluded.Source: §B.1 · p. 12
B.1.1 QUERIES SELECTED FOR FEEDBACK ABLATION STUDY
The appendix describes query sets used for four ablations, along with references to the benchmark inference prompt and a sample feedback trajectory. The studies use 150 queries for feedback and output-format comparisons, 60 for irrelevance, and 10 per environment for API complexity.
B.1.1 QUERIES SELECTED FOR FEEDBACK ABLATION STUDY
- The feedback ablation uses the same 150 queries and focuses on incorrect argument names, argument types, and function names.Source: §B.1.1 · p. 12
B.1.2 QUERIES SELECTED FOR IRRELEVANCE ABLATION STUDY
- The irrelevance ablation uses 60 queries from Spaceflight, Movie, and Patent environments.Source: §B.1.2 · p. 13
B.1.3 QUERIES SELECTED FOR OUTPUT FORMAT ABLATION STUDY
- The output-format ablation compares structured JSON with no JSON on the same 150 queries, reporting incorrect-format effects and drops in other error metrics.Source: §B.1.3 · p. 13
B.1.4 QUERIES SELECTED FOR API COMPLEXITY ABLATION STUDY
- For this study, 10 queries per environment are randomly collected from the original 150 base queries to maintain unanimous averaging of scores across all provided environments.Source: §B.1.4 · p. 13
B.2 PROMPT FOR BENCHMARK INFERENCE
- The appendix identifies Table 8 as a sample prompt used during benchmark inference.Source: §B.2 · p. 13
B.3 SAMPLE TRAJECTORY OF THE BENCHMARK
- Table 9 is identified as a sample benchmark trajectory that highlights feedback provided during query execution.Source: §B.3 · p. 13
B.4 DETAILS ON API SIMILARITY IMPACT ON MODEL PERFORMANCE
The passages describe measuring API-name similarity by comparing function-name pairs after refinement and averaging their pairwise similarities. A sample trajectory also shows argument and type errors before a valid call returns results.
B.4 DETAILS ON API SIMILARITY IMPACT ON MODEL PERFORMANCE
- For the Patent example, function names are stripped of common tokens and represented in shortened forms such as “patent title” and “patent title date.”Source: §B.4 · p. 13
- The method calculates Jaccard similarity for every unique function-name pair after refinement, then averages those pairwise values.The equation defines Jaccard similarity as intersection over union; the passage identifies the number of unique pairs used in the average.Source: §B.4 · pp. 13–14
- The sample trajectory moves from an invalid argument name and an incorrect function or argument type to a call with an integer page_num that returns sports results.Source: §B.4 · p. 18