Source-linked AI summary
ToolQA: A Dataset for LLM Question Answering with External Tools
Yuchen Zhuang, Yue Yu, Kuan Wang, Haotian Sun, Chao Zhang
TL;DR
LLM evaluations can conflate memorized knowledge with external-tool use, while hallucination and numerical-reasoning weaknesses motivate better tool assessment. ToolQA addresses this with a benchmark built from non-overlapping reference data, tool-dependent questions, 13 tools, and automated answer generation; existing systems perform substantially worse on hard questions, exposing challenges in tool composition and reasoning.
Problem
Existing evaluations may not distinguish internal memorization from genuine external-tool use because evaluation data can overlap with LLM pre-training data.
Method
ToolQA combines reference corpora selected to minimize pre-training overlap, human-guided template-based question generation, programmatic answer generation, and 13 external tools.
Results
The best tool-augmented model achieved 43.15% on easy questions and 8.2% on hard questions, while internal-knowledge baselines achieved approximately 5% and 2%, respectively.
Takeaways & Limitations
ToolQA provides a benchmark for diagnosing tool-use reasoning, with hard questions revealing limitations in tool composition, incorrect calls, and incorrect data-source use.
Takeaways & Limitations
As problem complexity and tool count increase, few-shot exemplars cannot cover all compositional tool uses, and innovative behaviors may be accompanied by hallucinations.
Abstract
from arXiv · showhide
Large Language Models (LLMs) have demonstrated impressive performance in various NLP tasks, but they still suffer from challenges such as hallucination and weak numerical reasoning. To overcome these challenges, external tools can be used to enhance LLMs' question-answering abilities. However, current evaluation methods do not distinguish between questions that can be answered using LLMs' internal knowledge and those that require external information through tool use. To address this issue, we introduce a new dataset called ToolQA, which is designed to faithfully evaluate LLMs' ability to use external tools for question answering. Our development of ToolQA involved a scalable, automated process for dataset curation, along with 13 specialized tools designed for interaction with external knowledge in order to answer questions. Importantly, we strive to minimize the overlap between our benchmark data and LLMs' pre-training data, enabling a more precise evaluation of LLMs' tool-use reasoning abilities. We conducted an in-depth diagnosis of existing tool-use LLMs to highlight their strengths, weaknesses, and potential improvements. Our findings set a new benchmark for evaluating LLMs and suggest new directions for future advancements. Our data and code are freely available to the broader scientific community on GitHub.
1 Introduction
LLMs can hallucinate and struggle with numerical reasoning, while existing evaluations may confuse memorized knowledge with genuine tool use. ToolQA addresses this gap with tool-dependent questions and reports low performance for internal-knowledge baselines and substantial difficulty for tool-augmented models.
- LLMs demonstrate broad capabilities but remain susceptible to hallucinations and weak numerical reasoning.
- Existing evaluations may not distinguish recalling pre-trained information from genuinely using external tools, partly because evaluation data may overlap with pre-training data.
- ToolQA contains questions answerable only through appropriate tools over external reference corpora, reducing the possibility of answers based solely on internal knowledge.
- ToolQA is curated through reference-data collection, human-guided question generation, and programmatic answer generation, requiring minimal human labeling effort.
- Approximately 5% and 2% success rates were achieved by ChatGPT and Chain-of-thoughts prompting on easy and hard questions, respectively.The best tool-augmented performance was 43.15% on easy questions and 8.2% on hard questions.
2 Related Work
Prior work augments LLMs with retrieval, code-based tools, and multi-tool reasoning, but evaluation has often emphasized individual tools or downstream benchmarks. ToolQA targets multi-tool composition and the logical reasoning required to coordinate different tools.
- Retrieval-augmented methods extract knowledge from sparse or dense corpora, mainly using free text rather than multiple tool types.
- Program-of-Thought, PAL, MathPrompt, and Code4Struct use code-based tools for question answering, especially on tabular and mathematical tasks.
- Other methods incorporate basic tools such as calculators, calendars, and machine translation, while ART, ReAct, and Reflexion generate intermediate reasoning steps and actions.
- Earlier evaluations primarily assessed single-tool usage through downstream performance on question answering, fact-checking, timely-information, and computation-intensive benchmarks.
3 ToolQA Dataset
ToolQA combines context-specific external corpora with 13 tools and an automated pipeline for generating tool-dependent questions and precise answers. Its easy questions require limited extraction, whereas hard questions require multi-step operations and tool composition.
- 3.1 Dataset Details: ToolQA spans 8 domains, and each instance contains a question, answer, reference corpora, and available tools.Reference corpora may be text, tabular databases, or graphs.
- 3.2 Reference Data and Tools: The benchmark provides 13 tools for text retrieval, database operations, code interpretation, mathematical computation, and graph operations.
- 3.2 Reference Data and Tools: Reference corpora are selected to avoid pre-training overlap, provide context-sensitive facts, and contain all information needed to answer the questions.
- 3.2 Reference Data and Tools: The corpora cover temporal, spatial, social, scientific, mathematical, and personal contextual dimensions using sources such as Flights, Coffee, Yelp, Airbnb, GSM8K, SciREX, Agenda, and DBLP.
- 3.3 Human-Guided Question Generation: Human-guided question generation uses LLM-proposed templates, manual validation, and sampled reference-data values to create concrete tool-answerable questions.
- 3.3 Human-Guided Question Generation: Easy questions extract one information piece with fewer tools, while hard questions require multi-piece operations, comparisons, and more complex tool reasoning.
- 3.4 Programmatic Answer Generation: Programmatic answer generation executes tool-specific operators and chains over reference data to produce accurate answers, including for multi-step questions.
4 Experiments
Experiments compare standard and tool-augmented LLMs on ToolQA, finding that tool use improves performance but remains substantially weaker on hard questions. ReAct performs best by iteratively using execution observations, while model scale affects easy and hard questions differently.
- Comparing Different Tool-Use LLMs: ReAct, Chameleon, ChatGPT, and CoT are evaluated on ToolQA, covering standard and tool-augmented question answering methods.ReAct is instantiated with gpt-3.5-turbo and text-davinci-003.
- Comparing Different Tool-Use LLMs: Tool-level demonstrations cover all tools at least once, providing concise usage tutorials within the LLM context limit.
- Comparing Different Tool-Use LLMs: ReAct performs best by using execution-trace observations to generate subsequent actions and iteratively refine its tool-use chain.
- Easy vs. Hard Questions: The best method averages 43.13% success on easy questions but only 8.24% on hard questions requiring more tool calls and complicated compositions.The results indicate that current tool-augmented LLMs struggle with reasoning about hard tasks and generating tool-use plans.
- GPT-3 vs. GPT3.5: ReAct with GPT-3 outperforms ReAct with GPT-3.5 on easy questions, whereas GPT-3.5 performs better on hard questions.The authors hypothesize that GPT-3 better follows demonstrated tool-call formats, while GPT-3.5 better supports reasoning and code understanding for novel solutions.
5 Result Analysis and Discussion
ToolQA error analysis shows that tool-augmented LLMs commonly select incorrect arguments or data sources, while harder questions increase hallucination and long-context errors. ReAct’s error patterns also vary by question difficulty and tool type.
- Argument Errors: 44.56% of ReAct’s 377 easy-question errors and 48.23% of its 436 hard-question errors were argument errors.Argument errors were the most common error type in both settings.
- Argument Errors: ReAct made nearly 10x more errors with code-related tools on hard questions than on easy questions.Database-tool errors instead totaled 120 on easy questions and 95 on hard questions.
- Incorrect Data Source: LLMs frequently selected the wrong reference corpus, confusing temporally related sources such as Flight, Coffee, Airbnb, and Yelp with Agenda.Scientific questions also caused confusion between the SciREX and DBLP corpora.
- Innovation and Hallucination: Innovation can help ReAct identify alternative tool strategies, but it is often accompanied by hallucinations when solving hard Coffee questions.The case study contrasts strict repetition of failed database operations with an innovative shift to SQL interpretation.
- Other Error Types: Other recurring errors include infeasible actions, overly long contexts, misunderstanding observations, and low-quality retrieval.These errors respectively involve unavailable tools, context-length limits, failed interpretation of observations, and failure to retrieve relevant information.
- Other Error Types: Hard questions show slightly higher hallucination and long-context error rates because they often require composing more tools.The overall error distribution remains similar across easy and hard questions.
6 Conclusion
ToolQA is an automated benchmark for evaluating whether LLMs can use external tools to solve complex questions. Experiments found limited hard-question performance and recurring errors in tool calls and data-source selection, motivating future tool-use fine-tuning.
- Conclusion: ToolQA assesses LLMs’ ability to use external tools for solving complex problems.The benchmark is curated through reference data collection, template-based question generation, and programmatic answer generation.
- Conclusion: The automated curation pipeline is general and can expand to additional areas of external knowledge.
- Conclusion: Even the strongest baseline achieved limited performance on ToolQA’s hard questions.
- Conclusion: Current tool-augmented LLMs commonly make incorrect tool calls and select incorrect data sources.
- Conclusion: ToolQA is particularly suited to evaluating multi-tool methods and the logical reasoning needed to compose different tools.
B.1 Different Data Source Introduction
ToolQA uses diverse reference datasets designed to require external information rather than internal recall. Its sources include contemporary, detailed, graph-based, mathematical, and synthetic agenda data.
- Reference Corpora: Flight Status (2022-2023) contains contemporary airline information that is too recent for LLMs’ internal knowledge.
- Reference Corpora: Daily Coffee Price (2000-2022) provides detailed daily coffee prices that are too contemporary for internal knowledge.
- Reference Corpora: Yelp Business Data and Airbnb Open Data contain detailed records from selected metropolitan areas and New York activities, respectively.
- Reference Corpora: DBLP Citation Network V14 constructs author-author and paper-paper graphs from records after 2020.
- Reference Corpora: GSM8k questions are sampled from ChatGPT’s error cases, while SciREX supplies document-level information-extraction data.
- Reference Corpora: Agenda is a synthetic corpus of 10,000 records created by generating names, events, and dates and composing them into virtual-character agendas.
B.2 Generation Details of Agenda Dataset
The Agenda dataset is designed around synthetically generated personal records to reduce privacy risks and pretraining-data overlap. The section also specifies template-based questions across multiple datasets, with harder templates requiring aggregation, comparison, recommendation, or multi-step reasoning.
- Agenda Dataset: The Agenda corpus is automatically synthesized to avoid prior LLM exposure and address privacy concerns surrounding personal data.Each record follows a structured NAME–EVENT–TIME–DATE pattern.
- Easy Questions: Agenda easy questions ask about events, locations, times, durations, and attendees using five templates.
- Hard Questions: Hard templates require multi-step operations such as counting, averaging, ranking, nearest-neighbor search, recommendations, and cross-record comparisons.They span flights, coffee prices, Yelp, Airbnb, SciREX, Agenda, and DBLP datasets.
- Hard Questions: Hard Agenda questions ask about event counts, availability, scheduling windows, named events, and the number of dates containing a person’s scheduled activities.
F.1 Implementation Details
The implementation uses fixed-generation settings and prompt-based demonstrations for tool-use methods. Examples show ReAct, Chameleon, and other workflows loading data, filtering records, retrieving values, calculating results, and finishing with answers.
- Environment: Experiments use Intel i7-5930K CPUs, NVIDIA RTX A5000 GPUs, Python 3.8, Huggingface 4.6.0, and PyTorch 1.10.ChatGPT API generation uses top_p = 1.0 and temperature t = 1.0.
- ReAct Prompts: ReAct examples interleave thoughts, tool actions, observations, calculations, and a final Finish action.The examples include Python-based arithmetic, SciREX retrieval, DBLP graph queries, agenda retrieval, and SQL execution.
- Tool Examples: Agenda examples retrieve a natural-language event description and extract its location, while coffee examples use SQL to return a date-specific trading volume.The coffee example returns volume 10115 for 2000-01-14.
- Chameleon Prompts: The Chameleon prompt defines modules for sequential execution, including Calculate[formula], within a policy model that selects a solvable module sequence.
G.5 Limitation
The paper identifies ToolQA as an expanding direction for tool-augmented LLM research. Future development is intended to add broader task, data, tool, and method coverage.
- Future Scope: ToolQA is planned to expand with more tasks, data, tools, and methods as tool-augmented LLM research develops.