PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Function Calling: Benchmarking Tool-Using Agents under Tool-Environment Unreliability

Yang Tian, Zhengpeng Shi, Yu Zhou, Bo Zhao

arXiv:2606.25819v2cs.CLcs.SE

TL;DR

Tool-use benchmarks have largely assumed clean, stable, trustworthy environments, leaving agent robustness to tool unreliability insufficiently examined. ToolBench-X evaluates multi-step tasks under recoverable hazards, and experiments find substantial agent fragility alongside improvements from guidance and recovery strategies.

  • Problem

    Recent tool-use benchmarks increasingly cover complex tasks but still largely assume clean, stable, trustworthy tools, leaving unreliability insufficiently examined.

    Source: Introduction · p. 1, Problem Formulation · p. 3

  • Method

    ToolBench-X evaluates executable multi-step tasks under five structured tool uncertainties, with each injected instance retaining a valid recovery path to its canonical answer.

    Source: Introduction · p. 1, Benchmark Construction · p. 4

  • Results

    All evaluated models achieved success rates below 60% under hazards, while targeted hints and additional failure-triggered reasoning improved task completion.

    Source: Introduction · p. 2

  • Takeaways & Limitations

    Tool-use evaluation should assess task completion under uncertain tool environments rather than function-call accuracy alone.

    Source: Introduction · pp. 1–2

    A residual gap between Hint and Oracle suggests current models still do not reliably execute some adaptive recovery behaviors.

    Source: Further Analysis · p. 6

Abstract

from arXiv · show

Large language models are increasingly deployed as agents that solve tasks by interacting with external tool environments. Although recent tool-use benchmarks increasingly cover complex task settings, they still largely assume clean, stable, and trustworthy tool environments, leaving tool-environment unreliability insufficiently examined. We introduce ToolBench-X, a benchmark for evaluating agents under recoverable reliability hazards. ToolBench-X contains executable multi-step tasks across diverse domains and sequential, parallel, and mixed workflows, each paired with deterministic tools and a canonical final answer for automatic evaluation. Starting from clean tool environments, ToolBench-X injects five structured hazard types: Specification Drift, Invocation Error, Execution Failure, Output Drift, and Cross-source Conflict. Crucially, each injected instance remains solvable through at least one valid recovery path, such as retrying, fallback, verification, or cross-checking. Experiments reveal a substantial reliability gap: agents that perform well with reliable tools often fail under recoverable hazards. Further analysis shows that failures are driven less by tool-use volume or inference budget than by limited hazard diagnosis and ineffective recovery. Targeted recovery hints recover many failed tasks, while test-time scaling yields more limited gains. These results suggest that tool-use evaluation should move beyond function-call accuracy toward task completion under unreliable tool environments. The code and data is available at https://github.com/Foreverskyou/ToolBench-X.

Introduction

Reliable tool-use evaluation must move beyond correct function calls to task completion when tools are uncertain. ToolBench-X tests agents on recoverable hazards, and results show substantial fragility alongside gains from targeted guidance and recovery strategies.

Introduction

  • Clean-tool evaluations miss real-world hazards such as stale documentation, renamed fields, service timeouts, incomplete outputs, and conflicting tool evidence.Agents may trust suspicious results, retry the wrong call, invent arguments, or finish before evidence is complete.Source: Introduction · p. 1
  • The paper argues that evaluation should measure whether agents can diagnose unreliable outputs, verify evidence, recover from runtime errors, and complete tasks.This shifts the focus from isolated call correctness to robust task completion under structured uncertainty.Source: Introduction · p. 1
  • ToolBench-X provides executable multi-step tasks across sequential, parallel, and mixed workflows, with deterministic tools, canonical answers, and five injected uncertainty types.Each task retains at least one recovery path, including retry, fallback, cross-checking, normalization, or evidence verification.Source: Introduction · p. 1
  • All evaluated models achieved success rates below 60% under reliability hazards.Targeted hints and failure-triggered additional reasoning improve task completion, while many failures reflect limited diagnosis, evidence verification, and recovery.Source: Introduction · p. 2

Related Work

Prior tool-use benchmarks assess sub-capabilities, parallel calls, or multi-turn sequential tasks, while studies of unreliable environments have often used shallow perturbations or limited task settings. ToolBench-X instead evaluates task progress through complex execution cycles under recoverable uncertainty.

Large language models agents

  • Tool use is the action interface through which agents select operations, supply inputs, interpret observations, and incorporate them into later reasoning.This interaction underlies agents operating on websites, code, databases, documents, and other external environments.Source: Related Work · p. 2

Benchmarks for tool-using agents

  • Existing benchmarks range from sub-capability tests to parallel and multi-turn sequential tool-use evaluations.T-EVAL, UltraTool, and MetaTool treat invocation as question answering; BFCL-V1/V2 cover parallel use in single-turn settings, while BFCL-V3 adds multi-turn sequential evaluation.Source: Related Work · p. 2
  • Prior work on unreliable tool environments has largely examined shallow perturbations, simple call chains, or a single type of untrusted environment.The paper positions its benchmark as evaluating uncertainty throughout complete, complex task-execution cycles.Source: Related Work · p. 2

ToolBench-X

Conventional function-calling benchmarks assume clean tool transitions, while real environments can introduce uncertainty that propagates through multi-step tasks. ToolBench-X evaluates executable tasks under structured hazards while preserving a valid path to the canonical answer.

Problem Formulation

  • Clean-transition evaluations miss how unreliable tool observations can disrupt multi-step task completion and change the decisions an agent should make.ToolBench-X addresses this gap by evaluating degradation under hazards and improvements enabled by reliability signals.Source: Problem Formulation · p. 3

Benchmark Construction

  • ToolBench-X builds tasks from seven topic categories, with sequential, parallel, and mixed workflows, executable tools, and canonical clean results.The benchmark pipeline also includes human review of task validity, tool correctness, hazard consistency, and answer reliability.Source: Benchmark Construction · pp. 3–4, Figure 1 · p. 3
  • Five hazards model mismatches between expected and observed tool behavior, including specification drift, invocation errors, execution failures, and output drift.Each type has a recovery approach, from reconstructing calls or retrying to extracting and canonicalizing returned values.Source: Benchmark Construction · p. 4
  • Cross-source conflict challenges multi-source aggregation: the canonical answer is recovered by checking completeness, normalizing outputs, resolving contradictions, and reconciling evidence rather than voting.The benchmark preserves at least one valid recovery path, which may involve retry, fallback, cross-checking, normalization, or verification.Source: Benchmark Construction · p. 4
  • The benchmark spans seven domains and all five hazard types, with balanced sequential, parallel, and hybrid workflows and most tasks using about four tools.Figure 2 summarizes domain, hazard, and number-of-tools distributions.Source: Benchmark Construction · p. 4, Figure 2 · p. 4

Experiments

ToolBench-X evaluations show that reliable tool use remains difficult under injected hazards, with results varying by model, task complexity, and exception type. Further analyses find that recovery depends more on diagnosing hazards and adapting actions than on simply making more calls or using more inference compute.

Main Result

  • Doubao-Seed-2.0-Lite solves only slightly more than half of ToolBench-X tasks, and no model exceeds 0.60 accuracy overall.GPT-5.4 achieves 0.453; open-source Qwen3.5-35B-A3B-Thinking reaches 0.419, improving 4.7 percentage points over its non-thinking counterpart.Source: Main Result · p. 5, Table 2 · p. 5
  • Parallel tasks average 0.421 accuracy, while Output exceptions average 0.581 and Invocation exceptions remain among the dominant failure modes.The Output–Invocation gap exceeds thirty percentage points; accuracy also fluctuates more across exception types than task types.Source: Main Result · p. 5

Further Analysis

  • Targeted hints raise Baseline accuracy by 25.5–35.5 points, recovering roughly 60–80% of the lost accuracy, while a residual Hint–Oracle gap remains.Figure 4 compares the exception-injected Baseline, clean Oracle upper bound, and Hint and Test-time scaling recovery strategies.Source: Further Analysis · p. 6, Figure 4 · p. 6
  • Most models lose accuracy on high-complexity tasks; Doubao-Seed-2.0-Lite falls from 0.577 on low-complexity tasks to 0.462 on high-complexity tasks.The overall decline is not perfectly monotonic across models, and exception type, workflow structure, and call dependencies also affect robustness.Source: Further Analysis · pp. 6–7, Table 3 · p. 6
  • Accuracy is weakly associated with call expansion ratio (Pearson r = 0.326); Doubao-Seed-2.0-Lite scores 0.513 at a ratio of 0.773.GPT-4o and Qwen-3.0-30B-A3B-Instruct make more calls, with ratios of 0.948 and 0.950, yet achieve lower accuracy.Source: Further Analysis · p. 7, Figure 5 · p. 7

Error Analysis

  • Ineffective Continuation accounts for 48.6% of failed trajectories, followed by Early Abandonment at 26.7%.The taxonomy uses final-answer validity, post-error tool continuation, post-error call count, and call expansion ratio; models with similar accuracy can exhibit different failure patterns.Source: Error Analysis · p. 7, Figure 6 · p. 7

Conclusion

ToolBench-X evaluates whether agents can complete tasks while diagnosing and recovering from unreliable tool behavior. Experiments find that current agents remain fragile, with failures tied more to limited anomaly awareness and ineffective recovery than to tool-use volume or inference budget.

Conclusion

  • ToolBench-X evaluates task completion under recoverable tool-environment hazards rather than only performance with reliable tools.It examines agents’ ability to diagnose unreliable behavior, adapt strategies, recover, and complete tasks.Source: Conclusion · p. 7
  • Experiments find that current agents remain fragile, with failures driven more by limited anomaly awareness and ineffective recovery than by tool-use volume or inference budget.The paper points to uncertainty estimation, verification, fallback, and recovery mechanisms as elements of reliability-aware agent design.Source: Conclusion · p. 7
  • ToolBench-X provides a foundation for evaluating and developing trustworthy, resilient tool-using agents in real-world settings.Source: Conclusion · p. 7

Human Validation and Quality Control

The benchmark’s quality control combines human review, automated checks, and consensus resolution. The final dataset contains 1,106 instances, and exact-match evaluation agreed with human semantic judgments on 97% of a sampled set.

Human Validation and Quality Control

  • Five reviewers completed 800 person-hours of validation over 20 working days, with each paid at least the local minimum hourly wage.Source: Human Validation and Quality Control · p. 9
  • Each candidate received primary review, with independent second review for ambiguous or higher-risk cases and discussion to resolve disagreements.Instances without reviewer consensus were removed.Source: Human Validation and Quality Control · p. 9
  • Reviewers checked task clarity, deterministic tool execution, reference-answer consistency, hazard fit, answer leakage, and a valid recovery path.Hazards were assigned only when appropriate to the task, and every retained injected instance had at least one viable recovery route.Source: Human Validation and Quality Control · p. 9
  • Automatic validation required the clean path to produce the reference answer and the injected environment to retain a recovery path to that answer.Source: Human Validation and Quality Control · p. 9
  • 1,106 instances remained in the final benchmark after validation, an 88.5% retention rate from 1,250 candidates.Of the retained tasks, 378 are sequential, 358 parallel, and 370 mixture tasks.Source: Human Validation and Quality Control · p. 9
  • 97 of 100 sampled instances matched between exact-match evaluation and independent human semantic judgments.Disagreements were inspected, and brittle or non-unique reference formats were revised and rechecked.Source: Human Validation and Quality Control · p. 9

Additional Dataset Statistics Details

The final dataset spans three workflow types, seven main topics, and 84 subtopics. Retained counts vary across task-type–subtopic combinations because some candidates failed validation or executability requirements.

Additional Dataset Statistics Details

Detailed Experimental Settings

The evaluation covers seven commercial and seven open-source language models. Tasks use a unified API setup, constrained tool interactions, and strict exact-match scoring.

Detailed Experimental Settings

  • The evaluation includes seven commercial models and seven representative open-source language models.Source: Detailed Experimental Settings · pp. 9–10
  • Models used a unified OpenAI-compatible API with default temperature, up to 10 rounds per task, 8 parallel workers, and 8,192 output tokens.Each request had a 120-second timeout, and each round allowed at most one tool call, retry, fallback, or finish.Source: Detailed Experimental Settings · p. 10
  • A task counted as correct only when the whitespace-stripped output exactly matched its required answer string.API requests were retried up to three times with exponential backoff; persistent failures counted as task failures.Source: Detailed Experimental Settings · p. 10

Case Studies

Six representative trajectory pairs compare success and failure on the same task and injected tool implementation. Hazard labels reflect failure location and required recovery, not just exception type.

Case Studies

  • Six representative no-hint trajectory pairs come from a matched 200-task subset, with each pair comparing success and failure on the same task and injected tool implementation.Source: Case Studies · p. 10
  • Hazard labels are assigned by the semantic location of failure and the recovery path needed to reach the answer, rather than by the surface Python exception alone.Source: Case Studies · p. 10
  • The same exception type can require different recovery behaviors across cases.Source: Case Studies · p. 10

Prompt Templates

This section catalogs prompt templates for task and tool generation, hazard and hint generation, evaluation-time decisions, tool arguments, final-answer fallback, and failure-triggered test-time scaling. Its examples show recovery choices in sequential and parallel tasks under tool failures.

Task Generation Prompts

Tool Generation Prompts

Hazard Injection and Hint Generation Prompts

Policy Decision Prompts

Tool Arguments Prompts

Final-Answer Fallback Prompts

Test-Time Scaling Prompts

  • Sequential examples show recovery by continuing validation after empty parser output or timeout, while manual reasoning or inconsistent normalization can produce wrong respondent IDs and row counts.In the specification-drift example, the recovered trajectory returns three valid rows by preserving downstream rule checks.Source: Test-Time Scaling Prompts · p. 12
  • Parallel examples show that recovering missing branch evidence or intersecting visible sources can support aggregation, whereas treating a local value as the global answer leads to failure.The coupon example recovers SAVE10 from visible source sets after two merchant calls fail; the meal example warns against mistaking a single branch for the aggregate.Source: Test-Time Scaling Prompts · p. 13
  • Table 16 presents a prompt for failure-triggered test-time scaling.The passage does not include the prompt’s wording or further details.Source: Table 16 · p. 18

Related papers

Loading 2606.25819v2…