PaperlayerView exact revision on arXiv

Source-linked AI summary

Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use

Yize Cheng, Chenrui Fan, Mahdi JafariRaviz, Keivan Rezaei, Soheil Feizi

arXiv:2605.14038v2cs.AI

TL;DR

Prior work often treats tool necessity as model-independent and focuses on obvious cases, despite differences in what individual models can solve without assistance. This paper defines necessity relative to each model’s empirical performance and finds substantial mismatches between necessity and tool calls, concentrated in the transition from cognition to action.

  • Problem

    Prior adaptive-tool-use research has largely treated tool necessity as model-independent and focused on obvious cases, although necessity can vary with a model’s capabilities.

    Source: §1 · p. 1

  • Method

    The authors define tool necessity from each model’s empirical performance and analyze tool use as internal cognition followed by execution, using hidden-state probes and trajectory tracing.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    26.5–54.0% of arithmetic queries and 30.8–41.8% of TruthfulQA queries showed mismatches between model-specific tool necessity and observed tool-use actions.

    Source: §4.2 · p. 5

  • Takeaways & Limitations

    Reliable tool use requires not only recognizing when tools are needed but also translating that recognition into action.

    Source: §6 · p. 9

    The study uses N=10 and T=0.7, and its reliance on hidden-state probing makes it inapplicable to closed-source models such as GPT or Gemini.

    Source: C Limitations · p. 16

Abstract

from arXiv · show

Large language models (LLMs) increasingly act as autonomous agents that must decide when to answer directly vs. when to invoke external tools. Prior work studying adaptive tool use has largely treated tool necessity as a model-agnostic property, annotated by human or LLM judge, and mostly cover cases where the answer is obvious (e.g., fetching the weather vs. paraphrasing text). However, tool necessity in the wild is more nuanced due to the divergence of capability boundaries across models: a problem solvable by a strong model on its own may still require tools for a weaker one. In this work, we introduce a model-adaptive definition of tool-necessity, grounded in each model's empirical performance. Following this definition, we compare the necessity against observed tool-call behavior across four models on arithmetic and factual QA dataset, and find substantial mismatches of 26.5-54.0% and 30.8-41.8%, respectively. To diagnose the failure, we decompose tool use into two stages: an internal cognition stage that reflects whether a model believes a tool is necessary, and an execution stage that determines whether the model actually makes a tool-call action. By probing the LLM hidden states, we find that both signals are often linearly decodable, yet their probe directions become nearly orthogonal in the late-layer, last-token regime that drives the next-token action. By tracing the trajectory of samples in the two-stage process, we further discover that the majority of mismatch is concentrated in the cognition-to-action transition, not in cognition itself. These results reveal a knowing-doing gap in LLM tool-use: improving tool-use reliability requires not only better recognition of when tools are needed, but also better translation of that recognition into action.

1 Introduction

The paper defines tool necessity relative to each model’s empirical capabilities and finds substantial gaps between that necessity and observed tool calls. It attributes most mismatches to failures to translate internal recognition into action.

1 Introduction

  • The paper defines tool necessity by each model’s empirical ability, since a query solvable by a stronger model may require tools for a less capable one.This contrasts with prior model-agnostic approaches that rely on human annotators or strong LLM judges [22] [13].Source: §1 · pp. 1–2
  • Necessity-action mismatches reached 26.5–54.0% on arithmetic and 30.8–41.8% on factual QA across four models.Models often called tools despite being able to answer directly, or answered directly despite lacking the requisite internal knowledge.Source: §1 · p. 2
  • Probes found cognition and execution intent often linearly decodable, but their directions became nearly orthogonal in late layers at the last token.The cognition stage reflects whether internal representations encode that a tool is necessary; execution concerns producing tool-triggering tokens.Source: §1 · p. 2, Figure 1 · p. 2
  • Most mismatches arose during the transition from cognition to action, revealing a knowing-doing gap in tool use.Models often represented awareness of their limitations but failed to produce the tool-triggering tokens.Source: §1 · pp. 2–3, Figure 1 · p. 2

2 Related work

Prior work often treats tool necessity as query-static and measures self-assessment explicitly, while this paper grounds necessity in each model’s capabilities and probes implicit cognition. It connects tool-calling errors to a broader knowing-doing gap in which internal beliefs can diverge from actions.

Tool calling in LLM agents

  • Earlier studies label tool necessity as query-static, whereas this work empirically grounds it in each model’s capabilities, extending Wang et al.’s model-dependent framing [27].Related work also examines tool-use accuracy [12], hallucinated calls [33] [23], and robustness to tool descriptions [25] [5]; the authors describe their pipeline as the first empirical grounding of necessity in a given model’s actual capabilities.Source: §2 · p. 3

Meta-cognition of LLMs and the “knowing-doing gap”

  • Because explicit self-assessment differs from task-oriented problem solving, the paper follows work probing internal states to measure implicit cognition of tool necessity [13] [28].Prior approaches elicit knowledge boundaries [31] or verbalized confidence [15], but verbal reports may not reflect internal activations [9]; the distinction is likened to system I and system II thinking [14].Source: §2 · p. 3
  • Prior hidden-state studies show internal beliefs can diverge from actions, and this work finds the same knowing-doing gap in tool calling, contributing substantially to end-to-end errors [34] [32].Zhao et al. [34] report failures to refuse harmful queries despite internal recognition, while Zhang et al. [32] find models recognize inability on some math problems yet continue unproductive reasoning.Source: §2 · p. 3

3 Defining model-adaptive tool necessity and two-stage modeling of tool-call

The section defines tool necessity relative to each model’s empirical ability to solve queries without tools and separates recognizing that a tool is needed from actually invoking one. It also decomposes necessity-action mismatch into recognition and execution mismatches.

Defining model-adaptive tool necessity

  • A query is tool-necessary for model f when it cannot solve the query consistently across N independent tool-free runs; otherwise, necessity is 0.The number of runs N and temperature T control how conservative the capability-boundary estimate is.Source: §3 · p. 3
  • The model-adaptive definition grounds tool necessity in empirical behavior, reflecting that occasional unaided correctness may still warrant tools for reliable performance.A model’s capability boundary differs across models, unlike a static query label independent of the model being evaluated.Source: §3 · pp. 3–4

The cognition-execution modeling of tool-call

  • Tool use is modeled as internal cognition about whether a tool is needed followed by execution of that cognition as an actual tool call.This distinguishes meta-cognition about the model’s capability boundary from its ability to act on that belief.Source: §3 · pp. 3–4

End-to-end error diagnosis

  • End-to-end necessity-action mismatch is decomposed into the mismatch between necessity and cognition and the mismatch between cognition and the actual tool-use decision.D(nf(x), af(zf(x))) is decomposed into D(nf(x), zf(x)) and D(zf(x), af(zf(x))).Source: §3 · p. 4

4 Dataset curation

The section curates arithmetic and factual QA datasets, defines tool necessity by each model’s no-tool performance, and measures whether tool calls align with that model-specific need. The observed mismatch rates are substantial and vary by model and domain.

4 Dataset curation

  • The study covers arithmetic and factual QA with Qwen3-8B, Qwen3-4B, Llama-3.1-8B-Instruct, and Llama-3.2-3B-Instruct.Arithmetic mixes simple and difficult operations across 4,000 instances; factual QA uses 817 TruthfulQA instances. Calculator and search API access serve as external assistance for the respective domains.Source: §4 · p. 4

4.1 Grounding tool necessity to model-specific capability boundaries

  • Across arithmetic and factual QA, models have different capability boundaries, so the same samples can be tool-necessary for one model and unnecessary for another.For each model, 10 no-tool inferences at temperature 0.7 define samples with at least one failure as tool-necessary and samples correct in all runs as tool-unnecessary; Figure 2 visualizes the model-dependent boundaries.Source: §4.1 · p. 4, Figure 2 · p. 5

4.2 Collecting tool-call behaviors on tool-necessary and tool-unnecessary instances

  • With tools enabled, greedy decoding collects tool-call actions, which are divided into four sets according to necessity and whether the model called a tool.Necessary-Called and Unnecessary-NotCalled are aligned with the model-dependent definition, while Necessary-NotCalled and Unnecessary-Called constitute the end-to-end mismatch; model-specific handlers expose the calculator and search API in each model’s expected syntax.Source: §4.2 · pp. 4–5
  • Mismatch rates range from 26.5% to 54.0% on arithmetic and from 30.8% to 41.8% on TruthfulQA.Table 1 breaks tool-call behavior into aligned and misaligned categories; the Mis. column aggregates the mismatches.Source: Table 1 · p. 5, §4.2 · p. 5
  • Error patterns vary by model and domain: Qwen3-8B overuses tools on arithmetic but underuses them on TruthfulQA, while Qwen3-4B reverses that pattern.On arithmetic, Qwen3-8B has UN-C 38.2% versus N-NC 3.5%; on TruthfulQA, its N-NC is 17.9% versus UN-C 13.2%. Qwen3-4B has arithmetic N-NC of 14.5%, exceeding UN-C, but TruthfulQA UN-C of 23.1% exceeds N-NC of 18.7%.Source: §4.2 · p. 5

5 From meta-cognition to execution ability: What went wrong?

Hidden states often encode both tool-necessity cognition and call/no-call action, but those signals become nearly orthogonal at the generation readout. Most mismatch arises when cognition fails to translate into action, even when cognition is confident.

5.1 Probing for model’s cognition

  • Necessity is more linearly separable on Arithmetic than TruthfulQA, where only near-last tokens in mid-late Llama layers show decent separability.On Arithmetic, broad mid-to-late-layer regions exceed MCC = 0.4; the heatmap patterns are similar within the Qwen and Llama model families.Source: §5.1 · p. 6, Figure 3 · p. 6

5.2 Probing for action

  • Action is highly decodable: the action probe reaches MCC ≥0.4 across broad regions of nearly every model on both datasets.The signal spans most layers and token positions, rather than appearing only in a narrow region.Source: §5.2 · p. 7, Figure 4 · p. 7

5.3 The gap between cognition and execution

  • Cognition and action directions show some intermediate-position alignment, but become nearly orthogonal at late layers of the final query token.This collapse occurs in the readout regime that drives the next-token decision, including for Qwen3-8B and Qwen3-4B, and holds more uniformly for other models and TruthfulQA.Source: §5.3 · pp. 7–8, Figure 5 · p. 8

5.4 Two stage error diagnosis and attribution

  • Most mismatch arises in cognition-to-action translation: stage-two-only errors dominate stage-one-only errors across all four panels.The mismatch persists even when the necessity probe strongly indicates that a sample is tool-necessary or tool-unnecessary.Source: Figure 6 · p. 8, §5.4 · p. 9
  • Cognition–execution mismatch does not primarily occur under uncertainty; many mismatches appear when cognition confidence is close to 0 or 1.The cognition probe’s post-sigmoid output measures necessity confidence, while the plotted call probability quantifies the model’s likelihood of making a tool call.Source: §5.4 · p. 9, Figure 7 · p. 9

6 Conclusion

The paper defines tool necessity relative to each model’s empirical capabilities and finds substantial mismatches between models’ needs and tool calls. Its analysis identifies a knowing-doing gap: internal recognition can become misaligned with execution intent, so reliable tool use requires recognition to translate into action.

6 Conclusion

  • The paper introduces a model-adaptive definition of tool necessity grounded in empirical capabilities and finds substantial mismatches between models’ needs and tool calls.Source: §6 · p. 9
  • Hidden-state analysis of cognition and execution reveals a knowing-doing gap, with later-layer cognitive representations becoming orthogonally misaligned with execution intent.Source: §6 · p. 9
  • Improving autonomous agents requires bridging this gap so internal meta-cognition reliably translates into appropriate action.Source: §6 · p. 9

A More details on arithmetic dataset curation

The arithmetic dataset spans easy calculations, larger-operand short expressions, and multi-step expressions designed to test tracking and order of operations. It contains 4,000 samples, generated with fixed sampling shares and a fixed random seed.

A More details on arithmetic dataset curation

  • The dataset groups problems into easy one- or two-step arithmetic, short expressions with larger operands, and multi-step precedence, parenthesized, multiplication, and long-chain expressions.The groups target calculator-unnecessary cases, digit-error risk, and intermediate-value or order-of-operations errors, respectively.Source: A More details on arithmetic dataset… · p. 13
  • 4,000 samples are allocated using fixed sampling shares across problem families, with repeated expressions skipped and resampled until each family reaches its share.The dataset is sampled with a fixed random seed; Table 2 reports the family breakdown and sampling shares.Source: A More details on arithmetic dataset… · p. 13, Table 2 · p. 13
  • Algorithms 1–13 specify the procedures for generating samples in each problem family, using discrete-uniform integer draws and continuous-uniform draws on [0, 1].The appendix defines U{m, . . . , n} as discrete uniform over integers m through n and U(0, 1) as continuous uniform on the unit interval.Source: A More details on arithmetic dataset… · p. 13

B Explicitly prompting for verbalized belief of tool-necessity

Explicitly asking models to verbalize whether tools are necessary produces judgments that poorly reflect capability-grounded necessity and can substantially change tool use. The results highlight a gap between prompted self-assessment and behavior under task-oriented prompts.

B Explicitly prompting for verbalized belief of tool-necessity

  • The two-stage procedure first elicits a yes/no judgment about tool necessity, then asks the model to answer the original request.Table 3 reports MCC against capability-grounded necessity, cognition–execution mismatch, and the proportion of changed tool-call behavior. The authors include explicit assessment for completeness while noting limitations in verbalizing internal processes and differences between self-assessment and problem solving [9], following internal-state probing work [13] [28].Source: B Explicitly prompting for verbalized… · p. 15
  • Llama-3.2-3B-Instruct had negative MCC on TruthfulQA, while Llama-3.1-8B-Instruct answered “no” for every sample, leaving MCC undefined.The all-no judgments conflict with capability measurements indicating that some samples require tools.Source: B Explicitly prompting for verbalized… · p. 15
  • Llama-3.1-8B-Instruct had a 0 mismatch rate, and explicit judgments produced noticeably lower cognition–execution mismatch rates than those in Section 5.4.After answering “no” for every sample, it consistently refrained from tool calls; the authors suggest the earlier answer encourages consistency during later generation.Source: B Explicitly prompting for verbalized… · pp. 15–16
  • Up to nearly 50% of samples changed tool-calling behavior under explicit self-assessment prompts versus task-oriented prompts.The authors caution that this shift may make explicit-prompt evaluations diverge from tool use in realistic task settings.Source: Table 3 · p. 16, B Explicitly prompting for verbalized… · p. 16

C Limitations

The study evaluates tool necessity at one setting, N = 10 and T = 0.7, and its reliance on hidden-state probing makes it inapplicable to closed-source models such as GPT or Gemini.

C Limitations

  • Using only N = 10 and T = 0.7 leaves how other settings affect mismatch rates untested, while hidden-state probing makes the work inapplicable to closed-source LLMs.The authors suggest testing other N and T values and note the limitation for models such as GPT and Gemini.Source: C Limitations · p. 16

Related papers

Loading 2605.14038v2…