Source-linked AI summary
Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives
Haibo Jin, Suijin Wang, Xucheng Yu, Haojing Luo, Haohan Wang
TL;DR
Existing LLM tool-use systems are brittle across dependent and multi-turn calls and lose accuracy with large tool catalogues. The paper introduces Tool Primitives, ToolFace, and HEART’s Planner-Router-Verifier orchestration, reporting higher performance than SFT-based and commercial baselines, lower API cost, and 84% completion on 50 real-world tasks.
Problem
Existing methods struggle with heterogeneous schemas and outputs in multi-step or multi-turn tool use, while accuracy falls as tool catalogues expand.
Method
The paper wraps tools in natural-language Tool Primitives, stores 25,519 functions in ToolFace for on-demand retrieval, and orchestrates them with Planner, Router, and Verifier agents.
Results
HEART outperforms SFT-based models by 10% on average, surpasses three commercial models by 6% on average, reduces token cost by up to 85%, and completes 84% of 50 real-world tasks.
Takeaways & Limitations
HEART provides a structured and iterative tool-use process that improves performance, efficiency, robustness, and failure recovery across diverse benchmarks.
Takeaways & Limitations
The multi-agent design consumes more tokens, ToolFace requires manually authored schemas, and robustness evaluation focuses on prompt injection at the tool-selection stage.
Abstract
from arXiv · showhide
Large language models (LLMs) augmented with external tools have demonstrated remarkable capability in solving complex real-world tasks. However, existing approaches suffer from two key challenges: brittle multi-step and multi-turn reasoning caused by incompatible tool output types and API schemas, and performance degradation under large tool catalogues. To address these, we introduce \textbf{Tool Primitives}, a design that replaces rigid API schema-based invocation with natural language as the interface for tool calling, where each tool is wrapped with an LLM interface that handles schema resolution and execution internally, enabling natural inter-tool communication for nested and multi-turn tool calling. Building on Tool Primitives, we host \textbf{ToolFace}, a centralized repository of 25,519 functions from which LLMs dynamically retrieve only the relevant tools at inference time, eliminating the need to enumerate raw API schemas in context. To orchestrate Tool Primitives and ToolFace reliably in complex settings, we further propose \textbf{HEART}, a \textbf{H}arness \textbf{E}ngineering framework via \textbf{A}gent-native, \textbf{R}eusable \textbf{T}ool Primitives, comprising a Planner, Router, and Verifier that jointly support dynamic tool invocation planning, multi-step execution, and feedback-driven recovery. Experiments on five benchmarks demonstrate that HEART outperforms SFT-based models by $10\%$ on average and surpasses GPT-5.4, Claude-4.6-Sonnet, and Gemini-3.1-Pro by $6\%$ on average while reducing API cost by up to $85\%$. On 50 real-world tasks, HEART achieves $84\%$ task completion, $3.8\times$ the average of three frontier commercial models ($22\%$).
1 Introduction
Existing tool-use methods struggle with brittle multi-step and multi-turn reasoning and degrade as tool catalogues grow. HEART addresses these challenges with Tool Primitives, ToolFace, and a Planner-Router-Verifier system, achieving strong benchmark and real-world results.
- Motivation: 28% full-sequence match accuracy is achieved by even the strongest models on nested API-call benchmarks such as NESTFUL.Performance also drops substantially as interaction depth increases on multi-turn benchmarks.
- Motivation: 7%-85% accuracy drops occur as tool catalogue size scales from 8K to 120K tokens.The challenge is amplified by real-world collections such as ToolBench, which contains over 16,000 APIs.
- Approach: HEART combines a Planner, Router, and Verifier to decompose queries, map parameters, execute tools, and support feedback-driven recovery.The Planner can request clarification when context is insufficient.
- Results: 10% average improvement over SFT-based models and 6% average improvement over three commercial models are reported across five benchmarks.API cost is reduced by up to 85%, and 50 real-world tasks achieve 84% task completion versus 22% for the commercial-model average.
- Approach: Tool Primitives provide natural-language interfaces that encapsulate schema resolution and execution, while ToolFace enables dynamic retrieval from 25,519 functions.The design avoids requiring explicit API-schema knowledge during invocation.
2 Related Work
Prior tool-use research addresses tool coverage and execution complexity but typically exposes raw schemas directly in the model context. HEART separates tool storage, representation, and invocation through natural-language Tool Primitives and on-demand ToolFace retrieval.
- Prior Work: Prior approaches span simple stateless invocation, multi-turn trajectory synthesis, and evaluation across structured calling, dialogue, and compositional planning.The related work is organized around tool coverage and execution complexity.
- Key Differences: Raw API schemas are typically exposed directly in the input context for prompting-based and fine-tuned tool-use methods.This couples tool storage, representation, and invocation.
- Key Differences: Tool Primitives replace direct raw-schema exposure with natural-language tool interfaces.This is HEART’s first stated departure from the prevailing paradigm.
- Key Differences: ToolFace retrieves only relevant Tool Primitives instead of requiring the full tool catalogue in context.This separates tool storage from the model’s input context.
3 Methodology
HEART combines ToolFace, Tool Primitives, and harness agents into an iterative pipeline for planning, routing, execution, verification, and recovery. Tool Primitives translate natural-language requests into schema-valid function calls and structured results, while the agents manage ambiguity and failures.
- ToolFace and Tool Primitives: A Tool Primitive wraps one ToolFace function, accepts a natural-language request, resolves its schema internally, executes the function, and returns a structured result.This provides an agent-native interface without exposing raw API schemas to the calling model.
- ToolFace and Tool Primitives: ToolFace is a centralized registry of structured schema-function pairs, including 25,519 tools collected across five benchmark ecosystems and manually crafted additions.Each manually authored schema records interface details such as parameter types, constraints, and return specifications.
- Planner: The Planner decomposes the query, checks information sufficiency, requests clarification when needed, and otherwise emits an ordered invocation plan.The plan is an ordered sequence of tool invocation steps.
- Router: The Router resolves arguments and execution configuration, constructs a natural-language request, and dispatches it to the corresponding Tool Primitive.The Primitive then handles internal schema validation and argument binding.
- Verifier: The Verifier evaluates completion, argument consistency, execution validity, and constraint satisfaction, returning structured failure feedback to the Planner for targeted re-planning.Successful steps continue through the plan, while failures enrich context for recovery.
4 Experiments
HEART is evaluated across five tool-use benchmarks, ablations, and a re-planning study. It consistently outperforms strong baselines, with component and budget analyses identifying the importance of Tool Primitives, verification, and iterative recovery.
- Main Results: HEART tops ToolBench with 75.1% average Pass Rate and 75.7% Win Rate, exceeding Claude-4.6-Sonnet and ToolLLaMA baselines.On the hardest I3-Inst subset, HEART achieves 77.2% Pass and 89.4% Win while generalizing without explicit DFSDT search.
- Main Results: HEART achieves 0.44 NESTFUL Full Acc., while all xLAM and ToolACE variants score 0.00 under one-shot and DeepSeek-V3 reaches 0.09.The result indicates that flat tool-use trajectories do not generalize to nested API sequences, whereas structured planning performs better.
- Main Results: HEART records the best Passk scores across τ 2-Bench domains and interaction horizons, reaching 0.50 Pass4 in Telecom versus 0.33 for the best baseline.Despite higher token consumption from its multi-agent architecture, HEART reduces API cost by up to 7.4× relative to GPT-5.4.
- Main Results: HEART achieves 86.9% ACEBench Overall, surpassing GPT-5.4 by 0.9%, with larger advantages on Agent, Special, and Multi-Turn categories.HEART scores 72.3% on Agent, 95.0% on Special, and 83.5% on Multi-Turn; the Multi-Turn margin over GPT-5.4 is +2.5%.
- Main Results: On BFCLv4, HEART achieves 84.0% Web Search Overall and 69.38% Memory Overall, outperforming Claude-Sonnet-4.6 and GPT-5.4 on reported comparisons.HEART exceeds Claude-Sonnet-4.6 by +3.0% on Web Search Overall and +1.10% on Memory Overall, and GPT-5.4 by +13.47% on Memory Overall.
- Ablation Study: Removing ToolFace and Tool Primitives causes the largest ablation drop, reducing ToolBench Avg. Pass Rate from 75.1 to 16.1 and NESTFUL Full Acc. from 0.44 to 0.06.Removing the Verifier produces the second-largest drop, while Planner and Router removals cause moderate but consistent degradation.
- Ablation Study: ToolBench Pass Rate rises from 51.4 to 65.9 and NESTFUL Full Acc. from 0.27 to 0.42 between one and two re-planning rounds, with performance saturating at B = 3.B = 5 yields negligible further improvement, so B = 3 is adopted as the default.
- Main Results: HEART achieves 84% task completion on 50 real-world tasks versus 20% for GPT-5.4, 24% for Claude-4.6-Sonnet, and 22% for Gemini-3.1-Pro.The reported results attribute the execution advantage to grounded argument mapping and stateful execution for tasks requiring action in the world.
5 Conclusion
HEART is presented as a harness-engineering framework for improving LLM tool use across scalability, interaction robustness, compositional reasoning, and failure recovery. Its coordinated planning, routing, verification, and Tool Primitive design support structured, iterative tool use.
- HEART addresses scalability to large tool catalogs, robustness in multi-turn interaction, compositional reasoning, and failure recovery.
- Tool Primitives and coordinated planning, routing, and verification transform tool use into a structured and iterative reasoning process.
- Experiments across multiple benchmarks report improvements in performance, efficiency, and robustness, including resilience to prompt injection.
A Detailed Agent Role Specifications
HEART assigns complementary responsibilities to a Planner, Router, Tool Primitive layer, and Verifier. Together, they acquire sufficient context, construct and execute invocations, and recover from failures through structured feedback.
- The Planner analyzes intent, checks information sufficiency, requests clarification when needed, and produces an ordered invocation plan.
- The Router resolves arguments from context and dispatches natural-language requests to the planned Tool Primitives.
- Direct dispatch to raw implementations is fragile because formatting deviations cause schema failures and raw schemas degrade selection with large catalogs.
- The Verifier evaluates execution results against structured criteria and sends diagnostic feedback to the Planner when verification fails.
B Verifier Evaluation Criteria
The Verifier checks each tool result against task completion, argument consistency, execution validity, and constraint satisfaction. HEART’s broader pipeline passes structured results between Tool Primitives and uses verifier feedback to re-plan failed executions.
- Verifier Evaluation Criteria: The Verifier evaluates task completion, argument consistency, execution validity, and constraint satisfaction independently.
- Verifier Evaluation Criteria: A failure in any single criterion produces criterion-specific diagnostics rather than an opaque pass/fail signal.
- Pipeline Interfaces: The Planner first obtains sufficient context, then emits an ordered invocation plan whose steps are executed sequentially.
- Pipeline Interfaces: The Router returns ordered dispatch entries containing target tools, natural-language requests, resolved bindings, and execution hyperparameters.
- Tool Primitive Interface: Tool Primitives resolve schemas and execute functions internally, returning natural-language summaries for downstream context and structured failures when execution cannot proceed.
- Verifier Evaluation Criteria: The Verifier issues PASS only when all four criteria succeed; otherwise, structured feedback is returned to the Planner for recovery.
E.1 Model Pricing
The cost analysis uses model input/output token prices reported in a dedicated table. Prices are expressed in USD per 1M tokens for models evaluated in the τ 2-Bench comparison.
- Table 10 lists input and output token prices for all models included in the cost analysis.
- The pricing unit is USD per 1M tokens.
- The listed models are those used in the τ 2-Bench cost comparison.
E.2 Additional ACEBench Comparisons
On ACEBench, HEART outperforms open-source and SFT-based tool-calling baselines overall, with especially large gains in Multi-Turn and Agent categories.
- 86.9% Overall, HEART surpasses Qwen2.5-Coder-32B-Instruct at 79.6% by 7.3 percentage points.
- 12.5 percentage points separate HEART and Qwen2.5-Coder-32B-Instruct on Multi-Turn, at 83.5% versus 71.0%.
- 11.5 percentage points separate HEART and Qwen2.5-Coder-32B-Instruct on Agent, at 72.3% versus 60.8%.
- SFT-based models perform poorly on Special and Agent, with Watt-Tool-8B reaching 6.0% and 2.8%, respectively.
E.3 Ablation Studies on Backbone LLM Scale
The backbone ablation shows that stronger models improve HEART’s planning and routing performance, while Qwen3-8B offers a strong cost-performance balance and Tool Primitives remain modular across backbones.
- Backbone capacity: HEART (GPT-5.4) achieves the highest Passk scores across all domains, showing that stronger backbone capacity benefits planning and routing.
- Evaluation setup: Table 12 evaluates backbone LLM scale across three domains on τ 2-Bench.
- Cost-performance trade-off: HEART (Qwen3-8B) matches the GPT-5.4 + Qwen3-8B hybrid on most metrics while costing over 12× less than HEART (GPT-5.4).
- Modularity: The hybrid configuration achieves performance close to full GPT-5.4 while retaining Qwen3-8B only for Tool Primitives.
- Modularity: Tool Primitives integrate with arbitrary backbone LLMs without architectural modification, allowing orchestration and execution layers to scale independently.
- Backbone quality: HEART (LLaMA-3.1-8B) substantially trails HEART (Qwen3-8B) across domains and k, indicating model quality matters more than parameter count within this size class.
G Limitation
HEART’s limitations concern deployment cost, ToolFace curation, and incomplete robustness coverage.
- The multi-agent design consumes more tokens than single-model baselines, so stricter latency budgets may require consolidating roles.
- ToolFace relies on manually authored schemas, and extending the registry to new domains requires additional curation effort.
- Robustness evaluation focuses on prompt injection during tool selection, leaving compromised tool outputs for future study.