PaperlayerView exact revision on arXiv

Source-linked AI summary

AgentWeave: Routing Before Reasoning for Efficient Function Calling in Tool-Rich Language Models

Saurav Singla, Aarav Singla, Advik Gupta, Parnika Gupta

arXiv:2608.23078v1cs.AIcs.CL

TL;DR

Growing tool catalogs force function-calling models to process more schemas and distinguish among more alternatives, while evaluations often leave the candidate catalog fixed. AgentWeave addresses this by routing a bounded candidate set before inference with the downstream model unchanged; in a frozen 48-task BFCL-derived study, it achieved six native successes versus none for three matched baselines while reducing exposure, tokens, and latency.

  • Problem

    Fixed candidate catalogs leave underexamined which functions a function-calling model should see as tool collections grow.

  • Method

    AgentWeave is a deterministic, model-independent pre-inference routing layer that maps requests and source catalogs to bounded candidate subsets for a fixed downstream model.

  • Results

    AgentWeave achieved 6/48 (12.5%) native BFCL successes, while all-tools, deterministic random top-8, and semantic top-8 baselines each achieved 0/48.

  • Takeaways & Limitations

    The evidence supports treating candidate-space construction as an experimentally meaningful stage and measuring routing separately from model reasoning.

  • Takeaways & Limitations

    The evidence is limited to a narrow BFCL-derived routing-pressure study and does not establish state-of-the-art BFCL performance or universal improvement across models.

Abstract

from arXiv · show

Large language models increasingly operate over large collections of tools, functions, APIs, and specialized agents. As the candidate action space grows, a function-calling model must process more schemas, consume more prompt tokens, and distinguish among increasingly similar or irrelevant alternatives. We study a complementary systems strategy: reduce the candidate set before language-model inference while leaving the downstream model unchanged. We introduce AgentWeave, a deterministic pre-inference routing layer that constructs a bounded model-visible action space using eligibility, requirement, capability, and routing signals. We evaluate AgentWeave with a frozen BFCL-derived routing-pressure protocol using the public MadeAgents/Hammer2.1-1.5b model. On 48 fresh BFCL V4 multiple-function tasks, AgentWeave achieves 6/48 (12.5%) native BFCL successes, whereas all-tools, deterministic random top-8, and semantic top-8 baselines each achieve 0/48. The paired success difference is +12.5 percentage points with a 10,000-resample paired bootstrap 95% confidence interval of +4.17 to +22.92 points and exact McNemar p=0.03125. Relative to all-tools exposure, AgentWeave presents 70.18% fewer tools, uses 61.70% fewer input tokens, and exhibits 50.95% lower mean local-model latency. The result is deliberately narrow: this is a BFCL-derived routing-pressure study rather than an official full BFCL leaderboard score, and absolute task success remains low. The evidence nevertheless shows that candidate-space construction can materially affect a fixed model's function-calling behavior and motivates evaluating routing as a distinct stage before model reasoning.

1. Introduction

Function-calling evaluations commonly expose a fixed tool catalog, leaving an important systems question underexamined: which functions should the model see? AgentWeave frames candidate-space construction as a distinct, model-independent stage before reasoning.

  • Fixed candidate catalogs obscure which functions a model should see as tool collections grow.
  • AgentWeave decomposes function calling into candidate-space construction followed by language-model reasoning.Applications can first remove unavailable tools, then route among the remaining candidates.
  • The routing layer is designed to operate without fine-tuning or changing the downstream model.This enables controlled same-model comparisons across candidate-presentation strategies.
  • The paper contributes a frozen 48-task BFCL-derived replication with matched baselines and measurements of success, retention, token use, and latency.

2. Background and Related Work

Prior work advances tool use through learning, reasoning, retrieval, and benchmarking, while AgentWeave instead holds the function-calling model fixed and changes the candidate functions visible before inference. This positions routing as a separate systems variable from final function selection.

  • Tool-augmented systems established tool use as structured decision-making under external constraints, not merely output formatting.
  • API-Bank, ToolBench, and BFCL broaden evaluation across planning, retrieval, invocation, and multiple function-calling settings.
  • Hammer and ToolACE improve the function-calling model itself, whereas AgentWeave holds the model fixed and changes visible candidate functions.
  • Retrieval success differs from final selection because the model must still choose a function and generate arguments from the returned set.
  • Multiple-function tasks require retaining every required function, making all-candidates-retained rate complementary to average recall.A bounded high-recall set is preferable to either a single prediction or an unbounded catalog.
  • AgentWeave asks whether selective context construction can improve function-calling behavior while reducing schema text processed by the model.

3. Problem Formulation and Architecture

AgentWeave formalizes routing as mapping a request and source catalog to a bounded subset before a fixed model makes its prediction. Its architecture combines policy filtering, requirement-aware routing, and separate authorization.

  • Problem Formulation and Architecture: Routing transforms the conventional prediction y=M(x,F) into y=M(x,F'), where F'=R(x,F) is a subset of the available catalog.Model weights, generation settings, and evaluation logic remain unchanged across conditions.
  • Problem Formulation and Architecture: A useful router balances compression against retention by preserving required functions while minimizing the visible candidate set.
  • Deterministic Scope and Policy Filtering: Deterministic scope and policy filtering can remove candidates using role, tenant, permission, deployment, jurisdiction, product, or capability constraints.Additional semantic routing is optional when deterministic filtering already produces a sufficiently small catalog.
  • Requirement-Aware Routing: Requirement-aware routing matches request-derived signals against candidate descriptions, capabilities, and provider groupings under a bounded exposure budget.The selected function list can be consumed by local, hosted, or provider-neutral models without changing the router interface.
  • Provenance and Failure Localization: Authorization remains a separate fail-closed gate after model selection because routing controls visibility while authorization controls permitted actions.

4. Experimental Design

The study uses a frozen, same-model BFCL-derived protocol that changes only the model-visible candidate context across matched routing conditions. It preserves native evaluation and reports success, efficiency, retention, and paired statistical comparisons.

  • Benchmark Source and Task Selection: The primary experiment evaluates 48 fresh BFCL V4 multiple-function tasks with no overlap with the earlier 12-task pilot.The multiple-function category tests cases where correct behavior may require selecting among several candidate functions.
  • Reproducibility Controls: Task selection and scoring artifacts are frozen before evaluation, with outcome-affecting changes requiring a new study identifier and untouched sample.
  • Model and Evaluation Setting: All routing strategies use the fixed MadeAgents/Hammer2.1-1.5b model in a local, keyless evaluation.This design isolates candidate-presentation effects from model scaling or provider differences.
  • Routing-Pressure Construction: Each task begins in a deterministic 16-tool pressure environment, while the intervention changes model-visible candidates rather than benchmark ground truth.
  • Metrics: The study measures native BFCL success, tools shown, input tokens, local-model latency, original-candidate recall, and all-candidates-retained rate.
  • Statistical Analysis: Paired bootstrap intervals and exact McNemar tests compare strategies evaluated on the same tasks.Semantic retrieval is also allowed to outperform AgentWeave on retention diagnostics rather than being optimized after final scoring.

5. Main Results

In the frozen 48-task replication, AgentWeave was the only condition producing native BFCL successes, while also reducing model-visible context and local processing measures relative to all-tools exposure.

  • Native Function-Calling Success: 6/48 (12.5%) native BFCL successes were achieved by AgentWeave, compared with 0/48 for each matched baseline.The paired advantage over each baseline was +12.5 percentage points, with a 10,000-resample paired bootstrap 95% confidence interval of +4.17 to +22.92 points and exact McNemar p=0.03125.
  • Native Function-Calling Success: AgentWeave’s absolute success rate remains low despite its relative advantage in this difficult experiment.All conditions used the same 1.5B model under a 16-tool pressure setting, and three reasonable alternatives failed on every task.
  • Efficiency Effects: 70.18% fewer model-visible tools, 61.70% fewer input tokens, and 50.95% lower mean local-model latency were observed against all-tools exposure.Latency is hardware- and implementation-dependent and should not be interpreted as a universal speedup.
  • Efficiency Effects: Random and semantic top-8 baselines also reduced token count and latency, so compression alone does not account for AgentWeave’s observed native successes.AgentWeave’s contribution is the combination of stronger observed native success and a smaller average visible tool set.

6. Candidate Retention and Failure Analysis

Retention diagnostics show that higher candidate recall alone did not explain native success, motivating attention to candidate composition and downstream selection behavior. The paper frames routing as retaining required functions while bounding and discriminating the visible set.

  • Retention Diagnostics: 86.81% mean original-candidate recall and 66.67% all-candidates-retained rate made semantic top-8 the strongest compressed retention baseline, yet it achieved 0/48 native successes.AgentWeave had lower retention diagnostics—76.91% and 54.17%, respectively—but achieved 6/48 native successes.
  • Failure Analysis: Native success may depend on candidate composition and competition structure, not merely whether required functions are retained.Closely related distractors can change how the same model behaves when selecting among visible functions.
  • Failure Analysis: Routing failures include removing required functions, retaining only part of a required multi-function set, schema competition, argument-generation errors, and downstream model failure.These categories separate retrieval failures from errors occurring after the candidate set is adequate.
  • Why Recall@K Is a Better Routing Objective Than Hit@1: Required-tool Recall@K and all-required-tools-retained rate are more faithful routing objectives than router Hit@1 alone.The language model remains the final function selector after routing.
  • Why Recall@K Is a Better Routing Objective Than Hit@1: A future evaluation should trace retention versus compression across candidate budget K before measuring downstream native success.This operating curve could distinguish retrieval quality, candidate composition, and budget effects while supporting policy-driven choices based on latency, token budget, or risk tolerance.

7. Replication, Robustness, and Scientific Boundaries

The v6 replication preserved the pilot’s directional result using a larger untouched sample and controlled experimental conditions. The evidence supports a narrow claim about candidate-space construction affecting a fixed model, not a universal benchmark or production advantage.

  • Relationship to the v5 Pilot: 48 fresh tasks with zero overlap with the 12-task v5 pilot increased the replication sample while preserving a frozen evaluation design.The v5 pilot produced 2/12 successes for AgentWeave versus 0/12 for each baseline, but exact McNemar p=0.5 was insufficient for a strong statistical claim.
  • Relationship to the v5 Pilot: 6/48=12.5% versus 0/48 for each baseline preserved the pilot’s directional finding under unchanged model, routing, baseline, generation, and evaluation settings.The larger number of discordant successes provides stronger evidence than repeatedly modifying the router on the same observed split.
  • Why the Negative Results Matter: The study retains weak and negative results, including earlier cases where simple embedding baselines outperformed the original router on already-observed data.This practice reduces selective presentation of favorable comparisons.
  • What the Study Does Not Claim: The study does not establish an official full BFCL leaderboard score, universal superiority to semantic retrieval, production-scale success, universal efficiency gains, security or authorization solutions, or generalization across models.These boundaries constrain interpretation to the tested routing-pressure setting.
  • Why the Negative Results Matter: The semantic baseline’s stronger retention but zero native success, alongside AgentWeave’s low 12.5% success rate, limits the claim to an experimentally meaningful candidate-set variable.The central contribution is not a state-of-the-art leaderboard claim.
  • What the Study Does Not Claim: The preprint presents explicit experimental boundaries and reproducibility detail rather than treating derived routing-pressure numbers as official benchmark rankings.This framing reflects the study’s responsibility to support reproduction and avoid overstating its evidence.

8. Deep Result Analysis and Mechanistic Interpretation

The frozen v6 evidence indicates that candidate-set construction changes a fixed model’s native function-calling outcomes while reducing schema exposure, token use, and latency. The result is narrow: retention alone does not predict success, and broader stage-wise and cross-model evaluation remains necessary.

  • Matched replication: 6/48 AgentWeave tasks succeeded natively versus 0/48 for each comparator, with an exact McNemar p=0.03125 and a paired bootstrap interval of +4.17 to +22.92 percentage points.The paired result supports an effect in this matched routing-pressure sample, not a population-wide accuracy estimate.
  • Efficiency: 70.18% fewer visible tools, 61.70% fewer input tokens, and 50.95% lower mean local-model latency were observed against all-tools exposure.Average visible tools fell from 16.00 to 4.77, tokens from 124,821 to 47,805, and latency from 55.92 s to 27.43 s.
  • Candidate composition: 86.81% original-candidate recall for semantic top-8 versus 76.91% for AgentWeave did not yield native successes for semantic top-8.The comparison rejects a simple higher-retention explanation and motivates testing distractor similarity, redundancy, ordering, and candidate budgets.
  • Pilot-to-replication consistency: The v5 pilot and v6 replication showed consistent descriptive direction, but v5 remained hypothesis-generating and was not pooled for a new inferential claim.v5 produced 2/12=16.67% for AgentWeave versus 0/12, while v6 produced 6/48=12.5% versus 0/48.
  • Stage decomposition: End-to-end success combines required-function retention, model selection, argument generation, and execution, so native success alone cannot localize the source of improvement.The proposed decomposition reports P(R), P(S|R), P(A|S,R), and P(E|A,S,R).
  • Boundaries and implications: The frozen evidence supports routing as a meaningful experimental variable, but not state-of-the-art BFCL performance, universal model improvement, or hardware-independent latency gains.The paper recommends reporting routing and reasoning metrics separately, including required-tool retention, candidate-set size, schema-token count, latency, native correctness, authorization, and execution success.

Data and Software Availability

AgentWeave's source code, benchmark materials, frozen results, and reproduction documentation are publicly available under Apache-2.0, while BFCL resources remain governed by their published terms.

  • Source code, benchmark manifests, frozen result summaries, and reproduction documentation are publicly available on GitHub under the Apache-2.0 license.
  • BFCL data and evaluation code remain subject to the Gorilla/BFCL project's published repository terms and versioned evaluation procedures.

Author Statement

The paper presents open-source benchmark research without implying endorsement and documents a controlled but deliberately limited evaluation design. Its evidence is bounded by confounding among routing choices, a narrow experimental setting, and limited statistical power.

  • The paper reports open-source research and benchmark evidence without implying endorsement by BFCL/Gorilla, MadeAgents/Hammer, arXiv, or affiliated organizations.
  • The study compares routing strategies on matched tasks using the same model, frozen configuration, benchmark revision, and native evaluator.
  • AgentWeave changes both candidate-set size and distractor selection, preventing separate attribution to compression, capability grouping, requirement extraction, ordering, or other routing details.
  • The evaluation uses one small model, one BFCL category, one pressure size, and 48 fresh tasks, limiting external validity across models, catalogs, schemas, agents, inference settings, and authorization constraints.
  • The paired significance result benefits from same-task comparisons and six AgentWeave-favoring discordances, but the confidence interval remains wide.

Appendix B. Result Interpretation Checklist

The appendix provides a checklist for reproducing and interpreting AgentWeave's result without extending empirical claims beyond the frozen evidence.

  • Matched comparison integrity: Reproduction requires the same 48 task identifiers, Hammer2.1-1.5b model, generation settings, benchmark revision, and 16-tool pressure construction for every strategy.
  • Primary outcome: Native BFCL success is the primary end-to-end outcome, while routing recall should not substitute for the scored result.
Loading 2608.23078v1…