Typed Federated Artifacts for the Agentic Web:Sharing Tool-Routing Knowledge Across Frozen,Heterogeneous LLM Agents

Abhijit Chakraborty; Ni Trieu; Vivek Gupta

Source-linked AI summary

Typed Federated Artifacts for the Agentic Web:Sharing Tool-Routing Knowledge Across Frozen,Heterogeneous LLM Agents

Abhijit Chakraborty, Ni Trieu, Vivek Gupta

arXiv:2609.06815v1Sep 6, 2026cs.CLcs.AI

TL;DR

The paper addresses how frozen, heterogeneous agents can share tool-routing knowledge without exchanging model updates, private traces, or opaque flat text. It proposes typed federated artifacts and implements them as SYNAPSE, finding near-centralized StableToolBench performance, substantial typed-format gains, and improved τ-bench tool-call accuracy. The paper also shows that labeled benchmark pools can let TF-IDF classifiers outperform LLM routers, limiting what those benchmarks measure.

Problem
Frozen heterogeneous models lack a portable, privacy-aware unit for exchanging tool-routing knowledge, while flat text cannot distinguish statistics, rules, and documentation.
Method
The paper defines schema-validated typed federated artifacts and instantiates SYNAPSE as a tool-routing compendium with typed fields, field-level merging, and inference-time retrieval.
Results
Federated compendia remain within 1.1 points of centralized routing on StableToolBench, typed fields outperform flat rendering by 8.5 points on clean data, and every τ-bench compendium arm exceeds PLAIN by at least 6.7 points.
Takeaways & Limitations
Typed structure transfers routing knowledge across frozen models, while τ-bench’s gains on well-documented tools come from compendium format rather than federated experience.
Takeaways & Limitations
StableToolBench’s instruction pool contains labeled queries for every test tool, including every test query verbatim before filtering, so it cannot measure routing without labels.

Abstract

from arXiv · show

An open, networked web will allow agents to run frozen models from multiple vendors, keep their history private, and teach each other which tool to call and when. Flat text (prompts, example pools) makes it difficult for the protocol to distinguish between noise statistics, merging rules, and documentation. Weights and adapters cannot transfer that knowledge between platforms. We suggest sharing typed federated artifacts, schema-validated objects with well-defined fields for per-field privacy (described here, but measured), dispute resolution, and cross-model transfer, and instantiating them as SYNAPSE1, a common tool-routing knowledge. After deleting 192 garbage entries and 1,916 training items that duplicate or almost duplicate test queries, a federated compendium routes within 1.1 points of a centralized one at 20 MB of JSON per client each round on StableToolBench (3,180 tools). The same experience merged and shown to the router as typed fields rather than one flat string is worth 8.5 points on clean data and 7.4 under 60% injected contradiction. Crossing merge and rendering shows the halves are inseparable (the typed merge shown flat is the worst arm), while three conflict policies are indistinguishable, so the conflict log that motivated this work is not the On τ-bench retail, each compendium arm improves GPT-4o agents' per-step tool-call accuracy by at least 6.7 points, attributed to format rather than federated experience. Two cautionary findings conclude the paper: on a topic-labeled math proxy and StableToolBench, a TF-IDF classifier over the same labeled experience beats every LLM routing arm (by 48 and 26 points, mostly retrieval recall) because the benchmark's pool holds labeled queries for every supposedly unseen tool and every test query verbatim before our filter. It cannot measure routing to tools without labels, which routing exists for.

1 Introduction

The paper argues that the agentic web lacks a portable unit for sharing tool-use knowledge across frozen, heterogeneous models. It proposes typed federated artifacts so privacy, conflict handling, and cross-model transfer operate on explicit fields rather than flat text.

Motivation and contribution: Flat prompts and example pools cannot distinguish numeric statistics, durable tool-use rules, and canonical documentation.That prevents the protocol from applying field-appropriate privacy, merging, and documentation behavior.
Motivation and contribution: Typed federated artifacts give exchanged knowledge a schema, field roles, types, and validation rules.This makes the artifact independently interpretable across organizations and models.
Method: SYNAPSE instantiates typed artifacts for tool routing using metadata, usage statistics, successful scenarios, precautions, templates, and a structured annex.Clients build local compendia, which aggregators merge before frozen models retrieve and rerank candidates.
Contributions: The contribution includes a typed merge operator, controlled typing-versus-structure experiments, real-catalog benchmarks, and two negative benchmarking findings.The negative findings concern classifier performance on a topic proxy and StableToolBench’s contaminated instruction pool.

2 Typed compendium and merge

SYNAPSE represents tool-routing experience as a validated, typed compendium whose fields can be merged, privacy-processed, and rendered separately. Invalid contributions are rejected, while field-level disagreement is retained as typed conflict information for inference.

Compendium structure: Each compendium contains metadata and usage statistics, successful scenarios, precautions, call templates, and a structured annex.The typed operator merges these fields while preserving dissent for the router.
Validation and merge: Every scenario, precaution, and template must reference a registered tool, and every numeric field must lie within its declared range.Contributions failing these checks are rejected before merging.
Validation and merge: The aggregator clusters scenarios per tool, computes per-field majorities, and attaches disagreements as a typed conflict_log.The router renders the conflict log as “do not use when” conditions in the same round.
Privacy and inference: The schema supports differential privacy on numeric metadata fields and masking on text fields, although noise was disabled in the reported runs.At inference, the router retrieves five distinct tools using canonical descriptions and scenarios.

3 Does typing matter, and why? Isolating type from structure

On StableToolBench, typed structure improves routing even without contradictions, and the benefit depends on preserving both typed merging and typed rendering. Conflict-policy variants remain statistically indistinguishable in these experiments.

Typing versus flat representation: 8.5 points separate typed-field rendering from flat-string rendering before contradiction injection: 0.583 versus 0.497.The typed/untyped gap is already present at 0% contradiction, so injected disagreement is not its sole source.
Contradiction experiment: At 40% contradiction, injected pairs touch 2,709 of 3,180 tools and the gold tool for 253/258 test queries.The injections are applied identically across arms for each contradiction rate and seed.
Contradiction experiment: Typed arms still lead on 253 seed-456 queries whose gold tool was contradicted: 0.629 versus 0.486.The typed merge reduces recall@5, but the reported gap appears in reranking.
Merge versus rendering: The typed merge rendered as a flat blob scores 0.460, below the flat merge rendered flat at 0.497.The crossed experiment therefore identifies both merging and rendering as necessary components of the observed advantage.
Conflict policy: Keeping dissent, delaying it one round, or discarding it produces results within noise at every contradiction rate.The paper does not claim that the conflict log improves accuracy on StableToolBench.

4 Benchmarks and transfer

Across StableToolBench and τ-bench retail, federated compendia perform close to centralized sharing and improve tool-call accuracy over a plain agent loop. The results also show that retrieval limits experience gains and that τ-bench’s improvement is attributable to compendium format rather than federated experience.

StableToolBench: SYNAPSE is never more than 1.1 points below CENTRALIZED and matches or exceeds it on five of six StableToolBench groups.LOCAL-ONLY trails by 12 to 27 points, while pooled experience adds 4–8 points over DOCS-ONLY on single-tool queries.
StableToolBench: Oracle retrieval raises experience’s advantage over DOCS-ONLY to 7–12 points on every group, including multi-tool queries.The reported multi-tool example is 0.762 versus 0.658, indicating retrieval is a major bottleneck.
τ-bench retail: Every compendium arm exceeds PLAIN by at least 6.7 points on τ-bench retail per-step tool-call accuracy.The three compendium arms lie within one standard deviation of one another across 55 held-out tasks.
τ-bench retail: On τ-bench retail, federated experience adds nothing measurable on 14 well-documented tools, so the gain is attributed to compendium format.The paper notes that one artifact under two models on one benchmark remains untested.

5 Two negative results for federated-agent benchmarking

The paper finds that benchmark pools can make tool-routing evaluations reward lexical classification rather than routing to unseen tools. On StableToolBench, overlap remained after filtering, and classifier performance exceeded every LLM routing arm.

0.92 versus 0.44: TF-IDF+SVM classification outperforms the LLM router on the GSM8k topic proxy.Randomly relabeling the routing artifact leaves the classifier at 0.82, indicating it reads the query rather than the compendium.
1.000 raw-pool accuracy: the classifier matches every StableToolBench test query in the unfiltered instruction pool.After removing exact and near-duplicate overlaps, it still reaches 0.817 on supposedly unseen splits.
26 points: the filtered classifier remains ahead of every LLM routing arm on StableToolBench.Under oracle retrieval, the best LLM arm reaches 0.770, leaving a 5-point gap; about 21 points are attributed to retrieval recall.
The benchmark pool contains generated queries for every test tool, sharing vocabulary with tool descriptions, so the splits are unseen only relative to ToolLLaMA’s training split.The paper therefore characterizes these evaluations as measuring text classification rather than routing.

6 Related work and outlook

The paper distinguishes SYNAPSE from prior federation approaches by making a typed artifact, rather than a model, adapter, prompt, index, or free-text compendium, the unit exchanged across organizations.

SYNAPSE introduces a type signature at the federation boundary, making the artifact rather than the model the unit of exchange.This distinguishes it from approaches that federate parameters, flat text, free-text summaries, or indices without typed fields.
Adapter federation requires shared architectures, while prompt and exemplar federation exchanges flat text.FICAL instead merges free-text knowledge compendiums with an LLM summarizer.
Federated RAG federates indices without typed fields, whereas SYNAPSE exchanges schema-typed artifacts.

A One round of SYNAPSE

One SYNAPSE round converts private tool-use experience into a privacy-processed typed compendium, validates and merges those artifacts at the edge, then uses the global compendium for frozen-model tool selection.

Each organization shares only a typed compendium derived from private tool-use experience, with numeric fields clipped and noised and text masked.The artifact is about 20 MB of JSON per round.
An edge node schema-validates each artifact, merges fields, records disagreements in a typed conflict log, and broadcasts one global compendium.
At inference, a frozen LLM retrieves five candidate tools using descriptions and pooled scenarios, then reads each candidate’s “when” and “when not” fields to choose one.The resulting call becomes experience for the next round.

B Threat model and privacy mechanisms

The threat model includes an honest-but-curious edge aggregator and clients that read the global compendium, while the privacy mechanism clips and noises numeric fields and masks text literals.

The adversary is an honest-but-curious edge aggregator or any client that reads the global compendium.The model assumes no trusted server.
Numeric usage counts and success rates are clipped to [0, c] and Laplace-noised with scale c/ε at the edge.Across R rounds, basic composition gives (Rε, 0)-DP for numeric fields.
Text scenarios and precautions remain short templated strings after masking numbers, identifiers, and quoted spans.The workshop version makes no formal claim about the text path.

C Experimental details and released artifacts

The experiments compare routing under real and oracle retrieval, crossed merge and rendering choices, and injected contradictions, with reproducible artifacts and recorded run provenance.

Released artifacts: The released experimental record includes hashed per-query routing logs, manifest-linked source runs, content hashes, and repository commits.The table caption also distinguishes the excluded classifier fit on the unfiltered pool from the separate filtered-pool run.
Evaluation setup: Routing accuracy is evaluated with both real retrieval and oracle retrieval, where the gold tool is forced into the five candidates.Results are reported as mean±SD over seeds 42/123/456, with DOCS-ONLY deterministic.
Merge versus rendering: A typed merge helps only when its fields are shown to the reranker; rendered flat, it performs worse than the flat merge and worst under contradiction.The merge-rendering crossing uses StableToolBench G1+G2-instruction with n=258.
Contradiction injection: Contradiction injection pairs scenarios and precautions about real tools, attributing each pair to different clients and adding positive scenarios as well.This design explains why recall@5 rises slightly with injection rate; precaution-only injection is reserved for the full study.
Run provenance: τ-bench results use vendored repository trees with recorded hashes, while provider settings and local-completion differences are documented for the reported rows.The DOCS-ONLY and PLAIN/SYNAPSE/CENTRALIZED rows correspond to different vendored tree hashes.

D Scale and latency measurements

At 3,180 tools, query-time retrieval and reranking remain relatively fast, while edge clustering dominates the unoptimized per-round computation.

Query-time latency: 37.8 ms is the end-to-end routing latency over 258 queries at 3,180 registered tools.This uses dense retrieval followed by 8B logitscoring reranking of five distinct candidates.
Query-time latency: 1.1 ms is the dense-retrieval cost, compared with 36.6 ms for reranking five distinct candidates.The measurements use one A100-class card and batch size 1.
Aggregation cost: Roughly an hour per aggregation round is required to cluster 28,000 scenarios on CPU in the unoptimized implementation.The edge merge dominates wall-clock cost, but occurs once per aggregation round rather than per query.