Source-linked AI summary
Tool-to-Agent Retrieval: Bridging Tools and Agents for Scalable LLM Multi-Agent Systems
Elias Lumer, Faheem Nizar, Anmol Gulati, Pradeep Honaganahalli Basavaraju, Vamse Kumar Subbiah
TL;DR
Agent-first retrieval can hide relevant tools whose parent-agent description does not align with a query, while tool-only retrieval ignores bundle benefits on multi-step tasks. Tool-to-Agent Retrieval embeds tools and parent agents in a shared vector space, links them through metadata, and traverses those links during retrieval. Across eight embedding models on LiveMCPBench, Tool-to-Agent Retrieval gained 19.4% Recall@5 and 17.7% nDCG@5 over prior agent retrievers.
Problem
Agent-first retrieval can hide relevant tools whose parent-agent description does not align with a query, while tool-only retrieval ignores bundle benefits on multi-step tasks.
Source: §1 · pp. 1–2Method
Tool-to-Agent Retrieval embeds tools and parent agents in a shared vector space, links them through metadata, and traverses those links during retrieval.
Source: §1 · p. 2Results
Across eight embedding models on LiveMCPBench, Tool-to-Agent Retrieval gained 19.4% Recall@5 and 17.7% nDCG@5 over prior agent retrievers.
Source: §4 · p. 7Takeaways & Limitations
The results point to unified tool and agent selection as a promising direction for retrieval across increasingly complex agent networks.
Source: §4 · p. 7
Abstract
from arXiv · showhide
Recent advances in LLM Multi-Agent Systems enable scalable orchestration of sub-agents, each coordinating hundreds or thousands of tools or Model Context Protocol (MCP) servers. However, existing retrieval methods typically match queries against coarse agent-level descriptions before routing, which obscures fine-grained tool functionality and often results in suboptimal agent selection. We introduce Tool-to-Agent Retrieval, a unified framework that embeds both tools and their parent agents in a shared vector space and connects them through metadata relationships. By explicitly representing tool capabilities and traversing metadata to the agent level, Tool-to-Agent Retrieval enables granular tool-level or agent-level retrieval, ensuring that agents and their underlying tools or MCP servers are equally represented without the context dilution that arises from chunking many tools together. Evaluating Tool-to-Agent Retrieval across eight embedding models, our approach achieves consistent improvements of 19.4% in Recall@5 and 17.7% in nDCG@5 over previous state-of-the-art agent retrievers on the LiveMCPBench benchmark.
1 Introduction
Large tool collections make retrieval essential, but agent-only and tool-only approaches each miss useful information or context. Tool-to-Agent Retrieval jointly represents tools and agents, enabling routing at either level; its LiveMCPBench evaluation reports gains over prior state-of-the-art approaches.
1 Introduction
- Forwarding all tool descriptions is impractical at scale: one MCP server with 26 tools can consume over 4,600 tokens.The introduction frames efficient retrieval as essential when assistants access large collections of tools and MCP servers.Source: §1 · p. 1
- Agent-first routing can hide relevant tools, while tool-only retrieval ignores the surrounding bundle that may benefit multi-step tasks.The paper calls for retrieval that can return a focused tool or an entire agent, depending on the query.Source: §1 · pp. 1–2
- Tool-to-Agent Retrieval embeds tools and parent agents in one vector space, linking them with metadata for joint retrieval and traversal.It aims to preserve fine-grained tool semantics while retaining agent context, without collapsing many tools into a coarse description.Source: §1 · p. 2
- Unlike tool-only or hierarchical agent-first pipelines, the approach retrieves from a single linked space without committing upfront to either level.Source: §1 · p. 3
- 17.7% Recall@5 and 19.4% nDCG@5 improvements over previous state-of-the-art approaches are reported across eight embedding models on LiveMCPBench.Source: §1 · p. 3
2 Approach: Tool-to-Agent Retrieval
Tool-to-Agent Retrieval models tools and agents together and uses ownership metadata to connect tool matches to executable agents. It searches a joint catalog and aggregates retrieved entities into unique agents.
2 Approach: Tool-to-Agent Retrieval
- The method represents agents and their tools as a bipartite graph, with ownership edges linking each tool to its agent.This representation is intended to preserve tool-level capabilities alongside agent context, including execution context such as authentication and access policies.Source: §2 · p. 4
- The unified catalog indexes tool names and descriptions alongside agent names and descriptions, with tool entries linked to their parent agent.Source: §2 · p. 4
- Retrieval ranks the top N entities from the joint catalog, maps tools to their owners, and returns the top K unique agents.The described search combines BM25 with dense vector retrieval.Source: §2 · p. 4
- Queries may be direct or decomposed into sub-tasks; step-wise querying is the primary evaluation setting.The method can take an original query, derived sub-steps, or both as input.Source: §2 · pp. 4–5
3 Experiments and Evaluation
On LiveMCPBench, Tool-to-Agent Retrieval consistently outperforms prior methods across reported retrieval metrics. The evaluation uses step-wise queries and tests performance across embedding models.
3 Experiments and Evaluation
- The evaluation uses LiveMCPBench, with 70 MCP servers, 527 tools, and 95 real-world questions annotated with step-by-step breakdowns and tool–agent mappings.Questions average 2.68 steps, 2.82 tools, and 1.40 MCP agents.Source: §3 · p. 5
- The comparison uses step-wise querying against BM25, ScaleMCP, and MCPZero on an agent-only baseline dataset.The agent-only baseline contains MCP server names and descriptions; evaluation spans eight embedding models.Source: §3 · pp. 5–6
- Tool-to-Agent Retrieval consistently outperforms prior methods across Recall, mAP, and nDCG; Table 2 compares performance by embedding against MCPZero.Source: Table 2 · p. 6, §3 · p. 6
- Across eight embedding models, improvements over MCPZero had standard deviations of 0.02 in Recall@5 and 0.01 in nDCG@5.Amazon Titan v2 showed a +28% relative Recall@5 gain, from 0.66 to 0.85; All-MiniLM-L6-v2 achieved +13%.Source: §3 · p. 6
4 Conclusion
Tool-to-Agent Retrieval unifies tool and agent representations to support granular retrieval while preserving context. On LiveMCPBench, it improves Recall@5 and nDCG@5 over prior agent retrievers across eight embedding models.
4 Conclusion
- 19.4% Recall@5 and 17.7% nDCG@5 gains over prior agent retrievers were observed across eight embedding models on LiveMCPBench.Source: §4 · p. 7
- The framework embeds tools and their parent agents in a shared vector space and links them through metadata relationships.Source: §4 · p. 7
- The conclusion identifies unified tool and agent selection as a direction for retrieval architectures that scale across increasingly complex agent networks.Source: §4 · p. 7