Source-linked AI summary
Beyond Semantic Similarity: Performance and Costs of Agentic Retrieval for Complex Tasks
Reza Esfandiarpoor, Radek Osmulski, Yauhen Babakhin, Gabriel de Souza P. Moreira, Oliver Holworthy, Jie He, Ronay Ak, Jiarui Cai, Ryan Chesler, Bo Liu, Even Oldridge
TL;DR
Emerging retrieval tasks need more than surface-level similarity, so this paper combines LLM reasoning with dense retrieval in a ReAct agent. Across ViDoRe v3 and BRIGHT, the agentic pipeline is competitive and improves nDCG@10, but incurs substantial inference costs.
Problem
Standard retrieval based on semantic similarity is insufficient for emerging applications that require high-level skills such as iterative exploration and intermediate reasoning.
Source: §4 · p. 4Method
Results
Across ViDoRe v3 and BRIGHT, the same agentic pipeline achieves competitive results without modification and improves nDCG@10 by 8.7 points over standard retrieval using the same embedding model.
Source: §1 · p. 1Takeaways & Limitations
Agentic retrieval is a promising direction for complex tasks, while broader deployment calls for more efficient approaches and cost controls adaptable to query complexity.
Source: §1 · p. 1, §4 · p. 4LLM inference overhead is the main limitation to agentic retrieval at scale, with reasoning steps and token costs varying with query complexity and agent behavior.
Source: §3.3 · p. 4, §4 · p. 4
Abstract
from arXiv · showhide
Modern information systems, including many agentic workflows, use dense retrieval to explore large amounts of unstructured data. However, dense retrieval relies on surface-level semantic similarity, which is insufficient for increasingly complex search applications. Here, we investigate agentic retrieval that combines the reasoning capabilities of Large Language Models (LLMs) with the efficient corpus exploration of retrievers in a ReAct agentic loop to solve complex retrieval tasks. In our experiments, we show that agentic retrieval is more effective than standard retrieval, improving nDCG@10 by 8.7 points using the same embedding model. Moreover, while specialized retrieval methods struggle on out-of-domain tasks, agentic retrieval is highly generalizable: the same pipeline achieves competitive results on both the ViDoRe v3 and BRIGHT leaderboards. However, this improvement comes at a cost. On average, agentic retrieval takes 107.4 seconds, compared to 0.67 seconds for standard retrieval, and consumes 764.1K input and 5.8K output tokens per query. In short, our study demonstrates the effectiveness of agentic retrieval in modern data systems and motivates future work on more cost-efficient retrieval agents for large-scale deployment.
1 Introduction
Emerging retrieval tasks require reasoning-intensive search beyond surface similarity; this paper studies an LLM-and-retriever ReAct agent and reports stronger retrieval and cross-domain competitiveness alongside higher costs.
1 Introduction
- Complex tasks can have limited surface-level semantic similarity between queries and relevant documents, requiring reasoning, world knowledge, and iterative exploration.Source: §1 · p. 1
- The proposed approach combines LLM capabilities and dense retrievers through a ReAct loop and is evaluated on enterprise retrieval and reasoning-intensive tasks.Source: §1 · p. 1
- 8.7 points better nDCG@10 than standard retrieval with the same embedding model is achieved by the best agentic retrieval agent, on average.Source: §1 · p. 1
- Without modification, agentic retrieval achieves competitive results on both the ViDoRe v3 and BRIGHT leaderboards, while specialized methods struggle out of domain.Source: §1 · p. 1
- Agentic retrieval incurs significantly higher costs than standard dense retrieval, making cost and efficiency limitations to widespread adoption.Source: §1 · p. 1
2 Retrieval Agent
The retrieval agent uses an LLM in a loop with dense retrieval to explore the corpus, adapt searches, and return relevant documents; the section also describes reliability and infrastructure choices.
2 Retrieval Agent
- Prior combinations often use static workflows such as query rewriting, which prevent the LLM from refining queries using newly discovered information.Source: §2 · p. 2
- The agent loops between LLM reasoning and retrieval, adapting when, how often, and with which queries it searches; Figure 1 depicts the think, retrieve, and final_results sequence.Source: §2 · p. 2, Figure 1 · p. 2
- Initial dense-retrieval results are given to the agent to avoid unnecessary searches and help it infer corpus document types for subsequent queries.Source: §2 · p. 2
- Retrieve searches by dense similarity and returns document IDs, Think supports reasoning and planning, and Final_Results submits ranked document IDs and a rationale.Source: §2 · p. 2
- If an agent error occurs, the pipeline falls back to Reciprocal Rank Fusion to combine ranked documents collected from prior searches.Source: §2 · p. 2
- To avoid MCP server startup, embedding-loading, lifecycle, and network overhead, the implementation uses a thread-safe in-process singleton retriever.Source: §2 · p. 2
3 Experiments
Across the evaluated complex-retrieval benchmarks, agentic retrieval outperforms standard retrieval and generalizes across domains, while requiring substantially greater per-query time and tokens.
3.1 Setup
- Evaluation spans ViDoRe v3's six languages, 10 domains, and seven query types, plus reasoning-intensive BRIGHT; models include Opus 4.5 and gpt-oss-120b.The experiments use dataset-specific retrievers for ViDoRe v3 and BRIGHT.Source: §3.1 · p. 3
3.2 Results
- Agentic retrieval outperforms standard retrieval across cases using the same embedding model.The authors attribute this to the agent better understanding query complexity and using embeddings more effectively.Source: §3.2 · p. 3
3.3 Analysis
- The unchanged agent ranks #1 on ViDoRe v3 and #2 on BRIGHT, while specialized INF-X-Retriever performs worse out of domain.The reported leaderboard ranks are as of March 13, 2026.Source: Table 2 · p. 3, §3.3 · p. 3
3.2 Results
- Opus 4.5 exceeds gpt-oss-120b by 6.2 nDCG@10 points on average (9.5 on BRIGHT); on ViDoRe v3, it averages 9.2 search calls versus 2.4.The authors suggest limited exploration by gpt-oss-120b may contribute to the performance difference.Source: §3.2 · p. 3
- 57%: agentic retrieval reduces the performance gap between weaker and stronger embedding models, on average.The authors note embedding models are often kept small for large corpora, limiting their capabilities.Source: §3.2 · p. 3
3.3 Analysis
- Agents refine and persistently rephrase queries as new information appears, and decompose multipart tasks into simpler goals.These are among the recurring search patterns described in the analysis.Source: §3.3 · pp. 3–4
- 107.45 seconds per query versus 0.67 seconds for standard retrieval; agentic retrieval also consumes 764.1k input and 5.8k output tokens per query.The paper identifies additional LLM inference overhead as the main limitation to agentic retrieval at scale.Source: Table 3 · p. 4, §3.3 · p. 4
4 Discussion
The discussion presents agentic retrieval as a response to complex applications, while emphasizing efficiency and task-specific system design as practical concerns.
4 Discussion
- RAG and DeepResearch applications demand iterative exploration and intermediate reasoning beyond standard retrieval based on semantic similarity.The discussion identifies these as high-level skills required by evolving retrieval applications.Source: §4 · p. 4
- Agentic retrieval costs include reasoning steps and tokens, which depend on query complexity and agent behavior, unlike index-size-driven standard retrieval costs.The authors suggest cost controls that let users adapt token and reasoning costs to their applications.Source: §4 · p. 4
- Current open-source models lag proprietary counterparts, motivating retrieval-specific models that are more effective and more efficient.The authors point to iterative exploration as a specialized skill retrieval agents require.Source: §4 · p. 4
- For retrieval, parallel search attempts in one turn can reduce steps and potentially provide more context for the next attempt.The authors contrast this task-specific pattern with agents' usual sequential tool calls.Source: §4 · p. 4
4.1 Related Work
- Prior LLM query rewriting remains static [5] [8] [17], whereas agentic retrieval dynamically adapts iterative reasoning and exploration to query complexity.The discussion distinguishes this dynamic pipeline from earlier static approaches.Source: §4.1 · p. 4
- Unlike agentic systems that use standard retrieval [2] [3] [9], this work makes retrieval itself agentic as a reusable search component for downstream applications.The distinction is whether the retrieval method itself changes, rather than only how an agent uses it.Source: §4.1 · p. 4
5 Conclusion
The paper concludes that combining LLM reasoning and iterative exploration with standard retrievers better addresses complex retrieval tasks across ViDoRe v3 and BRIGHT. The approach generalizes across domains, but adds compute requirements.
5 Conclusion
- A ReAct agent combines standard retrievers with LLM reasoning and iterative exploration for complex retrieval.The retrievers explore the corpus while LLMs provide higher-level capabilities.Source: §5 · p. 4
- Experiments on ViDoRe v3 and BRIGHT show improved performance over standard dense retrieval and cross-domain generalizability relative to specialized solutions.The same agent pipeline performs well across diverse tasks, while specialized solutions drop significantly out of domain.Source: §5 · p. 4
- The paper compares retrieval costs and identifies additional compute requirements as a limitation warranting further efficiency work.The conclusion encourages improvements to agentic retrieval's efficiency and effectiveness.Source: §5 · p. 4