Source-linked AI summary
Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs
Yangning Li, Weizhi Zhang, Yuyao Yang, Wei-Chieh Huang, Yaozu Wu, Junyu Luo, Yuanchen Bei, Henry Peng Zou, Xiao Luo, Yusheng Zhao, Chunkit Chan, Yankai Chen, Zhongfen Deng, Yinghui Li, Hai-Tao Zheng, Dongyuan Li, Renhe Jiang, Ming Zhang, Yangqiu Song, Philip S. Yu
TL;DR
LLMs need both external factual grounding and stronger multi-step reasoning, because RAG alone can struggle with complex inference while reasoning alone can mis-ground facts. This survey unifies the two through a taxonomy of one-way and synergized systems, concluding that iterative retrieval–reasoning coupling improves factual grounding, logical coherence, and adaptability. Its breadth-first synthesis also leaves finer technical trade-offs and method-specific limitations less developed.
Problem
LLMs struggle with hallucinated knowledge and complex reasoning, while existing RAG and reasoning approaches require a unified account of their integration.
Method
The survey organizes Reasoning-Enhanced RAG, RAG-Enhanced Reasoning, and Synergized RAG-Reasoning systems, including their methods, benchmarks, and open challenges.
Results
Tight retrieval–reasoning coupling improves factual grounding, logical coherence, and adaptability beyond one-way enhancement.
Takeaways & Limitations
Iteratively refining search and reasoning is the survey’s supported direction for more effective, multimodally adaptive, trustworthy, and human-centric systems.
Takeaways & Limitations
The survey favors breadth over depth, so its taxonomy may obscure fine-grained trade-offs, assumptions, and limitations of individual method classes.
Abstract
from arXiv · showhide
Retrieval-Augmented Generation (RAG) lifts the factuality of Large Language Models (LLMs) by injecting external knowledge, yet it falls short on problems that demand multi-step inference; conversely, purely reasoning-oriented approaches often hallucinate or mis-ground facts. This survey synthesizes both strands under a unified reasoning-retrieval perspective. We first map how advanced reasoning optimizes each stage of RAG (Reasoning-Enhanced RAG). Then, we show how retrieved knowledge of different type supply missing premises and expand context for complex inference (RAG-Enhanced Reasoning). Finally, we spotlight emerging Synergized RAG-Reasoning frameworks, where (agentic) LLMs iteratively interleave search and reasoning to achieve state-of-the-art performance across knowledge-intensive benchmarks. We categorize methods, datasets, and open challenges, and outline research avenues toward deeper RAG-Reasoning systems that are more effective, multimodally-adaptive, trustworthy, and human-centric. The collection is available at https://github.com/DavidZWZ/Awesome-RAG-Reasoning.
1 Introduction
LLMs face intertwined limitations in factual grounding and complex reasoning, motivating RAG, reasoning enhancement, and their eventual integration. The survey organizes this progression from one-way improvements toward iterative retrieval–reasoning systems.
- LLMs are limited by hallucinations from static parametric knowledge and difficulty with complex real-world reasoning.
- RAG supplies external knowledge, while reasoning methods strengthen LLMs’ inherent inference abilities.
- Early approaches improved RAG with reasoning or improved reasoning with retrieved knowledge, but both followed one-way enhancement patterns.
- Static Retrieval-Then-Reasoning can miss knowledge needs that emerge during reasoning, suffer from conflicting evidence, and lack adaptive feedback.
- Synergized systems iteratively interleave reasoning and retrieval so each can refine the other during multi-step investigation.
- The survey synthesizes this integration, covering one-way enhancements, synergized frameworks, benchmarks, and open challenges.
2 Background and Preliminary
RAG combines staged retrieval with multi-step generation, while Chain-of-Thought reasoning supplies structured inference over curated context. Their combination motivates the RAG-Reasoning paradigm.
- RAG proceeds through retrieval, integration, and generation to fetch, curate, and reason over task-relevant external content.
- Chain-of-Thought encourages models to reason step by step before answering, complementing the structured RAG pipeline.
3 Reasoning-Enhanced RAG
Reasoning-Enhanced RAG applies inference throughout retrieval, integration, and generation rather than merely concatenating initially retrieved documents. The survey groups these methods by how reasoning improves search, evidence handling, and grounded output.
- Traditional RAG retrieves documents once and concatenates them with the query, often missing deeper context and relationships needed for complex reasoning.
- Retrieval optimization: Reasoning can improve retrieval through query reformulation, retrieval planning, adaptive retrieval decisions, and reasoning-aware retriever enhancement.
- Retrieval optimization: Query decomposition, reformulation, and CoT-based expansion align retrieved context more closely with the generator’s reasoning needs.
- Integration enhancement: Integration methods assess evidence relevance and synthesize heterogeneous information to reduce irrelevant content and preserve useful support.
- Reasoning during generation: During generation, context-aware synthesis selects relevant knowledge while grounded controls verify factual consistency, add citations, and constrain reasoning to evidence.
4 RAG-Enhanced Reasoning
RAG-Enhanced Reasoning supplies external or internal contextual knowledge to fill factual gaps and support multi-step inference. Retrieval sources include knowledge bases, the web, tools, prior experience, and examples.
- External and in-context retrieval can reduce hallucinations, bridge logical gaps, and improve factual accuracy, interpretability, and consistency.
- External retrieval: Knowledge-base retrieval supports sequential reasoning by providing interconnected factual entries for general and specialized tasks.
- External retrieval: Web retrieval enables step-by-step claim verification and knowledge-intensive question answering using dynamic online content.
- External retrieval: Tool-using systems invoke calculators, libraries, APIs, or symbolic components to improve numerical accuracy and factual precision interactively.
- In-context retrieval: Prior experience retrieves past interactions, decisions, reinforcement signals, or conversational histories to support adaptive reasoning.
- In-context retrieval: Example-based reasoning selects demonstrations or training examples similar to the query to guide reasoning patterns and improve novel-question performance.
5 Synergized RAG-Reasoning
Synergized RAG-Reasoning systems tightly interleave retrieval and reasoning through iterative workflows and agentic orchestration. The survey organizes these approaches by reasoning structure and agent architecture, including single-agent and multi-agent systems.
- Iterative Retrieval–Reasoning: Iterative retrieval and reasoning allow newly obtained evidence to inform subsequent inference, while reasoning guides further retrieval.This addresses tasks where one retrieval step or one reasoning round may miss necessary information.
- Reasoning Workflows: Reasoning workflows span chain-based, tree-based, and graph-based structures, progressing from linear reasoning to branching and relational representations.Chain methods interleave retrieval between reasoning steps; tree and graph methods expand search structures or navigate linked evidence.
- Agent Orchestration: Single-agent systems interweave search with reasoning, enabling dynamic evidence lookup during problem solving through prompting, supervised fine-tuning, or reinforcement learning.RL-based approaches optimize what to search, how to integrate evidence, and when to stop, supporting complex knowledge-intensive tasks.
- Agent Orchestration: Multi-agent systems use centralized or decentralized orchestration to divide retrieval, reasoning, and knowledge integration across specialized agents.Decentralized systems broaden information coverage across sources, while centralized hierarchies support task decomposition, progressive refinement, routing, and reconfiguration.
6 Benchmarks and Datasets
The survey evaluates knowledge and reasoning together across benchmarks ranging from basic fact retrieval to intricate multi-step tasks in general and specialized domains.
- Benchmark Coverage: Representative benchmarks cover web browsing, single-hop QA, multi-hop QA, multiple-choice QA, mathematics, and code-centric evaluation.Examples include BrowseComp, TriviaQA, HotpotQA, MMLU-Pro, MATH, and LiveCodeBench.
7 Future Work
Future work targets more accurate, efficient, trustworthy, multimodal, adaptive, and human-aligned Synergized RAG-Reasoning systems. The survey emphasizes unresolved challenges in latency, retrieval trustworthiness, user interaction, agentic control, and multimodal retrieval.
- Reasoning Efficiency: Reasoning efficiency is constrained by latency from iterative retrieval and multi-step reasoning loops.A single deep research query can take over 10 minutes in practical settings, motivating compression methods such as quantization, pruning, and distillation.
- Retrieval Efficiency: Retrieval efficiency requires budget-aware query planning, evidence or belief-state caching, and adaptive control based on uncertainty.These mechanisms aim to reduce redundant access and wasteful retrieval operations.
- Human-Agent Collaboration: Human-agent collaboration should model uncertain user intent, support iterative clarification, and adapt reasoning to user expertise and preferences.The survey presents human-in-the-loop interaction as important for robust and user-aligned systems in open-ended domains.
- Agentic and Multimodal Capabilities: Future systems should improve dynamic tool selection, retrieval planning, adaptive orchestration, and multimodal grounding and cross-modal reasoning.Most current systems remain text-only, while real-world applications increasingly require multimodal retrieval and integration.
- Retrieval Trustworthiness: Trustworthy retrieval remains necessary because poisoned or misleading sources can undermine downstream reasoning.Future work combines traceability, uncertainty quantification, robust generation, and multidimensional trust benchmarks beyond accuracy.
8 Conclusion
The survey describes retrieval and reasoning as converging through three stages: one-way reasoning-enhanced RAG, one-way RAG-enhanced reasoning, and synergized systems that iteratively refine both. It identifies future directions toward systems that are more effective, multimodally adaptive, trustworthy, and human-centric.
- Three Evolutionary Stages: The survey organizes the field into Reasoning-Enhanced RAG, RAG-Enhanced Reasoning, and Synergized RAG-Reasoning systems.The third stage uses single or multiple agents to iteratively refine search and reasoning, exemplified by Deep Research platforms.
- Conclusion: Tight retrieval–reasoning coupling is presented as improving factual grounding, logical coherence, and adaptability beyond one-way enhancement.The conclusion frames these properties as the collective direction of the surveyed approaches.
Limitations
The survey prioritizes breadth over detailed treatment, and its broad taxonomy can obscure method-specific trade-offs, assumptions, and limitations.
- The survey synthesizes over 200 papers but favors breadth over depth, limiting discussion of individual methods’ technical nuances and implementation details.The authors specifically note reduced depth in specialized retrieval and reasoning subfields.
- Its taxonomy abstracts across reasoning-enhanced RAG, RAG-enhanced reasoning, and synergized systems, which may hide class-specific trade-offs and assumptions.
A Full Benchmark
The survey compiles and organizes benchmarks that jointly test external knowledge retrieval and internal reasoning across diverse tasks and domains. It also identifies major gaps in realism, multimodal robustness, reasoning coverage, and end-to-end evaluation.
- Benchmark compilation: The appendix consolidates benchmark metadata, including venue, code, task category, domain, knowledge sources, knowledge type, and reasoning capabilities.This organization is intended to support dataset selection and comparison for future RAG-reasoning studies.
- Benchmark compilation: The collection contains 46 benchmarks spanning 13 tasks and 12 domains, including newer datasets designed to better mirror real-world applications.Examples include Humanity’s Last Exam and BrowseComp, which require extensive retrieval and expert-level or domain-specific reasoning.
- Task coverage: The benchmark set ranges from single-hop fact retrieval to multi-hop document chaining, deductive reasoning, mathematical problem-solving, summarization, and iterative web navigation.TriviaQA emphasizes single evidence retrieval, while HotpotQA and MuSiQue require linking information across documents.
- Coverage gaps: Current coverage is limited for personalized industrial settings, heterogeneous or multimodal content, noisy or conflicting information, and causal, counterfactual, decision-oriented, or analogical reasoning.The survey also notes missing standardized benchmarks and metrics for evaluating complete reasoning processes, including retrieval efficiency, query quality, and multi-step logical consistency.
B Deep Research Implementations
Deep research implementations use agentic RAG systems that decompose complex questions, interleave retrieval with reasoning, and synthesize evidence through single- or multi-agent architectures. The survey compares these implementations, retrievers, reasoning workflows, and orchestration strategies while highlighting trade-offs in robustness, efficiency, complexity, and cost.
- Deep Research Implementations: Deep research systems autonomously break down complex questions, iteratively gather diverse evidence, and synthesize information through multiple reasoning steps.They plan multistep queries, interleave retrieval with reasoning, and coordinate specialized tools or agents.
- Agent Architectures: Single-agent architectures manage decomposition, retrieval, and synthesis within one model, while multi-agent systems distribute responsibilities among specialized agents.Single-agent designs offer simplicity and shared context; multi-agent designs provide modularity and potentially greater robustness.
- Agent Architectures: Multi-agent orchestration increases coordination and communication complexity, with higher computational costs than simpler designs.These costs arise alongside the distribution of responsibilities among specialized agents.
- Retrieval Implementations: Deep research retrievers have evolved from sparse keyword search to dense semantic matching, web search, and LLM-callable retrieval tools.Web-search retrievers provide real-time access to open-domain information.
- Reasoning Workflows: Reasoning workflows range from efficient linear chains to tree- and graph-based methods with higher recall and transparency but greater computational overhead.Linear approaches remain vulnerable to error propagation, whereas more complex workflows incur additional costs.
- Agent Orchestration: Agent orchestration ranges from single-agent setups to multi-agent systems that enhance robustness and scalability but add communication overhead and conflict-resolution complexity.The comparison emphasizes balancing efficiency, accuracy, and resource constraints when selecting workflows and architectures.