PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Knowledge-Oriented Retrieval-Augmented Generation

Mingyue Cheng, Yucong Luo, Jie Ouyang, Qi Liu, Huijie Liu, Li Li, Shuo Yu, Bohou Zhang, Jiawei Cao, Jie Ma, Daoyu Wang, Enhong Chen

arXiv:2503.10677v3cs.CLcs.AI

TL;DR

Existing surveys overlook RAG’s knowledge utilization, motivating this paper’s knowledge-centric synthesis. The survey reviews notable RAG work and proposes an organizing framework, reporting improved NLP applications through relevant external context while identifying knowledge selection and retrieval as key challenges.

  • Problem

    Current surveys generally overlook RAG’s essence—knowledge utilization—leaving a gap this paper aims to fill.

  • Method

    The paper conducts a systematic, extensive review of notable RAG works from a knowledge-centric perspective and proposes an overarching organizing framework.

  • Results

    RAG systems have demonstrated significant improvements across NLP applications, including open-domain question answering, by incorporating relevant external context.

  • Takeaways & Limitations

    RAG can address language models’ limited access to real-time or domain-specific knowledge and produce more informed, contextually accurate outputs.

  • Takeaways & Limitations

    RAG remains challenged by selecting relevant knowledge and retrieving information from real-world knowledge corpora.

Abstract

from arXiv · show

Retrieval-Augmented Generation (RAG) has gained significant attention in recent years for its potential to enhance natural language understanding and generation by combining large-scale retrieval systems with generative models. RAG leverages external knowledge sources, such as documents, databases, or structured data, to improve model performance and generate more accurate and contextually relevant outputs. This survey aims to provide a comprehensive overview of RAG by examining its fundamental components, including retrieval mechanisms, generation processes, and the integration between the two. We discuss the key characteristics of RAG, such as its ability to augment generative models with dynamic external knowledge, and the challenges associated with aligning retrieved information with generative objectives. We also present a taxonomy that categorizes RAG methods, ranging from basic retrieval-augmented approaches to more advanced models incorporating multimodal data and reasoning capabilities. Additionally, we review the evaluation benchmarks and datasets commonly used to assess RAG systems, along with a detailed exploration of its applications in fields such as question answering, summarization, and information retrieval. Finally, we highlight emerging research directions and opportunities for improving RAG systems, such as enhanced retrieval efficiency, model interpretability, and domain-specific adaptations. This paper concludes by outlining the prospects for RAG in addressing real-world challenges and its potential to drive further advancements in natural language processing.

1 Introduction

RAG augments language models with dynamically retrieved external knowledge, improving contextual accuracy while addressing limitations involving real-time, domain-specific, and rare-entity knowledge. This survey organizes RAG around the lifecycle of knowledge and reviews its methods, challenges, applications, evaluation, and future directions.

  • RAG foundations: RAG dynamically retrieves documents, databases, or structured data during generation to produce more informed and contextually accurate outputs.This distinguishes RAG from models relying solely on pre-trained parameters.
  • RAG foundations: External knowledge helps RAG address limited access to real-time or domain-specific information and difficulties handling rare or out-of-vocabulary entities.
  • Applications and capabilities: RAG improves applications including open-domain question answering, document summarization, specialized-topic handling, dialogue, and knowledge-intensive tasks.Retrieved passages support more precise answers, while external documents can produce richer summaries reflecting source nuances.
  • Knowledge-centric perspective: Knowledge-centric RAG can retrieve, incorporate, and reason over multiple information pieces, enabling more nuanced, informed, and contextually grounded outputs.The survey frames this as a lifecycle spanning identification, retrieval, integration, and reasoning with external information.
  • Challenges: RAG still faces challenges in selecting relevant information from large, noisy, diverse sources and retrieving it efficiently and accurately at generation time.Models must also integrate and reason over retrieved knowledge coherently, especially in specialized or ambiguous contexts.
  • Survey scope and contribution: The survey structures prior work around knowledge utilization and the knowledge lifecycle, covering basic and advanced methods, evaluation, applications, emerging trends, and research gaps.It particularly highlights multimodal knowledge integration and domain-specific applications as areas for further exploration.

2 Related Work

RAG research evolved from early retrieval-enhanced generation into a broad field spanning external knowledge access, multimodal and graph-based methods, and evaluation. This survey addresses gaps in prior surveys by organizing RAG through a comprehensive knowledge-centric perspective.

  • RAG evolution: RAG was formally introduced in 2020 for knowledge-intensive tasks, while REALM achieved state-of-the-art results in open-domain question answering.The field expanded rapidly after ChatGPT, with retrieval-augmented approaches increasingly shaping modern NLP.
  • RAG evolution: RAG combines retrieval with generation so models can dynamically access external information during complex reasoning.The integration is associated with more informative and factually accurate generated outputs.
  • Survey gaps: Existing surveys often focus on specific foundations, enhancement methods, text-based tasks, or individual application areas rather than a comprehensive view.They also frequently omit multimodal coverage and the central role of knowledge utilization.
  • This survey: This survey fills that gap by systematically examining RAG through a knowledge-centric approach.Its scope includes RAG foundations, enhancement methods, and applications across different domains.

3 Overview of Retrieval-Augmented Generation

RAG extends conventional generation by retrieving external knowledge and integrating it with an LLM’s internal knowledge. The overview covers retrieval across heterogeneous sources, generation through denoising and reasoning, and multiple integration strategies.

  • Core formulation: RAG transforms generation from y = f(x) into y = f(x, z) = f(x, g(x)) by adding retrieved context z to input x.This formulation enriches generation for tasks requiring complex contextual understanding.
  • Retrieval: Retrieval obtains relevant information from external knowledge bases regardless of modality, storage format, or structural organization.Sources may include text, images, video, audio, code, tables, databases, search engines, HTML, and graphs.
  • Generation: Generation combines internal and external knowledge through denoising and reasoning to filter unreliable information and synthesize multiple sources.These processes support coherent and accurate outputs on complex tasks.
  • Knowledge integration: Knowledge integration merges retrieved external knowledge with LLM knowledge through input-layer, output-layer, or intermediate-layer strategies.The strategies respectively concatenate documents with queries, adjust logits, or modify hidden states.
  • Knowledge integration: Integration strategies offer different advantages and should be selected according to task requirements and computational constraints.This frames integration as a design choice rather than a single fixed mechanism.

4 Fundamentals and Objectives of RAG

RAG systems depend on coordinated user-intent understanding, retrieval, knowledge integration, answer generation, and evaluation. The section emphasizes relevance, efficiency, consistency, adaptability, and holistic assessment across these stages.

  • User intent understanding: User-intent understanding must move beyond keyword matching because queries can be ambiguous, implicit, personalized, or culturally dependent.Contextual and semantic techniques such as query rewriting and intent clustering support more precise retrieval and generation.
  • Knowledge retrieval: Retrieval must balance precision, recall, accuracy, and efficiency while handling large, heterogeneous, noisy, and frequently changing knowledge sources.Indexing, approximate nearest-neighbor search, scalable representations, preprocessing, and continual adaptation address these pressures.
  • Knowledge integration: Knowledge integration must align retrieved information with internal LLM knowledge while preventing inconsistencies, factual inaccuracies, and conflicts.Merging text chunks and structured data into a unified representation and resolving contradictions are central challenges.
  • Answer generation: Answer generation synthesizes heterogeneous information into coherent, contextually appropriate outputs tailored to user intent and communicative setting.The model may need to provide concise answers, summaries, or dialogue while preserving fluency and naturalness.
  • RAG evaluation: RAG evaluation must assess both retrieval and generation because BLEU and ROUGE alone may not capture retrieval effectiveness.Comprehensive assessment combines retrieval precision, recall, and F1-score with generation coherence, fluency, and related qualitative judgments.

5 A Taxonomy of RAG Methods

The survey proposes a taxonomy that organizes RAG methods into basic approaches, advanced approaches, and evaluation strategies. It covers the end-to-end basic workflow, extensions such as multimodal and agentic RAG, and effectiveness–efficiency assessment.

  • Taxonomy structure: The taxonomy organizes the RAG landscape into three branches: basic approaches, advanced approaches, and evaluation strategies.This structure provides a reference framework for the field’s rapidly growing methods and architectures.
  • Basic approaches: Basic approaches cover the workflow from user intent understanding and knowledge preparation through retrieval, integration, generation, and citation.The workflow includes parsing, embedding, indexing, and producing a final cited response.
  • Advanced approaches: Advanced approaches extend the basic pipeline through RAG training, multimodal RAG, memory RAG, and agentic RAG.These methods enhance the capabilities of the core retrieval-augmented workflow.
  • Evaluation strategies: Evaluation strategies measure effectiveness through relevance, coherence, and accuracy, and efficiency through latency, throughput, and resource utilization.The framework therefore assesses both generated-content quality and computational performance.

6 Basic RAG Approaches

Basic RAG approaches form a multi-stage pipeline that interprets user intent, processes and indexes heterogeneous knowledge, retrieves relevant information, integrates it, and generates cited answers. The survey also describes query decomposition and rewriting, diverse knowledge sources, and structured-data methods as ways to improve retrieval, reasoning, and response quality.

  • Pipeline: The basic RAG pipeline covers intent understanding, knowledge parsing, embedding, indexing, retrieval, integration, answer generation, and citation.It converts diverse information into searchable representations before producing answers with supporting citations.
  • Query Understanding: Query decomposition breaks complex questions into simpler subproblems, supporting multi-step and compositional reasoning.Self-ask extends this strategy by generating and answering follow-up questions for multi-hop reasoning.
  • Query Understanding: Reasoning-integrated methods combine retrieval with verification or chained actions to improve accuracy, traceability, and robustness in knowledge-intensive tasks.CoVe verifies intermediate results, SearChain retrieves during Chain-of-Query reasoning, and CoA unifies decomposition, retrieval, and verification.
  • Query Understanding: Query rewriting improves retrieval by narrowing semantic gaps between user queries and target knowledge.Approaches include reinforcement-learning-based rewriting, supervised methods for long-tail queries, hypothetical-document generation, and high-level abstraction.
  • Knowledge Sources: RAG uses structured, semi-structured, unstructured, and multimodal knowledge, but parsing and integrating heterogeneous sources remain significant challenges.Knowledge graphs support retrieval precision, reasoning depth, and domain adaptability, while tables require semantic interpretation and structured-query generation.
  • Structured Knowledge: Structured-data methods such as TableRAG use query expansion, schema retrieval, and cell retrieval to identify relevant tabular information before generation.The survey also notes challenges from implicit table relationships, domain-specific table structures, and translating natural-language questions into database queries.

Input Layer Integration.

Input-layer integration combines retrieved information with the original query before generation. Methods range from direct text concatenation to feature-level representations, with refinement techniques addressing noise, context limits, and efficiency.

  • Input Layer Integration.: Input-layer methods operate on raw text or feature representations to enhance the context supplied to the generator.
  • Input Layer Integration.: Text-level integration concatenates the top-k retrieved documents with the original query.This straightforward approach can introduce noise and exceed the model’s context window.
  • Input Layer Integration.: Reranking and weighted filtering improve concatenated context by prioritizing relevant content and removing irrelevant information.
  • Input Layer Integration.: Context compression preserves essential information while reducing token count, enabling more efficient generation within a fixed window.
  • Input Layer Integration.: Feature-level integration transforms queries and retrieved documents into dense or sparse vectors before combining them with the model.This provides finer control over external-knowledge fusion than simple textual concatenation.

Intermediate Layer Integration.

Intermediate-layer integration injects retrieved knowledge into the generator’s hidden representations. It offers computationally cheaper fusion or context-aware attention, but these choices involve specificity, architectural, and computational trade-offs.

  • Intermediate Layer Integration.: Intermediate-layer methods condition the model’s hidden layers on retrieved information during generation.
  • Intermediate Layer Integration.: Attention-based integration merges retrieved information with intermediate representations through dedicated cross-attention or specialized memory structures.RETRO, TOME, and LongMem exemplify these approaches.
  • Intermediate Layer Integration.: Attention-based methods provide context-aware fusion but increase computational complexity.
  • Intermediate Layer Integration.: Some intermediate integration methods incorporate knowledge through simple weighted additions rather than attention.
  • Intermediate Layer Integration.: Weighted embedding integration sums learned weights for top-k retrieved documents into intermediate layers as a computationally cheaper fusion method.It may lack cross-attention’s dynamic, token-level specificity.

Output Layer Integration.

Output-layer integration modifies the generator’s final output distribution rather than its input or intermediate representations. Ensemble and calibration methods use retrieved information to broaden knowledge or refine prediction confidence without altering the core architecture.

  • Output Layer Integration.: Logit-level methods use external information as verification or correction for final predictions in translation, code summarization, and image captioning.
  • Output Layer Integration.: Output-layer integration influences final token probabilities without altering the core generator architecture.
  • Output Layer Integration.: Ensemble-based integration interpolates the language model’s predictions with probabilities derived from retrieved neighbors.kNN-LM is presented as the canonical example.
  • Output Layer Integration.: kNN-LM broadens the model’s knowledge at the decision point, improving generalization and robustness.
  • Output Layer Integration.: Calibration-based integration uses retrieval-based logits to adjust or refine the model’s prediction confidence.

Contrastive Decoding as Post-hoc Integration.

Contrastive Decoding is a post-hoc, inference-time integration strategy that contrasts token likelihoods with and without retrieved context. The broader generation pipeline pairs denoising with reasoning, selecting strategies according to task demands and knowledge structure.

  • Contrastive Decoding as Post-hoc Integration.: Contrastive Decoding balances internal and retrieved knowledge by contrasting token likelihoods with and without retrieved context.The adjustment is controlled by α and operates at inference time without additional training.
  • Contrastive Decoding as Post-hoc Integration.: The contrastive formulation penalizes generation paths that deviate from external evidence, helping mitigate hallucinations and factual errors during knowledge conflicts.
  • Contrastive Decoding as Post-hoc Integration.: Contrastive Decoding offers lightweight factual-consistency and conflict-resolution support without architectural modification or retraining.
  • Contrastive Decoding as Post-hoc Integration.: RAG generation requires denoising irrelevant or contradictory retrievals and reasoning over retained information to produce coherent, relevant responses.These capabilities involve a trade-off between fidelity to retrieved knowledge and intrinsic generative ability.
  • Contrastive Decoding as Post-hoc Integration.: Denoising spans confidence-based filtering, discriminator quality gates, explicit supervision, iterative self-correction, and adaptive retrieval.The spectrum ranges from efficient real-time filtering to computationally intensive self-correction for complex tasks.
  • Contrastive Decoding as Post-hoc Integration.: Reasoning strategies include cross-attention across retrieved documents, memory modules for longitudinal consistency, and graph-based reasoning for structured relationships.These approaches target multi-document synthesis, long-context coherence, and domain-specific relational inference.
  • Contrastive Decoding as Post-hoc Integration.: Think-on-Graph 2.0 combines knowledge graphs with unstructured text for relational reasoning that is difficult to infer from text alone.The approach is suited to expert systems and tasks requiring deep domain-specific knowledge.
  • Contrastive Decoding as Post-hoc Integration.: Cross-attention-based generation can align answers with citations, reducing hallucinations and improving factual citations.

7 Advanced RAG Approaches

Advanced RAG approaches extend basic systems through improved training, multimodal processing, memory, agentic reasoning, and parameter-level knowledge integration. These methods aim to expand RAG’s ability to address complex, dynamic, and specialized tasks while balancing retrieval and generation optimization.

  • Advanced RAG methods target training optimization, multimodal processing, memory enhancement, agentic reasoning, and parameter-level knowledge integration.The survey identifies five advancement areas spanning retrieval-generation synergy, multiple sensory modalities, long-term memory, dynamic optimization, and knowledge integrated into model parameters.
  • RAG Training: RAG training balances retriever relevance and generator quality through static, unidirectional guided, and collaborative optimization strategies.Static training fixes one component, guided training lets one component direct the other, and collaborative training refines both components together.
  • Static Optimization Training: Static optimization enables faster training and lower resource use but may limit retrieval-generation synergy and task adaptability.The approach is useful when resources are limited or rapid deployment is required, though fixing one component can prevent full interaction between components.
  • Memory RAG: Memory RAG introduces explicit memory as an intermediate layer between raw-text retrieval and model parameters, preserving holistic understanding while remaining updateable and efficient.Explicit memory provides compressed, structured knowledge that is more updateable than model parameters and more efficient than raw-text processing.
  • Parametric RAG: Parametric RAG integrates retrieved knowledge into LLM parameters, reducing online computation and improving reasoning efficiency through offline precomputation.The survey reports that parameterization can internalize document-level knowledge more effectively than prompt concatenation while keeping computational cost affordable.
  • Parametric RAG: Parametric knowledge integration reduced inference time by up to 36% while maintaining or exceeding in-context RAG baselines.The reported comparison describes lower latency alongside complementary knowledge injection and in-context retrieval paradigms.

8 Comprehensive Evaluation Strategies for RAG

Comprehensive RAG evaluation examines retrieval quality, answer grounding, factual accuracy, efficiency, and citation reliability. The survey emphasizes that automated metrics remain limited by semantic shortcuts, surface-level agreement, and computational cost.

  • RAG evaluation assesses both effectiveness and efficiency, including retrieval relevance, answer coherence and accuracy, computational resources, latency, and scalability.Effectiveness concerns how retrieved context supports generation, while efficiency concerns the resources and time required by retrieval and generation.
  • Retrieval Evaluation: Context-oriented benchmarks such as ARES, RAGAS, TruLens, and KILT measure query-context alignment, relevance, richness, and completeness.ARES, RAGAS, and TruLens emphasize immediate alignment, whereas KILT provides broader measures including R-precision and Recall@k.
  • Evaluation Limitations: Semantic-similarity metrics may treat topically similar documents as relevant even when they lack the facts needed to answer a query.Automated faithfulness checks can also miss subtle hallucinations or contradictions, while granular verification introduces significant computational overhead.
  • Answer Evaluation: Answer-oriented metrics evaluate factual correctness, faithfulness to retrieved passages, misleading content, and claim-level evidential support.The reviewed frameworks include Recall and Precision, Answer Faithfulness, Accuracy, Misleading Rate, and claim-level Groundedness.
  • Citation Evaluation: Citation-focused evaluation measures attribution correctness, citation redundancy, verifiability, consistency, relevance, and conciseness.Frameworks such as RECLAIM, RARR, WebGPT, and ALCE extend evaluation toward transparent and grounded attribution.
  • Efficiency Trade-offs: RAG evaluation reveals a trade-off between reasoning adaptivity, computational cost, and latency efficiency across retrieval paradigms.Agentic RAG can improve factual accuracy but increase latency, Graph RAG faces synchronization and freshness bottlenecks, and Parametric RAG can reduce latency while increasing training cost.

9 Downstream Tasks and Applications

RAG supports downstream tasks by combining retrieval with generation to provide factual grounding, contextual enrichment, and access to timely or domain-specific information. Applications span question answering, information extraction, text understanding, scientific research, finance, education, healthcare, law, and industrial settings.

  • RAG downstream applications include question answering, information extraction, text understanding, text generation, and summarization.Table 5 is described as organizing representative datasets across question answering, information extraction, and text understanding with corresponding sub-task categories.
  • Question Answering: In question answering, RAG retrieves targeted evidence for factual answers and synthesizes multiple sources for multi-hop reasoning.The survey distinguishes single-hop questions requiring one passage from multi-hop questions requiring evidence gathered and combined across sources.
  • Information Extraction: In information extraction, retrieved evidence improves entity linking, disambiguation, relation identification, and construction of structured knowledge resources.Retrieval supplies contextual information about entities and their relationships, supporting more accurate extraction.
  • Text Understanding and Generation: In text understanding and generation, retrieval adds contextual depth for classification, summarization, and incorporation of timely or domain-specific information.Retrieved evidence helps models focus on salient points for concise summaries and enrich generated content with additional context.
  • Scientific Applications: Scientific applications use RAG to retrieve and synthesize domain-specific knowledge for materials science, chemistry, physics, and life-science tasks.Examples include material design, synthesis planning, reaction prediction, molecular-interaction analysis, physics question answering, and specialized biological question answering.
  • Financial Applications: Financial RAG applications improve retrieval and decision support for information extraction, question answering, sentiment analysis, invoices, and reports.The survey describes more accurate financial QA and sentiment predictions enabled by retrieved context and improved retrieval methods.

10 Prospects and Future Directions

Future RAG research addresses structured knowledge, multimodal data, personalization, agentic reasoning, generative integration, edge deployment, and long-context reasoning. The survey repeatedly identifies scalability, interpretability, updating, privacy, and efficiency as practical boundaries.

  • GraphRAG: GraphRAG connects entities and relations to retrieval, supporting more interpretable and robust performance for complex factual and reasoning tasks.Graph nodes and edges can serve as composable contexts for multi-hop reasoning and entity alignment.
  • GraphRAG: GraphRAG remains limited by heterogeneous cross-domain or cross-lingual integration, dynamic graph correction, real-time responsiveness, and scalability.The survey identifies efficient retrieval and high-concurrency operation as continuing research directions.
  • Multimodal RAG: Multimodal RAG integrates visual, auditory, textual, and sensor information to improve multimodal question answering, scene understanding, and product recommendation.Future work requires scalable representation learning, cross-modal retrieval optimization, adaptive fusion, privacy, robustness, and incremental updates.
  • Personalized RAG: Personalized RAG uses user-specific knowledge and memory mechanisms to produce more relevant, context-aware responses while maintaining efficiency and scalability.Open issues include memory extraction, storage, updating, fusion, privacy, security, and adaptation across multilingual and cross-domain settings.
  • Agentic RAG: Agentic RAG enables autonomous planning, tool use, iterative retrieval, and multi-step reasoning for tasks beyond traditional RAG frameworks.The survey describes agents that decide when and how to search, refine evidence across cycles, and progressively analyze information to produce comprehensive reports.
  • Agentic RAG: Agentic RAG remains constrained by error propagation, limited interpretability, reliance on publicly accessible information, and weaker nuanced analysis than summarization.The survey also identifies efficiency, correctness, distributed deployment, and hardware heterogeneity as open challenges.

11 Conclusion

This survey organizes RAG research around the lifecycle of knowledge, from sourcing and retrieval through integration and generation. It analyzes stage-specific challenges, classifies emerging approaches, and identifies open issues and research directions for knowledge-intensive applications.

  • The survey organizes existing RAG research around the lifecycle of knowledge, from sourcing and retrieval to integration and generation.
  • The framework clarifies relationships and trade-offs between different RAG methods across the knowledge lifecycle.
  • Analyzing challenges in knowledge parsing, integration, and context adaptation supports a classification scheme for multimodal and memory-augmented RAG.
  • The survey identifies critical open issues and promising research directions for RAG across diverse knowledge-intensive domains.
  • The authors aim to provide researchers with understanding of RAG’s components, challenges, developments, and opportunities for future studies.
Loading 2503.10677v3…