Source-linked AI summary
Knowledge Graph-Guided Retrieval Augmented Generation
Xiangrong Zhu, Yuexiang Xie, Yi Liu, Yaliang Li, Wei Hu
TL;DR
RAG can mitigate LLM hallucinations, but semantic retrieval often returns isolated chunks while ignoring their relationships. KG2RAG uses semantic seeds, KG-guided expansion, and KG-based organization to produce diverse, coherent context, with experiments on HotpotQA and variants showing superior response and retrieval quality.
Problem
Semantic-based RAG retrieves isolated relevant chunks while overlooking their intrinsic relationships, limiting the organization and usefulness of retrieved knowledge.
Method
KG2RAG associates chunks with a knowledge graph, expands semantic seed chunks through graph relationships, and organizes selected chunks into coherent paragraphs.
Results
KG2RAG consistently outperforms RAG-based baselines in response quality and retrieval quality on HotpotQA and its variants.
Takeaways & Limitations
KG2RAG produces retrieved chunks that are diverse, intrinsically related, and self-consistent for LLM response generation.
Takeaways & Limitations
KG2RAG focuses on retrieval optimization and does not optimize other RAG modules, although it is described as compatible with them.
Abstract
from arXiv · showhide
Retrieval-augmented generation (RAG) has emerged as a promising technology for addressing hallucination issues in the responses generated by large language models (LLMs). Existing studies on RAG primarily focus on applying semantic-based approaches to retrieve isolated relevant chunks, which ignore their intrinsic relationships. In this paper, we propose a novel Knowledge Graph-Guided Retrieval Augmented Generation (KG$^2$RAG) framework that utilizes knowledge graphs (KGs) to provide fact-level relationships between chunks, improving the diversity and coherence of the retrieved results. Specifically, after performing a semantic-based retrieval to provide seed chunks, KG$^2$RAG employs a KG-guided chunk expansion process and a KG-based chunk organization process to deliver relevant and important knowledge in well-organized paragraphs. Extensive experiments conducted on the HotpotQA dataset and its variants demonstrate the advantages of KG$^2$RAG compared to existing RAG-based approaches, in terms of both response quality and retrieval quality.
1 Introduction
RAG mitigates LLM hallucinations by retrieving external knowledge, but semantic retrieval can produce redundant, disconnected chunks. KG2RAG addresses this by expanding and organizing chunks through knowledge-graph relationships, and experiments report gains in response and retrieval quality.
- RAG retrieves relevant document knowledge for LLM prompts, helping mitigate hallucinations caused by outdated information or missing domain-specific knowledge.
- Semantic-based RAG may retrieve homogeneous, redundant chunks without intrinsic relationships, limiting comprehensive and reliable responses.
- Knowledge graphs represent entities and relations as linked triplets, supplying fact-level relationships and highlighting facts semantic retrieval may miss.
- KG2RAG first retrieves semantic seed chunks, then uses graph-guided expansion to include chunks connected by overlapping or related entities and triplets.
- KG-based context organization filters retrieved information and arranges chunks into internally coherent paragraphs before feeding them to LLMs.
- Experiments on HotpotQA and variants show KG2RAG consistently outperforms RAG-based baselines in response quality and retrieval quality.
2 Methodology
KG2RAG combines semantic retrieval with knowledge-graph expansion and organization to retrieve factually connected chunks and present them as coherent context. Its workflow includes offline chunk–KG association, graph-guided expansion, relevance filtering, and paragraph arrangement before LLM response generation.
- Document Offline Processing: Documents are split into chunks and associated with a knowledge graph linking extracted entities and relations to their source chunks.The association can use an existing KG or extract entities and relations directly from chunks; KG2RAG adopts the latter approach with LLM prompts.
- KG-enhanced Chunk Retrieval: Semantic retrieval selects the top-k chunks most similar to the user query as seed chunks for subsequent graph-guided expansion.Similarity is computed from embedding-based representations using cosine similarity.
- KG-enhanced Chunk Retrieval: Graph-guided expansion traverses m-hop neighborhoods around the seed chunks and retrieves chunks associated with the resulting expanded subgraph.This gathers chunks containing the same or related entities or triplets, rather than requiring high semantic similarity or nearby document positions.
- KG-enhanced Chunk Retrieval: Graph-guided expansion is designed to reduce redundancy and excessive homogeneity while increasing diversity and developing a more comprehensive knowledge network.The design addresses the risk that purely semantic retrieval produces isolated chunks and misses factual connections.
- KG-based Context Organization: KG-based context organization filters redundant linking information to retain relevant facts and improve the informativeness of retrieved chunks.Expanded chunks are converted into an undirected weighted graph, divided into connected components, and filtered using maximum spanning trees.
- KG-based Context Organization: The organization module arranges retrieved chunks into intrinsically related, self-consistent paragraphs using the knowledge graph as a skeleton.Chunks linked to each tree's edges are concatenated through depth-first search, while triplet representations support relevance reranking.
3 Experiments
Experiments evaluate KG2RAG on HotpotQA and shuffled variants under distractor and fullwiki settings, comparing response and retrieval quality with multiple RAG baselines and ablations. Results show consistent gains, while ablations clarify complementary roles for graph expansion and context organization.
- Datasets and setup: HotpotQA experiments cover distractor and fullwiki settings, including variants that reduce reliance on LLM prior knowledge by replacing entities consistently across queries, triplets, and documents.The fullwiki setting searches 66,581 Wikipedia documents, whereas distractor supplies ten supporting documents; shuffled variants require identifying relevant document content rather than relying on memorized facts.
- Response quality: KG2RAG consistently outperforms RAG baselines in response quality, especially in fullwiki and Shuffle-HotpotQA settings.Against LLM-only generation, RAG methods improve F1 by more than 29.1% on original HotpotQA and 26.4% on Shuffle-HotpotQA.
- Response quality: At least 8% improvement over baselines occurs in fullwiki response quality, while Shuffle-HotpotQA gains reach at least 2.5% in distractor and 6.4% in fullwiki settings.The fullwiki comparison involves a candidate pool thousands of times larger than distractor, making retrieval and context organization especially important.
- Retrieval quality: KG2RAG achieves similar recall but over 7.9% and 6.9% higher precision on HotpotQA and Shuffle-HotpotQA distractor settings, respectively, and improves both precision and recall in fullwiki.Retrieval evaluation uses F1, precision, and recall against referenced facts; response evaluation compares generated answers with ground truth.
- Ablation analysis: Removing context organization preserves similar answer quality but worsens retrieval quality, because graph expansion retrieves more irrelevant chunks and consumes additional tokens.The organization module filters and arranges expanded chunks to preserve relevant information.
- Ablation analysis: Removing graph expansion yields high retrieval precision and F1 with fewer chunks but weaker responses because semantic retrieval can miss necessary fact-linked chunks.The ablation supports complementary roles: organization selects and structures evidence, while expansion recovers information missed by semantic similarity.
4 Related Work
Prior RAG work retrieves relevant chunks using semantic or keyword similarity, while KG-based approaches introduce structured relationships to improve retrieval and context organization.
- Retrieval-Augmented Generation: RAG retrieves relevant chunks from candidate documents to assist LLM response generation.
- Retrieval-Augmented Generation: Sentence-window retrieval enriches context by fetching neighboring chunks around retrieved chunks and concatenating them.
- Retrieval-Augmented Generation: Reranking measures semantic relevance more precisely, while KG2RAG organizes chunks into paragraphs using KGs as a skeleton.
- LLMs with Knowledge Graph: KG research also examines LLM applications in knowledge graph completion and knowledge graph question answering.
- LLMs with Knowledge Graph: KGP uses a document KG for retrieval expansion, whereas GraphRAG extracts KGs to analyze document semantics for query-focused summarization.
5 Conclusion
KG2RAG integrates knowledge graphs into RAG through graph-guided expansion and context organization, producing diverse, related, self-consistent paragraphs. Experiments and ablations report stronger response and retrieval quality than existing RAG approaches.
- 5 Conclusion: KG2RAG links chunks to a specific KG to provide fact-level relationships among retrieved chunks.
- 5 Conclusion: KG-guided chunk expansion and KG-based context organization make retrieved chunks diverse, intrinsically related, and self-consistent.
- 5 Conclusion: KG2RAG outperforms existing RAG-based approaches in both response quality and retrieval quality.
- 5 Conclusion: An ablation study indicates that KG-guided expansion and KG-based context organization collaboratively enhance KG2RAG effectiveness.
Limitations
KG2RAG focuses on retrieval optimization rather than other RAG modules, although it is described as compatible with them.
- Limitations: KG2RAG optimizes KG-guided retrieval expansion and KG-based context organization, not query rewriting or multi-turn dialogue.
- Limitations: The framework is described as orthogonal and compatible with other RAG modules, with future work targeting plug-and-play integration.
A.1 Results on More Datasets
Additional experiments evaluate KG2RAG on MuSiQue and TriviaQA, extending assessment beyond HotpotQA. The reported results support its effectiveness across multiple datasets and a long-context setting.
- A.1 Results on More Datasets: Additional experiments assess KG2RAG on two datasets to examine effectiveness and generality across scenarios.
- A.1 Results on More Datasets: KG2RAG maintains superiority on MuSiQue in response F1, response exact match, and retrieval F1.
- A.1 Results on More Datasets: TriviaQA documents average 2,895 words, compared with 917 words for HotpotQA.
- A.1 Results on More Datasets: Results on TriviaQA confirm KG2RAG’s effectiveness in a typical long-context setting.
A.2 Efficiency Analysis
KG²RAG is more efficient than LightRAG and GraphRAG in knowledge-graph construction costs, while its retrieval and generation times remain competitive with these approaches and close to Semantic RAG.
- KG²RAG has lower token cost, fewer LLM calls, and lower time cost than LightRAG and GraphRAG for KG construction.These costs are compared per chunk during knowledge-graph construction.
- KG²RAG requires less retrieval and response-generation time than LightRAG.The comparison uses average retrieval and generation time per query.
- KG²RAG requires less retrieval and response-generation time than GraphRAG and is very close to Semantic RAG.The response-generation advantage may reflect use of a condensed, informative context.