PaperlayerView exact revision on arXiv

Source-linked AI summary

Retrieval-Augmented Generation with Hierarchical Knowledge

Haoyu Huang, Yongfeng Huang, Junjie Yang, Zhenyu Pan, Yongqiang Chen, Kaili Ma, Hongzhi Chen, James Cheng

arXiv:2503.10150v3cs.CLcs.AI

TL;DR

Graph-based RAG methods have not adequately incorporated hierarchical knowledge, while graph structure can still leave semantically related entities distant. HiRAG uses hierarchical indexing and bridging in retrieval; the paper reports superior performance over existing methods.

  • Problem

    Existing approaches overlook hierarchical knowledge, and graph structure can leave semantically related entities distant, undermining RAG utility where contextual coherence matters.

    Source: §1 · pp. 1–2

  • Method

    HiRAG combines hierarchical indexing of clustered entities with a bridging mechanism that integrates knowledge across layers.

    Source: §1 · p. 2

  • Results

    HiRAG achieves superior performance than existing methods, and the authors report extensive experiments demonstrating effectiveness and efficiency.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The paper's approach pairs hierarchical indexing with knowledge bridging to address structural and local-global knowledge gaps in graph-based RAG.

    Source: §1 · p. 2

    HiRetrieval relies solely on LLM-generated weights for relation ranking, which may affect query relevance.

    Source: §7 · p. 9

Abstract

from arXiv · show

Graph-based Retrieval-Augmented Generation (RAG) methods have significantly enhanced the performance of large language models (LLMs) in domain-specific tasks. However, existing RAG methods do not adequately utilize the naturally inherent hierarchical knowledge in human cognition, which limits the capabilities of RAG systems. In this paper, we introduce a new RAG approach, called HiRAG, which utilizes hierarchical knowledge to enhance the semantic understanding and structure capturing capabilities of RAG systems in the indexing and retrieval processes. Our extensive experiments demonstrate that HiRAG achieves significant performance improvements over the state-of-the-art baseline methods.

1 Introduction

Graph-based RAG can miss relationships among entities and struggles with gaps between local and global knowledge. HiRAG addresses these challenges through hierarchical indexing and retrieval, and the paper reports experimental effectiveness and efficiency.

1 Introduction

  • GraphRAG's communities capture structural proximity, KAG depends on manual annotation, and LightRAG does not bridge disparities between local and global knowledge.Naive RAG also tends to overlook relationships among entities in retrieved text chunks.Source: §1 · p. 1
  • Existing graph-based RAG can leave semantically related entities structurally distant and fail to align local with global knowledge.The paper illustrates the second gap with Amazon: local entity details and global community summaries may produce disjointed or contradictory answers when unaligned.Source: §1 · pp. 1–2
  • HiRAG combines hierarchical indexing, where higher-layer entities summarize lower-layer clusters, with retrieval that bridges local entity descriptions and global community knowledge.The proposed three-level context comprises global, bridge, and local knowledge.Source: §1 · p. 2
  • HiRAG targets both distant structural relationships between semantically similar entities and the gap between local and global information.Its stated contributions are unsupervised hierarchical indexing and a bridging mechanism for knowledge integration.Source: §1 · p. 2
  • The authors report that extensive experiments demonstrate HiRAG's effectiveness and efficiency, with ablations validating each component's contribution.The supplied introduction gives no quantitative results.Source: §1 · p. 2

2 Related Work

Related work surveys graph-augmented LLM methods that use graph reasoning, retrieval, pruning, and task-specific structures. It also describes approaches that retrieve communities, align knowledge, or refine supervision.

2 Related Work

  • GNN-RAG retrieves query-related entities and uses shortest paths, while LightRAG combines dual-level retrieval with graph-enhanced text indexing.GRAG applies soft pruning and graph soft prompts; StructRAG selects a task-suitable structure before response generation.Source: §2 · p. 3
  • Microsoft GraphRAG retrieves related communities and supports global and local search; KAG uses conceptual semantic reasoning and human-annotated schemas.KAG is described as a professional-domain knowledge service framework.Source: §2 · p. 3
  • ReG uses LLMs to refine weak supervision signals for fine-grained graph-based RAG, while GNN-RAG constructs reasoning paths between retrieved entities and candidate answers.The section situates these methods among graph-augmented LLM approaches.Source: §2 · pp. 2–3

3 Preliminary and Definitions

This section formulates graph-structured RAG in terms of generation, retrieval, and graph indexing. It describes selecting a retrieved subgraph to approximate the answer-generation objective.

3 Preliminary and Definitions

  • The RAG framework assigns generation to the LLM, retrieval to R, graph indexing to φ, and graph retrieval to ψ.The graph database G is constructed from an external database D.Source: §3 · p. 3
  • M(a|q,G) denotes the target distribution for an answer given query q and graph G, with retrieval and LLM generation components.A is the set of possible responses.Source: §3 · p. 3
  • The framework approximates the answer objective by retrieving an optimal subgraph G* from the external graph database.The formulation says that, in most cases, only the most relevant subgraph needs to be retrieved.Source: §3 · p. 3

4 The HiRAG Framework

HiRAG builds a layered knowledge graph that aggregates semantically related entities, then retrieves local, global, and bridging knowledge for answer generation.

4.1 Indexing with Hierarchical Knowledge

  • HiIndex starts with entity-centric triples, then embeds and GMM-clusters each layer, summarizes clusters with an LLM, and links summaries to their source entities.Source: §4.1 · pp. 3–4
  • Leiden communities over the hierarchical graph use higher-layer semantic hubs to group related entities even when they are distant in the lower layer.Source: §4.1 · pp. 4–5

4.2 Retrieval with Hierarchical Knowledge

  • HiRetrieval selects query-relevant entities for local knowledge and retrieves reports from communities connected to those entities for global knowledge.Source: §4.2 · p. 5
  • It selects query-related key entities within retrieved communities and connects them with shortest paths, whose triples provide bridge-level knowledge.Source: §4.2 · p. 5

4.3 Why is HiRAG effective?

  • Higher-layer summaries shortcut distant but semantically related entities, while retrieval paths connect local entity descriptions with global community insights.Source: §4.3 · p. 5
  • HiRAG combines semantic shortcuts, global community context, and local-to-global paths to generate comprehensive, context-aware answers.Source: §4.3 · pp. 5–6

5 Experimental Evaluation

The section evaluates HiRAG against five baseline methods on four UltraDomain datasets using LLM-judged win rates. Results favor HiRAG broadly, while ablations and cost analysis examine its components and trade-offs.

5.1 Overall Performance Comparison

  • Graph-augmented methods surpass NaiveRAG, while methods with global knowledge significantly surpass FastGraphRAG, which relies on local knowledge.Source: §5.1 · p. 6
  • HiRAG outperforms existing approaches in the majority of cases across four datasets and four evaluation dimensions, and ranks highest among graph-enhanced systems across them.The reported comparisons appear in Table 1.Source: §5.1 · pp. 6–7, Table 1 · p. 7

5.2 Hierarchical KG vs. Flat KG

  • Removing HiIndex lowers win rates in almost all cases, significantly in at least half; bridge-level retrieval remains effective on a flat KG.Source: §5.2 · p. 7

5.3 HiRetrieval vs. Gapped Knowledge

  • Without bridge knowledge, win rates fall significantly across all datasets and evaluation dimensions; bridge descriptions connect local- and global-level knowledge.In the case study, descriptions from different communities support an answer identifying Amazon as both an e-commerce and cloud provider.Source: §5.3 · pp. 7–8, Table 2 · p. 8, Figure 3 · p. 8

5.4 Determining the Number of Layers

  • HiIndex construction stops when the change rate in cluster sparsity falls below 5%, indicating little or no further cluster-quality improvement.Cluster sparsity rises as clusters contain fewer entities, then stabilizes across layers.Source: Figure 4 · p. 8, §5.4 · p. 8

5.5 Efficiency and Costs Analysis

  • About 7.55 USD total indexing cost on Mix with DeepSeek-V3; HiRAG uses no retrieval tokens, unlike KAG and LightRAG, making online retrieval more efficient.Indexing costs are accumulated across the process, while retrieval costs are averaged per query.Source: Table 3 · p. 9, §5.5 · p. 9

6 Conclusions

HiRAG uses hierarchical knowledge in graph-based RAG through HiIndex and HiRetrieval, and the paper reports superior performance over existing methods.

6 Conclusions

  • HiRAG uses graph structures with hierarchical knowledge to enhance RAG systems.Source: §6 · p. 9
  • HiIndex enhances structural and semantic connectivity across hierarchical layers.Source: §6 · p. 9
  • HiRetrieval bridges global conceptual abstractions with localized entity descriptions.Source: §6 · p. 9
  • The paper reports that HiRAG achieves superior performance over existing methods.Source: §6 · p. 9

7 Limitations

HiRAG’s indexing and retrieval stages have distinct costs and quality constraints: layered summarization consumes tokens and time, while relation ranking may affect query relevance.

7 Limitations

  • Layered entity summarization can make hierarchical KG construction costly in tokens and time.The authors suggest parallelizing indexing to reduce time overhead.Source: §7 · p. 9
  • HiRetrieval relies on LLM-generated relation weights, which may affect query relevance.The authors identify more effective query-aware ranking as future work.Source: §7 · p. 9
  • The authors also propose incorporating causality into HiRAG to enhance LLM reasoning capabilities.Source: §7 · p. 9

A Experimental Datasets

The experimental datasets are summarized statistically and share a consistent tokenization setup.

A Experimental Datasets

B Evaluations with Objective Metrics

The objective-metric evaluation compares HiRAG and baselines on two multi-hop QA datasets using EM and F1; the reported prose says HiRAG answers are more accurate.

B Evaluations with Objective Metrics

C Implementation Details of HiRAG

HiRAG combines iterative hierarchical indexing with local, global, and bridge-layer retrieval; appendix evidence includes structural, case-based, and evaluator cross-checks.

C Implementation Details of HiRAG

  • HiIndex constructs the hierarchical graph iteratively, while retrieval supplies local, global, and bridge-layer knowledge as LLM context.The number of clustered layers depends on changes in cluster sparsity at each layer.Source: C Implementation Details of HiRAG · p. 12

D The Clustering Coefficients of HiIndex

E A Simple Case of Hierarchical KG

  • A case study groups semantically similar entities under a shared summary entity, strengthening their connections from a semantic perspective.The example uses BIG DATA and RECOMMENDATION SYSTEM, summarized as DISTRIBUTED COMPUTING.Source: E A Simple Case of Hierarchical KG · p. 12

F Cross Validation for LLM as a Judge

  • Qwen-turbo and Claude-3.5-sonnet judge evaluations on Mix consistently rank HiRAG above the compared graph RAG methods.The authors report similar result patterns on the other datasets.Source: F Cross Validation for LLM as a Judge · p. 13

G Knowledge Coverage of Bridge-Level Descriptions

Bridge-level retrieval captures both local/entity-level and global/community-level information across four datasets. The appendix describes prompts for constructing hierarchical knowledge and evaluating retrieval.

G Knowledge Coverage of Bridge-Level Descriptions

  • Bridge-level context captures a significant portion of both entity-level and community-level information across four datasets.The reported recall is the percentage of local- or global-level information captured in bridge-level context; Figure 6 depicts a shortest path in the hierarchical KG between entities from the introduction case.Source: G Knowledge Coverage of Bridge-Level… · p. 13, Figure 6 · p. 13

H.1 Prompt Templates for Entity Extraction

  • The entity-extraction prompt processes text chunks and includes three examples intended to guide the LLM toward higher-accuracy extraction.Source: §H.1 · p. 13

H.2 Prompt Templates for Entity Extraction

  • The entity-extraction prompt processes text chunks and includes three examples intended to guide the LLM toward higher-accuracy extraction.Source: §H.2 · p. 13

H.3 Prompt Templates for Relation Extraction

  • Relation extraction combines previously extracted entities with their text chunks to produce triples in the basic knowledge graph, the hierarchical graph’s 0-th layer.Source: §H.3 · p. 13

H.4 Prompt Templates for Entity Summarization

  • At each hierarchical layer, the LLM generates summary entities from the previous layer and relations between layers that explain the summaries.Figure 9 presents the prompt template for generating summary entities and corresponding relations.Source: §H.4 · p. 13, Figure 9 · p. 17
  • Table 6 reports win rates for HiRAG and baselines across four tasks, using three powerful LLMs as evaluators.Source: Table 6 · p. 14

H.5 Prompt Templates for RAG Evaluation

  • Evaluation uses the LightRAG prompt design, asking the LLM to return evaluation results and reasons in JSON.Figures 7–9 show templates for entity extraction, relation extraction, and summary-entity generation, respectively.Source: §H.5 · p. 14, Figure 7 · p. 15, Figure 8 · p. 16

Related papers

Loading 2503.10150v3…