PaperlayerView exact revision on arXiv

Source-linked AI summary

Youtu-GraphRAG: Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoning

Junnan Dong, Siyu An, Yifei Yu, Qian-Wen Zhang, Linhao Luo, Xiao Huang, Yunsheng Wu, Di Yin, Xing Sun

arXiv:2508.19855v3cs.IR

TL;DR

GraphRAG systems have often improved graph construction or retrieval separately, while their alignment and evaluation on data potentially seen during LLM pretraining remain challenges. Youtu-GraphRAG unifies both stages through a shared graph schema, with dual-perception community organization and an agentic retriever. Across five challenging benchmarks, it reports up to 90.71% token-cost savings and 16.62% higher accuracy than state-of-the-art baselines.

  • Problem

    Prior GraphRAG work often optimizes construction or retrieval in isolation, and evaluation datasets may already have been seen by scaled LLMs.

    Source: §1 · p. 2

  • Method

    Youtu-GraphRAG unifies schema-guided graph construction and retrieval, combining dual-perception community detection with an agentic retriever and an Anonymity Reversion evaluation task.

    Source: §1 · pp. 2–3

  • Results

    Up to 90.71% saving of token costs and 16.62% higher accuracy over state-of-the-art baselines are reported across five challenging benchmarks.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The reported results indicate adaptability for domain transfer with minimal intervention on the graph schema.

    Source: §1 · p. 2

Abstract

from arXiv · show

Graph retrieval-augmented generation (GraphRAG) has effectively enhanced large language models in complex reasoning by organizing fragmented knowledge into explicitly structured graphs. Prior efforts have been made to improve either graph construction or graph retrieval in isolation, yielding suboptimal performance, especially when domain shifts occur. In this paper, we propose a vertically unified agentic paradigm, Youtu-GraphRAG, to jointly connect the entire framework as an intricate integration. Specifically, (i) a seed graph schema is introduced to bound the automatic extraction agent with targeted entity types, relations and attribute types, also continuously expanded for scalability over unseen domains; (ii) To obtain higher-level knowledge upon the schema, we develop novel dually-perceived community detection, fusing structural topology with subgraph semantics for comprehensive knowledge organization. This naturally yields a hierarchical knowledge tree that supports both top-down filtering and bottom-up reasoning with community summaries; (iii) An agentic retriever is designed to interpret the same graph schema to transform complex queries into tractable and parallel sub-queries. It iteratively performs reflection for more advanced reasoning; (iv) To alleviate the knowledge leaking problem in pre-trained LLM, we propose a tailored anonymous dataset and a novel 'Anonymity Reversion' task that deeply measures the real performance of the GraphRAG frameworks. Extensive experiments across six challenging benchmarks demonstrate the robustness of Youtu-GraphRAG, remarkably moving the Pareto frontier with up to 90.71% saving of token costs and 16.62% higher accuracy over state-of-the-art baselines. The results indicate our adaptability, allowing seamless domain transfer with minimal intervention on schema.

1 Introduction

The introduction identifies a gap in GraphRAG systems that optimize construction or retrieval in isolation, and proposes Youtu-GraphRAG as a schema-guided integration of both. It also introduces an anonymous evaluation task and reports improved accuracy and token efficiency across benchmarks.

1 Introduction

  • Prior work often optimizes graph construction or retrieval separately, despite the importance of both for coherent knowledge organization and complex reasoning.The introduction also notes that widely used datasets may already have been seen by scaled LLMs, complicating evaluation of GraphRAG performance.Source: §1 · p. 2
  • Youtu-GraphRAG integrates graph construction and retrieval in a vertically unified, schema-guided agentic framework.The schema bounds extraction and query decomposition, and the framework combines construction and retrieval agents.Source: §1 · p. 2
  • Its dually perceived community detection combines graph topology and subgraph semantics to organize knowledge hierarchically.The resulting knowledge tree supports top-down filtering and bottom-up reasoning with community summaries.Source: §1 · pp. 2–3, Figure 2 · p. 3
  • The retrieval agent uses the graph schema to split complex queries into parallel sub-queries and iteratively reflect.This supports multi-route retrieval and more advanced reasoning.Source: §1 · p. 2, Figure 2 · p. 3
  • An anonymous dataset and Anonymity Reversion task are introduced to address knowledge leakage in pretrained LLMs during evaluation.The task is intended to measure GraphRAG performance when evaluation data has not been exposed through pretraining.Source: §1 · pp. 2–3
  • Up to 90.71% saving of token costs and 16.62% higher accuracy over state-of-the-art baselines are reported across five challenging benchmarks.The paper also reports state-of-the-art performance across diverse reasoning tasks and domains.Source: §1 · p. 3

2 Task Definition

This section defines GraphRAG as retrieving structured knowledge from a corpus to answer a natural-language question, then generating a response. It describes graph construction and retrieval as stages whose costs, retrieval performance, and answer accuracy are evaluated.

2 Task Definition

  • GraphRAG answers a natural-language question by retrieving structured knowledge from a corpus and then generating a response.The framework includes both graph construction and graph retrieval.Source: §2 · p. 3

2.1 Construction Stage

  • Graph construction extracts document-level triples into a fine-grained graph and clusters graph elements into summarized communities.Community summaries become higher-level meta-nodes; construction is evaluated by time and token consumption.Source: §2.1 · pp. 3–4

2.2 Retrieval Stage

  • At inference, graph-based retrieval returns a subgraph for multi-hop traversal, while conventional retrieval returns the top-k similar documents.The section evaluates retrieval recall and answer accuracy, alongside retrieval efficiency.Source: §2.2 · p. 4

3 Approach: Youtu-GraphRAG

Youtu-GraphRAG unifies graph construction and retrieval around a shared schema. Its pipeline adds hierarchical community organization and schema-guided, reflective retrieval for reasoning across knowledge granularities.

3.1 Schema-Bounded Agentic Extraction

  • Schema-bounded extraction constrains graph construction to targeted entity, relation, and attribute types, then expands the schema using document patterns accepted above a confidence threshold.The resulting graph includes named entities and attributes connected by entity relations and has_attribute relations.Source: §3.1 · pp. 4–5

3.2 Upon Schema: Graph Indexing with Knowledge Tree

  • Dually-perceived community detection combines topological relation overlap with semantic similarity to identify representative centers and iteratively merge communities into a hierarchy.The overview depicts initial communities from triple embeddings, center identification, and pairwise merging.Source: Figure 3 · p. 5, §3.2 · pp. 6–7

3.2.1 Knowledge Tree

  • The Knowledge Tree organizes communities, keywords, entity-relation triples, and attributes across four levels, supporting high-level filtering and fine-grained retrieval.Community descriptions and structurally-semantic keywords are incorporated into the hierarchy.Source: §3.2 · p. 6, §3.2.1 · pp. 7–8

3.3 Agentic Retriever

  • The schema-enhanced retriever decomposes complex queries into valid atomic sub-queries, then uses iterative reflection and specialized retrieval routes for different query types.The routes include entity matching for single-hop facts, triple matching for few-hop relations, community filtering for global queries, and depth-limited DFS for complex constraints.Source: §3.3 · pp. 8–9

4 Experiments

The evaluation distinguishes retrieval effectiveness from end-to-end answering and addresses knowledge leaking, where pretrained knowledge can substitute for retrieved evidence. It uses reject and open modes to assess these capabilities and semantic response similarity to account for textual variation.

4.1 Evaluation Metrics

  • DeepSeek-V3-0324 assesses response similarity against ground-truth references to account for semantic deviations from minor character variations.The evaluation also mentions character-based recall, EM, and F1 protocols.Source: §4.1 · p. 9
  • Knowledge leaking occurs when answers rely on pretrained knowledge rather than the retrieval mechanism, with prompt differences affecting reproduced results.Some prompts require rejection when evidence is insufficient, while others allow parametric knowledge or leave instructions ambiguous.Source: §4.1 · p. 9
  • Reject mode requires abstention when retrieved evidence is insufficient, strictly testing retrieval effectiveness and hallucination prevention.Open mode permits answers from retrieved content or parametric knowledge to measure overall capability in practical deployment.Source: §4.1 · p. 9
  • The authors report reproducing representative baselines and argue that AnonyRAG supports fair and comprehensive assessment of GraphRAG methods.Evaluation prompts are provided in Appendix A.Source: §4.1 · p. 9

4.2 Datasets

The evaluation uses multi-hop QA and textbook-corpus benchmarks, alongside bilingual anonymous datasets designed to test performance without pretrained-memory shortcuts. It compares retrieval and graph-based baselines under consistent model settings.

4.2 Datasets

  • HotpotQA, MuSiQue, and 2WikiMultiHopQA provide three multi-hop question-answering benchmarks for evaluation.Source: §4.2 · p. 9
  • GraphRAG-Bench (G-Bench) adds a benchmark constructed from textbook corpora for evaluation across diverse domains.Source: §4.2 · p. 9
  • AnonyRAG-CHS and AnonyRAG-ENG are bilingual anonymous datasets introduced with an Anonymous Reversion evaluation task.Source: §4.2 · p. 10
  • The datasets anonymize selected entities to disrupt model-memory shortcuts while entity linking preserves discourse comprehension.The paper documents dataset construction details in Appendix B.Source: §4.2 · p. 10

4.3 Baselines

  • Baselines span Naive RAG, flat-graph methods, and hierarchical tree-based methods, including GraphRAG, LightRAG, RAPTOR, and E2GraphRAG.The flat-graph group also includes G-Retriever and HippoRAG 1&2.Source: §4.3 · p. 10
  • Baselines and Youtu-GraphRAG use the same evaluation settings, DeepSeek-V3-0324 and Qwen3-32B backbones, and an all-MiniLM-L6-v2 embedding model.Source: §4.3 · p. 10

4.4 Overall Evaluation

Youtu-GraphRAG achieves top-20 accuracy gains across open and reject evaluation and combines lower token consumption with strong performance. Its construction and community-detection stages also show low token costs and efficient runtimes.

4.4.1 Comparison of Time and Token Consumption

  • Graph construction uses the fewest tokens across six datasets and efficient time on five; community detection uses no more than 10,000 tokens and is time-efficient across all datasets.The comparison used DeepSeek-V3-0324 on identical hardware with 32-thread concurrent inference.Source: §4.4.1 · p. 10

4.4.2 Main Performance Comparison

  • Youtu-GraphRAG attains the highest top-20 accuracy across virtually all datasets and settings in open and reject modes, using two LLM backbones.Source: Table 1 · p. 11, §4.4.2 · p. 11
  • Open mode improves 2 to 8 points over the strongest competitor; DeepSeek-V3-0324 top-20 accuracy reaches 86.5% on HotpotQA, 85.5% on 2Wiki, and 53.6% on MuSiQue.Source: §4.4.2 · p. 11
  • Reject-mode top-20 accuracy is 81.2% on HotpotQA, 77.6% on 2Wiki, and 47.5% on MuSiQue, 7–14 points above the strongest baseline.Source: §4.4.2 · p. 11
  • Open-mode top-20 accuracy reaches 42.88% on Annoy-CHS and 43.26% on Annoy-ENG, surpassing all baselines.Source: §4.4.2 · p. 12
  • Youtu-GraphRAG shifts the performance–token-cost Pareto frontier, attaining the best benchmark performance with up to an order of magnitude fewer graph-construction tokens.Source: Figure 6 · p. 12, §4.4.2 · p. 12

4.5 Analysis of Generalizability

Youtu-GraphRAG shows strong domain transfer across six heterogeneous benchmarks without task-specific fine-tuning. Its top-10 results also show gains across open and reject settings, including on anonymous datasets.

4.5 Analysis of Generalizability

  • The authors attribute generalizability to schema-guided domain-adaptive graphs, hierarchical community structures, and query decomposition that adapts retrieval strategies without manual tuning.The framework integrates graph construction and retrieval, with dually-perceived community detection yielding hierarchical knowledge structures.Source: §4.5 · p. 12
  • The passage identifies HotpotQA and 2Wiki as open-setting hop-reasoning datasets and reports superior abstention by Youtu-GraphRAG on MuSiQue and 2Wiki in reject settings, indicating robustness in complex reasoning and uncertainty calibration.Source: §4.5 · p. 13
  • Top-10 open accuracies are 83.4% on HotpotQA, 82.3% on 2Wiki, and 52.1% on MuSiQue, surpassing the strongest competitor by 4˜8 points; reject-mode gains are 8˜12 points.On Annoy-CHS and Annoy-ENG, the agent-enhanced model attains 38.08% and 42.57%, respectively.Source: §4.5 · p. 13
  • Youtu-GraphRAG achieves the best Open Accuracy and Reject Accuracy on all six datasets, surpassing state-of-the-art GraphRAG baselines.The evaluation uses six heterogeneous benchmarks without task-specific fine-tuning.Source: §4.5 · p. 12, Figure 7 · p. 13

4.6 Ablation Studies

The ablations remove community detection, agent reasoning and reflection, or schema guidance. The reported results indicate that each component contributes, with agent reasoning especially important for multi-hop inference.

4.6 Ablation Studies

  • Removing community detection drops performance across datasets, including around 1.7% on HotpotQA and 2.5% on 2Wiki.The authors state that coherent communities facilitate more accurate retrieval and reasoning for global questions.Source: §4.6 · p. 13
  • Removing agent reasoning and reflection yields the most severe degradation on complex reasoning datasets, with reported differences of 19.8% on 2Wiki and -7.5% on MuSiQue.The authors link the iterative reasoning-feedback loop to resolving ambiguous intermediate steps.Source: §4.6 · p. 13
  • Removing schema guidance produces a 7.27% decrease on AnonyRAG-CHS, a knowledge-intensive setting.The authors highlight the seed schema's role in handling new domains with minimal manual intervention.Source: §4.6 · p. 13
  • Youtu-GraphRAG consistently outperforms all ablated variants, and the three components are described as complementary.The authors characterize agent reasoning as most critical for multi-hop inference and schema guidance as important in low-resource and domain-specific scenarios.Source: §4.6 · pp. 13–14

5 Related Work

Related work develops retrieval and knowledge-organization approaches largely along separate trajectories. The paper frames its framework as a response to this lack of unified design.

5 Related Work

6 Conclusions

Youtu-GraphRAG unifies graph construction and retrieval through a shared graph schema, using agents for extraction and query decomposition and dually-perceived detection to build hierarchical knowledge. The paper also introduces Anonymity Reversion and reports efficiency, accuracy, and domain-transfer gains.

6 Conclusions

  • Youtu-GraphRAG advances the Pareto frontier with up to 90.71% reduction in token costs and 16.62% higher accuracy than state-of-the-art baselines.Source: §6 · p. 14
  • A schema-guided agent continuously extracts knowledge using predefined entity types, relations, and attributes.The seed schema can be expanded, while the shared schema also guides the agentic retriever's query decomposition.Source: §6 · p. 14
  • Dually-perceived community and keyword detection combines structural topology with subgraph semantics to form a hierarchical knowledge tree for top-down filtering and bottom-up reasoning.Source: §6 · p. 14
  • Anonymity Reversion and a curated anonymous dataset are proposed to mitigate knowledge leakage and measure GraphRAG performance.The framework is reported to transfer across domains with minimal schema adjustments.Source: §6 · p. 14

A Prompt templates in LLMs generation

The templates use minimal prompts to compare generation modes that differ in whether the LLM may use parametric knowledge. Reject mode declines when retrieved knowledge is insufficient, while open mode permits an answer from the LLM’s own knowledge.

A Prompt templates in LLMs generation

  • Minimalistic prompts differ between modes to evaluate whether allowing LLMs to use parametric knowledge within RAG affects performance.The prompt design aims to minimize confounding factors.Source: A Prompt templates in LLMs generation · p. 17

A.1 Reject mode

  • Reject mode presents the question alongside extracted knowledge from different retrieval paths.Source: §A.1 · p. 17
  • When that extracted knowledge is insufficient, reject mode instructs the LLM to decline answering.Source: §A.1 · p. 17
  • The reject-mode response prompt places the extracted context before an Answer field.Source: §A.1 · p. 17

A.2 Open mode

  • Open mode permits answering from the LLM’s own knowledge when the extracted knowledge is insufficient.Source: §A.2 · p. 17
  • The open-mode response prompt likewise places the extracted context before an Answer field.Source: §A.2 · p. 17

B Data Collection and Processing

The dataset uses four classic novels selected for multilingual coverage and complex entity representations, with anonymized corpora and question-answer pairs. Its evaluation formats include Anonymity Reversion and multiple choice, followed by zero-shot filtering for objective, unique answers.

B Data Collection and Processing

  • The study draws raw data from Water Margin, Dream of the Red Chamber, Moby-Dick, and Middlemarch, selected for multilingual coverage and complex entity representations.The works are described as public domain; Table 4 reports question-type and difficulty-distribution statistics.Source: B Data Collection and Processing · p. 17, Table 4 · p. 18
  • DeepSeek extracts corpus entities for anonymization and constructs question-answer pairs from 2Wiki and MuSiQue query templates, which are anonymized using the same dictionary.Source: B Data Collection and Processing · p. 17
  • Anonymized questions substantially degrade LLM performance relative to non-anonymized questions, directing evaluation toward retrieved context rather than parametric memorization.Table 5 presents representative comparisons of anonymized and non-anonymized questions.Source: B Data Collection and Processing · p. 18, Table 5 · p. 18
  • Anonymity Reversion asks models to reconstruct anonymized entities from question-answer pairs provided as context, testing contextual-clue use for entity recovery.Source: B Data Collection and Processing · p. 18
  • A subset of questions becomes multiple choice to diversify question types and make evaluation objective.Source: B Data Collection and Processing · p. 18
  • Zero-shot filtering checks performance on transformed questions to preserve contextual-reasoning objectives while ensuring objective, unique answers and mitigating LLM-as-judge unreliability.Table 6 presents representative cases of the two question-answer formats.Source: B Data Collection and Processing · p. 18, Table 6 · p. 19

Related papers

Loading 2508.19855v3…