Source-linked AI summary
Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learning
Haoran Luo, Haihong E, Guanting Chen, Qika Lin, Yikai Guo, Fangzhi Xu, Zemin Kuang, Meina Song, Xiaobao Wu, Yifan Zhu, Luu Anh Tuan
TL;DR
RAG and GraphRAG face limitations in representing structural knowledge and conducting flexible retrieval, alongside constraints in construction and answer generation. Graph-R1 uses lightweight hypergraph construction and multi-turn retrieval optimized with end-to-end RL. Across the reported evaluations, it outperforms traditional GraphRAG and RL-enhanced RAG methods in accuracy, retrieval efficiency, and generation quality.
Problem
GraphRAG methods face costly construction and semantic loss, fixed one-time retrieval, and dependence on long-context reasoning and prompt quality.
Source: §1 · p. 1Method
Graph-R1 constructs a lightweight knowledge hypergraph, models retrieval as multi-turn agent-environment interaction, and optimizes the process with an end-to-end reward.
Source: §1 · p. 2Results
Across standard RAG datasets, Graph-R1 outperforms traditional GraphRAG and RL-enhanced RAG methods in reasoning accuracy, retrieval efficiency, and generation quality.
Source: §1 · p. 2Takeaways & Limitations
Experiments across six benchmarks support Graph-R1’s superiority in accuracy, retrieval efficiency, and generation quality.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Retrieval-Augmented Generation (RAG) mitigates hallucination in LLMs by incorporating external knowledge, but relies on chunk-based retrieval that lacks structural semantics. GraphRAG methods improve RAG by modeling knowledge as entity-relation graphs, but still face challenges in high construction cost, fixed one-time retrieval, and reliance on long-context reasoning and prompt design. To address these challenges, we propose Graph-R1, the first agentic GraphRAG framework via end-to-end reinforcement learning (RL). It introduces lightweight knowledge hypergraph construction, models retrieval as a multi-turn agent-environment interaction, and optimizes the agent process via an end-to-end reward mechanism. Experiments on standard RAG datasets show that Graph-R1 outperforms traditional GraphRAG and RL-enhanced RAG methods in reasoning accuracy, retrieval efficiency, and generation quality. Our software and data are publicly available at https://github.com/LHRLAB/Graph-R1.
1. Introduction
Graph-R1 addresses construction cost and semantic loss, fixed one-shot retrieval, and dependence on long-context reasoning and prompt quality in GraphRAG. It uses lightweight hypergraph construction, multi-turn retrieval, and end-to-end reinforcement learning, with reported gains over GraphRAG and RL-enhanced RAG baselines.
1. Introduction
- RAG adds external knowledge to improve factual consistency, while graph-based representations address chunk retrieval’s difficulty capturing complex entity relationships.The introduction describes LLM hallucinations in knowledge-intensive or proprietary-knowledge applications as motivation.Source: §1 · p. 1
- GraphRAG faces costly knowledge construction with semantic loss, fixed one-time retrieval, and dependence on long-context reasoning and prompt quality.Fixed retrieval can limit performance on complex queries.Source: §1 · pp. 1–2
- Graph-R1 combines lightweight hypergraph construction, multi-turn agentic retrieval, and an end-to-end reward integrating generation quality, retrieval relevance, and graph-path reliability.The agent repeatedly follows a “think-retrieve-rethink-generate” loop in the hypergraph environment.Source: §1 · p. 2
- Graph-R1 outperforms traditional GraphRAG and RL-enhanced RAG methods in reasoning accuracy, retrieval efficiency, and generation quality.Figure 2 compares F1 scores across RAG benchmarks and states that graph-structured knowledge with RL achieves a higher performance ceiling than chunk-based knowledge.Source: §1 · p. 2, Figure 2 · p. 2
- The work presents Graph-R1 as a foundation for future knowledge-driven, strategy-optimized agent-based generation systems.Source: §1 · p. 2
2. Related Work
Related work covers graph-structured retrieval and reinforcement learning for LLMs and interactive agents. Graph-R1 is framed as learning a multi-turn retrieval policy over graph knowledge through end-to-end RL.
2. Related Work
- Graph-R1 is presented as the first agentic GraphRAG framework to learn a multi-turn retrieval policy via end-to-end RL.The related-work discussion situates it among graph-structure and retrieval-optimization approaches.Source: §2 · p. 2
- Prior work explores graph-structured knowledge, richer graph representations, and path-based retrieval optimization for GraphRAG.Source: §2 · p. 2
- RL has been used to enhance LLM reasoning and interactive agents, motivating its application to agentic GraphRAG.The section cites DeepSeek-R1 and GRPO among end-to-end RL developments.Source: §2 · p. 2
- Figure 3 depicts an agent iteratively thinking, querying the graph, retrieving n-ary facts, and answering the question.Source: Figure 3 · p. 3
3. Preliminaries
The preliminaries formalize GraphRAG as three stages: knowledge-graph construction, graph retrieval, and answer generation.
3. Preliminaries
- Knowledge-graph construction extracts facts from each semantic unit into a unified graph.An LLM-based extractor parses each unit into relation–entity pairs, with each relation associated with a set of entities.Source: §3 · p. 3
- Graph retrieval first obtains candidate reasoning paths and then selects a relevant subset to maximize their joint likelihood.The selected paths are used as graph evidence for answering.Source: §3 · p. 3
- Answer generation produces a natural-language answer from the query and selected graph paths.The formulation combines answer-generation likelihood with the retrieval-pruning distribution.Source: §3 · p. 3
4. Methodology: Graph-R1
Graph-R1 constructs a knowledge hypergraph and treats reasoning as multi-turn interaction between an LLM agent and that graph. It optimizes the agent’s reasoning trajectories with end-to-end reinforcement learning.
4.1. Agent Initialization
- Each corpus chunk yields n-ary relational facts whose hyperedges connect participating entities; shared encoder embeddings structure the resulting knowledge hypergraph.Source: §4.1 · p. 3
- The agent state tracks the query and prior state-action history; actions comprise thinking, query generation, graph retrieval, and final answering.Source: §4.1 · pp. 3–4
4.2. Reasoning via Multi-turn Graph Interaction
- At each multi-turn step, Graph-R1 retrieves facts through entity-linked hyperedges and direct query-hyperedge similarity, then merges both ranked lists by reciprocal rank aggregation.Source: §4.2 · p. 4
4.3. Outcome-directed End-to-end RL Optimization
- GRPO optimizes trajectories with outcome-directed rewards for valid structured steps and answer token-level F1, coupling format compliance with answer correctness.Source: §4.3 · p. 5
4.2. Reasoning via Multi-turn Graph Interaction
- For Qwen2.5 models at 1.5B, 3B, and 7B, Figure 4 reports step-wise F1 scores showing Graph-R1 outperforms both the baselines and GPT-4o-mini variants.Source: Figure 4 · p. 5
5. Experiments
The experiments evaluate Graph-R1 across standard RAG datasets, multiple model scales, and five research questions. They report comparative results on reasoning, construction cost, retrieval efficiency, and generation quality.
5.1. Experimental Setup
- Experiments address five research questions across six standard RAG datasets, comparing correctness, retrieval relevance, and generation quality against RAG, GraphRAG, and RL-based baselines.The baselines include multiple model scales and both conventional and graph-based retrieval approaches.Source: §5 · p. 6, §5.1 · p. 6
5.2. Main Results (RQ1)
- 57.28 F1 under Qwen2.5-7B-Instruct, above StandardRAG (32.05), HyperGraphRAG (29.40), and Search-R1 (46.19); Graph-R1 also consistently outperforms baselines across base models.The study compares Graph-R1 with baselines across different base models.Source: §5.2 · p. 7
5.3. Ablation Study and Comparative Analysis (RQ2)
- Removing knowledge construction, multi-turn interaction, or reinforcement learning degrades Graph-R1’s performance.Figure 5a presents the component ablation study.Source: §5.3 · p. 7, Figure 5 · p. 7
- With RL, hypergraph knowledge achieves the highest performance ceiling, ahead of binary-relation graphs, chunk-based knowledge, and no external knowledge.Figure 5b compares performance across knowledge representations.Source: §5.3 · p. 7, Figure 5 · p. 7
- Graph-R1 consistently outperforms baselines across datasets and model sizes; GRPO achieves the highest F1 among the tested RL strategies.The analysis also reports a slightly lower performance ceiling when Graph-R1 is trained on Qwen3 (4B), which is already well trained by RL.Source: §5.3 · p. 7
5.4. Analysis of Graph-R1’s Construction Cost (RQ3)
- 5.69 seconds and $2.81 per 1K tokens for construction; generation takes 7.0s per query at $0 cost.Construction costs are below GraphRAG (8.04s, $3.35) and HyperGraphRAG (6.76s, $4.14); the system generates over 120K nodes and 98K edges.Source: §5.4 · pp. 7–8, Figure 6 · p. 8
5.5. Analysis of Graph-R1’s Retrieval Efficiency (RQ4)
- Graph-R1 averages 1200-1500 tokens and 2.3-2.5 interaction turns, with the highest F1 at a moderate average retrieved-content length.Figure 7 compares response length, interaction turns, and performance against retrieval-content length.Source: §5.5 · p. 8, Figure 7 · p. 8
5.6. Analysis of Graph-R1’s Generation Quality (RQ5)
- Graph-R1 outperforms RL-based baselines and achieves generation quality comparable to GPT-4o-mini-based HyperGraphRAG, with 86.9 Correctness, 95.2 Relevance, and 88.5 Logical Coherence.Its Overall score is 82.4 versus Search-R1’s 70.3.Source: §5.6 · p. 8, Figure 6 · p. 8
6. Conclusion
The conclusion presents Graph-R1 as an end-to-end RL-powered agentic GraphRAG framework combining lightweight hypergraph construction and multi-turn retrieval. Across six benchmarks, it reports gains in accuracy, retrieval efficiency, and generation quality.
6. Conclusion
- Graph-R1 combines lightweight knowledge hypergraph construction, multi-turn retrieval, and a unified reward mechanism for outcome-directed reasoning.The conclusion characterizes the framework as bridging graph-structured knowledge with natural-language generation.Source: §6 · p. 8
Impact Statement
This material states the expected impact of Graph-R1, describes its construction and reasoning prompts, and presents a proof claim about graph-structured knowledge and agent accuracy.
Impact Statement
- The work relies on publicly available datasets and expects positive societal impact from advancing reliable, structured knowledge-grounded reasoning.The impact statement reports no significant ethical concerns.Source: Impact Statement · p. 9
A.1. Knowledge HyperGraph Construction Prompt
- HyperGraphRAG’s n-ary relation-extraction prompt is reused, while construction skips confidence-score calculations and uses simpler semantic retrieval.The authors state these changes reduce construction costs while maintaining equivalent knowledge representation.Source: §A.1 · p. 13
A.2. Agentic Knowledge Reasoning Prompt
- The agent repeatedly interacts with the environment, adding information to form the prompt for its next exchange.Figure 9 shows a final prompt produced by the full reasoning process.Source: §A.2 · p. 13, Figure 9 · p. 13
B.1. Proof of Proposition 1
- The proof argues that graph-structured knowledge provides denser evidence and faster belief convergence, yielding at least as high answer accuracy as linear chunks.It states strict inequality when the graph contains structural relations not recoverable from text.Source: §B.1 · p. 14
B.2. Proof of Proposition 2
The proof argues that adaptive multi-turn graph retrieval can use evidence gathered so far to improve information efficiency and answer accuracy under a fixed token budget. It also frames end-to-end RL as a way to connect graph retrieval with grounded answer generation.
B.2. Proof of Proposition 2
- Under a fixed token budget, adaptive multi-turn retrieval targets informative graph regions using prior evidence, improving retrieval efficiency and answer accuracy when early evidence permits pruning.The proof compares information gain per token and relates lower conditional entropy to lower expected error.Source: §B.2 · pp. 14–15
B.3. Proof of Proposition 3
- Trajectory-level rewards combine format and answer quality, reinforcing graph-retrieval trajectories that produce correct, grounded answers.The proof links this reward design to lower conditional entropy and error.Source: §B.3 · p. 15
C. Graph-R1 Algorithm Details
- Graph-R1 has three phases: n-ary fact extraction into a hypergraph, multi-turn agent reasoning with retrieval or answer actions, and GRPO policy optimization.The algorithm’s optimization uses sampled trajectories and format and answer rewards.Source: C. Graph-R1 Algorithm Details · p. 16
- Construction scales as O(TK)+O(F), each reasoning step as O((|V|+|EH|)d), and policy optimization as O(NTd).The analysis identifies parallelization and localized retrieval over compact hypergraph subsets as efficiency benefits.Source: C. Graph-R1 Algorithm Details · p. 16
D. Dataset Details
- Evaluation covers six widely-used RAG benchmarks selected from FlashRAG (Jin et al., 2025b), spanning single-hop and multi-hop question answering; each dataset has 5,120 training and 128 testing instances sampled uniformly.Source: D. Dataset Details · p. 17
E.1. Baselines with GPT-4o-mini
- GPT-4o-mini baselines span chunk retrieval and graph methods using entity graphs, compact graphs, path pruning, hierarchical planning, or hypergraphs.The listed graph approaches include GraphRAG, LightRAG, PathRAG, HippoRAG2, and HyperGraphRAG.Source: §E.1 · p. 17
E.2. Baselines with Qwen2.5 (1.5B, 3B, 7B)
- Qwen2.5 comparisons include direct generation, supervised fine-tuning, answer-only GRPO, and chunk-based retrieval methods trained with GRPO.The retrieval baselines include Search-R1 and the two-stage R1-Searcher.Source: §E.2 · pp. 17–18
F. Evaluation Details
The section describes answer and retrieval evaluation measures and points to the experiment configuration table. It also includes a figure about generation evaluation dimensions.
F. Evaluation Details
- The evaluation uses Exact Match and Retrieval Similarity alongside other metrics, with EM checking exact answer agreement and R-S assessing semantic similarity to gold knowledge.The section also refers to seven dimensions for generation evaluation.Source: F. Evaluation Details · p. 18, Figure 10 · p. 18
G. Implementation Details
- The hyperparameter table summarizes model backbones, input limits, training configurations, and retrieval settings.The provided passage identifies the table’s scope but does not give its individual settings.Source: G. Implementation Details · p. 18, Table 3 · p. 18
H. Case Study Analysis
A case study reports Graph-R1’s answer quality relative to several retrieval baselines. The section also describes hypergraph construction and experiments designed to examine cross-domain transfer.
H. Case Study Analysis
- Graph-R1 identifies both the director and birthdate in the case study and achieves the highest G-E score, 88.57.The passage contrasts this with failures by NaiveGeneration and R1, and shortcomings by StandardRAG and HyperGraphRAG.Source: H. Case Study Analysis · p. 19
J. Details of Knowledge HyperGraph Construction
- The construction pipeline chunks documents, extracts n-ary relations as hyperedges with coherence scores, then embeds entities and hyperedges for retrieval.Documents use 1,200-token windows with 50-token overlap; GPT-4o-mini performs relation extraction and bge-large-en-v1.5 supplies embeddings.Source: J. Details of Knowledge HyperGraph… · p. 20, Figure 12 · p. 20
- Compared with HyperGraphRAG, engineering changes to parallelization, prompts, storage, and vectorization reduce token consumption and computational overhead.The passage points to Table 6a for evidence of these efficiency improvements.Source: J. Details of Knowledge HyperGraph… · p. 20
- The O.O.D. analysis includes in-domain training, training on a different dataset, and combined-dataset training, controlling the combined regime to match I.I.D. data volume.The combined training pool is uniformly downsampled to one sixth, with balanced data from six domains.Source: K. More Experimental Results on O.O.D.… · p. 20
K.1. Results and Analysis on Six Open-Domin Datasets
Across six multi-hop and open-domain datasets and three training regimes, Graph-R1 consistently improves over Search-R1. Additional domain-wise results report gains on five specialized datasets.
K.1. Results and Analysis on Six Open-Domin Datasets
- Graph-R1 consistently improves over Search-R1 across six datasets and all three training regimes.The comparison covers multi-hop and open-domain datasets.Source: Table 5 · p. 21, §K.1 · p. 21
- Across Medicine, Agriculture, CS, Legal, and Mixed, Graph-R1 (3B) surpasses Search-R1 in both EM and F1, including on specialized domains not seen during training.The authors describe these findings as evidence of cross-domain robustness and transferability.Source: Table 6 · p. 21, §K.2 · p. 21