PaperlayerView exact revision on arXiv

Source-linked AI summary

LivingRAG: Augmenting Graph RAG with Experience

Yuzhuo Cui, Zongye Zhang, Qingjie Liu

arXiv:2608.25960v1cs.AI

TL;DR

Existing Graph RAG systems often discard query-level reasoning after inference, leaving later related queries to retrieve and reason from scratch. LivingRAG stores verified experiences and reuses their graph signals for retrieval and compact summaries for generation. Experiments report improved accuracy over strong RAG baselines and reduced completion-token use when relevant prior experience is reused.

  • Problem

    Most RAG and Graph RAG systems answer queries independently and discard query-level reasoning signals, limiting reuse across related online QA queries.

  • Method

    LivingRAG adds a writable experience store that passes grounding and novelty checks, reusing activation maps for retrieval and compact reasoning summaries for generation.

  • Results

    LivingRAG improves accuracy over strong RAG baselines and reduces completion-token use when later queries retrieve useful prior experience.

  • Takeaways & Limitations

    Verified experience reuse augments Graph RAG by transferring useful graph neighborhoods and reasoning patterns across online QA queries.

  • Takeaways & Limitations

    Long-term experience growth can increase matching cost and destabilize reuse, while unchanged stored experiences may become invalid in dynamic domains.

Abstract

from arXiv · show

Graph-based RAG improves multi-hop question answering by organizing evidence as a knowledge graph. However, most existing RAG systems process each query in isolation and discard useful reasoning from the LLM's response after inference. As a result, later related queries need to retrieve evidence and reason from scratch. We propose LivingRAG, a Graph RAG framework with writable and reusable reasoning experience. LivingRAG adds a writable experience store to a graph-based retrieval backbone, enabling verified experiences to be reused during inference in two ways. Stored graph signals help retrieval find entities and passages that were useful in earlier related queries. Stored summaries provide a reference reasoning pattern for answer generation. We analyze online QA streams and find reusable signals from shared entities, graph neighborhoods, and question templates. Experiments on multi-hop QA benchmarks show that LivingRAG improves accuracy over strong RAG baselines and reduces completion-token use when relevant prior experience is reused.

1 Introduction

LivingRAG addresses the loss of reusable reasoning in query-by-query Graph RAG by storing verified experiences that guide later retrieval and generation. It reports improved accuracy and lower completion-token use when relevant prior experience is reused.

  • Existing RAG systems commonly answer queries independently, discarding query-level reasoning signals and forcing later related queries to retrieve and reason again.
  • Related queries can share entities, nearby graph regions, or reasoning templates, so reusable experience should preserve graph-neighborhood priors and reasoning patterns beyond final answers.
  • LivingRAG adds a writable experience store whose verified entries retain source queries, activated entities, reasoning summaries, and validation metadata.
  • Stored activation maps guide retrieval toward useful evidence neighborhoods, while reasoning summaries scaffold answer generation for later queries.
  • Quality gates store candidates only when claims are supported by retrieved evidence and sufficiently novel, reducing the risk of propagating unsupported reasoning.
  • Experiments on multi-hop QA benchmarks report improved accuracy over strong RAG baselines and reduced completion-token use and estimated API cost when prior experience is useful.

2 Related Work

Related work improves RAG through graph structure, reasoning, and memory, whereas LivingRAG targets online Graph RAG by reusing verified reasoning experiences across QA streams.

  • Graph-based RAG methods organize entities, passages, or associations to support retrieval for complex questions.
  • Reasoning-enhanced RAG uses decomposition, intermediate notes, self-reflection, or tool use to improve single-query reasoning.
  • Memory-based agents and memory-augmented RAG systems reuse observations, feedback, episodic histories, or global clues for later retrieval and reasoning.
  • LivingRAG focuses on online Graph RAG for QA streams and stores verified reasoning experiences with graph activation information and compact reasoning scaffolds.

3 Method

LivingRAG extends a graph-based retriever with online experience reuse: earlier verified experiences influence retrieval and generation, while quality gates control write-back. Its added retrieval cost grows linearly with the experience store.

  • 3.1 Motivating Analysis: LivingRAG operates causally on online QA streams, allowing query q_t to reuse only earlier queries q_1, . . . , q_{t−1}.
  • 3.1 Motivating Analysis: Entity, graph-neighborhood, and template reuse measure complementary relationships between current and earlier queries.
  • 3.1 Motivating Analysis: Graph-neighborhood reuse is common across datasets, while direct entity reuse is limited and template reuse varies with dataset formality.
  • 3.2 Graph Retrieval Backbone: The base LinearRAG backbone constructs a passage-sentence-entity retrieval graph and ranks passages using relevance propagation and Personalized PageRank.
  • 3.3 Experience-Augmented Retrieval: During retrieval and generation, LivingRAG uses experience-augmented graph retrieval and an optional reasoning scaffold; without useful experience, it falls back to base Graph RAG.
  • 3.5 Experience Write-Back: After answering, the system forms a candidate containing the query, retrieved passages, explanation, and answer, then stores it only if grounding and novelty checks pass.
  • 3.3 Experience Store: A verified experience stores the source query, embedding, sparse activation map, compact reasoning summary, answer, timestamp, and grounding confidence.
  • 3.3 Experience Store: Activation maps preserve graded graph-neighborhood information for later retrieval, while compact summaries select entity-anchored explanation sentences as generation scaffolds.

4 Experiments

Experiments evaluate LivingRAG on online multi-hop QA streams, measuring accuracy, token efficiency, realized reuse, and component contributions. LivingRAG improves accuracy and reduces completion-token use and cost, while ablations distinguish retrieval, generation, and quality-gating roles.

  • Experimental setup: The evaluation uses four multi-hop QA streams, with LivingRAG initialized empty and updated online so only earlier queries contribute experience.The benchmarks are 2WikiMultiHopQA, HotpotQA, MuSiQue, and MuSiQue-full; WixQA is additionally compared only with LinearRAG.
  • Overall QA accuracy: LivingRAG achieves the best overall accuracy on four multi-hop benchmarks and in the WixQA comparison against LinearRAG.The writable experience store does not weaken the graph-RAG backbone's base QA ability under the evaluation setting.
  • Online token efficiency: 22.7% weighted completion-token reduction accompanies a 3.5% weighted prompt-token increase across the evaluated datasets.LivingRAG reduces completion tokens on all five datasets, while added reasoning scaffolds or experience context increase prompt tokens.
  • Online token efficiency: 12.1% total run-cost reduction lowers the estimate from $51.05 to $44.87 despite prompt-token overhead.Prompt-token increases partly offset completion-token savings, but total cost decreases on every dataset.
  • Online token efficiency: After startup, LivingRAG usually uses fewer completion tokens, with savings becoming more stable as experiences accumulate.Early stream segments may use more completion tokens because few stored experiences are available; green shading marks segments with lower LivingRAG usage.
  • Reuse signals and ablations: HotpotQA has low values under both strict reuse-trace metrics, suggesting those measurements may not capture all experience effects on that dataset.The realized metrics measure graph-neighborhood transfer and scaffold reuse during inference, not all possible experience effects.
  • Reuse signals and ablations: Activation maps transfer graph neighborhoods, scaffolds reuse reasoning patterns, and quality gates preserve accuracy by filtering stored experiences.Removing activation-map fusion hurts MuSiQue-full and WixQA more, removing scaffolds hurts 2WikiMultiHopQA most and reduces token savings, and quality gates store 27.4% of candidates; removing them lowers accuracy on all datasets.

5 Conclusion

LivingRAG extends Graph RAG with writable, reusable reasoning experience. Experiments show improved accuracy and reduced completion-token use over strong RAG baselines, suggesting verified experience reuse as an augmentation to Graph RAG.

  • Conclusion: LivingRAG stores verified experiences and reuses them through activation-map fusion and reasoning scaffolds.The framework combines a writable experience store with graph retrieval and generation support.
  • Conclusion: Experiments on multi-hop QA benchmarks show improved accuracy and reduced completion tokens over strong RAG baselines.The conclusion reports both performance and inference-efficiency gains.

6 Limitations

LivingRAG’s limitations concern experience-store growth, stale stored knowledge, and the use of controlled benchmarks as proxies for natural online streams.

  • Experience-store growth is linear in accepted experiences but may increase matching cost and destabilize reuse in much larger, long-running deployments.The authors suggest pruning, compression, or indexed experience search as possible maintenance strategies.
  • Stored experiences remain accepted without updates or deletion, so time-sensitive facts can become invalid and influence later queries.The fixed-corpus benchmarks cannot directly evaluate expiration under changing knowledge; time-aware refresh and adaptive pruning remain future work.
  • The evaluation uses controlled QA benchmarks as proxies for online query streams, which may differ from natural traffic in reuse density, topic drift, and noise.Evaluation on naturally timestamped streams with topic drift and controlled noise remains future work.
  • Related-work methods mainly improve reasoning for the current query, whereas LivingRAG focuses on storing and reusing reasoning from earlier queries.Its stored unit combines a sparse activation map, compact reasoning summary, and validation metadata.

C Detailed Complexity Analysis

LivingRAG preserves LinearRAG’s corpus-side complexity while adding selective experience-store computation, validation, and storage costs tied to accepted experiences.

  • LivingRAG leaves the LinearRAG corpus graph and retrieval procedure unchanged, adding experience-store operations around the existing pipeline.It changes the initial entity activation vector before invoking the same graph retrieval procedure.
  • Experience matching scans stored experiences using dense query-embedding similarity and sparse activation-map similarity, then fuses only the top-K matches.Optional top-H truncation limits activation-map fusion cost; template-aware selection may add one additional store scan.
  • Grounding validation is selective: Novelty Score uses a store scan, while Grounding Score and NLI run only after the novelty gate passes.The combined expression separates unchanged corpus-side LinearRAG cost from added experience-store cost.
  • 1592 of 5817 candidates were written, producing a 27.4% write-back ratio and tying store growth to verified, novel experiences.Only 1806 candidates passed NS and reached GS, while 4011 skipped the NLI check.
  • Persistent experience storage is linear in accepted experiences and uses sparse activation maps, compact scaffolds, one query embedding, and metadata.Transient template embeddings are excluded from persistent storage, and the overhead remains separate from the static corpus graph.

D Quality Gates for Experience Write-Back

LivingRAG uses novelty and grounding gates to control experience write-back, evaluating novelty first so failed candidates avoid the NLI check.

  • A candidate is stored only when both the novelty and grounding gates pass; novelty failure skips grounding validation.This ordering changes computation order but not the acceptance requirement.
  • As the store grows, NS pass and write-back rates decrease; across five streams, 1806 of 5817 candidates reached GS.NS controls store growth by filtering candidates before the more expensive grounding check.
  • On WixQA, all checked candidates passed GS because the grounding threshold was permissive while novelty controlled repeated support patterns.The average checked-candidate GS was 0.886 and the lowest score was 0.653, above the 0.6 threshold.
  • Case studies show that novelty blocks near-duplicate experiences, while grounding rejects correct answers lacking supporting retrieved entities.LivingRAG can return an answer without storing the candidate when grounding fails.

E Dataset Details

The evaluation spans multi-hop and support-oriented QA streams, with dataset-specific analyses of reuse structure, entity extraction, and passage-length variation.

  • Datasets: The evaluation uses five streams, including 2WikiMultiHopQA, HotpotQA, MuSiQue, MuSiQue-full, and WixQA.The standard benchmarks test combining evidence across multiple facts, while MuSiQue-full and WixQA emphasize online experience reuse.
  • Gate dynamics: Figure 3 tracks NS pass, GS pass, experience write-back, and cumulative stored experiences over online inference windows.GS is measured only for candidates reaching the GS check, and the first 10% of each stream is expanded for startup behavior.
  • MuSiQue: MuSiQue-full contains 2,417 answerable questions and exposes reusable reasoning through decomposed single-hop questions.Experience clusters are connected components whose questions share at least one decomposed single-hop question.
  • MuSiQue: MuSiQue-full provides stronger reuse structure than the smaller MuSiQue subset, with more clusters, higher coverage, and larger average cluster size.This makes it a stress test for systems that reuse prior reasoning components.
  • WixQA: WixQA contains support questions over the Wix Help Center and shares an enterprise knowledge base with recurring products, settings, and help-article neighborhoods.The main evaluation uses 400 balanced ExpertWritten and Simulated questions and excludes the Synthetic subset.
  • Entity extraction: WixQA requires product-domain query terms because its important cues are often UI labels, plans, features, settings, and user actions rather than ordinary named entities.LLM-based extraction with Qwen3.5-9B supplies seed terms while leaving downstream graph retrieval unchanged.
  • Entity extraction: On WixQA, spaCy extracts no entity for 52.5% of questions, whereas LLM NER removes this failure case by extracting useful product-domain terms.The same extractor is used for LinearRAG and LivingRAG in WixQA comparisons.
  • Passage structure: WixQA has substantially shorter chunks than the other datasets: 44.4% contain fewer than 1,000 characters and 59.6% fewer than 2,000.This mix of short and long passages creates a dataset-specific retrieval challenge.

F.3 Evaluation Metric for WixQA

WixQA requires a task-oriented LLM accuracy metric because its support-style answers are longer and more varied than standard short-answer QA. In a 400-question evaluation, LLM-based accuracy distinguishes answer quality that contain-match largely misses.

  • Entity extraction: WixQA’s product-domain retrieval cues motivate LLM entity extraction because spaCy often misses or broadens them.The prompts extract Wix product-documentation entities and article-level action-topic terms from passages, while question prompts map queries to canonical product names and action-topic phrases.
  • Why contain-match is unsuitable: WixQA expects support solutions with product-specific guidance, operation steps, conditions, and caveats rather than short answer spans.Its median answer length is 59 words, compared with 2 words for the other datasets; 85.0% exceed 30 words.
  • Why contain-match is unsuitable: Contain-match accuracy is too strict for WixQA because a correct support answer need not contain the full official answer verbatim.A generated answer may solve the user’s problem without copying the entire gold support answer.
  • LLM-based evaluation: 66.00% versus 70.25%: LLM-based accuracy reports these WixQA results for LinearRAG and LivingRAG, respectively.Contain-match records 0 hits for LinearRAG and 1 for LivingRAG in the same 400-question evaluation.
  • LLM-based evaluation: LLM evaluation checks whether standard QA answers contain key gold information without contradiction, while WixQA evaluation checks whether the support issue is correctly resolved.For WixQA, the answer must recommend the right product or feature and describe the core operation correctly.

G Runtime Breakdown

LivingRAG adds experience matching and asynchronous write-back around an unchanged LinearRAG retrieval path. Retrieval remains a small runtime component, while validation is more expensive but can occur after answer delivery.

  • Runtime components: LivingRAG keeps LinearRAG’s CPU retrieval path unchanged and separates answer generation from experience write-back.Candidates are checked by novelty and grounding gates after answer generation, with LLM calls issued concurrently.
  • Runtime components: 1.7% to 5.1%: retrieval accounts for this share of LLM time across the measured datasets, and LivingRAG retrieval is close to LinearRAG.The added experience-augmented retrieval step is therefore not the main runtime bottleneck.
  • Validation cost: Write-back validation is more expensive than retrieval because it performs local NLI checks and maintains the experience store.The checks run on a single NVIDIA GeForce RTX 3060 GPU.
  • Validation cost: Validation need not increase user-visible answer latency because it begins after answer generation and can finish asynchronously.The asynchronous design keeps online answering separate from experience-store maintenance.
  • Prompt-token trade-off: LivingRAG incurs a small later increase in input prompt tokens when it adds reasoning scaffolds, but this is smaller than its completion-token reduction.At stream startup, no useful experiences exist, so LivingRAG’s prompt matches LinearRAG’s earliest prompt.

J Direct Case Study: LinearRAG vs. LivingRAG

Direct case studies show LivingRAG reuses prior graph neighborhoods and reasoning scaffolds to recover answer-bearing evidence or shorten generation. The benefits include corrected answers and substantial completion-token savings in selected cases.

  • Experience-augmented retrieval: LivingRAG corrected a failed MuSiQue-full case, changing the answer from Francisco de Orellana to Sebastian Cabot while reducing completion tokens from 8,739 to 1,264.The stored experience supplied a Cabot-centered entity neighborhood that led retrieval to the gold evidence chain.
  • Experience-augmented retrieval: LivingRAG corrected another MuSiQue-full case, changing the answer from 113 BC to 406 while reducing completion tokens from 4,360 to 847.The reused experience transferred the Central Europe–406 event connection to the new question.
  • Reasoning-scaffold reuse: 3,628 to 325: scaffold reuse preserved a correct MuSiQue-full answer while sharply reducing completion tokens.The scaffold explicitly decomposed the Moon River performer and Audrey Hepburn awards steps before generation.
  • Reasoning-scaffold reuse: 7,054 to 1,042: a reused scaffold preserved the correct answer 45 while transferring the same second-hop form to a different surface entity.Both cases bind the entity to Clarksville, so the distance hop and answer form remain the same.

K.2 Query-Order Robustness

LivingRAG’s gains persist across alternative online query orders. It achieves higher accuracy and lower completion-token use than LinearRAG in every reported metric and tested order.

  • Robustness across query orders: LivingRAG outperforms LinearRAG on every reported accuracy metric across the tested query orders.The evaluation covers complete 2Wiki, MuSiQue-full, and WixQA streams under the paper order and two random permutations.
  • Robustness across query orders: LivingRAG uses fewer completion tokens than LinearRAG in every tested query order.Accuracy and token cells report LinearRAG / LivingRAG, with tokens measured as per-query averages.
  • Robustness across query orders: Realized reuse patterns remain stable after reordering, providing evidence that benefits are not driven only by one favorable stream arrangement.The reported means and sample standard deviations are computed over the paper order and two permutations.
  • Transfer beyond LinearRAG: The separate PPR test provides preliminary retrieval-level evidence of interface transfer rather than end-to-end generation compatibility.The adapter treats verified states as sparse priors while leaving PPR and ranking unchanged.
  • Transfer beyond LinearRAG: The retrieval-level transfer test uses all 400 WixQA queries and 7,132 corpus chunks with earlier quality-gated states mapped into the graph.R@5 measures top-five gold-article coverage, Hit@5 any gold hit, and MRR@5 the first-hit rank.
Loading 2608.25960v1…