PaperlayerView exact revision on arXiv

Source-linked AI summary

HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, Yu Su

arXiv:2405.14831v3cs.CLcs.AI

TL;DR

Retrieval-augmented generation (RAG) systems struggle to integrate new knowledge across passages. HippoRAG addresses this through a hippocampal-inspired memory index built from an LLM-generated knowledge graph. It reports improvements of around 3 and 20 points over current RAG methods [10] [71] on MuSiQue and 2WikiMultiHopQA [33], respectively.

  • Problem

    Current RAG methods encode passages independently, limiting integration of new knowledge across documents needed for tasks such as scientific literature review.

    Source: §1 · p. 1

  • Method

    HippoRAG emulates hippocampal memory indexing through an LLM-built knowledge graph and query-entity linking for online retrieval.

    Source: §2.2 · p. 3

  • Results

    Around 3 and 20 points of improvement over current RAG methods [10] [71] are achieved by HippoRAG on MuSiQue and 2WikiMultiHopQA [33], respectively.

    Source: §1 · p. 2

  • Takeaways & Limitations

    HippoRAG offers a promising middle ground between standard RAG and parametric memory for continuously updating LLM long-term memory.

    Source: §7 · p. 10

    HippoRAG uses off-the-shelf components without additional training, and most system errors arise from named-entity recognition and open information extraction.

    Source: §7 · p. 10

Abstract

from arXiv · show

In order to thrive in hostile and ever-changing natural environments, mammalian brains evolved to store large amounts of knowledge about the world and continually integrate new information while avoiding catastrophic forgetting. Despite the impressive accomplishments, large language models (LLMs), even with retrieval-augmented generation (RAG), still struggle to efficiently and effectively integrate a large amount of new experiences after pre-training. In this work, we introduce HippoRAG, a novel retrieval framework inspired by the hippocampal indexing theory of human long-term memory to enable deeper and more efficient knowledge integration over new experiences. HippoRAG synergistically orchestrates LLMs, knowledge graphs, and the Personalized PageRank algorithm to mimic the different roles of neocortex and hippocampus in human memory. We compare HippoRAG with existing RAG methods on multi-hop question answering and show that our method outperforms the state-of-the-art methods remarkably, by up to 20%. Single-step retrieval with HippoRAG achieves comparable or better performance than iterative retrieval like IRCoT while being 10-30 times cheaper and 6-13 times faster, and integrating HippoRAG into IRCoT brings further substantial gains. Finally, we show that our method can tackle new types of scenarios that are out of reach of existing methods. Code and data are available at https://github.com/OSU-NLP-Group/HippoRAG.

1 Introduction

HippoRAG uses an LLM-built knowledge graph and query-seeded Personalized PageRank to integrate information across passages in one retrieval step. The introduction reports multi-hop QA improvements and cheaper, faster retrieval than iterative methods.

1 Introduction

  • Current RAG encodes passages independently, limiting cross-document knowledge integration despite supplying new knowledge to static LLMs [36].Current AI systems lack continuously updating long-term memory. Tasks requiring cross-passage integration include scientific literature review, legal case briefing, and medical diagnosis.Source: §1 · p. 1
  • Even perfectly executed multi-step RAG can be insufficient for path-finding multi-hop questions; a case study explores existing methods’ limitations and HippoRAG’s potential.Existing systems use iterative retrieval and LLM generation to join disparate passages.Source: §1 · pp. 2–3
  • HippoRAG transforms a corpus into a schemaless knowledge graph with an LLM, then uses query-seeded Personalized PageRank [30] for single-step multi-hop retrieval.The design follows hippocampal memory indexing theory, in which the neocortex stores memory representations and the hippocampus indexes their associations. PPR explores graph paths and identifies relevant subgraphs for retrieval.Source: §1 · p. 2
  • Around 3 and 20 points of improvement over current RAG methods [10] [71] are reported on MuSiQue and 2WikiMultiHopQA [33], respectively.These gains come from HippoRAG’s single-step multi-hop retrieval.Source: §1 · p. 2
  • 10 to 30 times cheaper and 6 to 13 times faster online retrieval than iterative methods like IRCoT accompanies comparable performance.These efficiency comparisons concern HippoRAG’s online retrieval process.Source: §1 · p. 2
  • Up to 4% and 20% complementary gains on MuSiQue and 2WikiMultiHopQA [33], respectively, result from combining HippoRAG with IRCoT.The combination also improves performance on HotpotQA, described as a less challenging multi-hop QA dataset.Source: §1 · p. 2

2 HippoRAG

HippoRAG models memory encoding and retrieval using an LLM, an open knowledge graph, and retrieval encoders, with Personalized PageRank [30] supporting pattern completion. Node specificity further adjusts retrieval using local passage counts.

2.1 The Hippocampal Memory Indexing Theory

  • The hippocampal indexing theory describes pattern separation for distinct experiences and pattern completion for retrieving complete memories from partial cues.Encoding routes neocortical features through parahippocampal regions into an associative hippocampal index; retrieval returns relevant memories through this pathway. Updating the index integrates new information without changing neocortical representations, although HippoRAG omits many theoretical details for simplicity.Source: §2.1 · p. 3, §2.3 · p. 4

2.2 Overview

  • HippoRAG models the neocortex with an LLM, the hippocampal index with an open knowledge graph, and parahippocampal regions with retrieval encoders.Passage-level triple extraction stores discrete noun phrases for fine-grained pattern separation; encoder-derived links between similar phrases aid pattern completion.Source: §2.2 · p. 3

2.3 Detailed Methodology

  • Offline indexing first extracts named entities, then conditions triple extraction on them, adding synonymy edges when encoder cosine similarity exceeds a threshold.The two-step, one-shot prompting procedure balances generality with a bias toward named entities while retaining other noun-phrase concepts. Indexing also records noun-phrase occurrence counts in a |N| × |P| passage matrix.Source: §2.3 · p. 4
  • Query entities link to their highest-cosine-similarity graph nodes, and Personalized PageRank [30] propagates their weights before node probabilities are aggregated into passage rankings.Before node-specificity weighting, query nodes receive equal initial probability and other nodes receive zero. PPR distributes probability toward query nodes’ joint neighborhoods, and multiplying its output by the passage matrix produces retrieval scores.Source: §2.2 · p. 3, §2.3 · pp. 4–5
  • Node specificity weights each query node by the inverse number of passages containing it before PPR, modulating both its own probability and its neighborhood’s.The weight is s_i = |P_i|^-1, where P_i contains passages from which node i was extracted. Unlike IDF, this proposed alternative uses only locally available counts rather than a global passage total.Source: §2.3 · p. 5

3 Experimental Setup

The experimental setup evaluates HippoRAG on three multi-hop QA datasets against single- and multi-step retrieval baselines, measuring both retrieval and question-answering performance.

3.1 Datasets

  • 1,000 validation questions each from MuSiQue (answerable), 2WikiMultiHopQA [33], and HotpotQA define the evaluation sets.Following IRCoT, each dataset’s retrieval corpus pools supporting and distractor passages from the selected questions. HotpotQA is included for completeness despite being a weaker multi-hop reasoning test because of spurious signals.Source: §3.1 · p. 5

3.2 Baselines

  • Baselines include BM25, Contriever, GTR, ColBERTv2, Propositionizer [10], RAPTOR [71], and the multi-step method IRCoT.Propositionizer [10] rewrites passages into propositions, while RAPTOR [71] constructs summary nodes to facilitate retrieval from long documents.Source: §3.2 · p. 5

3.3 Metrics

  • Retrieval is measured with recall@2 and recall@5 (R@2 and R@5), while question answering uses exact match (EM) and F1.Tables 2 and 3 cover single-step and multi-step retrieval, respectively; the multi-step setting combines HippoRAG with methods such as IRCoT.Source: §3.3 · p. 5, §3.2 · p. 5, Table 2 · p. 6

3.4 Implementation Details

  • 100 MuSiQue training examples tune HippoRAG’s synonymy threshold to 0.8 and Personalized PageRank damping factor to 0.5.Defaults use GPT-3.5-turbo-1106 at temperature 0 with Contriever or ColBERTv2. The damping factor controls restarting from query nodes rather than continuing graph exploration, and performance is reported as generally robust to hyperparameters.Source: §3.4 · p. 6

4 Results

HippoRAG improves retrieval and question answering, while combining it with IRCoT further improves retrieval. Single-step HippoRAG matches or outperforms IRCoT with cheaper, faster online retrieval.

Single-Step Retrieval Results

  • 11% R@2 and 20% R@5 improvements on 2WikiMultiHopQA, and around 3% gains on MuSiQue, accompany HippoRAG outperforming all single-step baselines, including Propositionizer and RAPTOR.The larger 2WikiMultiHopQA gains partly reflect its entity-centric design. HotpotQA performance is competitive but limited by lower knowledge-integration requirements and a concept-context tradeoff, which ensembling alleviates.Source: §4 · p. 6

Multi-Step Retrieval Results

  • Around 4%, 18% and 1% R@5 improvements on MuSiQue, 2WikiMultiHopQA and HotpotQA, respectively, result from using HippoRAG as the retriever for IRCoT.These gains demonstrate that HippoRAG and iterative retrieval are complementary.Source: §4 · p. 6

Question Answering Results

  • Up to 3%, 17% and 1% F1 improvements on MuSiQue, 2WikiMultiHopQA and HotpotQA, respectively, follow HippoRAG’s retrieval improvements using the same QA reader.QA experiments use the best-performing retrieval backbone, ColBERTv2, and compare HippoRAG, ColBERTv2, IRCoT, and IRCoT using HippoRAG.Source: §4 · pp. 6–7, Table 4 · p. 7
  • 10-30 times cheaper and 6-13 times faster online retrieval accompanies single-step HippoRAG matching or outperforming IRCoT in question answering.The efficiency comparison concerns online retrieval rather than the complete QA pipeline.Source: §4 · p. 7

5 Discussions

HippoRAG’s single-step multi-hop retrieval improves recovery of complete supporting evidence and can find associative paths missed by ColBERTv2 and IRCoT. The analyses support flexible LLM extraction, PPR, and dataset-dependent contributions from node specificity and synonymy edges, while identifying competitive open-weight indexing alternatives.

5.1 What Makes HippoRAG Work?

  • Two of three datasets favor Llama-3.1-70B over GPT-3.5, while its 2Wiki performance remains competitive, supporting cheaper open-weight indexing.Llama-3.1-8B is also competitive with GPT-3.5 except on 2Wiki, where performance drops substantially.Source: Table 5 · p. 7, §5.1 · p. 7
  • PPR outperforms query-node weighting and direct-neighbor expansion on all three datasets; adding neighbors without PPR performs worse than using query nodes alone.The query-only alternative multiplies query-node probabilities by node specificity; the neighborhood alternative additionally distributes a small amount of probability to direct neighbors.Source: Table 5 · p. 7, §5.1 · p. 8
  • Node specificity considerably improves MuSiQue and HotpotQA but barely affects 2WikiMultiHopQA, where synonymy edges have their largest effect.The authors attribute the specificity pattern to 2WikiMultiHopQA’s reliance on named entities with little term-weight variation, and suggest synonymy edges help standardize noisy entity mentions.Source: Table 5 · p. 7, §5.1 · p. 8
  • Twice as many triples are produced by GPT-3.5 as REBEL, whose substitution substantially degrades retrieval performance.An intrinsic OpenIE evaluation using 239 gold triples from 20 MuSiQue training examples finds both Llama-3.1-Instruct models slightly below GPT-3.5, but all LLMs vastly outperform REBEL.Source: Table 5 · p. 7, §5.1 · pp. 7–8

5.2 HippoRAG’s Advantage: Single-Step Multi-Hop Retrieval

  • 6% on MuSiQue and 38% on 2WikiMultiHopQA are HippoRAG’s top-5 all-recall advantages over ColBERTv2, increasing from the reported 3% and 20% retrieval gaps.All-recall measures the percentage of queries for which every supporting passage is retrieved, highlighting single-step multi-hop retrieval rather than partial evidence recovery. IRCoT can also solve multi-hop retrieval, but its online retrieval is 10-30 times more expensive and 6-13 times slower.Source: Table 6 · p. 8, §5.2 · p. 8

5.3 HippoRAG’s Potential: Path-Finding Multi-Hop Retrieval

  • HippoRAG’s associative index and graph search retrieve relevant passages in a path-finding example where ColBERTv2 and IRCoT fail.Path-finding questions require identifying a relevant path among many possibilities, rather than following a specified path. HippoRAG connects disparate information about Thomas Südhof to retrieve his passages.Source: Table 7 · p. 8, §5.3 · p. 9

6 Related Work

HippoRAG distinguishes its continuously updated knowledge graph from summarization-based memory indexing and supervised graph retrieval. It combines LLM-based graph construction with structured retrieval, complementing graph-based improvements to answer generation.

6.1 LLM Long-Term Memory

  • Parametric updating lacks a robust continual-learning solution [46], while long-context memory faces engineering hurdles and observed limitations [21].Existing updating approaches include fine-tuning, model editing [15] [49] [52] [95], and external parametric memory modules [32]. Expanding context windows suggest another memory mechanism, but its viability remains uncertain.Source: §6.1 · p. 9
  • Multi-step RAG integrates new or updated knowledge [38, 64, 72, 78, 88, 90, 92], but cannot solve the paper’s more complex path-finding multi-hop questions.RAG provides a simple mechanism for updating stored knowledge over time [36]; the limitation concerns its online information integration.Source: §6.1 · p. 9
  • Unlike RAPTOR [71], MemWalker [9], and GraphRAG, which repeat summarization when data arrives, HippoRAG integrates new knowledge by adding knowledge-graph edges.These methods integrate information during offline indexing and might handle the more complex knowledge-integration tasks, but their summaries must be regenerated when new data is added.Source: §6.1 · p. 9

6.2 Multi-Hop QA & Graphs

  • Graph-augmented reading comprehension improves generation, making it complementary to HippoRAG’s retrieval-only improvements.Earlier supervised methods combine hyperlink or co-occurrence graphs with language models through GNNs; newer methods place knowledge-graph triples directly into LLM prompts [57] [43] [47].Source: §6.2 · p. 10
  • HippoRAG constructs its own knowledge graph with LLMs and retrieves without supervision, unlike trained graph-traversal rerankers using Wikipedia hyperlinks [16] [4] [44].The paper attributes greater adaptability to this combination of graph construction and unsupervised multi-hop retrieval.Source: §6.2 · p. 10

6.3 LLMs & KGs

  • HippoRAG combines LLM knowledge-graph construction with structured knowledge retrieval, continuing research on language-model–knowledge-graph synergy.Related work augments LLMs with knowledge graphs [48] [84], or augments graphs through parametric-knowledge distillation [7] and direct text parsing [8].Source: §6.3 · p. 10

7 Conclusions & Limitations

HippoRAG shows strong path-following multi-hop QA results and promise on path-finding, alongside efficiency improvements and continuous updating. Its limitations include extraction errors, graph-search errors, and unvalidated scalability beyond current benchmarks.

Conclusions

  • HippoRAG’s knowledge integration, efficiency improvements, and continuous updating position it as a middle ground between standard RAG and parametric memory.Its knowledge integration capabilities are demonstrated by strong path-following multi-hop QA results and promise on path-finding multi-hop QA.Source: §7 · p. 10

Limitations

  • Scalability remains unvalidated: the synthetic hippocampal index’s efficiency and efficacy have not been empirically established at sizes far beyond current benchmarks.Llama-3.1 achieved similar performance to closed-source models, offering considerable cost reductions, but this does not establish scalability.Source: §7 · p. 10
  • All components are off-the-shelf without extra training, and most errors arise from NER and OpenIE, motivating component-specific fine-tuning.OpenIE consistency also needs improvement for longer compared with shorter documents.Source: §7 · p. 10
  • The remaining errors involve graph search, motivating improvements beyond simple PPR, such as allowing relations to guide graph traversal directly.Relation-guided traversal is proposed as a future improvement, rather than demonstrated here.Source: §7 · p. 10

Appendices … B Dataset Comparison

The appendices illustrate HippoRAG’s graph indexing and PPR-based retrieval through a MuSiQue example and compare dataset distractor quality. Contriever similarity distributions indicate that HotpotQA’s distractors are relatively ineffective.

Appendices

  • Beyond the worked pipeline example, the appendices cover ablations, OpenIE evaluation, path-finding cases, error analysis, efficiency, implementation and compute requirements, and LLM prompts.The worked example explicitly traces HippoRAG’s indexing and retrieval processes.Source: Appendices · p. 19, A HippoRAG Pipeline Example · p. 20

A HippoRAG Pipeline Example

  • The MuSiQue path-following example links Alhandra’s birthplace, Vila Franca de Xira, to Lisbon District through OpenIE triples from two supporting passages.The worked example follows both indexing and retrieval; its distractor excerpts concern the district mentioned in the question.Source: A HippoRAG Pipeline Example · pp. 20–21, Figure 3 · p. 20

Indexing: Subgraph Related to the Question

  • Indexing sequentially applies NER and OpenIE to each corpus passage, forming an open knowledge graph whose relevant subgraph connects information across passages.Figure 4 displays only the subgraph relevant to the example question.Source: Figure 4 · p. 21

Retrieval: Query NER & Node Retrieval

  • Retrieval extracts query entities, selects graph nodes with a retrieval encoder, propagates personalized probabilities using PPR, and sums node probabilities into passage rankings.In the example, probability spreads from Alhandra toward its birthplace node, helping retrieve passages beyond the query entity itself.Source: Figure 5 · p. 22

B Dataset Comparison

  • Contriever scores question–passage matches to assess distractor quality, with effective distractors ideally scoring near the mean of supporting passages.The comparison examines score distributions across all three datasets rather than relying on individual distractor examples.Source: B Dataset Comparison · p. 23
  • HotpotQA distractor scores cluster nearer the lower bound of supporting-passage scores than in the other two datasets, indicating relatively ineffective distractors.Their similarity scores are not substantially higher than those of the least similar supporting passages.Source: Figure 6 · p. 23, B Dataset Comparison · p. 23

C Ablation Statistics

REBEL generates around half as many graph nodes and edges as the tested LLMs. Both Llama-3.1 versions produce similar numbers of OpenIE triples to GPT-3.5 Turbo.

C Ablation Statistics

D Intrinsic OpenIE Evaluation · E Case Study on Path-Finding Multi-Hop QA · E.1 Path-Finding Multi-Hop Question Construction Process

Intrinsic OpenIE quality correlates imperfectly with retrieval performance, although recall gains accompany better retrieval for the larger Llama model. The path-finding case study describes cross-passage entity search and constructs challenging questions using paired traits to select Wikipedia distractors.

D Intrinsic OpenIE Evaluation

  • 239 gold triples from 20 MuSiQue training documents supported an intrinsic OpenIE evaluation using the CaRB [6] framework.The annotation exercise was designed to better understand the interaction between OpenIE and retrieval.Source: D Intrinsic OpenIE Evaluation · p. 23, Table 9 · p. 24
  • GPT-3.5 has much stronger intrinsic OpenIE performance than Llama-3.1-70B-Instruct but only slightly higher retrieval performance, demonstrating imperfect correspondence between the evaluations.The intrinsic evaluation also reveals a large performance difference between REBEL and LLMs.Source: D Intrinsic OpenIE Evaluation · p. 24, Table 9 · p. 24
  • Llama-3.1-70B-Instruct improves retrieval over its 8B counterpart, with intrinsic OpenIE gains confined to recall.The evaluation suggests that recall is particularly important for retrieval performance.Source: Table 9 · p. 24, D Intrinsic OpenIE Evaluation · p. 24

E Case Study on Path-Finding Multi-Hop QA

  • Path-finding questions require integrating passages to identify relevant entities among many candidates, challenging both single-step ColBERTv2 and multi-step IRCoT.An example asks for all Stanford professors working on the neuroscience of Alzheimer’s.Source: E Case Study on Path-Finding Multi-Hop… · p. 24

E.1 Path-Finding Multi-Hop Question Construction Process

  • The first two questions pair a book/movie trait with an author/director trait, using both traits to extract Wikipedia distractors.Construction begins by identifying a book or movie and then its author or director.Source: §E.1 · p. 24
  • The third question and motivating example start with a randomly chosen professor or drug, then use two associated traits to extract Wikipedia distractors.Professor questions combine university and research topic; drug questions combine treated disease and mechanism of action.Source: §E.1 · p. 24

E.2 Qualitative Analysis … F.2 Concepts vs. Context Tradeoff

HippoRAG’s qualitative examples demonstrate cross-passage knowledge integration, but its entity-centric design leaves useful context unused. Uncertainty-based ensembling improves retrieval, although it lowers performance in some cases.

E.2 Qualitative Analysis

  • HippoRAG identifies Mark Haddon and Black Hawk Down from cross-passage clues that ColBERTv2 and IRCoT fail to resolve.Black Hawk Down appears within HippoRAG’s first four passages; IRCoT’s three-step restriction prevents identifying the film while exploring two directors.Source: §E.2 · p. 24
  • HippoRAG ranks the Chlorambucil passage highest for the leukemia-mechanism query, while ColBERTv2 and IRCoT retrieve only leukemia-associated passages.IRCoT guesses that Venetoclax uses the requested cytosolic p53 interaction mechanism, although no passage in the curated dataset explicitly supports that guess.Source: §E.2 · p. 25

F.1 Overview

  • 100 MuSiQue errors fall into NER, OpenIE, and PPR categories, with NER limitations accounting for nearly half and OpenIE ranking second.NER can discard essential query information: extracting only “Windows 8” loses signals about browsers and accessibility.Source: §F.1 · p. 25
  • PPR can miss relevant subgraphs even when NER and OpenIE work correctly because similar passages create confounding signals and PPR cannot directly use query context.Correctly extracted “European” and “Huguenots” nodes still fail to guide retrieval toward the relevant refugee-emigration passage.Source: §F.1 · p. 25

F.2 Concepts vs. Context Tradeoff

  • HippoRAG’s entity-centric indexing enables single-step multi-hop retrieval and can prevent contextual distractions from overwhelming salient entities.For the Sergio Villanueva example, HippoRAG retrieves a relevant boxer passage, while ColBERTv2 follows contextual cues toward famous Spanish navigators.Source: §F.2 · p. 26
  • Around 48% of errors in the small-scale analysis arise from ignored context, particularly when extracted concepts are general.For a query reduced to “protons,” HippoRAG retrieves uranium and nuclear-weapons passages, whereas ColBERTv2 retrieves passages about discovering atomic numbers.Source: §F.2 · p. 26
  • Uncertainty ensembling improves MuSiQue and beats HotpotQA baselines in R@5; with IRCoT, the ColBERTv2 ensemble beats previous HotpotQA baselines in R@2 and R@5.Ensembling can also lower performance, especially on 2WikiMultiHopQA.Source: Table 13 · p. 26, Table 14 · p. 27, §F.2 · p. 27
  • Uncertainty ensembling averages normalized HippoRAG and dense-retrieval scores only when at least one query–knowledge-graph entity similarity falls below threshold θ.Both passage-score sets are normalized to the range 0 to 1 before averaging, increasing reliance on standard retrieval when entity linking is uncertain.Source: §F.2 · pp. 26–27

F.3 OpenIE Limitations … Open Information Extraction

HippoRAG trades higher offline indexing overhead for cheaper, faster online retrieval than IRCoT, while OpenIE omissions and longer passages expose extraction limitations. These sections also specify retrieval settings, compute requirements, and prompts for named-entity recognition and structured knowledge extraction.

F.3 OpenIE Limitations

  • OpenIE omissions include a crucial song title and war boundary years, creating knowledge gaps that can impair retrieval and question answering.GPT-3.5 Turbo misses “Don’t Let Me Wait Too Long” and fails to capture the war’s beginning and ending years accurately. The authors suggest long-entity insensitivity and neglected temporal properties as explanations.Source: §F.3 · p. 27, Table 15 · p. 27

F.4 OpenIE Document Length Analysis

  • GPT-3.5 Turbo’s OpenIE quality deteriorates substantially on longer passages in a small-scale intrinsic evaluation.The CaRB [6] evaluation compares the 10 longest and 10 shortest annotated passages using GPT-3.5 Turbo (1106). Greater sentence and paragraph complexity is a proposed explanation; further chunking risks disrupting sentence interdependence.Source: §F.4 · p. 27, Table 16 · p. 28

G Cost and Efficiency Comparison

  • 10 to 30 times higher retrieval costs for IRCoT accompany HippoRAG’s 6 to 13 times faster online retrieval using GPT-3.5 Turbo.Measurements cover 1,000 queries, with 2–4 sequential IRCoT rounds and single-threaded API calls; HippoRAG extracts query entities instead of processing retrieved documents. The authors expect token-count differences to preserve efficiency advantages even as LLM generation prices fall.Source: Table 17 · p. 28, G Cost and Efficiency Comparison · p. 28
  • 10 times slower indexing and $15 higher costs per 10,000 passages approximately characterize HippoRAG’s offline overhead relative to IRCoT.The authors suggest using open-source LLMs to reduce this overhead.Source: G Cost and Efficiency Comparison · p. 28
  • 4 hours approximately suffice to index 10,000 documents using locally deployed Llama-3.1-70B-Instruct, vLLM [40], and four H100 GPUs.The authors report performance similar to GPT-3.5 Turbo in their ablation study, supporting local deployment as an alternative for indexing.Source: Table 18 · p. 28, G Cost and Efficiency Comparison · p. 28

H Implementation Details & Compute Requirements

  • 10 passages are retrieved per IRCoT step, with at most two steps for HotpotQA and 2WikiMultiHopQA and four for MuSiQue.IRCoT + HippoRAG replaces BM25 with HippoRAG; the QA reader receives five passages and a one-shot chain-of-thought demonstration. Compute includes four 48GB RTX A6000 GPUs for ColBERTv2 and Contriever, four 80GB H100 GPUs for Llama-3.1 indexing, and two AMD EPYC 7513 processors for Personalized PageRank; OpenAI compute requirements are largely undisclosed.Source: H Implementation Details & Compute… · p. 29

I LLM Prompts

  • Separate prompts handle passage and query named-entity recognition, while iterative retrieval uses beam search retaining each candidate passage’s highest historical score.Beam search accommodates the original IRCoT implementation’s lack of per-passage scores.Source: I LLM Prompts · p. 29, Figure 7 · p. 29, Figure 8 · p. 30

Open Information Extraction

  • The OpenIE prompt requests RDF triples as a JSON list, requiring at least one supplied named entity per triple and explicit pronoun resolution.Each triple should preferably contain two supplied named entities.Source: Open Information Extraction · p. 31, Figure 9 · p. 31

Related papers

Loading 2405.14831v3…