PaperlayerView exact revision on arXiv

Source-linked AI summary

Don't Lose Entities from Retrieval to Generation: Dual Entity Recovery RAG for multi-hop QA

Heechang Lee, Dong-Young Lim

arXiv:2609.06065v1cs.CL

TL;DR

Multi-hop RAG decompositions can discard entity information, disrupting retrieval or leaving retrieved sentences without the context needed to ground pronouns. DER-RAG keeps entities explicit across both stages with entity-aware sub-questions and sentence prefixes. It matches or exceeds strong baselines on three benchmarks, while fixed-evidence tests show that prefixes improve exact match.

  • Problem

    Question and corpus decomposition can remove entities needed for retrieval or the pronoun context needed to interpret a retrieved sentence.

    Source: §1 · pp. 1–2

  • Method

    DER-RAG uses entity-explicit query decomposition and prefixes retrieved sentences with the subject entity of their source passage.

    Source: §3 · p. 4, §3.2 · p. 5

  • Results

    Across three multi-hop QA benchmarks, DER-RAG matches or exceeds strong baselines, including graph-based methods.

    Source: §5 · p. 9

  • Takeaways & Limitations

    A retrieval-controlled experiment shows that subject-entity prefixes improve exact match even when the retrieved evidence is fixed.

    Source: §1 · p. 2

    DER-RAG reuses one passage-level subject entity across sentences and queries, although useful entity granularity can vary and other entities may appear in a passage’s coreference chains.

    Source: §6 · p. 10

Abstract

from arXiv · show

Retrieval-augmented multi-hop question answering (QA) decomposes a query into sub-questions and decomposes the corpus into smaller retrieval units such as sentences. Both forms of decomposition improve the pipeline, but we show that both share the same vulnerability, the loss of entity information, and that this loss breaks the pipeline at two separate points. The first point is retrieval, where a sub-question loses the entity resolved at the previous hop, leaving the retriever with nothing to match against. The second point is harder to see, because retrieval still appears to succeed. Once a passage is split into sentences, an isolated sentence loses the context that grounds its pronouns, so even with the correct sentence in hand the LLM cannot tell which entity the sentence is about. We isolate this second point as a distinct failure mode that we call lost-in-generation, and a retrieval-controlled experiment shows that it degrades answers even when the gold evidence is fixed in the context. We then propose Dual Entity Recovery RAG (DER-RAG), which keeps the grounding entity explicit from retrieval through to generation with two lightweight components, a two-way query decomposition that carries the resolved entity across sub-questions and a subject entity prefix attached to each sentence at generation time. DER-RAG needs no graph construction, no corpus modification, and no fine-tuning, yet on three multi-hop QA benchmarks it matches or exceeds strong baselines, including graph-based methods that depend on costly offline structures.

1 Introduction

Multi-hop RAG decomposes both questions and corpora, but each decomposition can discard the entity needed to ground retrieval or generation. DER-RAG addresses both losses, and a fixed-evidence experiment shows that sentence prefixes improve answer exact match.

1 Introduction

  • Entity loss creates two distinct failures: later sub-questions can omit the resolved entity needed for retrieval, while split sentences can lose the context grounding their pronouns.The latter failure persists after retrieval succeeds and is invisible to retrieval-side metrics.Source: §1 · pp. 1–2
  • With retrieved evidence held fixed, subject-entity prefixes improve exact match by +9.24 on MuSiQue, +9.08 on 2Wiki, and +5.18 on HotpotQA.Because the evidence is identical across conditions, the gains isolate a generation-side failure rather than a retrieval difference.Source: §1 · p. 2
  • Across three multi-hop QA benchmarks, DER-RAG matches or exceeds strong baselines, including graph-based methods that rely on offline graph construction.It uses two lightweight components without graph construction, corpus modification, or fine-tuning.Source: §1 · p. 2
  • DER-RAG carries resolved entities across sub-questions and prefixes retrieved sentences with their source passage’s subject entity, keeping grounding explicit from retrieval through generation.The approach requires no graph construction, corpus modification, or fine-tuning.Source: §1 · p. 2

2 Related Work

Prior multi-hop RAG work improves retrieval and reasoning through decomposition, reranking, compression, and offline corpus structures. DER-RAG instead examines where entity information is lost across the pipeline, including after correct evidence has been retrieved.

2 Related Work

  • Prior RAG work improves systems through iterative retrieval, reranking, and context compression, while multi-hop methods decompose questions and retrieve at each step.Retrieval units range from passages to sentences to longer grouped units.Source: §2 · pp. 2–3
  • Graph-based methods organize corpora into trees or entity graphs and retrieve by traversal, but require offline graph construction whose cost grows with corpus size.ChainRAG identifies lost-in-retrieval when entity underspecification in a sub-question causes retrieval to fail.Source: §2 · p. 3
  • For decomposition depth, DER-RAG predicts the number of sub-questions with a prompted LLM call, avoiding both a fixed two-hop assumption and a fine-tuned termination policy.This supports arbitrary reasoning depths without training data.Source: §2 · p. 3
  • DER-RAG treats referential failure as a pipeline-generated problem after retrieval, unlike static decontextualization methods that rewrite and re-index sentences.It prefixes sentences at generation time, leaving the retrieval index unchanged and adding one offline LLM call per passage.Source: §2 · p. 3

3 DER-RAG

DER-RAG keeps the entity grounding evidence explicit through query decomposition, sentence retrieval, and final answer reasoning. Its components recover entities at both retrieval and generation stages without graph construction, corpus modification, or fine-tuning.

3 DER-RAG

  • Entity information is lost at retrieval when later sub-questions omit the previously resolved entity, and at generation when isolated sentences lose the context grounding their pronouns.The generation failure can persist even when the sentence has been retrieved correctly.Source: §3 · p. 3
  • The method uses one offline LLM call per passage to extract subject entities; it requires no graph construction, corpus modification, or fine-tuning, and extraction scales linearly with corpus size.The extraction treats passages independently and is parallelizable.Source: §3 · p. 4

3.1 Two-Way Query Decomposition

  • Comparison questions generate all sub-questions in one call from entities already named in the question; other questions are decomposed sequentially, with prior answers supplying explicit entities for later steps.After classifying the question type, a prompted LLM predicts decomposition depth without training data or a learned termination policy.Source: §3.1 · p. 5

3.2 Prefixed-Sentence Generation

  • DER-RAG retrieves original sentences, then prefixes each with the subject entity extracted once per source passage before sending the prefixed context to the LLM.Subject entities are cached as passage metadata; the retrieval index remains unchanged and the prefix can be used as a drop-in modification.Source: §3.2 · p. 5

3.3 Final Answer Reasoning

  • DER-RAG also prefixes evidence during final-answer reasoning, so the LLM can aggregate grounded sentences and sub-answers under either integration strategy.The same loss of entity grounding can recur when reasoning over accumulated evidence at the final-answer stage.Source: §3.3 · p. 6

4 Experiments

Experiments on MuSiQue, 2WikiMultiHopQA, and HotpotQA compare DER-RAG with retrieval baselines and isolate its components using controlled tests and ablations. DER-RAG matches or exceeds strong baselines, while controlled retrieval and component ablations examine entity grounding, routing, retrieval units, and efficiency.

4.2 Main Results

  • Across three benchmarks and both backbone LLMs, DER-RAG matches or exceeds the strongest baselines on average, and its better variant ranks first or second on every dataset.It also outperforms graph-based HippoRAG w/ IRCoT on all three datasets and performs strongly under both answer integration strategies.Source: §4.2 · pp. 6–7

4.3 Lost-in-Generation under Controlled Retrieval

  • With evidence fixed, sentence splitting lowers F1 by 2.2–2.9 points, while subject prefixes recover 1.6–3.2 points and bring results near the full-passage level.The retriever is bypassed and the underlying passages remain unchanged, isolating the generation effect; finer units can still help under ordinary retrieval.Source: Table 2 · p. 7, §4.3 · p. 7

4.4 Ablation Study

  • Two-way query decomposition improves F1 over post-hoc rewriting by 4.4 points on MuSiQue and 2Wiki, while gains over one-step decomposition are 10.5 and 5.2 points, respectively.On HotpotQA, the strategies are close and two-way decomposition is slightly behind, consistent with its mostly two-hop questions.Source: Table 3 · p. 7, §4.4 · p. 8
  • On the NaiveRAG base, prefixing sentences adds 3.0, 9.5, and 4.0 F1 points over unmodified sentences on MuSiQue, 2Wiki, and HotpotQA.The compared sentences are retrieved identically, so the gain isolates generation-time entity grounding rather than retrieval.Source: §4.4 · p. 8, Table 4 · p. 8

4.5 Effect of Two-Way Routing

  • Removing routing has virtually no effect on MuSiQue, but causes a notable drop on 2Wiki, where 82 of 200 questions are comparison-type.The reported benefit is smaller on HotpotQA, where comparison-question accuracy is lower.Source: §4.5 · pp. 8–9, Table 5 · p. 8

4.7 Top-k Study

  • Across integration types and datasets, F1 is relatively stable as k ranges from 5 to 25, with diminishing returns or slight declines generally appearing beyond k=10.The exception is 2Wiki under ANSINT, where F1 gradually rises with k; the study adopts k=10 as the default.Source: Figure 4 · p. 9, §4.7 · p. 9

4.6 Effect of Sentence Prefixing

  • Sentence prefixing improves performance on all three datasets under both integration strategies, unlike routing’s more concentrated benefit on 2Wiki.The paper attributes the broad pattern to lost-in-generation arising whenever passages are split into sentences, regardless of question type or reasoning depth.Source: Table 6 · p. 8, §4.6 · p. 9

4.8 Online Efficiency

  • DER-RAG uses around 25–33 retrieved sentences per question versus ChainRAG’s 67–104, with a comparable but slightly higher number of LLM calls.The authors attribute the smaller context to prefixing restoring entity grounding without retrieving additional sentences.Source: Table 8 · p. 9, Table 7 · p. 9, §4.8 · p. 9

5 Conclusion

The conclusion identifies entity loss during query and corpus decomposition as a two-stage problem in multi-hop RAG. DER-RAG addresses both stages and matches or exceeds strong baselines across three benchmarks without graph construction, corpus modification, or fine-tuning.

5 Conclusion

  • Lost-in-retrieval occurs when a later sub-question drops the entity resolved at the previous hop, while lost-in-generation occurs when sentence splitting removes the context grounding pronouns.A retrieval-controlled experiment finds that lost-in-generation degrades answers even when gold evidence is fixed in context.Source: §5 · p. 9
  • DER-RAG keeps grounding entities explicit across retrieval and generation using two lightweight components, without graph construction, corpus modification, or fine-tuning.Source: §5 · p. 9
  • Across three multi-hop QA benchmarks, DER-RAG matches or exceeds strong baselines, including graph-based methods.The conclusion presents direct entity-loss remedies as a worthwhile alternative to heavier retrieval structures.Source: §5 · p. 9

6 Limitations

DER-RAG has three design limitations: its passage-level subject entity can be too rigid, its decomposition-depth decision is made only once, and evaluation is limited to Wikipedia-derived benchmarks.

6 Limitations

  • A single subject entity is reused for every sentence and query, although the most useful entity can vary in granularity and with the query.A passage-level entity also cannot resolve every coreference chain when sentences refer to entities other than the main subject.Source: §6 · p. 10
  • Decomposition depth is decided once after question-type classification, limiting adaptation when a question’s reasoning depth is hard to predict.The paper identifies a train-free per-step policy as a future direction.Source: §6 · p. 10
  • Evaluation uses only Wikipedia-derived MuSiQue, 2WikiMultiHopQA, and HotpotQA, so generalization to real-world domain-specific corpora remains unvalidated.The authors call for empirical validation on such corpora before assuming results generalize beyond these benchmarks.Source: §6 · p. 10

A Dataset Statistics and Implementation Details

The appendix describes benchmark passage statistics, shared evaluation settings, offline entity-extraction costs, and the fixed-context setup for the controlled experiment.

A Dataset Statistics and Implementation Details

  • 43.69 and 47.93 sentences per passage are the averages for MuSiQue and HotpotQA, versus 22.75 for 2WikiMultiHopQA.Each dataset contains 200 questions; the passage-density difference is relevant because denser passages yield more pronoun-bearing sentences after splitting.Source: A Dataset Statistics and… · p. 12, Table 9 · p. 12
  • All methods share text-embedding-3-small, BGE-Reranker-large, spaCy sentence splitting, and the same preprocessed corpus.DER-RAG extracts subject entities offline in one pass and caches them; Table 10 reports the resulting cost.Source: A Dataset Statistics and… · p. 12, Table 10 · p. 12
  • 2Wiki has the lowest total extraction cost because its passages average fewer sentences, reducing input tokens per full-passage extraction call.The per-call cost varies across corpora and does not track the number of calls alone.Source: A Dataset Statistics and… · p. 12
  • The controlled experiment fixes each question’s context to gold passages plus same-pool distractors, then compares sentence contexts with and without source-passage subject prefixes.The fixed contexts average 8.6 to 11.1 passages across datasets and range from 2 to 20; all datasets and models were used under their licenses and intended research use.Source: A Dataset Statistics and… · p. 12, Table 11 · p. 12

B Question Type Classification Details

The LLM-based binary classifier routes questions to decomposition paths with over 99% accuracy on each benchmark. The few errors are mainly bridge questions classified as comparison questions.

B Question Type Classification Details

C Decomposition Depth

DER-RAG predicts decomposition depth after question-type classification rather than fixing it in advance. On these benchmarks, most multi-step questions receive two sub-questions and none receive three or more.

C Decomposition Depth

  • DER-RAG predicts the number of sub-questions after question-type classification instead of fixing depth in advance.The analysis records the predicted count for every question and reports its distribution by dataset.Source: C Decomposition Depth · p. 13
  • No question is assigned three or more sub-questions, while the vast majority of multi-step questions receive exactly two across all three datasets.Questions assigned zero are comparison-type questions routed to one-step decomposition and excluded from multi-step decomposition.Source: C Decomposition Depth · p. 13, Table 13 · p. 13
  • The authors report that predicted depth aligns with benchmark reasoning requirements and terminates compactly without overdecomposing.They interpret the absence of three-or-more-step assignments alongside the predominance of two-step assignments.Source: C Decomposition Depth · p. 13

D Portability of Two-Way Query Decomposition

Replacing ChainRAG’s post-hoc rewriting with two-way query decomposition improves or matches performance across three datasets, supporting its use as a portable component in other RAG pipelines.

D Portability of Two-Way Query Decomposition

  • The transfer test replaces ChainRAG’s post-hoc rewriting with two-way query decomposition while leaving the rest of ChainRAG unchanged.The study distinguishes this transfer test from the query-decomposition ablation on a NaiveRAG base.Source: D Portability of Two-Way Query… · p. 13
  • Two-way query decomposition improves or matches ChainRAG’s performance across all three datasets.Table 14 compares ChainRAG with post-hoc rewriting against the version that replaces rewriting with two-way query decomposition.Source: D Portability of Two-Way Query… · p. 13, Table 14 · p. 14
  • The results indicate that two-way query decomposition can serve as a portable drop-in replacement within existing RAG frameworks.The reported comparison finds it resolves lost-in-retrieval more effectively than post-hoc rewriting.Source: D Portability of Two-Way Query… · p. 13

E Title vs. Subject Entity

Subject entity prefixes are comparable to or better than title prefixes in most tested settings, while extracted entities can provide fuller or more relevant passage subjects. The reported extraction errors are rare and have limited aggregate impact in the evaluated setting.

E Title vs. Subject Entity

  • Token-level F1 scores of 82 to 86 across all three datasets indicate that extracted subject entities closely match passage titles in most cases.The reported gap between token-level F1 and exact match reflects partial surface-form differences, such as “Daniel Day-Lewis” versus “Daniel Day Lewis”.Source: E Title vs. Subject Entity · p. 14, Table 15 · p. 14
  • On 2WikiMultiHopQA under CX-TINT, subject entity prefixing improves F1 from 68.84 to 72.04.Across most settings, subject entity prefixing is comparable to or better than title prefixing; title prefixing is marginally higher on MuSiQue under ANSINT.Source: E Title vs. Subject Entity · p. 14, Table 16 · p. 14
  • Case 1, where the title is correct and the extracted subject is incorrect, accounts for fewer than 3% of passages on every dataset.Correcting all identified extraction errors changes F1 by at most 1.04 points and EM by at most 2.0 points, with no consistent directional effect.Source: E Title vs. Subject Entity · p. 14, Table 18 · p. 15, Table 19 · p. 15
  • Correcting every identified extraction error has limited aggregate impact on final-answer performance in the evaluated setting.The passage attributes this bound to Case 1’s rarity, which accounts for 2.3–2.5% of passages across the three datasets.Source: E Title vs. Subject Entity · p. 14, Table 19 · p. 15
  • Extracted entities sometimes identify a passage’s primary subject when its title names a sub-entity, and can provide fuller forms than abbreviated titles.The human evaluation reports both cases: extracted entities can correct inaccurate titles and can complement titles judged correct.Source: E Title vs. Subject Entity · pp. 14–15, Table 17 · p. 15, Table 18 · p. 15
  • The most effective subject entity may vary in specificity and with the query being answered.The authors propose identifying candidate entities at varying granularity and selecting the most query-relevant one at retrieval time.Source: E Title vs. Subject Entity · p. 15

F Case Study

A multi-hop QA case study contrasts how NaiveRAG with query decomposition, ChainRAG, and DER-RAG handle entity grounding. The appendix also describes DER-RAG’s prompt-based components and discloses the use of AI assistants for writing assistance.

F Case Study

  • DER-RAG resolves both entity-loss failures in the case study by carrying the resolved entity into the second sub-question and prefixing retrieved sentences with their passage’s subject entity.NaiveRAG with one-step query decomposition misses the resolved entity at retrieval; ChainRAG repairs that query but still leaves retrieved sentences without an explicit subject, while DER-RAG answers correctly.Source: F Case Study · pp. 15–16, Table 20 · p. 17

G Prompts

  • DER-RAG uses task-specific prompts for subject entity extraction, question type classification, and sub-question generation, without task-specific fine-tuning.The prompts use few-shot demonstrations; Figures 7–9 show comparative-question decomposition and the first and final sub-questions for non-comparative questions.Source: G Prompts · p. 16, Figure 6 · p. 16, Figure 7 · p. 17

H Use of AI Assistants

  • AI assistants were used for writing assistance, including grammar checking, sentence-level editing, and improving clarity and conciseness.The paper states that all technical content, experimental results, and claims are the authors’ own.Source: H Use of AI Assistants · p. 16
Loading 2609.06065v1…