Source-linked AI summary
EnSI-RAG: Entity-Structure-Indexed Retrieval-Augmented Generation for Long-Document Question Answering
Xuanyu Meng, Jiashuo Sun, Jash Rajesh Parekh, Jiawei Han
TL;DR
Long-document question answering is difficult when evidence spans multiple entities, relationships, and passages, while chunk-based retrieval can limit answer quality. EnSI-RAG uses entity-centered passages and a query-independent structure index to localize supporting evidence, then has an LLM synthesize answers from the original text; it achieves competitive or superior performance across Loong and Oolong.
Problem
Long-document question answering requires locating and connecting dispersed evidence while preserving entity identity, document structure, and relationships.
Method
EnSI-RAG constructs entity-centered passages and a query-independent entity-structure index whose retrieval handles point to original passages for LLM-based answer synthesis.
Results
EnSI-RAG achieves competitive or superior performance against strong retrieval baselines across Loong and Oolong's heterogeneous reasoning and information-aggregation settings.
Takeaways & Limitations
EnSI-RAG provides a flexible middle ground between unstructured chunk-based RAG and fully structured database-style reasoning while preserving the LLM's role as semantic integrator.
Takeaways & Limitations
Indexed records are retrieval-oriented and need not encode every detail required by future questions, so their usefulness depends on leading retrieval to the correct original evidence.
Abstract
from arXiv · showhide
Question answering (QA) over long, connected documents remains challenging because relevant evidence may span multiple entities and their relationships. Existing retrieval-augmented generation (RAG) methods typically index documents as raw chunks and retrieve them through embedding similarity. Their performance degrades when chunk boundaries separate entities from supporting evidence or when a question requires multi-hop reasoning across the corpus. We propose EnSI-RAG (Entity-Structure-Indexed Retrieval-Augmented Generation), a framework that constructs a query-independent, entity-centered index. Each record (e, t, k, v) represents an entity e, its type t, a semantic category k in {property, relation, aspect}, and a value v, while retaining links to the original source passages. At query time, these records serve as retrieval handles, and an LLM synthesizes the retrieved passages into the final answer. This design separates evidence localization from answer synthesis while preserving traceable source evidence. Across Loong and Oolong, EnSI-RAG achieves an average accuracy of 78.24. Relative to the published baseline scores used as references, this is 6.62 points higher, suggesting its effectiveness across these settings. The code is available at https://github.com/RamonMeng/EnSI-RAG.
Introduction
Long-document QA requires locating and integrating evidence distributed across entities, passages, and relationships. EnSI-RAG addresses this challenge with entity-centered passages and a query-independent structure that guides retrieval while leaving synthesis to the LLM.
- Motivation: Long-document questions often require evidence aggregation across multiple entities, documents, or relationships rather than a single passage.Applications include comparing financial reports, aggregating legal evidence, and tracing citations, methods, or findings across papers.
- Motivation: Larger context windows do not guarantee reliable evidence use, while RAG shifts answer-quality dependence to the retriever.Long-context models remain sensitive to evidence position, and retrieved evidence limits generation quality.
- Motivation: Fixed-length chunking can separate entities from supporting evidence, detach table rows from headers, or combine unrelated topics.Such boundaries may yield locally similar passages that remain insufficient for answering the question.
- Approach: EnSI-RAG defines retrieval units semantically as entity-centered passages and organizes them in a query-independent representation of semantic structure.This design aims to preserve coherent evidence while reducing fragmentation and contamination.
- Approach: EnSI-RAG extracts records for entities, types, properties, relations, and aspects, then uses them as retrieval handles pointing to original passages.The LLM integrates retrieved evidence and synthesizes the final answer, rather than reasoning over a normalized database.
- Evaluation: EnSI-RAG is evaluated on Loong and Oolong, covering heterogeneous multi-document reasoning and large-scale information aggregation.The framework uses unified retrieval while allowing domain-specific passage construction.
Related Work
Prior work addresses long-document QA through offline corpus structuring, larger context windows, and retrieval-augmented generation. EnSI-RAG is positioned against fixed-size chunk retrieval and its reliance on retriever recall.
- Offline corpus structuring: Classical and recent systems construct offline corpus representations using inverted indexes, entity and relation extraction, or hierarchical summaries.These approaches organize information before individual questions are received.
- Retrieval-augmented generation: Larger context windows do not fully resolve long-document QA because models can use evidence unevenly across input positions.The related work links this issue to the “lost in the middle” effect and aggregation-intensive questions.
- Retrieval-augmented generation: Adaptive, iterative, and reinforcement-learning-trained retrieval policies improve RAG, but these methods still rely on near-perfect retriever recall.The burden of finding sufficient evidence remains central to answer quality.
- Retrieval-augmented generation: The dominant RAG strategy ranks fixed-size chunks using lexical or dense similarity, despite poor alignment with long-document structure.Chunk boundaries can sever coherent discussions or detach table rows from their headers.
EnSI-RAG
EnSI-RAG preprocesses long documents into entity-centered passages, extracts structured retrieval records, and indexes them to preserve links to source evidence. Query-time retrieval follows entity structure, while the LLM synthesizes the original passages.
- Passage Construction: Offline preprocessing converts long documents into semantically contained, entity-centered passages and extracts records describing their entities and semantic fields.The workflow includes passage construction, information extraction, and index-entry building.
- Passage Construction: Passage boundaries vary by task, domain, and document type because semantic containment, rather than a fixed length, determines the needed context.Each passage balances retaining an entity’s supporting evidence against mixing unrelated entities or topics.
- Passage Construction: Compared with arbitrary chunking, entity-centered passages reduce semantic fragmentation by keeping entities with their evidence and reduce contamination from unrelated facts.Preserved provenance lets later stages retrieve grounded source evidence rather than an extraction-only surrogate.
- Information Extraction: Records encode entity mentions or canonical names, entity types, and passage-supported properties, relations, and aspects.Properties represent attributes, relations connect entities, and aspects identify information facets without requiring a single normalized value.
- Information Extraction: Extraction is retrieval-oriented: records need not capture every future answer detail, but every indexed field must be supported by local passage evidence.Partial records remain useful when they lead to the correct original passage, while unsupported information is excluded.
- Index Entry Building: The entity-structure index maps structured keys to sets of supporting passage identifiers, retaining repeated, refined, contradictory, or complementary evidence.Set-valued indexing supports lookup, aggregation, comparison, and exploration of ambiguous or multi-hop evidence paths.
- Retrieval: At query time, structured entity-centered keys constrain retrieval, and multi-hop results can expose intermediate entities for subsequent hops.This follows the corpus’s entity structure rather than relying on one embedding search over anonymous chunks.
- Answer Synthesis: The framework separates evidence localization from answer synthesis: records retrieve original passages, and the LLM produces the final answer from those passages.Structured records function as retrieval handles rather than a reasoning substrate.
Experiments
Experiments evaluate EnSI-RAG on Loong and Oolong using benchmark-specific protocols and compare its design choices through controlled ablations. Results favor row-level passages, domain-dependent label granularity, and shallower retrieval on the evaluated English Financial subset, while efficiency concentrates most cost in offline preprocessing.
- Benchmark setup: Loong evaluates heterogeneous multi-document reasoning, whereas Oolong emphasizes large-scale information aggregation over long inputs.
- Benchmark setup: Loong uses average accuracy, while Oolong uses an LLM-based judge for non-numeric questions and a deviation-based metric for numeric aggregation.
- Passage granularity: 100.00 accuracy from row-level financial passages outperforms 91.50 from table-level passages by 8.50 points.The study selects row-level passages for Financial documents.
- Property, relation, and aspect granularity: Coarse labels improve English Financial from 92.00 to 93.50 and Paper from 88.00 to 91.50, but Legal decreases from 71.25 to 68.75.Coarse labels are selected for English Financial and Paper, while fine labels are retained for Legal.
- Retrieval depth: 94.00 accuracy from Top-5 retrieval exceeds Top-15 at 92.50 and Top-12 at 92.00 on English Financial questions.Additional retrieved evidence provides no improvement in this subset, although main results use Top-12 for a common configuration.
- Efficiency: Offline Passage Construction and Information Extraction account for 157,031 input tokens, 468.51 seconds, and 32.35 LLM calls per query after amortization.Online Retrieval uses 656 tokens, 19.01 seconds, and one call; Generation uses 5,460 tokens, 6.73 seconds, and one call.
Conclusion
EnSI-RAG uses an entity-structure index to connect future questions with original supporting passages, leaving the LLM to integrate evidence and synthesize answers.
- EnSI-RAG replaces arbitrary fixed-size chunks with semantically contained passages and a query-independent entity-structure index.The index maps entity, entity type, and property, relation, or aspect values to sets of original passage identifiers.
- The index provides structured access paths from questions to supporting passages while preserving the LLM’s role as final semantic integrator.
- Across financial, legal, academic, and aggregation-heavy QA settings, EnSI-RAG is designed as a middle ground between chunk-based RAG and database-style reasoning.
- The results suggest that separating evidence localization from answer synthesis benefits long-document question answering.The index guides retrieval toward relevant passages, and the LLM integrates those passages into the final answer.
Appendix: Experimental Details
The appendix documents the benchmark descriptions, evaluation protocols, and baseline methods used in the experiments.
- Table A1 describes the two benchmarks and reports evaluated subset sizes and task types in the main paper.
- Table A2 summarizes the evaluation protocols, which follow each benchmark’s official protocol.
- Table A3 describes baselines spanning conventional RAG, long-context inference, document transformation, agent-based reasoning, recursive reasoning, and structured retrieval.
Case Studies
The case studies show how preprocessing preserves structure needed for legal matching and citation-chain construction, including provenance and directed relations.
- Case Studies: The case studies cover legal judgment-result matching and paper citation-chain construction.The legal task and citation task are presented as representative cases where preserved preprocessing structure affects complete-answer recovery.
- Citation-chain construction: Each paper receives a canonical title and aliases derived from arXiv identifiers and filename variants.
- Citation-chain construction: Reference passages generate explicit forward cites and inverse cited-by records, with each relation retaining provenance to its original passage.
- Citation-chain construction: EnSI-RAG recovers the four consecutive citation edges required by the gold chain and selects the longest linear path after ignoring the requested non-linear branch.
- Citation-chain construction: The response reproduces the complete five-paper ground-truth chain in canonical-title form.
- Citation-chain construction: Treating cites and cited-by as directed first-class relations preserves complete edges for exact graph traversal and canonical-title output.The comparison states that a missing citation edge makes the full chain unrecoverable in SLIDERS.
Additional Reproducibility Details
The appendix fixes the Oolong evaluation slice and records execution settings, while noting that each reported result comes from one complete run.
- The Oolong slice filters the test split by the 262,144-token document-length condition and selects the first 192 eligible questions.The procedure verifies split membership, document length, manifest size, reconstructed question strings, and strict slice consistency.
- Oolong uses a deterministic dataset slice and therefore does not require a sampling seed.
- Loong uses Top-12 retrieval, while Oolong-Synth selects records deterministically by context-window identifier without instantiating an embedding model.GPT-4.1-mini handles LLM-based preprocessing; GPT-4.1 handles retrieval planning, final generation, and Oolong evaluation, with temperature 0.
- Each reported EnSI-RAG result comes from one complete run, so across-run standard deviations and statistical significance tests are not reported.
Prompts
The appendix reports the complete prompts used in the Loong and Oolong experiments, including runtime substitutions and non-LLM deterministic processing.
- Prompts: The appendix reproduces the complete prompts used in the Loong and Oolong experiments.Braced expressions denote runtime substitutions in the corresponding templates.
- Prompts: Deterministic parsing, index construction, filtering, and score aggregation do not invoke an LLM and therefore have no prompts to report.
Loong Prompts
The Loong prompt identifies bilingual company-name variants so financial records referring to the same company can share a canonical retrieval handle.
- Loong Prompts: The prompt extracts English or Chinese company-name variants that refer to the reporting company.
- Loong Prompts: These variants allow financial records for the same company to share a canonical retrieval handle.
Company-Name Alias Extraction
The company-name alias extraction prompt identifies current and explicitly documented former registrant names while excluding unrelated entities and returning a constrained JSON output.
- Company-Name Alias Extraction: The system prompt directs a bilingual extractor to identify English or Chinese names referring to the reporting company.
- Company-Name Alias Extraction: The user prompt requires extracting company or registrant names from a filing excerpt.
- Company-Name Alias Extraction: The rules include the exact current registrant name and former names only when the excerpt explicitly documents a prior name.
- Company-Name Alias Extraction: The rules exclude subsidiaries, auditors, creditors, customers, officers, locations, and stock tickers unless they are also registrant company names.
- Company-Name Alias Extraction: The output is restricted to a JSON schema containing a company_names array of clean display names.
- Company-Name Alias Extraction: A filing excerpt is supplied through the context placeholder.
Financial-Table Location
The paper describes prompt-driven indexing and answer generation across financial, academic, legal, Loong, and Oolong data, with structured retrieval records linked to original passages.
- Financial-Table Location: Financial-table location identifies financial-statement tables and returns their global line ranges plus metadata such as titles, years, and metric labels.The prompts exclude tables of contents and prohibit outputting row values, dollar values, percentages, share counts, or final answers.
- Financial-Table Location: Reference-section location finds the bibliography at the paper's end and returns exact start and end quotations, or false when absent.
- Retrieval Planning: Retrieval planning converts questions into structured items containing entities, types, association categories, aliases, values, operations, and answer requirements.The shared schema uses property, relation, and aspect records that point to passages; dataset conventions differ for paper, financial, and legal corpora.
- Retrieval Planning: The generic retrieval planner uses a shared entity-structure schema while applying dataset-specific conventions for paper, financial, and legal documents.
- Final-Answer Generation: The final-answer generator answers only from retrieved original passages and index records, preserves the requested format, and applies dataset-specific output rules.
- Final-Answer Generation: Oolong processing classifies every atomic instance into an allowed label and produces records retaining provenance and context-window identifiers.