PaperlayerView exact revision on arXiv

Source-linked AI summary

RIT-RAG: Navigating Document Corpora with Retrieval-Induced Trees

Meghanadh Pulivarthi, Swaraj Kumar Biswal, Kushagra Bhushan, Yatin Nandwani, Sachindra Joshi, Dinesh Raghu

arXiv:2610.11370v1cs.AIcs.IR

TL;DR

Agentic RAG searches iteratively but lacks document structure, while structure-aware methods may commit to a single document before seeing content. RIT-RAG induces navigable sub-trees from retrieved chunks; it achieves the highest accuracy across financial, scientific, and customer-support benchmarks, with gains on EntQABench.

  • Problem

    Agentic RAG reasons over isolated chunks without document structure, while structure-aware methods must select a document because the full corpus hierarchy may not fit in context.

    Source: §1 · p. 1

  • Method

    RIT-RAG uses retrieved chunks to induce query-specific sub-trees from document hierarchies, which an LLM agent navigates to select promising content.

    Source: §1 · p. 2

  • Results

    RIT-RAG achieves the highest accuracy against vanilla, graph-based, and agentic baselines on financial, scientific, and customer-support benchmarks, and improves EntQABench accuracy by 6.8–11.4 points over the strongest baseline.

    Source: §6 · p. 8

  • Takeaways & Limitations

    RIT-RAG balances the precision and recall of the text it reads while retaining high accuracy on benchmarks including EntQABench’s 2.84 million webpages.

    Source: §6 · p. 8

    RIT-RAG depends on a usable document hierarchy; weak headings or poor parsing can limit the quality of the map shown to the agent.

    Source: §7 · p. 9

Abstract

from arXiv · show

Retrieval-augmented generation (RAG) grounds language models in external corpora. Agentic RAG enables iterative search, yet exposes the model to isolated chunks without document structure, making it difficult to distinguish relevant evidence from chunks that merely resemble the query. Structure-aware methods such as PageIndex navigate document structure but cannot scale to the structures of large corpora, which do not fit in the LLM context. Hence, they first commit to a single document using a document retriever and cannot recover from a wrong choice. We propose RIT-RAG (Retrieval-Induced Tree RAG), which combines content retrieval with structural navigation. Offline, RIT-RAG builds a tree for each document from its table of contents or sitemap. At query time, it retrieves a broad set of chunks and uses their positions to induce manageable sub-trees, potentially across multiple documents. An LLM agent navigates these sub-trees, selectively reads promising nodes, and reformulates queries when needed. Thus, retrieval proposes where to look, while the agent decides what to read. Across financial, scientific, and customer-support benchmarks, RIT-RAG achieves the highest answer accuracy among vanilla, graph-based, and agentic baselines. On EntQABench, our new benchmark of 2.84 million technical-documentation webpages, it improves accuracy by 6.8 to 11.4 points over the strongest baseline across three LLMs.

1 Introduction

Chunk-based agentic RAG lacks document structure, while structure-first systems may commit to one document before examining its content. RIT-RAG uses retrieved evidence to induce navigable sub-trees, and the paper reports accuracy gains across public and enterprise benchmarks.

Motivation

  • Agentic RAG reasons over retrieved chunks without explicit document organization, making it difficult to distinguish useful evidence from merely similar content.The paper links this to context overload from unfocused retrieval.Source: §1 · p. 1
  • Structure-aware methods such as PageIndex first select a document because the whole corpus structure cannot fit in the LLM context, risking an irrelevant navigation path with little recovery.Before seeing document content, the system has limited ability to compare competing documents.Source: §1 · pp. 1–2

Results

  • RIT-RAG achieves the highest accuracy on WixQA, FinanceBench, and QASPER, and improves EntQABench accuracy by 6.8–11.4 points over the strongest baseline.The EntQABench corpus contains 2.84 million enterprise product-documentation webpages.Source: §1 · p. 2

Approach

  • RIT-RAG combines content-based retrieval with structure-aware navigation by using retrieved chunks to induce query-specific sub-trees, potentially across multiple documents.The LLM selects promising regions and can reformulate the query when no region offers a promising path.Source: §1 · p. 2

Contributions

  • The paper introduces EntQABench, a benchmark over 2.84 million product-documentation webpages, to evaluate RAG systems at realistic corpus scale.It also evaluates across multiple models and datasets, reporting consistent accuracy improvements on three public benchmarks and EntQABench.Source: §1 · p. 2

2 Related Work

Prior RAG work either retrieves chunks without exposing their document organization or navigates structure after selecting a document. RIT-RAG retrieves across the corpus and lets the agent navigate sub-trees around retrieved chunks.

Agentic RAG

  • Agentic RAG iteratively retrieves content, but the model interacts with the corpus through chunks and does not see how they are organized.The paper describes iterative retrieval-generation, adaptive retrieval, and hybrid retrieval as examples of agentic approaches.Source: §2 · p. 3

Structured Retrieval

  • Structured retrieval includes tree methods that cluster, summarize, or extract document structure and graph methods that organize entities and related information.The paper also describes inference-time methods that transform retrieved content into task-suited data structures.Source: §2 · p. 3

Structure-Aware Agents

  • Structure-aware agents such as PageIndex navigate a document hierarchy and selectively read sections, but over large corpora they must select and commit to one document first.PDFTriage similarly provides tools to fetch sections, pages, and tables within a document.Source: §2 · p. 3

RIT-RAG

  • RIT-RAG retrieves across the corpus and presents sub-trees around retrieved chunks, possibly from several documents, so the agent decides what to read.This separates evidence discovery from content reading.Source: §2 · p. 3

3 Preliminaries

The paper represents a corpus as a forest of document trees, with nodes corresponding to semantically organized units. It contrasts chunk-level agentic search with structure-first navigation and outlines limits of navigating large document hierarchies.

Corpus Representation

  • A corpus is represented as a forest of per-document trees whose nodes are chapters, sections, or subsections linked by parent-child relationships.Each node owns the text under it that is not under any of its children; each piece of document text belongs to exactly one node.Source: §3 · p. 3
  • Node titles are mandatory metadata, while summaries and keywords are optional because extracting them may be infeasible for corpora of millions of documents.A node's text may be empty when all its content lies under its children.Source: §3 · pp. 3–4
  • Document hierarchies may come from tables of contents and headings, or be inferred for webpage collections from sitemaps or other metadata.The serialized tree exposes structure without the full node contents; its node identifiers let the LLM request content.Source: §3 · p. 4

Retrieval Approaches

  • Agentic Search retrieves top-ranked chunks iteratively but discards document hierarchy, leaving the agent to process retrieved chunks without structural cues or control over which it reads.PageIndex instead selects a document, presents its tree, and lets the LLM selectively fetch node contents.Source: §3 · p. 4
  • PageIndex must select a document before navigation, and oversized tree representations may need truncation or partitioning, weakening the structural information available to the LLM.The paper attributes these constraints to corpus structures exceeding the LLM context window.Source: §3 · p. 4

4 RIT-RAG: Retrieval-Induced Tree RAG

RIT-RAG attaches lightweight structural metadata to indexed chunks, then uses retrieval results to build structural context for an agent to navigate through selective node reads.

4 RIT-RAG: Retrieval-Induced Tree RAG

  • RIT-RAG associates each indexed chunk with its node, title, and ancestor identifiers and titles, allowing document hierarchy to be reconstructed at inference time.The index also retains chunk content.Source: §4 · p. 4
  • Rather than showing retrieved chunk contents directly, RIT-RAG reconstructs their corresponding subtrees and lets the LLM select nodes to read.The structures are induced by content retrieval, so the system need not inspect every document in the corpus.Source: §4 · pp. 4–5
  • The LLM need not receive the full corpus structure or commit to one document before seeing content-level evidence; presented structure is bounded by selected nodes and their ancestors.Initial retrieval can surface relevant structures from multiple documents simultaneously.Source: §4 · p. 5
  • The query-dependent sub-forest includes the top-scoring nodes and their ancestors, placing retrieved evidence in document context and grouping nodes that share an ancestor.Subtrees are ordered by their highest selected-node score.Source: §4 · p. 5
  • In each round, the agent examines the sub-forest, reads selected nodes, then answers, reads more nodes, or reformulates its query.A reformulated query searches the entire corpus, allowing different documents to surface rather than committing to an earlier sub-forest.Source: §4 · p. 5

5 Experiments

Experiments evaluate RIT-RAG on existing and induced document structures, at million-page scale, and analyze evidence reading and costs. RIT-RAG achieves the highest accuracy across the public datasets and improves over the strongest scalable baseline on EntQABench.

5.1 Experimental Setup

  • Experiments cover existing structures in WixQA and QASPER, induced FinanceBench ToCs, and EntQABench’s 2.84-million-page corpus; methods share a hybrid retriever and up to 10 tool calls.Accuracy is judged against gold answers, alongside passage precision and recall, preprocessing cost, API cost, and latency.Source: §5 · p. 6, §5.1 · p. 6

5.2 Results

  • RIT-RAG achieves the highest accuracy across WixQA, QASPER, and FinanceBench with all three LLMs, and improves over same-retriever Agentic Search by 2.4–10.5 points.On WixQA and QASPER, both methods use the same retriever, LLM, and budget of 10 tool calls; they differ in whether the agent sees retrieved chunks or an induced sub-forest.Source: §5.2 · p. 7, Table 1 · p. 7
  • On EntDocs, RIT-RAG improves over the best baseline by 6.6, 7.9, and 11.4 points; EntDocs is a single large tree whose string representation does not fit any LLM context.Source: Table 3 · p. 7, §5.2 · p. 8
  • With Haiku inducing FinanceBench ToCs instead of Qwen3-32B, RIT-RAG accuracy changes by at most 1.4 points, versus up to 4.7 points for baselines.The induced ToCs also determine chunk boundaries, making baselines sensitive to structure changes.Source: §5.2 · p. 7, Table 2 · p. 7

5.3 Retrieval Analysis

  • RIT-RAG achieves the highest passage-reading F1 on all three analyzed datasets, with the highest precision and recall on WixQA and FinanceBench.On QASPER, it ranks second on both precision and recall; PageIndex recall drops to 50.4% when it selects the wrong document.Source: §5.3 · p. 8, Table 4 · p. 8

5.4 Cost Analysis

  • Preprocessing costs $15.5 on WixQA, $12.9 on QASPER, $590 on FinanceBench, and $0 on EntDocs; graph-method estimates reach $31K–$38K on EntDocs.Graph-method preprocessing costs grow linearly with corpus size; the paper excludes those methods from EntDocs evaluation.Source: §5.4 · p. 8

6 Conclusion

RIT-RAG combines content-based retrieval with structure-aware navigation, using retrieved chunks to induce query-specific sub-trees for an agent to navigate. It achieves the highest accuracy across financial, scientific, and customer-support benchmarks, while costing more per question than Agentic Search.

6 Conclusion

  • RIT-RAG uses retrieved chunks to induce query-specific sub-trees from corpus structure, letting the agent decide what to read rather than exposing chunks or committing to one document first.The method combines content-based retrieval with structure-aware navigation.Source: §6 · p. 8
  • On EntQABench’s 2.84 million webpages, RIT-RAG improves accuracy by 6.8–11.4 points over the strongest baseline, using title-only nodes without LLM preprocessing.The agent reads the induced sub-tree strings after every search, making RIT-RAG more costly per question than Agentic Search.Source: §6 · p. 8
  • RIT-RAG achieves the highest accuracy against vanilla, graph-based, and agentic baselines across financial, scientific, and customer-support benchmarks.The conclusion also characterizes the method as balancing the precision and recall of the text it reads.Source: §6 · p. 8
  • The authors propose compacting the sub-tree string representation to reduce query cost and studying richer node metadata at million-page scale.These are proposed future-work directions.Source: §6 · p. 9

7 Limitations

RIT-RAG has limitations in context management, representation of document relationships, and dependence on usable document hierarchies.

7 Limitations

  • RIT-RAG accumulates fetched nodes until reaching its context budget and does not support evidence compaction or chat-history eviction.Source: §7 · p. 9
  • RIT-RAG preserves hierarchical structure but ignores cross-links between nodes at the same tree level.Source: §7 · p. 9
  • RIT-RAG depends on a usable document hierarchy; weak headings or poor parsing can limit the quality of the map shown to the agent.Structure induction can recover a hierarchy for documents such as SEC filings.Source: §7 · p. 9

Reproducibility Statement

The reproducibility statement points to the paper and appendices for the experimental protocol, corpus construction, implementation, settings, prompts, and metrics. The appendix describes corpus-specific approaches to constructing navigable structures and the retrieval implementations.

Reproducibility Statement

  • The paper and appendices provide the algorithm, protocol, corpus processing, tree construction, EntQABench details, implementation choices, model settings, prompts, cost assumptions, and metric definitions.The statement directs readers to Sections 4 and 5.1 and Appendices A, B.1–B.2, C, D, E, F, G, K, L, and N.Source: Reproducibility Statement · p. 9

A Corpus-Specific Structure Extraction

  • WixQA trees use URL-based family-to-product-to-article hierarchy and HTML headings; FinanceBench uses all 364 filings and finer induced section hierarchies; QASPER uses markup headings; EntDocs preserves product sitemap trees.QASPER has no inter-document hierarchy, while EntDocs webpages are single nodes with no intra-document structure to induce.Source: A Corpus-Specific Structure Extraction · p. 11

B.1 RIT-RAG

  • RIT-RAG chunks each corpus node into fixed 512-token units and uses BM25 plus dense retrieval, fused by reciprocal rank fusion with κ = 60.Source: §B.1 · p. 11

B.2 PageIndex

  • The corpus-level PageIndex adaptation retrieves chunks by vector search, scores filings by summing their retrieved-chunk scores, then selects the top filing for ToC-based navigation.Source: §B.2 · p. 11

C Dataset and Structure Statistics

The dataset-statistics section identifies what Tables 5–7 report: corpus and evaluation-set sizes, hierarchical structure, and model and inference configuration.

C Dataset and Structure Statistics

D Model and Inference Configuration

  • Table 7 lists generation and offline-preprocessing models, decoding settings, and per-token prices used for cost accounting.Its settings include temperature and maximum tokens, and prices are effective API costs per one million tokens.Source: Table 7 · p. 12, D Model and Inference Configuration · p. 12

E Cost and Latency Analysis

RIT-RAG’s query-time cost and latency are compared with other methods, while answer-quality metrics are evaluated across generation models using a shared judge.

E Cost and Latency Analysis

  • The one-time preprocessing costs are reported at the API prices listed in Table 7.The supplied passage identifies Table 9 as the preprocessing-cost comparison.Source: E Cost and Latency Analysis · p. 12
  • 15.5–42.7 accuracy points over vanilla RAG accompany RIT-RAG’s 0.2–0.3-cent per-question cost.RIT-RAG takes 15–19 seconds per question, compared with 23–55 seconds for Agentic Search and 42–79 seconds for Interact-RAG; PageIndex takes 11–15 seconds and costs 1.2 cents on QASPER.Source: E Cost and Latency Analysis · p. 12

F Answer-Quality Metrics

  • Answer quality covers lexical token recall and seven normalized 0–1 G-E dimensions, including correctness, relevance, factuality, and comprehensiveness.The seven dimensions also include knowledgeability, logical coherence, and diversity.Source: F Answer-Quality Metrics · p. 12
  • All methods and generation models are scored by the same Gemini 2.5 Flash judge for comparability.The evaluation uses Haiku, GPT-5-mini, and Luna, with one preprocessing model per corpus.Source: F Answer-Quality Metrics · p. 12

G Retrieval Metrics

The retrieval analysis compares how well methods read gold evidence, with recall and precision showing different strengths across methods and datasets.

G Retrieval Metrics

  • The F1 score is computed as 2PR/(P + R), where passage precision and recall are measured against gold evidence.Figure 3 plots answer accuracy against mean API cost, and Figure 4 plots accuracy against mean LLM latency; both show dataset-specific panels and mark RIT-RAG with a star.Source: G Retrieval Metrics · p. 12, Figure 3 · p. 13, Figure 4 · p. 13
  • On QASPER, Agentic Search has the highest recall (85.5%) but only 11.5–17.3% precision, while PageIndex has the highest precision (54.9%) but 50.4% recall.PageIndex recall is 56.8% on FinanceBench; RIT-RAG is second on both QASPER measures, at 80.6% recall and 47.9% precision.Source: G Retrieval Metrics · p. 12
  • With GPT-5-mini, RIT-RAG has the highest recall on all three datasets, although PageIndex has slightly higher F1 on QASPER and FinanceBench.PageIndex’s F1 is 50.0% versus RIT-RAG’s 47.0% on QASPER, and 43.3% versus 42.9% on FinanceBench.Source: G Retrieval Metrics · pp. 12–13
  • Table 8 reports per-query accuracy, mean API cost, and mean LLM latency; Table 9 reports one-time preprocessing cost.Table 9 notes that other baselines need no preprocessing and marks one estimate as scaled linearly from WixQA.Source: Table 8 · p. 13, Table 9 · p. 13

H Complementarity Analysis

The complementarity analysis finds that RIT-RAG answers many questions uniquely answered by either PageIndex or Agentic RAG, as well as some missed by both.

H Complementarity Analysis

  • PageIndex’s structural navigation and Agentic RAG’s broad content search provide complementary strengths, whose correct-answer overlap with RIT-RAG is examined.Figure 5 depicts overlap among correct answers from the three methods.Source: H Complementarity Analysis · p. 13
  • RIT-RAG answers 22 of 27 WixQA, 11 of 15 FinanceBench, and 57 of 67 QASPER questions that PageIndex answers but Agentic RAG does not.For questions answered by Agentic RAG but not PageIndex, RIT-RAG answers 9 of 10, 14 of 18, and 168 of 192, respectively.Source: H Complementarity Analysis · p. 14
  • RIT-RAG also answers 2 WixQA, 11 FinanceBench, and 20 QASPER questions that both baselines miss.These counts show that its correct answers are not limited to the questions either baseline answers correctly.Source: H Complementarity Analysis · p. 14

I Why graph baselines fail on QASPER and FinanceBench

This section examines graph-baseline failures on QASPER and FinanceBench and presents a rubric for evaluating the diversity and originality of reasoning.

I Why graph baselines fail on QASPER and FinanceBench

  • On QASPER, HyperGraphRAG’s corpus-wide graph merges generic scientific entities across papers, while on FinanceBench, fragmented tables and bare numeric entities prevent retrieval of gold evidence.For QASPER, hubs include DATASET linked to 1,728 passages, MODEL to 4,046, and BERT to 1,397; for FinanceBench, about a fifth of graph nodes are pure numbers, which are poor retrieval keys, leading to hallucinations.Source: I Why graph baselines fail on QASPER… · pp. 14–15

J RIT-RAG Behavior Statistics

  • RIT-RAG makes 2.70–2.99 tool calls per query across benchmarks, reading 3.7–5.5 nodes and batching about 2–3 nodes per Read call.GetToC calls stay near 1.3–1.5 and Read calls near 1.2–1.7 on every benchmark; the agent stops early when it finds the answer, so corpus size does not add iterations.Source: J RIT-RAG Behavior Statistics · pp. 15–16

K Sensitivity to the Number of Nodes

  • With Haiku on held-out WixQA and QASPER, accuracy peaks at n = 50 for WixQA (88.0%) and at n = 25 and 50 for QASPER (88.5%); experiments use n = 50.Accuracy rises with n up to 50 and then decreases slightly; n controls how many retrieved nodes induce the subtree, balancing missed relevant nodes against added context.Source: K Sensitivity to the Number of Nodes · p. 16

L EntQABench Creation

  • EntQABench contains 317 questions spanning 350 IBM Documentation webpages, with question topologies from single-webpage to multi-step evidence across 3–4 webpages.The source corpus comprises approximately 3M documents and 9M chunks; questions are generated from retrieved documents with source citations, and each includes a gold answer, supporting citations, and answer nuggets.Source: L EntQABench Creation · pp. 16–17

N.5 G-E Judge Prompts (Seven Dimensions)

  • The diversity rubric scores reasoning from 10 for exceptionally rich, original ideas to 0 for no diversity or originality.Scores of 8–9 indicate a few novel angles, 6–7 some variety but generally safe thinking, 4–5 mostly standard thinking with minimal diversity, and 1–3 predictable or monotonous reasoning.Source: §N.5 · p. 23

Related papers

Loading 2610.11370v1…