PaperlayerView exact revision on arXiv

Source-linked AI summary

A Tree-based RAG Framework for Evidence-Intensive QA via Adaptive Planning and Topology-Aware Evidence Gathering

Songeun Lee, Kyungjin Min, Injae Na, Suyeong Lee, Chiyoung Kim, Woohwan Jung

arXiv:2609.04981v1cs.AIcs.IR

TL;DR

Evidence-intensive QA requires synthesizing information distributed across many documents, exposing rigidity and topology-ignorant evidence gathering in existing structured RAG. APT-RAG adaptively expands reasoning trees, gathers evidence according to their topology, and batches answer generation; it consistently outperforms structured RAG baselines on evidence-intensive QA benchmarks.

  • Problem

    Evidence-intensive QA requires synthesizing evidence distributed across dozens or hundreds of documents, exposing structural rigidity and topology-ignorant evidence gathering in existing structured RAG methods.

  • Method

    APT-RAG recursively constructs reasoning trees with adaptive planning, topology-aware evidence gathering, and evidence-guided batched answer generation.

  • Results

    APT-RAG consistently outperforms existing structured RAG methods on evidence-intensive QA benchmarks.

  • Takeaways & Limitations

    Recursive, topology-aware evidence gathering is a promising direction for answering questions over large, dispersed document spaces.

  • Takeaways & Limitations

    APT-RAG performance depends on planning quality, and inaccurate planning modules may create unnecessary branches or propagate errors.

Abstract

from arXiv · show

Recent structured RAG methods leverage tree- or graph-based reasoning structures to improve multi-hop QA. However, they face key limitations in evidence-intensive QA, where answering a question requires synthesizing information scattered across dozens or even hundreds of documents: structural rigidity, which limits adaptive reasoning expansion, and topology-ignorant evidence gathering, which prevents effective integration of evidence across different reasoning nodes. To address these issues, we propose APT-RAG, an Adaptive Planning and Topology-aware evidence gathering RAG framework. Adaptive planning dynamically expands the reasoning structure based on question dependencies and evidence requirements, while topology-aware evidence gathering improves evidence coverage through sibling evidence reuse, direct retrieval, and evidence aggregation from child nodes. We further introduce evidence-guided batched answer generation to reduce significant generation overhead in evidence-intensive QA. In the experiments on evidence-intensive QA benchmarks, APT-RAG outperforms existing structured RAG methods. Our code is available at https://github.com/hyudsl/APT-RAG.

1 Introduction

Evidence-intensive QA exposes structural rigidity and topology-ignorant evidence gathering in existing structured RAG methods. APT-RAG addresses these limitations through adaptive reasoning-tree expansion, topology-aware evidence gathering, and batched answer generation.

  • Motivation: Evidence-intensive QA requires synthesizing evidence from dozens or hundreds of documents, far beyond conventional multi-hop QA.HotpotQA averages 2.0 supporting pages, whereas MoNaCo averages 43.3 pages.
  • Limitations: Fixed upfront reasoning plans create structural rigidity, forcing individual nodes to process excessive evidence as questions become complex.Plan*RAG fails on an example requiring synthesis of 19 documents about 13 wars.
  • Limitations: Topology-ignorant methods gather evidence only at the root and fail to reuse answers from sub-question nodes.ToQ can fail on a subquestion even when its answer is directly inferable from another sub-question’s answer.
  • APT-RAG: APT-RAG recursively constructs a reasoning tree and adaptively chooses whether to decompose, answer directly, or reuse previously gathered evidence.This depth-first planning adjusts reasoning depth and breadth to sub-problem complexity.
  • APT-RAG: APT-RAG gathers evidence across tree topology through sibling reuse and child-to-parent aggregation, while batched generation reduces redundant inference overhead.The framework couples lateral and vertical evidence gathering with evidence-guided answer batching.
  • Results: APT-RAG consistently outperforms structured RAG baselines on evidence-intensive QA benchmarks.The contribution statement reports this result without specifying a numerical score.

2 Related Work

Evidence-intensive QA extends conventional multi-hop QA by requiring substantially more evidence and reasoning across widely distributed documents. Structured RAG methods address multi-hop complexity with tree- or graph-based decomposition and reasoning structures.

  • Evidence-intensive QA: Evidence-intensive QA requires substantially more evidence and reasoning steps than conventional multi-hop QA benchmarks.Benchmarks such as MoNaCo, QAMPARI, and DeepSearchQA require evidence from dozens to hundreds of documents.
  • Evidence-intensive QA: Evidence-intensive QA benchmarks require collecting and synthesizing distributed evidence that is difficult to retrieve in a single step from the initial question.The evidence may span dozens to hundreds of documents.
  • Structured RAG for Multi-Hop QA: Structured RAG decomposes complex questions into sub-questions and organizes reasoning with tree- or graph-based structures.Tree-based methods build reasoning trees, while graph-based approaches use directed acyclic graphs to control retrieval and reasoning flows.

3 Method

APT-RAG recursively builds and resolves a reasoning tree through adaptive planning and topology-aware evidence gathering. It combines sibling reuse, direct retrieval, child-answer aggregation, and batched generation to support broad evidence synthesis efficiently.

  • Overview: APT-RAG interleaves depth-first planning, evidence gathering, and answer generation instead of constructing the complete reasoning tree upfront.Each node represents a sub-question needed to resolve its parent, and traversal repeatedly applies this process across the tree.
  • Adaptive Planning: The planner chooses sibling reuse, direct retrieval, or decomposition according to question dependencies and evidence requirements.An answerability checker first tests whether preceding sibling QA pairs suffice; otherwise, a decomposer keeps the question atomic or splits it into sub-questions.
  • Topology-aware Evidence Gathering: Topology-aware gathering integrates evidence laterally from sibling QA pairs, externally from retrieved documents, or vertically from aggregated child answers.Vertical gathering propagates child answers through the tree, while lateral gathering reuses information already produced at the same depth.
  • Topology-aware Evidence Gathering: Generated node answers become evidence for sibling and parent nodes, allowing recursive accumulation and integration of information toward the final answer.The framework gathers evidence from a node’s own subtree and siblings’ subtrees rather than from ancestors’ regions.
  • Topology-aware Evidence Gathering: In the reasoning example, child answers are aggregated upward while a later sibling reuses an earlier answer to rank candidate species.The example combines external gathering for fine-grained population statistics, vertical aggregation, and lateral reuse.
  • Evidence-Guided Batched Answer Generation: Batched answer generation jointly answers sibling sub-questions with highly overlapping evidence, reducing LLM calls, input tokens, and latency.A motivating case decomposes one parent question into 35 entity-specific sub-questions, with 20 answerable from a single shared evidence source.

4 Experiments

APT-RAG is evaluated on evidence-intensive benchmarks using answer quality, retrieval coverage, and inference-cost measures. It achieves stronger performance across evidence scales while adaptive planning and topology-aware components improve the quality–efficiency trade-off.

  • Experimental Setup: APT-RAG is evaluated on MoNaCo and QAMPARI, which require synthesis across an average of 43.3 and 13.0 evidence pages, respectively.Baselines include basic and structured RAG methods, with answer precision, recall, F1, retrieval recall, and inference cost measured.
  • Main Results: APT-RAG achieves the best overall performance across benchmarks and model scales, with all answer F1 gains over baselines statistically significant at p ≤0.001.The reported comparisons use the same retrieval setup and backbone models across methods.
  • Main Results: On MoNaCo with the 30B setting, APT-RAG exceeds Plan*RAG by 8% in answer F1 and reaches retrieval recall 50.79 versus 39.28.APT-RAG also remains relatively stable under the 4B setting, while a similar trend appears on QAMPARI.
  • Main Results: As gold evidence requirements grow, APT-RAG maintains the highest answer performance, with 21–30-page F1 comparable to baselines’ 11–20-page range and stable recall at 31+ pages.All methods degrade as evidence requirements increase, but APT-RAG shows a smaller drop.
  • Adaptive Retrieval Scaling: APT-RAG increases retrieved documents as gold evidence grows, unlike baselines that retrieve nearly constant amounts under changing evidence requirements.The reported analysis links this behavior to adaptive expansion of the reasoning tree.
  • Performance-oriented Modules: Fixing tree breadth to 2 lowers latency but reduces answer F1 and retrieval recall, whereas breadth 6 increases latency and APT-RAG achieves higher F1 with less than half that latency.Removing vertical gathering decreases answer F1 by 4.20 points.
  • Efficiency-oriented Modules: Lateral gathering reduces average inference latency by 41.4%, while evidence-guided clustering reduces it by 8.2%.Clustering also produces larger reductions in the 75th and 95th latency percentiles as gold-document counts increase, including queries exceeding 100 seconds.

5 Conclusion

APT-RAG addresses evidence-intensive QA with adaptive planning and topology-aware evidence gathering. Experiments show consistent improvements in answer quality and retrieval recall, supporting recursive, topology-aware gathering for dispersed document spaces.

  • Conclusion: APT-RAG is an adaptive planning and topology-aware evidence gathering framework for evidence-intensive QA.It dynamically adjusts the search space and uses reasoning topology during evidence gathering.
  • Conclusion: APT-RAG consistently improves answer quality and retrieval recall on evidence-intensive QA benchmarks.The conclusion reports these improvements as the experimental outcome of the framework.
  • Conclusion: Recursive, topology-aware evidence gathering is presented as a promising direction for questions over large, dispersed document spaces.This conclusion follows the reported benchmark improvements and the framework’s design focus.

6 Limitations

APT-RAG’s inference cost still grows with evidence volume and reasoning branches, while performance depends on the accuracy of its planning modules.

  • In the 31+ range, larger merged evidence raises long-context processing cost enough to outweigh latency savings from clustering.
  • Inference cost increases as evidence volume and reasoning branches grow, even with evidence-aware batched answer generation.Tree pruning is suggested as a way to control unnecessary branches and evidence expansion.
  • APT-RAG performance depends on the quality of the planning process that constructs its reasoning tree.Inaccurate query decomposition or answerability checking may create unnecessary branches or propagate errors.

A Datasets

The appendix describes benchmark examples, corpus and indexing choices, baseline implementations, and APT-RAG’s recursive and batched inference procedures. Its clustering method groups evidence-overlapping subquestions under a context-window limit.

  • Datasets: The examples include questions requiring lists of Roman emperors’ deaths and reign lengths, and famous drag queens’ stage names.
  • Datasets: The retrieval corpus uses English Wikipedia, with benchmark-specific corpus construction and structure-preserving rule-based chunking.MoNaCo adds sampled non-gold pages from the January 2026 English Wikipedia dump.
  • Inference: APT-RAG resolves nodes through contextualization, adaptive planning, topology-aware evidence gathering, and answer generation during recursive depth-first traversal.The planner selects lateral, external, or vertical processing, with vertical processing recursively expanding the tree.
  • Batched Answer Generation: Batched answer generation plans all children first, batches consecutive external-gathering children, clusters their evidence, and jointly answers them.The method preserves sibling-order dependencies for recursive children while jointly processing independent external children.
  • Clustering: The clustering objective minimizes the number of subquestion clusters while requiring evidence compatibility and compliance with the context-window limit.The context limit is expressed through merged-evidence token length, and compatibility is represented in a graph.

E.4 Approximation Analysis

The constrained clustering analysis establishes bounds for the greedy approximation and evaluates how the context-window constraint affects cluster counts. The surrounding appendix material also documents implementation prompts and cost reporting.

  • Approximation Analysis: The constrained clustering optimum is bounded below by the relaxed optimum and above by the constrained greedy solution.The relaxed problem has a larger feasible set, while the greedy constrained solution is an upper bound by definition.
  • Approximation Analysis: The unconstrained greedy procedure produces no more clusters than the constrained greedy procedure because it does not enforce the context-window constraint.
  • Approximation Analysis: The context-window constraint keeps clustering behavior close to the relaxed solution, with a gap below one cluster even at the 21+ point.
  • Implementation Details: APT-RAG prompt templates cover contextualization, adaptive planning, evidence gathering, and answer generation across the recursive traversal.The templates specify component roles, input fields, and integration rules for different gathering modes.
  • Implementation Details: Batched answer generation clusters external subquestions by evidence overlap and context budget, then redistributes JSON outputs to their corresponding nodes.Each question is linked to allowed documents to prevent cross-question evidence confusion.

G Subset Evaluation

On a 300-example subset, APT-RAG is compared with RT-RAG for inference cost, answer quality, retrieval coverage, and structural adaptation. APT-RAG generally offers lower cost and adapts tree structure to evidence requirements.

  • Inference Cost: On MoNaCo with the 30B backbone, RT-RAG uses 115.6 LLM calls and 1.13M input tokens per question, about 3.6× and 8.4× APT-RAG’s costs.
  • Answer Quality: APT-RAG leads in Answer F1 across most settings, while RT-RAG’s retrieval recall is much lower on MoNaCo despite a slight 4B F1 advantage.On the 30B MoNaCo setting, the retrieval-recall gap widens to nearly 3×.
  • Answer Quality: Scaling APT-RAG from a 4B to 30B backbone improves MoNaCo F1 by +13.0, compared with +7.0 for RT-RAG.
  • Structural Adaptation: APT-RAG depth and breadth increase with required evidence volume, whereas baseline structures remain nearly flat.The baselines use fixed or bounded structures, early stopping, or limited expansion before or during retrieval.

H Full Evaluation

APT-RAG’s inference cost scales with evidence requirements rather than a fixed budget, producing statistically significant improvements while incurring higher cost on the most evidence-intensive setting.

  • Inference Cost: 33.61 LLM calls and 104.59s per question make APT-RAG the most expensive method on MoNaCo with the 30B backbone.Each MoNaCo question requires 43.3 gold pages on average, whereas structured RAG baselines issue 5.52–8.20 calls and take 17.77–81.27s.
  • Inference Cost: 2.9× fewer LLM calls and 4.3× lower latency on QAMPARI show that APT-RAG’s cost decreases when fewer evidence pages are required.QAMPARI questions require 13.0 pages on average; baseline call reductions reach at most 1.5×.
  • Inference Cost: 3.60s versus 9.67s and 7.19s makes APT-RAG faster than LogicRAG and ToQ on QAMPARI with the 4B backbone.The comparison indicates that APT-RAG can be faster than structured baselines when the backbone and evidence requirements are smaller.
  • Statistical Significance: Table 8 reports Answer F1 differences between APT-RAG and each baseline across the full benchmarks.The table defines ∆Answer F1 as APT-RAG − baseline and includes 95% confidence intervals.
  • Statistical Significance: All confidence intervals exclude zero and every comparison has p ≤0.001, with significance retained after Bonferroni correction.The paired permutation tests use average per-question F1 differences and percentile-bootstrap 95% confidence intervals with 10,000 resamples.

J Additional Ablation Results

Additional analyses examine fine-grained retrieval, topology-aware gathering, and evaluator robustness. They show that lateral gathering is infrequent but can reduce latency, while APT-RAG’s leading performance is stable across evaluators.

  • Adaptive Expansion: MoNaCo’s fine-grained criterion evaluates retrieval at the document section and content-type level, including sentences, lists, and tables.The analysis uses gold evidence annotated beyond document title or URL because the corpus is constructed from these fine-grained evidence units.
  • Topology-Aware Gathering: Table 10 reports the proportion of nodes resolved by each evidence-gathering strategy across datasets and model scales.The table provides the usage statistics underlying the topology-aware gathering analysis.
  • Topology-Aware Gathering: 1.6–6.2% of generated nodes use lateral gathering, while most nodes rely on external or vertical gathering.The low lateral-gathering proportion means most sibling nodes remain independent and can be processed in parallel.
  • Topology-Aware Gathering: Lateral gathering reduces latency by avoiding redundant retrieval and tree expansion, as shown by the Figure 7 ablation.Its limited usage therefore has a measurable efficiency role despite accounting for a small fraction of generated nodes.
  • Evaluator Robustness: APT-RAG consistently achieves the highest F1 under both Gemini-3.5-Flash and GPT-5.4 evaluators across backbone scales.The comparable scores from the two evaluators indicate that the main performance conclusions are robust to evaluator choice.

K Case Study

The MoNaCo case study traces APT-RAG’s adaptive decomposition and evidence gathering on a counting question about Mexico’s wars. The process builds war descriptions, resolves soldier-count evidence, and produces the final answer of three qualifying wars.

  • Case Study Setup: The case asks which Mexican wars involved more than 100,000 but fewer than 500,000 soldiers, illustrating evidence-intensive counting over Mexico’s wars.The same question is used to illustrate the limitations of existing structured RAG methods.
  • Final Answer: 3 wars satisfy the final soldier-count range in the illustrated MoNaCo question.The final answer is produced after combining the evidence for the listed wars and their troop counts.
  • Adaptive Decomposition: Mexico’s wars are first enumerated across independence, territorial, intervention, revolutionary, religious, global-war, and internal conflicts.The major-war list includes the Mexican War of Independence, Texas–Indian Wars, Mexican–American War, French Intervention in Mexico, Mexican Revolution, Cristero War, World War II involvement, and the Dirty War.
  • Adaptive Decomposition: The framework decomposes the broad request into per-war questions about causes, key events, and outcomes.The decomposition creates sub-questions such as the Mexican War of Independence, Texas–Indian Wars, and Dirty War.
  • Evidence Gathering: The sub-answers synthesize retrieved evidence, including independence from Spain, Indigenous displacement, and repression during Mexico’s Dirty War.Examples include the Cry of Dolores and Treaty of Cordoba for the Mexican War of Independence, conflicts involving Comanche and Apache groups, and PRI-era repression.
  • Evidence Gathering: Exact troop counts are unavailable for some wars, while the Dirty War evidence reports about 784,300 personnel.The answerability step preserves this distinction by noting that Mexican War of Independence troop counts are not stated and casualties are estimated at 250,000–600,000.
  • Prompted Pipeline: APT-RAG’s case-study prompts cover contextualization, answerability checking, decomposition, query rewriting, topology-specific gathering, and batched answer generation.The prompt templates correspond to figures for contextualization, answerability, decomposition, lateral/external/vertical gathering, root answering, retrieval rewriting, and evidence-guided batching.
Loading 2609.04981v1…