Source-linked AI summary
A Tree-based RAG Framework for Evidence-Intensive QA via Adaptive Planning and Topology-Aware Evidence Gathering
Songeun Lee, Kyungjin Min, Injae Na, Suyeong Lee, Chiyoung Kim, Woohwan Jung
TL;DR
Evidence-intensive QA requires synthesizing evidence scattered across dozens or hundreds of documents, while structured RAG methods can be rigid and ignore evidence across reasoning nodes. APT-RAG adaptively builds reasoning trees and gathers evidence along their topology, and consistently outperforms structured RAG baselines on evidence-intensive QA benchmarks.
Problem
Evidence-intensive QA requires synthesizing evidence across dozens or hundreds of documents, while structured RAG methods can be rigid and gather evidence without using information across reasoning nodes.
Source: §1 · pp. 1–2Method
APT-RAG recursively expands a reasoning tree according to subproblem complexity and gathers evidence across sibling branches and from child to parent nodes.
Source: §1 · p. 2Results
Takeaways & Limitations
Recursive, topology-aware evidence gathering is a promising direction for answering questions over large, dispersed document spaces.
Source: §5 · p. 9APT-RAG depends on planning quality, as inaccurate query decomposition or answerability checks may create unnecessary branches or propagate errors.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Recent structured RAG methods leverage tree- or graph-based reasoning structures to improve multi-hop QA. However, they face key limitations in evidence-intensive QA, where answering a question requires synthesizing information scattered across dozens or even hundreds of documents: structural rigidity, which limits adaptive reasoning expansion, and topology-ignorant evidence gathering, which prevents effective integration of evidence across different reasoning nodes. To address these issues, we propose APT-RAG, an Adaptive Planning and Topology-aware evidence gathering RAG framework. Adaptive planning dynamically expands the reasoning structure based on question dependencies and evidence requirements, while topology-aware evidence gathering improves evidence coverage through sibling evidence reuse, direct retrieval, and evidence aggregation from child nodes. We further introduce evidence-guided batched answer generation to reduce significant generation overhead in evidence-intensive QA. In the experiments on evidence-intensive QA benchmarks, APT-RAG outperforms existing structured RAG methods. Our code is available at https://github.com/hyudsl/APT-RAG.
1 Introduction
Evidence-intensive QA requires synthesizing evidence spread across dozens or hundreds of documents, exposing structural rigidity and topology-ignorant gathering in existing structured RAG. APT-RAG adaptively builds a reasoning tree, gathers evidence along its topology, and uses evidence-guided answer batching; it achieves higher answer F1 than existing structured RAG methods on MoNaCo and QAMPARI.
1 Introduction
- Evidence-intensive QA requires synthesizing evidence across dozens or hundreds of documents, exposing structural rigidity and topology-ignorant gathering in existing structured RAG methods.Fixed plans can force a node to process extensive evidence, while root-only gathering can miss evidence available from subquestion answers.Source: §1 · pp. 1–2
- Evidence-intensive QA raises the evidence burden from 2.0 pages on average in HotpotQA to 43.3 pages in MoNaCo.The introduction contrasts conventional multi-hop QA, which requires only a few supporting pages on average, with evidence distributed across dozens or hundreds of documents.Source: §1 · p. 1
- APT-RAG achieves higher answer F1 scores than existing structured RAG methods on the evidence-intensive QA benchmarks MoNaCo and QAMPARI.As evidence requirements increase, APT-RAG adaptively expands its reasoning tree and retrieves more supporting evidence, while baselines retrieve nearly constant numbers of documents.Source: §1 · p. 2
- APT-RAG recursively decides whether to decompose a node, answer it directly, or resolve it using gathered evidence, flexibly adjusting reasoning depth and breadth.It gathers evidence laterally across sibling branches and vertically from child to parent nodes so each node can use evidence relevant to its reasoning context.Source: §1 · p. 2
- Evidence-guided answer batching reduces redundant generation over overlapping evidence and substantially lowers LLM calls.The paper introduces batching to reduce inference latency.Source: §1 · p. 2
2 Related Work
Evidence-intensive QA requires substantially more evidence and reasoning than conventional multi-hop QA, often drawing on dozens to hundreds of documents. Existing structured RAG organizes reasoning with trees or graphs but does not sufficiently address dynamic evidence expansion or hierarchical and lateral evidence integration.
Evidence-intensive QA
- Evidence-intensive QA requires collecting and synthesizing evidence across dozens to hundreds of documents, beyond conventional multi-hop tasks such as HotpotQA and MuSiQue.Recent benchmarks include MoNaCo (Wolfson et al., 2026), QAMPARI, and DeepSearchQA (Gupta et al., 2026).Source: §2 · p. 2
Structured RAG for Multi-Hop QA
- Tree-based structured RAG decomposes questions into sub-questions and reasons over tree structures (Lee et al., 2024; Shi et al., 2026; Jiao et al., 2026).Source: §2 · p. 2
- Graph-based methods use DAGs to control retrieval and reasoning, including fixed atomic-subquestion graphs and dependency graphs with topological sorting (Verma et al., 2025; Chen et al., 2026).Source: §2 · p. 2
- These methods leave dynamic evidence expansion and management, as well as hierarchical and lateral evidence integration, insufficiently addressed for evidence-intensive QA.Source: §2 · p. 2
3 Method
APT-RAG addresses evidence-intensive QA by building and resolving a reasoning tree that gathers and integrates evidence across many documents. Its adaptive planning and topology-aware gathering coordinate evidence reuse, retrieval, and decomposition, while evidence-guided batching reduces redundant answer generation.
3.1 Task Formulation
- Evidence-intensive QA synthesizes evidence from dozens to hundreds of documents, aggregating evidence and intermediate reasoning results across steps to answer the original question.Unlike conventional multi-hop QA, the required evidence sets are substantially larger than a small number of paragraphs.Source: §3.1 · p. 3
3.2 Overview of APT-RAG Framework
- APT-RAG recursively builds a reasoning tree depth-first, interleaving contextualization, adaptive planning, topology-aware evidence gathering, and answer generation until the root is resolved.Contextualization resolves references to preceding sibling answers when a question depends on them; node answers can then serve as evidence for sibling or parent nodes.Source: §3.2 · pp. 3–4, Figure 2 · p. 3
3.3 Adaptive Planning
- APT-RAG first checks whether sibling QA pairs suffice; otherwise, it keeps an atomic question as a retrieval query or decomposes it into child sub-questions.If sibling answers suffice, the question follows the lateral-gathering path without further retrieval or decomposition.Source: §3.3 · p. 4
3.4 Topology-aware Evidence Gathering
- APT-RAG gathers evidence laterally from preceding siblings, externally through retrieval, or vertically by aggregating child answers to cover broadly dispersed evidence.Vertical gathering lets child nodes focus on narrower evidence scopes; sibling reuse avoids redundant retrieval or decomposition, while ancestors handle evidence from their own regions of the tree.Source: §3.4 · p. 4, §3.2 · p. 4
3.5 Evidence-Guided Batched Answer Generation
- APT-RAG batches independent sibling leaves with overlapping retrieved evidence, answering them jointly with deduplicated documents to reduce LLM calls, input tokens, and latency.It clusters by evidence overlap subject to a context-length constraint; its approximation relaxes that constraint for a clique-cover reduction, then uses modified greedy coloring to enforce the context window.Source: §3.5 · pp. 5–6
4 Experiments
Across evidence-intensive QA benchmarks, APT-RAG achieves the best reported overall performance. Its adaptive expansion and evidence-gathering modules improve evidence coverage and reduce inference latency.
4.1 Experimental Setup
- Experiments cover MoNaCo (43.3 evidence pages per question) and QAMPARI (13.0), comparing basic and structured RAG baselines using answer precision, recall, F1, and retrieval recall.All methods share a retrieval setup using Qwen3-Embedding-0.6B and the top 20 passages; RT-RAG is evaluated on a representative 300-example subset of each benchmark.Source: §4.1 · pp. 6–7
4.2 Main Results
- APT-RAG achieves the best overall performance across benchmarks and model scales; on MoNaCo 30B it beats Plan*RAG by 8% in answer F1, with retrieval recall 50.79 versus 39.28.All answer F1 gains over baselines are statistically significant (p ≤0.001). Under the 4B setting, APT-RAG maintains relatively stable performance while most baselines degrade substantially; a similar trend appears on QAMPARI.Source: Table 2 · p. 7, §4.2 · p. 7
4.3 Analysis
- As gold evidence requirements grow, APT-RAG maintains the highest performance with smaller drops, increasing retrieved documents, tree depth, and breadth while baselines remain nearly constant.APT-RAG’s answer F1 in the 21–30-page range is comparable to baselines in the 11–20-page range, and its retrieval recall remains stable in the 31+ range.Source: §4.2 · pp. 7–8, Figure 5 · p. 7, §4.3 · p. 8
4.4 Ablation Studies
- Fixing tree breadth at 2 yields the lowest answer F1 and retrieval recall, while removing vertical gathering reduces answer F1 by 4.20 points.Using fixed breadth 6 substantially increases latency; APT-RAG achieves the highest answer F1 with retrieval recall comparable to that variant and less than half its latency.Source: §4.4 · p. 8, Table 3 · p. 8
- Lateral gathering and evidence-guided clustering reduce average inference latency by 41.4% and 8.2%, respectively.Lateral gathering reduces redundant tree expansion; clustering’s latency reduction grows at the 75th and 95th percentiles as gold-document scale increases, including for queries exceeding 100 seconds.Source: §4.4 · pp. 8–9, Table 3 · p. 8, Figure 7 · p. 8
5 Conclusion
APT-RAG combines adaptive planning with topology-aware evidence gathering to address structural rigidity and topology-ignorant evidence gathering in structured RAG. Experiments show improved answer quality and retrieval recall on evidence-intensive QA benchmarks.
5 Conclusion
- APT-RAG combines adaptive planning and topology-aware evidence gathering to address structural rigidity and topology-ignorant evidence gathering in existing structured RAG methods.It dynamically adjusts the search space and uses the reasoning topology.Source: §5 · p. 9
- Experiments show APT-RAG consistently improves answer quality and retrieval recall on evidence-intensive QA benchmarks.The conclusion identifies recursive, topology-aware evidence gathering as a promising direction for questions over large, dispersed document spaces.Source: §5 · p. 9
6 Limitations
APT-RAG’s inference cost grows with evidence and reasoning branches, while performance also depends on the quality of planning that constructs its reasoning tree.
6 Limitations
- Evidence-intensive QA incurs substantial inference cost and latency; batched generation reduces overhead, but costs still rise with evidence volume and reasoning branches.Tree pruning could control unnecessary branches and evidence expansion.Source: §6 · p. 9
- In the 31+ range, larger merged evidence raises long-context processing cost enough to outweigh latency savings from clustering.Source: §6 · p. 9
- APT-RAG depends on planning quality: inaccurate query decomposition or answerability checks can create unnecessary branches or propagate errors.Future work could improve performance by training planning-related modules.Source: §6 · p. 9
A Datasets … E.3 Greedy Approximation
The appendix documents APT-RAG’s datasets, retrieval and baseline setup, recursive and batched inference, and evidence-guided clustering, including its constrained graph-coloring approximation.
MoNaCo
- MoNaCo examples ask how Roman emperors died and how long they reigned; QAMPARI asks for famous drag queens’ stage names.The supplied snippets include Basil I and Romanos III Argyros entries, plus A'keria Chanel Davenport.Source: MoNaCo · p. 10, QAMPARI · p. 10
B Corpus and Indexing
- Both benchmarks use English Wikipedia; MoNaCo pages are chunked by content structure, with title and section context, into over one million passages capped at 512 tokens.QAMPARI uses its official pre-chunked corpus; MoNaCo combines gold pages with sampled pages from the January 2026 dump, preserving sentence, line, or table-row boundaries.Source: B Corpus and Indexing · p. 11
C.1 Method Details
- Plan*RAG, LogicRAG, Tree-of-Question, and RT-RAG use different DAG or tree decomposition and evidence-synthesis procedures; RT-RAG’s official repeated construction and rewriting is slow.Plan*RAG (Verma et al., 2025) generates node sub-answers; LogicRAG (Chen et al., 2026) incrementally refines a summary; Tree-of-Question filters unreliable nodes; RT-RAG (Shi et al., 2026) builds bounded-depth binary trees.Source: §C.1 · p. 11
C.2 Reproduction Details
- Plan*RAG and Tree-of-Question are re-implemented from their published algorithms and prompts, while LogicRAG and RT-RAG use the authors’ official implementations.For comparison, Plan*RAG and Tree-of-Question use the APT-RAG final-answer prompt; the original reasoning algorithms and final-answer prompts are retained for LogicRAG and RT-RAG.Source: §C.2 · pp. 11–12
D.1 Inference without Batched Answer Generation
- Each recursive APT-RAG node contextualizes its question, plans a gathering mode, gathers evidence, and generates an answer, with vertical planning recursing on child questions.Evidence for generation comes from preceding sibling QA pairs, retrieved documents, or aggregated child QA pairs, depending on the gathering mode; the appendix presents the inference pseudocode.Source: D Algorithms · p. 12, §D.1 · p. 12
D.2 Inference with Batched Answer Generation
- Batched inference groups consecutive external-gathering children, clusters overlapping evidence under the context budget, and jointly answers them while preserving sibling-order dependencies.It flushes a batch before a child requiring non-external gathering and generates answers from the union of each cluster’s evidence.Source: §D.2 · pp. 12–13
D.3 Evidence-Guided Subquestion Clustering
- Evidence-guided clustering seeks the fewest mutually evidence-compatible subquestion groups that satisfy the language model’s context-window limit.The greedy method colors the complement graph in largest-first order, admitting a vertex only when pairwise compatibility and the context budget hold; otherwise it creates a singleton cluster.Source: §E.1 · p. 13, §D.3 · p. 13
E.2 Reduction to Graph Coloring
- The constrained problem reduces to coloring the complement of the compatibility graph, and its largest-first greedy approximation runs in O(m + n log n).The constrained optimum is at least the relaxed clique-cover optimum because the context-window constraint narrows the feasible set; the greedy procedure addresses the NP-hard problem.Source: §E.2 · p. 14, §E.1 · p. 14, §E.3 · pp. 14–15
E.4 Approximation Analysis · F Prompt Templates
The appendix establishes bounds for the constrained greedy clustering method and describes prompt designs for APT-RAG’s node-level stages. The prompts support contextualization, adaptive planning, evidence gathering, and answer generation.
E.4 Approximation Analysis
- At 21+ candidate external child nodes, the relaxed and constrained settings differ by less than one cluster, and the gap remains small as candidates increase.The reported result suggests that the context-window constraint preserves clustering behavior close to the relaxed solution.Source: Figure 10 · p. 15, §E.4 · p. 15
- The relaxed optimum is a lower bound and the constrained greedy result is an upper bound on the constrained optimum; unconstrained greedy produces no more clusters than constrained greedy.The inequalities are ˜K* ≤ ˆK* ≤ ˆK and ˜K ≤ ˆK.Source: §E.4 · p. 15
F Prompt Templates
- The prompt templates cover four node-level stages and specify each prompt’s role in recursive traversal, its instruction structure, and its design rationale.The stages are contextualization, adaptive planning, evidence gathering, and answer generation.Source: F Prompt Templates · p. 15
F.1 Contextualization
- Contextualization identifies conditions and extracts items from a referenced sibling’s answer before rewriting the sub-question with those items.Listing the extracted items before rewriting is intended to avoid leaving the placeholder verbatim or vaguely paraphrased.Source: §F.1 · pp. 15–16
F.2 Adaptive Planning
- Adaptive planning checks answerability before decomposition, using factual sufficiency and operational feasibility to avoid redundant retrieval for simple aggregations.It splits for bundled facts, stepwise dependencies, or operations such as sorting, ranking, or comparison, rather than following a fixed depth or branching factor.Source: §F.2 · p. 16
F.3 Evidence Gathering
- Evidence gathering rewrites a root or other non-decomposed question into a compact keyword query, while sibling questions routed to external gathering are rewritten together in one call.The batched path returns a JSON dictionary keyed by question index, so rewriting cost scales with decomposition levels rather than sibling count.Source: §F.3 · p. 16
F.4 Answer Generation
- Answer generation uses prompts tailored to lateral, external, and vertical gathering, plus dedicated root and batched-EC prompts with shared system instructions.Lateral answers use relevant sibling QA pairs, external answers include the search query with retrieved documents, and vertical answers combine child QA pairs into a single parent answer. Root output follows benchmark-specific formats, while batched EC answers are restricted to each question’s allowed documents and returned as an indexed JSON dictionary.Source: §F.4 · pp. 16–17
G Subset Evaluation
On randomly sampled 300-example benchmark subsets, APT-RAG is compared with RT-RAG on inference cost, answer quality, and how reasoning structures adapt to evidence requirements.
Inference Cost
- On MoNaCo with the 30B backbone, RT-RAG uses 115.6 LLM calls and 1.13M input tokens per question—about 3.6× and 8.4× APT-RAG’s costs, respectively.The cost gap is consistent across model scales and benchmarks; the paper attributes it to repeated tree construction, consensus sampling, and sub-question rewriting.Source: G Subset Evaluation · p. 17
Answer Quality
- On 300-example subsets, APT-RAG leads Answer F1 in most settings; RT-RAG’s MoNaCo 4B F1 edge comes with about half its Retrieval Recall, and nearly 3× lower recall at 30B.On MoNaCo, scaling from 4B to 30B improves APT-RAG’s F1 by +13.0, versus +7.0 for RT-RAG; the paper attributes the recall asymmetry to RT-RAG’s iterative refinement narrowing search at the cost of evidence coverage.Source: G Subset Evaluation · pp. 17–18
Structural Adaptation
- APT-RAG’s tree depth and breadth increase with required evidence volume, while baseline structures remain nearly flat on the 300-example subset.Depth is the maximum root-to-node distance, and breadth is the average number of generated child sub-questions over decomposed parent nodes. Plan*RAG and LogicRAG fix their graph before retrieval, RT-RAG uses a limited-depth binary tree, and ToQ stops expansion early at sufficiency checks; this rigidity limits reasoning space and contributes to the retrieval coverage gap.Source: G Subset Evaluation · p. 18, Figure 11 · p. 18
H Full Evaluation · I Statistical Significance
APT-RAG’s inference cost rises with evidence requirements, making it costly on high-evidence MoNaCo but faster than LogicRAG and ToQ on QAMPARI with a 4B backbone. Its full-benchmark Answer F1 differences from baselines are statistically significant under paired testing.
H Full Evaluation
- On MoNaCo with the 30B backbone, APT-RAG issues 33.61 LLM calls and takes 104.59s per question, versus baselines’ 5.52–8.20 calls and 17.77–81.27s.Questions require 43.3 gold pages on average.Source: H Full Evaluation · p. 18
- On QAMPARI, APT-RAG’s LLM calls drop 2.9× and latency drops 4.3×, while baseline LLM calls drop by at most 1.5×.QAMPARI questions require 13.0 pages on average.Source: H Full Evaluation · p. 18
- With the 4B backbone on QAMPARI, APT-RAG takes 3.60s, faster than LogicRAG at 9.67s and ToQ at 7.19s.Source: H Full Evaluation · p. 18
- Baselines cost less than APT-RAG on MoNaCo because their fixed structures do not scale with increasing evidence requirements, not because of stricter budget constraints.Source: H Full Evaluation · p. 18
I Statistical Significance
- Per-question average F1 differences were tested with a two-sided paired permutation test, using 95% confidence intervals from 10,000 percentile-bootstrap resamples.Source: I Statistical Significance · pp. 18–19
- All full-benchmark APT-RAG-versus-baseline Answer F1 differences have 95% confidence intervals excluding zero and p ≤0.001, remaining significant after Bonferroni correction.The paper concludes the Table 2 improvements are statistically significant and not attributable to evaluation-set variation.Source: Table 8 · p. 19, I Statistical Significance · p. 19
J Additional Ablation Results
APT-RAG’s retrieval and answer-quality findings hold under fine-grained evidence matching and across two MoNaCo evaluators. Lateral gathering is used sparingly but reduces latency by avoiding redundant retrieval and tree expansion.
Adaptive Expansion
- Under MoNaCo’s fine-grained evidence-matching criterion, APT-RAG achieves the highest retrieval recall, supporting adaptive expansion’s role in recovering more complete evidence.The criterion matches evidence at the corpus’s title or URL, section, and content-type granularity; the passage links this result to higher Answer F1 in Table 3.Source: J Additional Ablation Results · p. 19
Topology-Aware Gathering Usage
- Lateral gathering accounts for 1.6–6.2% of generated nodes, while most nodes rely on external or vertical gathering.Despite its limited use, lateral gathering reduces latency by avoiding redundant retrieval and tree expansion, as shown by the ablation in Figure 7.Source: Table 10 · p. 19, J Additional Ablation Results · pp. 19–20
Evaluator Robustness
- APT-RAG consistently achieves the highest MoNaCo Answer F1 under both evaluators, whose F1 scores are comparable across methods and backbone scales.This indicates that the main performance conclusions are robust to evaluator choice.Source: Table 11 · p. 19, J Additional Ablation Results · p. 20
K Case Study
The MoNaCo case study traces APT-RAG answering a count question about Mexico’s wars by organizing nested questions and evidence gathering; the final answer is three wars.
Inference Case: Evidence-Intensive Counting over Mexico’s Wars
- The MoNaCo inference trace returns 3 wars with more than 100,000 but fewer than 500,000 soldiers.The trace is presented in Figure 12.Source: K Case Study · pp. 20–21, Figure 12 · p. 22
- The trace first identifies Mexico’s wars, then expands to a major-war list and a contextualized question about each war’s causes, events, and outcomes.The contextualized question is marked unanswerable, and the trace shows evidence and answers for ten per-war questions.Source: K Case Study · p. 21
- The case’s war descriptions range from the Independence and Texas–Indian wars to the Dirty War, illustrating evidence gathered across distinct historical conflicts.The Independence-war answer cites Spain’s invasion and events including the Cry of Dolores and Treaty of Cordoba; the Texas–Indian answer describes conflict with Comanche and Apache sovereignty and Indigenous displacement; the Dirty War answer describes PRI repression, disappearances, and extrajudicial violence.Source: K Case Study · p. 21