PaperlayerView exact revision on arXiv

Source-linked AI summary

BEAR: Budgeted Evidence Allocation for Multi-Document Reasoning

Lin Sun, Linglin Zhang, Jingang Huang, Change Jia, Zhengwei Cheng, Xiangzheng Zhang

arXiv:2601.18116v2cs.CL

TL;DR

Multi-document reasoning requires selective allocation of limited query-time evidence because long-context inference is costly and flat retrieval weakly organizes cross-document evidence. BEAR builds hierarchical semantic indices offline and performs coarse-to-fine query-time access through exploration and recovery under a fixed evidence budget. It performs particularly strongly on DragonBall, remains competitive on HotpotQA, and achieves the best retrieval-based result on 2Wiki under the evaluated protocol.

  • Problem

    Multi-document reasoning must allocate limited query-time evidence across documents and semantic granularities, since relevant evidence may be dispersed or weakly organized by flat retrieval.

  • Method

    BEAR builds hierarchical semantic trees offline and uses complementary LLM-guided exploration and TreeExpansion-based recovery for coarse-to-fine evidence access under a fixed budget.

  • Results

    BEAR performs particularly strongly on DragonBall, remains competitive on HotpotQA, and yields the best retrieval-based result on 2Wiki under the evaluated protocol.

  • Takeaways & Limitations

    The findings suggest that hierarchy serves as an allocation substrate and that complementary exploration and recovery support selective evidence exposure beyond semantic chunking alone.

  • Takeaways & Limitations

    BEAR is most effective when document collections have enough semantic structure for hierarchical organization to be informative, and it incurs nontrivial offline indexing cost and system complexity.

Abstract

from arXiv · show

We argue that multi-document reasoning is constrained not only by how much text a model can read, but also by how limited query-time evidence budget is allocated across documents and semantic granularities. Full-context inference exposes the model to broad evidence non-selectively and at high per-query cost, while flat chunk retrieval often returns locally relevant passages that are weakly organized for cross-document synthesis. We present \textbf{BEAR}, a framework for structured evidence allocation that builds hierarchical semantic indices offline and performs coarse-to-fine evidence access at query time through complementary \emph{exploration} and \emph{recovery} paths. This coarse-to-fine design can be viewed as structured evidence allocation under a fixed evidence-context budget. Across synthetic and real-world benchmarks, BEAR performs particularly strongly on DragonBall, remains competitive with strong retrieval-based baselines on HotpotQA, and yields the best retrieval-based result on 2Wiki under our evaluated protocol, while operating under substantially smaller \emph{query-time evidence budgets} than the reported long-context references. Additional analyses suggest that the gains are associated with hierarchy as an allocation substrate together with complementary exploration and recovery, rather than semantic chunking alone.

1 Introduction

Multi-document reasoning must allocate limited query-time evidence across documents and semantic granularities, because long-context and flat retrieval leave evidence selection weakly organized. BEAR addresses this with hierarchical, coarse-to-fine access through complementary exploration and recovery paths.

  • Motivation: Limited evidence budgets must be allocated across documents and semantic granularities for multi-document reasoning.Relevant evidence may be dispersed or buried in distracting context, while flat retrieval returns locally similar passages weakly organized for cross-document synthesis.
  • Motivation: Flat retrieval over-retrieves weakly useful chunks, whereas naive long-context inference leaves evidence prioritization and localization implicit inside generation.A financial comparison example motivates combining global semantic allocation with localized evidence recovery.
  • BEAR: BEAR organizes documents into hierarchical semantic trees and accesses them through LLM-guided exploration and TreeExpansion-based recovery.The hierarchy supports coarse semantic selection followed by finer-grained access to detailed supporting evidence.
  • BEAR: BEAR formulates evidence access under a fixed query-time budget, using document-level fusion or node-level refinement when finer evidence discrimination is needed.The framework treats hierarchy as a mechanism for budget-aware access rather than only an indexing structure.

2 Related Work

Prior work uses retrieval, graphs, and hierarchies to organize evidence, but BEAR emphasizes coarse-to-fine query-time traversal and fusion under a fixed evidence budget.

  • Background: Longer context does not reliably improve knowledge-intensive reasoning and increases computational cost, motivating selective retrieval.Retrieval-based systems are especially relevant when evidence must be selected rather than merely concatenated.
  • Structured retrieval: Graph-based and hierarchical retrieval methods organize evidence around entities, relations, or treelike document abstractions.Examples include GraphRAG, LightRAG, HippoRAG, RAPTOR, and HiRAG.
  • BEAR’s positioning: BEAR differs by studying hierarchical-index use at query time under a fixed evidence budget rather than hierarchy construction in isolation.It preserves coarse document evidence when sufficient and invokes node-level refinement only when finer aggregation is required.
  • BEAR’s positioning: BEAR combines exploration and recovery at the node level instead of relying on a single traversal mechanism.This design allocates limited query-time evidence more selectively while making evidence organization explicit.

3 Method

BEAR builds a semantic forest offline and performs budget-aware coarse-to-fine retrieval online. Complementary exploration and recovery paths combine semantic reasoning with embedding and structural evidence recovery.

  • Offline organization: BEAR segments each document into coherent chunks and organizes them into a semantic tree with summary-bearing internal nodes and content-preserving leaves.LLM-guided chunking aligns boundaries with semantic units rather than fixed lengths.
  • Offline organization: Separate internal- and leaf-node embeddings support dense retrieval at document and node levels.Internal embeddings use hierarchical summary signals, while leaf embeddings represent original chunk content.
  • Coarse access: The exploration path reasons over high-level summaries, while the recovery path performs dense retrieval to form a deduplicated fused candidate set.Their union combines semantic reasoning with embedding-space coverage.
  • Budget-aware routing: BEAR returns fused document evidence when it fits the token budget Bmax; otherwise, it routes to finer node-level retrieval.This implements query-time evidence allocation through coarse-to-fine routing.
  • Fine-grained refinement: During refinement, exploration navigates summaries and TreeExpansion supplements query-node similarity with ancestor and descendant signals.The recovery score combines direct similarity, propagated ancestor relevance, and aggregated child support.
  • Fine-grained refinement: BEAR uses α = β = γ = 1 as an untuned default and deduplicates and truncates the final evidence list to satisfy the budget.Equal weighting preserves a fixed, lightweight retrieval policy despite query-category differences.

4 Experimental Setup

BEAR is evaluated on synthetic, real-world, and agentic multi-document reasoning benchmarks against retrieval, structure-enhanced, full-context, and long-context baselines. The protocol controls generators and reports answer quality alongside retrieval cost and latency.

  • Datasets: DragonBall, HotpotQA, and 2Wiki evaluate synthetic and real-world multi-hop retrieval, while BrowseComp-plus tests transfer to downstream agentic reasoning.DragonBall isolates retrieval quality more directly because its knowledge is LLM-synthesized.
  • Baselines: Baselines include BM25, BGE-M3, RAPTOR, LongRefiner, HippoRAG2, a same-generator full-context control, and Gemini long-context references.Shared implementations and generator settings are used where applicable, with LongRefiner as the main exception.
  • Table 1: Table 1 compares BEAR(docs), which returns fused document context, with BEAR(nodes), which adds node-level refinement, across three benchmarks.Retrieval-based methods share the Qwen3-32B generator, while DragonBall reports Recall, Completeness, Hallucination, and Irrelevance.
  • Protocol: The comparison protocol reports matched retrieval comparisons, a same-generator full-context control, controlled ablations, and long-context references.The main claim concerns the operating point under a shared-generator, budget-matched retrieval protocol rather than universal superiority across model families.
  • Metrics: Evaluation reports task-specific answer metrics plus token cost, end-to-end latency, and bi-path retrieval contributions.Results are generally averaged over 3 repeated runs with standard deviations.

5 Results

BEAR improves budget-constrained retrieval quality across evaluated benchmarks and evidence-context budgets, with gains linked to hierarchical access and complementary exploration–recovery paths. Its cost analyses show favorable online latency among retrieval-based methods, alongside nontrivial offline indexing overhead.

  • BEAR performs particularly strongly on DragonBall, remains close to HippoRAG2 on HotpotQA, and achieves the strongest retrieval-based result on 2Wiki under the evaluated protocol.
  • 85.55% Recall and 91.86% Completeness on DragonBall accompany 5.64% Hallucination and 2.40% Irrelevance for BEAR(docs).
  • Under 1K–2K budgets, BEAR(llm-nodes) is the strongest node-level variant, while at 4K–8K structured variants become competitive with reported long-context references using 4K–8K retrieved tokens rather than 512K-token inputs.
  • Why exploration–recovery helps: Bi-path fusion improves over the stronger single path on every answerable query type, with gains from +2.08 to +8.99 points and an average of +6.01.
  • Why exploration–recovery helps: The largest ablation jump occurs from leaf-only to tree-aware evidence access, increasing coverage from 50.55% to 87.91% before further gains from path fusion.
  • Cost and Latency Analysis: BEAR provides a favorable online quality–latency profile among evaluated retrieval-based methods, but hierarchy construction adds offline workload and system overhead.

6 Conclusion

BEAR combines hierarchical semantic indexing with complementary exploration–recovery allocation for budget-aware multi-document reasoning. Across evaluated benchmarks, it achieves strong retrieval results while using smaller query-time evidence budgets than reported long-context references.

  • BEAR combines hierarchical semantic indexing, complementary exploration–recovery allocation, and fixed-budget query-time evidence access.
  • BEAR performs particularly strongly on DragonBall, remains competitive on HotpotQA, and achieves the best retrieval-based result on 2Wiki under the matched protocol.
  • Coarse-grained evidence access provides a favorable completeness–faithfulness trade-off.

Limitations

BEAR depends on informative document structure and introduces substantial preprocessing and maintenance requirements. Its evidence organization also does not by itself eliminate downstream generator reasoning errors.

  • BEAR is most effective when document collections contain enough semantic structure for hierarchical organization to be informative.
  • On noisy, weakly structured, or rapidly changing corpora, hierarchy may become less reliable and selective-allocation gains may diminish.
  • BEAR incurs offline indexing cost and added system complexity from semantic chunking, tree construction, and multi-granularity indexing.
  • Dynamic deployment may require subtree reconstruction, index maintenance under document drift, and summary consistency across abstraction levels.
  • BEAR improves evidence selection and organization but does not by itself eliminate downstream reasoning errors of the generator.

A Additional Methods, Experimental Details, and Supplementary Results

The appendix collects material moved from the main paper, including expanded methods, supplementary results, and analyses of BEAR’s operating behavior.

  • The appendix gathers fuller method details, supplementary results, and additional analyses clarifying BEAR’s operating behavior.

A.1 Neither-Case Failure Analysis

Neither-path failures concentrate in deep leaf evidence and structurally homogeneous medical cases requiring cross-branch collection. The two paths fail through distinct selection, retrieval, and budget-truncation mechanisms.

  • 44 missed nodes across 27 neither-path cases were all leaf nodes; 93% occurred at depth 3 and 93% of cases came from medicine.
  • LLM-guided selection missed 43/44 nodes by choosing the correct document but the wrong tree node.
  • Conclusion-node selection bias can omit sibling branches containing evidence needed for multi-hop and comparison answers.
  • TreeExpansion misses arise from entity-sparse retrieval in homogeneous medical corpora and truncation under the fixed evidence budget.
  • The 4K token evidence budget can prevent expansion from covering deep leaves across parallel evidence-bearing branches.

A.2.1 Qwen3-32B Full-Context Control on DragonBall

The full-context control performs worse than structured evidence allocation despite receiving more raw context, with lower completeness and higher hallucination and irrelevance.

  • Lower Completeness and higher Hallucination and Irrelevance occur when Qwen3-32B receives the full document set instead of structured evidence allocation.The comparison indicates that increasing available context alone is insufficient.

A.2.2 Hierarchical Evidence Organization and Multi-Granularity Indexing

BEAR organizes documents into hierarchical semantic structures for multi-granularity retrieval and combines complementary retrieval paths. Analyses indicate that internal hierarchy and asymmetric path complementarity contribute to the strongest retrieval behavior.

  • Hierarchical Evidence Organization: Offline indexing combines semantic-aware chunking, LLM-based tree generation, and vector indexing of internal and leaf nodes.The same structure supports summary-level and fine-grained evidence retrieval.
  • Hierarchical Evidence Organization: BEAR constructs hierarchical semantic trees with typed internal summary nodes and leaf nodes retaining original semantic chunks.Each document forms one tree, with internal nodes capturing higher-level abstractions.
  • Query-Time Access: Only 1 of 8 gold supporting nodes is covered when semantic node selection chooses a matched conclusion instead of evidence-bearing nodes across sibling branches.The failure case illustrates the risk of semantic shortcuts in node selection.
  • Hierarchical Evidence Organization: Semantic chunking aligns chunks with coherent units rather than fixed token windows, while tree construction generates a table-of-contents-like hierarchy and node summaries.Long documents can be handled through progressive partitioning and merging of partial trees.
  • Bi-Path Retrieval: The node-level bi-path retriever remains stronger than either single path across domains and languages, with gains over the stronger single path from +0.00 to +13.10 points.The average improvement is +3.80 points, indicating complementarity across slices.
  • Bi-Path Retrieval: Tree-aware retrieval improves over leaf-only retrieval, while combining TreeExpansion and LLM-guided retrieval performs best in the progressive ablation.The staged pattern identifies access to internal semantic structure as an important improvement step.
  • Bi-Path Retrieval: BEAR(nodes) remains comparatively stable across four settings, with recall varying from 96.84% to 98.19%.The analysis attributes the main gains to consistent semantic routing interacting with complementary structural evidence recovery rather than simply using a stronger LLM.
  • Scoring Analysis: Increasing direct similarity tends to help multi-document questions, child aggregation helps factual questions, and stronger ancestor inheritance can reduce overall recall.Equal weights α = β = γ = 1/3 remain a stable compromise across query types.
Loading 2601.18116v2…