PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking Retrieval in the Long-Context Era: Training-Free Semantic Trees for Adaptive Context Construction

Zhe Xu, Tianhao Liao, Zhenqing Ling, Ying Shen, Yaliang Li, Daoyuan Chen

arXiv:2610.34146v1cs.CLcs.AI

TL;DR

When a full document fits in a long-context model, it remains an open question whether selecting and organizing evidence improves answering while preserving needed support. TREEFUSE uses a reusable, source-grounded semantic hierarchy to select answer context. Across four frontier answer models, it improves average QA-F1 over full-context prompting on three LongBench tasks while reducing answer-context length.

  • Problem

    When full documents fit within a context window, it remains uncertain whether retrieval improves answering and how to select context that retains support without unrelated material.

    Source: §1 · p. 1

  • Method

    TREEFUSE builds a reusable semantic hierarchy linked to source spans and combines structural–semantic scoring with fine-grained lexical attribution to select source text for answering.

    Source: §1 · p. 2

  • Results

    Across four frontier answer models, TREEFUSE improves average QA-F1 by 1.72 points over full-context prompting on three LongBench tasks while reducing answer-context length by 58.55%.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Retrieval can complement long-context capacity: selecting and organizing document content can improve average answering when the complete document is available.

    Source: §5 · p. 9

    The evaluation centers on long-document question answering; multimodal extension would require linking textual evidence with tables and figures.

    Source: §5 · p. 9

Abstract

from arXiv · show

Long-context language models can increasingly process entire documents, making full-context prompting a practical alternative to passage retrieval. However, greater capacity determines how much information a model can receive, but not which information best supports a query. This distinction motivates rethinking retrieval: when the full document already fits, can selecting and organizing its evidence improve answering? The challenge is to retain supporting facts and relations while keeping the context focused on the question. To reconcile these goals, we introduce \ours, a training-free framework that uses semantic abstraction to guide selection while retaining original text for answering. \ours constructs a reusable semantic hierarchy from the full document in a single generation, linking node descriptions to source spans. Given a query, structural--semantic scoring and fine-grained lexical attribution nominate shared source nodes; their union with ancestor context preserves structural relations and complete local text. Across four frontier answer models, \ours improves average QA-F1 by 1.72 points over full-context prompting on three LongBench tasks, surpassing the evaluated baselines on the four-model average while reducing answer-context length by 58.55\%. Our code is available at the $\href{https://anonymous.4open.science/r/treefuse-ECFE}{repository}$.

1 INTRODUCTION

When a full document fits in a model’s context window, selecting evidence may still improve answering, but the selection must retain supporting facts and relations without introducing unrelated material. TREEFUSE guides selection with a reusable semantic hierarchy while preserving source text for answering.

Motivation

  • Full-context prompting can leave relevant evidence hard to use, while narrow retrieval may omit needed facts or separate details from their interpretation.The introduction notes that irrelevant information can interfere with predictions and that performance can depend on where relevant information appears in a long context.Source: §1 · p. 1

Approach

  • TREEFUSE builds a query-independent semantic hierarchy linked to source spans, then selects nodes and their ancestors while retaining complete local text.Its selection combines structural–semantic scoring with fine-grained lexical attribution; the same constructed context can be used by different answer models without model-specific fine-tuning.Source: §1 · p. 2

Motivation

  • Figure 1 contrasts full context with distractors and incomplete retrieval with missing support, against TREEFUSE’s focused context retaining both supporting statements.Source: Figure 1 · p. 2

Evaluation and contributions

  • 1.72 QA-F1 points over full-context prompting on three LongBench tasks, with 58.55% less answer context across four models.TREEFUSE also has the highest four-model average QA-F1 among evaluated methods on each of the three LongBench tasks.Source: §1 · p. 2

2 RELATED WORK

Related work spans long-context retrieval and compression, semantic chunking and hierarchical indexes, and structure-aware or hybrid retrieval. TREEFUSE frames its approach as query-focused, source-preserving selection guided by a reusable semantic hierarchy.

Retrieval and compression for long-context QA

  • Long-context QA work evaluates retrieval-based answering, while compression methods prune tokens, select sentences, or generate compact contexts.The discussion also covers adaptive compression rates and combinations of textual and compressed representations.Source: §2 · pp. 2–3

Chunking, hierarchical indexing, and structure-aware retrieval

  • Hierarchical approaches range from semantic chunking and tree indexes to structure-aware retrieval, refinement, and hierarchy-guided search.Examples include recursive clustering, bidirectional tree traversal, heading-aware indexing, and discourse-aware retrieval.Source: §2 · p. 3

Positioning

  • TREEFUSE treats retrieval as a complement to long-context capacity, targeting contexts that retain supporting evidence while concentrating on the query.The related-work discussion places this framing alongside evaluated baselines and selected related methods.Source: §2 · p. 3

3 METHOD

TREEFUSE constructs a reusable, source-grounded semantic hierarchy and uses query-dependent semantic and lexical selection to build a focused context. The resulting pruned tree preserves structural relations and complete local source text for answering.

3.1 PROBLEM FORMULATION AND OVERVIEW

  • Given a document and query, TREEFUSE selects source-grounded semantic units to retain supporting information while limiting unrelated content, even when the full document fits.The pipeline separates the information used to select evidence from the text delivered to the answer model.Source: §3.1 · p. 3

3.2 SOURCE-GROUNDED SEMANTIC TREE CONSTRUCTION

  • A single generation builds an ordered hierarchy with node descriptions and source spans, so semantic scoring and lexical matching can nominate the same source unit.Each node links its semantic view, source interval, and position in the hierarchy; the hierarchy is query-independent and reusable.Source: §3.2 · pp. 3–4, Figure 2 · p. 4

3.3 QUERY-ADAPTIVE DUAL-GRANULARITY SELECTION

  • Structural–semantic scoring evaluates node relevance and supporting role, while lexical attribution maps localized matches in source text to their enclosing nodes.All nodes remain eligible for semantic scoring; lexical matching can nominate a complete node even when its full span has little lexical overlap.Source: §3.3 · pp. 4–5

3.4 STRUCTURE-PRESERVING PRUNING AND CONTEXT CONSTRUCTION

  • TREEFUSE unions the candidate sets, adds their ancestors, and prunes unselected branches while preserving retained parent–child relations and complete local source text.Added ancestors contribute only their own local spans rather than expanding to every surrounding branch; construction and selection require no method-specific parameter updates.Source: §3.4 · p. 5

4 EXPERIMENTS

The experiments test whether query-adaptive context construction helps across long-document QA tasks and answer models, then examine which selection and construction components contribute. TREEFUSE improves over full-context prompting on LongBench while shortening answer contexts, with ablations comparing semantic and lexical selection, hierarchy, semantic units, and tree construction.

4.1 EXPERIMENTAL SETUP

  • Evaluation spans Qasper, HotpotQA, 2WikiMQA, and LaRA-32K, using four answer models and shared query sets; Raw and five compression or retrieval baselines are compared.LongBench uses normalized token-overlap QA-F1, while LaRA uses official task-specific grading prompts.Source: §4.1 · pp. 5–6

4.2 MAIN RESULTS

  • TREEFUSE has the highest four-model average QA-F1 on each of three LongBench tasks, improving 1.72 QA-F1 points over Raw while reducing answer-context length by 58.55%.Source: Table 1 · p. 6, §4.2 · pp. 6–7
  • TREEFUSE improves the LongBench average for all four answer models; Kimi-K2.6 gains 3.77 points, from 52.42 to 56.19.The same prepared contexts are reused across models, without model-specific context construction or fine-tuning.Source: §4.2 · p. 7
  • On HotpotQA, TREEFUSE's Overall QA-F1 is 71.23 versus 68.18 for the higher-purity method; on 2WikiMQA, it combines second-highest All-hit and purity for highest ECP.These comparisons characterize a balance between support retention and concentration rather than dominance in either component alone.Source: §4.2 · p. 8

4.3 ABLATION STUDIES

  • TREEFUSE reaches 81.12 average QA-F1, exceeding Semantic-only by 0.84 points, Lexical-only (BM) by 6.16, and Semantic-only (BM) by 2.72.Random supplementation scores 1.66 and 2.10 points below Semantic-only on HotpotQA and 2WikiMQA; ablation scores average three runs.Source: Table 4 · p. 8, §4.3 · p. 8
  • On annotated subsets, TREEFUSE exceeds Lexical-only (BM) by 21.50 percentage points in Span Recall and 4.45 QA-F1 points; versus Semantic-only, differences are 1.17 and 0.36.Source: §4.3 · p. 8
  • Hierarchy-enabled TREEFUSE scores 81.12 average QA-F1 versus 77.50 for the flat pipeline; Fixed 200 and Chonkie score 76.12 and 76.55, respectively.The flat comparison jointly tests hierarchy-aware scoring, ancestor completion, and structured serialization; retained content and length can also change.Source: Table 5 · p. 8, §4.3 · pp. 8–9
  • The single-pass constructor scores 81.12 average QA-F1, versus 78.90 for Multi-stage and 78.34 for the HiChunk adapter under the tested selector and two multi-hop tasks.The HiChunk adapter maps its units into this node representation rather than evaluating the complete HiChunk retrieval system.Source: §4.3 · p. 9

5 CONCLUSION

TREEFUSE treats retrieval as query-specific selection and organization of evidence, complementing long-context capacity. Its evaluations show improved QA-F1 with less answer context, while the paper identifies evaluation scope and generated-structure dependence as limitations.

5 CONCLUSION

  • TREEFUSE improves average QA-F1 by 1.72 points over full-context prompting on three LongBench tasks, with 58.55% less answer context.It also achieves the highest four-model average accuracy on LaRA with 43.87% less context; compared baselines fall below Raw in average performance on each LongBench task and LaRA.Source: §5 · p. 9
  • The evaluation centers on long-document question answering; multimodal use would require linking textual evidence with tables and figures.Selection also depends on generated structure and descriptions, and the authors identify perturbation studies as a way to quantify that sensitivity.Source: §5 · p. 9

AI USE STATEMENT

The authors describe generative AI assistance as limited to language editing and routine coding tasks.

AI USE STATEMENT

  • Generative AI assisted with polishing, grammar, exposition clarity, and routine implementation tasks, but not research ideas, method design, results, or scientific conclusions.The authors state that the research ideas, hypotheses, method, experiments, and conclusions were carried out by the authors.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The ethics statement describes the work's data and study scope and notes limits of source-preserving selection.

ETHICS STATEMENT

  • TREEFUSE uses public benchmarks, collects no new personal data, and does not guarantee factual correctness or privacy protection.The statement says sensitive applications require appropriate safeguards.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The reproducibility materials describe the repository, prompts, selection procedures, serialization, and evaluation setup. They also note that API updates and stochastic generation may affect exact replication.

REPRODUCIBILITY STATEMENT

  • The code repository and appendices document prompts, selection rules, experimental settings, and evaluation procedures; API updates and stochastic generation may affect exact replication.Fixed contexts are reused across answer models and repeated runs.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 COMPARISON WITH RELATED METHODS

  • The comparison covers methods' document representation, query-time selection, and evidence supplied for answering; the overview contrasts flat, hierarchical, and reusable source-grounded tree paradigms.The table caption distinguishes representation reuse and method-specific training.Source: §A.1 · p. 15, Table 7 · p. 15, Figure 3 · p. 15

A.2 PROMPT TEMPLATES AND OUTPUT SCHEMAS

  • The prompt materials specify a query-informed keep threshold using the scored outline, score histogram, and retention profile, while the construction prompt requests a hierarchical tree and source-supported facts.The all-node scoring prompt rates each node's contribution using semantic and structural evidence, including supporting relations rather than word overlap alone.Source: §A.2 · p. 16

A.3 TREE CONSTRUCTION, SCORING, AND SELECTION

  • Tree construction aligns model-proposed section paths and boundary clues to source spans; selection combines semantic ratings with node-level BM25 scores and ancestor context.The lexical cutoff is the node-score distribution's P90, with ties included, so it need not select exactly 10% of nodes.Source: §A.3 · pp. 16–17

A.4 STRUCTURE-PRESERVING SERIALIZATION

  • Serialization retains selected nodes and their ancestors without changing parent–child links or sibling order, supplying complete local source text in benchmark-specific formats.The example excerpt notes that displayed ellipses omit text and that gold answers and relevance scores are not answer-model inputs.Source: §A.4 · pp. 17–18

B.1 BENCHMARKS AND SAMPLE ALIGNMENT

  • Evaluation methods, answer models, and three answer-generation runs use the same question set, with scores averaged over the same question–model combinations.The four model scores are equally weighted in the Overall result.Source: §B.1 · p. 19

B.2 ANSWER MODELS AND JUDGE CONFIGURATIONS

  • LongBench uses task-specific answer templates, while LaRA uses full-document prompts for Raw and retrieved-context prompts for compressed inputs.Frozen context files are reused across answer runs, and LaRA uses a fixed gpt-oss-20b judge.Source: §B.2 · p. 19

B.3 BASELINE IMPLEMENTATIONS AND CONTEXT BUDGETS

The comparisons adjust context budgets and clarify how main results are averaged and how compression relates to QA-F1. TREEFUSE leads the LongBench compression–performance comparison, while adjusted budgets are not exact per-question matches.

B.3 BASELINE IMPLEMENTATIONS AND CONTEXT BUDGETS

  • Context-budget adjustments target TREEFUSE’s retained length, but do not impose identical per-question input sizes.LongLL, Provence, HiChunk, BRIEF-Pro, and LongRefiner use different adjusted settings.Source: §B.3 · p. 19, Table 10 · p. 19

C.2 COMPRESSION–PERFORMANCE TRADE-OFF

  • TREEFUSE achieves the highest average QA-F1 in all three panels while removing 58.55% of context on average.Figure 4 relates four-model average QA-F1 to the proportion of raw-context characters removed.Source: Figure 4 · p. 20, §C.2 · p. 20

C.1 MAIN-COMPARISON DETAILS AND RUN-LEVEL INTERVALS

  • Main-comparison scores average three answer runs; Overall first averages models within each task, and LongBench weights its three tasks equally.Table 11 reports four-model Overall means with 95% Student-t intervals over three decoding runs.Source: §C.1 · p. 20, Table 11 · p. 20

C.3 COMPARISONS WITH ADJUSTED CONTEXT LENGTHS

  • Adjusted-length comparisons use the same question set per dataset across three runs, but are not exact per-query budget matches.Table 12 reports mean percentages of raw-context characters retained.Source: §C.3 · p. 20, Table 12 · p. 21

C.4 CONTEXT SAVINGS AND CONSTRUCTION AMORTIZATION

The analysis models whether reduced reader-context input can offset tree construction and query-selection input work. Its modeled reductions depend on reader-to-preprocessor weights and preprocessing reuse; evidence annotations use mapped source material.

C.4 CONTEXT SAVINGS AND CONSTRUCTION AMORTIZATION

  • P1 counts inputs for one successful tree construction per document, query selection, and the final answer request to assess amortization against reader prefill.LaRA reuses 64 document trees across 887 questions, reducing construction input to 2,209 tokens per question; selection is charged per question.Source: §C.4 · p. 21, Table 13 · p. 21
  • Under the stated κ proxies, all 16 combinations show positive modeled reductions in successful-path input work, ranging from 1.9% to 56.9%.Break-even thresholds are 5.40–6.83 on HotpotQA and 2WikiMQA, 10.24–10.64 on Qasper, and 1.66–1.82 on LaRA.Source: §C.4 · p. 22, Table 14 · p. 22
  • The smallest modeled margin is Qasper with Kimi-K2.6: κ = 11.089 is only 5.2% above κ∗ = 10.51, yielding a 1.9% modeled reduction sensitive to proxy misspecification.The threshold lets readers compare an assumed reader-to-preprocessor weight with the break-even weight.Source: §C.4 · pp. 22–23

D.1 EVIDENCE ANNOTATIONS AND SOURCE ALIGNMENT

  • Qasper annotations use mapped textual evidence with alternative annotation sets, while HotpotQA and 2WikiMQA use mapped supporting passages.The reported component metrics measure annotated source preservation rather than semantic answer sufficiency.Source: §D.1 · p. 24

D.2 EVIDENCE METRIC DEFINITIONS

The section defines source-coverage and concentration diagnostics, then describes variants and conditional evidence analyses. Its saved examples show that added source coverage can help recover answers, connect supporting facts, or leave an already-correct answer unchanged.

D.2 EVIDENCE METRIC DEFINITIONS

  • All-hit requires one completely covered eligible evidence set, while Evidence Purity measures mapped-source concentration; neither establishes semantic answer sufficiency or all useful information.Purity can omit valid paraphrases and unannotated context, and its denominator uses deduplicated recovered source positions.Source: §D.2 · p. 24

D.3 EQUAL-WEIGHT ECP RATIONALE

  • ECP equally averages All-hit and Evidence Purity, treating completeness and concentration as complementary diagnostics on a 0–100 scale rather than modeling answer correctness.The same aggregation is applied across methods and datasets.Source: §D.3 · p. 24

E.1 SELECTION, REPRESENTATION, AND CONSTRUCTION VARIANTS

  • The ablation comparisons use Qwen3.8-Max over three answer runs; BM denotes approximate character-budget matching, not exact per-query token equality.Table 15 summarizes changes made by each variant.Source: §E.1 · p. 25, Table 15 · p. 25

E.2 TREE-CONSTRUCTION CONFIGURATIONS

  • Single-pass construction jointly generates hierarchy and node descriptions; the alternative builds them through chunked outline, anchoring, and enrichment stages, while the HiChunk adapter tests its representation with TREEFUSE selection.The adapted HiChunk representation is not the complete HiChunk retrieval and Auto-Merge system.Source: §E.2 · p. 25

E.3 CONDITIONAL EVIDENCE AND ANSWER ANALYSIS

  • Conditional comparisons report Full-minus-variant changes in Span Recall and QA-F1; BM length matching is approximate, and generation-setting differences do not isolate every change.Span Recall measures length-weighted literal coverage of annotated source units.Source: §E.3 · p. 26, Table 16 · p. 26

E.4 SEMANTIC SUPPORT AND LEXICAL SUPPLEMENTATION CASES

  • In saved cases, lexical supplementation raised Qasper Span Recall from 0 to 100 and QA-F1 from 0 to 71.11; semantic support raised HotpotQA QA-F1 from 0 to 100, while added coverage sometimes left scores unchanged.The examples are configuration-specific, not population-level success estimates or isolated causal tests.Source: §E.4 · p. 26

E.5 SEMANTIC ADEQUACY VERSUS BOUNDARY EXACTNESS

This case shows that a coherent source node can retain required evidence without matching annotation boundaries. The authors distinguish evidence containment from boundary-perfect reconstruction, while noting that this is not an estimate of tree-construction error rates.

E.5 SEMANTIC ADEQUACY VERSUS BOUNDARY EXACTNESS

  • One annotated four-unit evidence set maps to a single coherent node, n8, which contains the mechanism, its motivation, and its interpretation.The tree organizes source regions for retrieval rather than reproducing a unique gold segmentation; titles and structural paths navigate, while summaries and facts guide selection.Source: §E.5 · p. 27
  • Span Recall rose from 0 to 100 and mean QA-F1 from 17.57 to 35.33 when Full retained all four evidence units, unlike Lexical-only.The QA-F1 gain accompanies improved evidence coverage; it does not establish a perfectly correct answer or isolate boundary placement as its cause.Source: Table 17 · p. 27, §E.5 · p. 27
  • Boundary or metadata differences need not remove support when relevant text remains in selectable nodes, is retrieved, and its original text reaches the reader.Incorrect metadata or structure can still cause a selection miss; the example is an observed containment case, not a controlled corruption experiment or error-rate estimate.Source: §E.5 · p. 27

Related papers

Loading 2610.34146v1…