PaperlayerView exact revision on arXiv

Source-linked AI summary

More Context, Same Budget: Dual-Bounded Relational Recall Beyond Top-K Retrieval

Thomson D. Nguy

arXiv:2608.18448v1cs.IRcs.CL

TL;DR

With a fixed context budget, the paper asks whether bounded graph-adjacent allocation can recover complete evidence sets more often than additional flat-ranked items. Using DBRR under matched retrieval-unit and token ceilings, it finds a 23.8-percentage-point increase in complete supporting-evidence recovery across 7,405 questions.

  • Problem

    With a fixed context ceiling, it remains unclear whether bounded graph-adjacent allocation recovers complete evidence sets more often than additional flat-ranked items.

  • Method

    DBRR allocates the same maximum retrieval units and tokens between relevance-selected seeds and bounded graph-adjacent context, comparing it with matched flat retrieval.

  • Results

    23.8 percentage points increased complete supporting-evidence recovery over additional flat-ranked context across 7,405 questions, with the advantage concentrated in bridge questions.

  • Takeaways & Limitations

    Under the frozen HotpotQA FullWiki design, how a finite retrieval budget is allocated materially changes whether the complete supporting-evidence set is available.

  • Takeaways & Limitations

    The study uses one frozen HotpotQA FullWiki design and relationship graph, without direct comparison against contemporary retrieval systems, so general superiority is outside the evidence.

Abstract

from arXiv · show

More context does not require a larger retrieval budget. Under the same ceiling, a retrieval system can recover more of the evidence a question requires by following relationships between evidence that flat top-k ranking leaves behind. We test that proposition with Dual-Bounded Relational Recall (DBRR), which allocates a fixed retrieval budget between relevance-selected seeds and bounded graph-adjacent context, against matched flat top-k retrieval using the same relevance-ranking stage and the same maximum number of retrieval units and tokens. The outcome is complete recovery of the official HotpotQA supporting-evidence set for each question. Across 7,405 FullWiki questions, the Primary DBRR allocation increased complete supporting-evidence recovery by 23.8 percentage points over its matched flat baseline (paired risk difference 0.2377; question-level bootstrap 95% interval 0.2269 to 0.2489). It improved 1,952 questions, tied on 5,261, and harmed 192. Bridge questions drove the effect, with a 28.7-point increase; comparison questions showed a smaller 4.2-point difference. In a prespecified, evaluation-only diagnostic population, real relationships also outperformed random-neighbor and degree-preserving shuffled-graph controls. The result is straightforward: under the same context budget, complete-evidence retrieval depends not only on which items rank highest, but on how context is allocated around them. Relational allocation recovered complete evidence sets that flat top-k retrieval left incomplete.

1 Introduction

With matched object and token ceilings, this study tests whether bounded graph-adjacent allocation recovers complete HotpotQA support sets more often than flat-ranked allocation. Across 7,405 FullWiki questions, Primary relational allocation improved complete supporting-evidence recovery by 23.8 percentage points, with the effect concentrated in bridge questions.

  • Motivation: A fixed context ceiling makes retrieval an allocation problem because every admitted item displaces another candidate.Ranking selects candidates, but a separate rule determines how the bounded context is spent.
  • Experimental question: The matched experiment asks whether bounded graph-adjacent allocation recovers complete support sets more often than additional flat-ranked items under identical object and token ceilings.Both routes share the relevance-ranked candidate list and terminate at the same allowed context ceiling.
  • Main result: 23.8 percentage points: Primary relational allocation increased complete supporting-evidence recovery over its matched flat baseline across 7,405 questions.The paired risk difference was 0.2377, with a 95% paired question-level bootstrap interval from 0.2269 to 0.2489.
  • Stratified result: 28.7 percentage points: the difference among 5,918 bridge questions exceeded the 4.2-point difference among 1,487 comparison questions.The comparison estimate was positive but inconclusive under the adjusted practical-margin rule; the Primary allocation harmed 192 baseline-recovered questions.
  • Scope: The result concerns complete support recovery in a frozen experiment, not downstream answer accuracy, model learning, or production retrieval-system performance.The matched flat baseline is an experimental control rather than a stand-in for learned path retrieval, GraphRAG, agentic search, web retrieval, or modern production RAG.

2 Related Work and Field Position

Study A positions bounded relational allocation as a distinct, controlled design variable after common relevance ranking, rather than claiming structure universally beats ranking. Its matched contrast measures complete support recovery while remaining separate from learned retrieval, generation, and GraphRAG systems.

  • Controlled allocation contrast: Both allocations begin from the same frozen relevance-ranking stage; the experiment changes post-ranking context composition, not candidate scores.This separates individual candidate relevance from the finite combination of items selected for downstream context.
  • Relation to learned retrieval: Prior graph-aware and multi-hop dense retrieval establish related relational mechanisms, but Study A neither reproduces nor directly compares against those learned systems.Its relational allocation is frozen, with different budgets, graph construction, endpoints, and evaluation scope; the result therefore does not establish superiority over learned path or dense retrieval.
  • Relation to generation systems: Study A evaluates evidence recovery rather than generated-answer accuracy, and its frozen HotpotQA support graph is a different research object from GraphRAG’s entity graphs, summaries, and answer workflow.A generator may ignore or misuse available evidence, so support recovery is not interchangeable with end-to-end generation outcomes.
  • Study A’s field position: 23.8 percentage points: bounded relational allocation recovered complete official support sets more often than additional flat-ranked context under matched object and token ceilings.The contribution is framed as a matched empirical contrast that makes the allocation decision visible, not as a new relational-retrieval primitive.
  • Claim boundaries: Under this frozen design, allocation around relevant seeds was consequential; the identified contribution is a measurable post-ranking design variable, not the broad claim that structure beats ranking.The endpoint is strict complete support-set recovery, while the observed effect is presented as a strong empirical claim with modest novelty.

3 Study Design

The study evaluated paired retrieval allocations on 7,405 frozen HotpotQA FullWiki questions, using complete recovery of each question’s official supporting-evidence set as the binary endpoint. Relational and flat arms shared relevance ranking and matched maximum object and token ceilings, while prespecified diagnostics and multiplicity-adjusted decision rules bounded interpretation.

  • Evaluation population: 7,405 FullWiki questions formed the frozen evaluation population, including 5,918 bridge and 1,487 comparison questions.Evidence had to be located in a resident Wikipedia-derived corpus; the benchmark did not test open-world discovery.
  • Primary endpoint: Complete recovery scored one only when retrieved context contained the entire official support set, distinguishing the endpoint from partial recall, F1, and answer metrics.The binary endpoint deliberately treats retrieving all but one support item the same as retrieving none.
  • Retrieval comparison: The relational arm used relevance-selected seeds plus bounded graph-adjacent context, whereas the matched flat baseline used additional ranked items under common maximum object and token ceilings.Thus, the primary contrast was context allocation rather than an arbitrarily larger context window.
  • Diagnostic population: 3,562 questions formed a separate opportunity population defined after retrieval outputs were immutable using official support and reachability in the real graph.This gold-derived, real-graph-conditioned population supported evaluation-only diagnostics rather than deployment selection.
  • Analysis and decision rule: Risk difference was computed as relational mean binary recovery minus matched-flat recovery on the same questions, with improve, tie, and harm counts also reported.The primary decision rule required the multiplicity-adjusted lower confidence bound to exceed the practical margin of +0.03; the Primary adjusted interval was [0.224173, 0.251047].

4 Results

Primary DBRR substantially increased complete supporting-evidence recovery over matched flat retrieval across 7,405 questions, with the largest benefit among bridge questions. Real relationships also outperformed neutral graph controls in prespecified full-population and evaluation-only diagnostic comparisons, while sensitivity analyses remained positive.

  • Primary result: 0.237677 paired risk difference favored Primary DBRR over the matched flat baseline across 7,405 questions (95% interval 0.226874 to 0.248886).Primary recovered complete support on 4,315 questions (58.3%), versus 2,555 (34.5%) for the matched flat baseline.
  • Primary result: 1,952 questions improved, 5,261 tied, and 192 were harmed under Primary DBRR.Improvements substantially outnumbered harms, but the harm cases rule out a uniformly beneficial effect.
  • Heterogeneity: 0.286921 paired risk difference favored Primary DBRR on bridge questions, compared with 0.041695 on comparison questions.Bridge questions had 1,789 improvements and 91 harms; comparison questions had 163 improvements and 101 harms, with the latter classified as INCONCLUSIVE after adjustment.
  • Graph controls: 0.165132 and 0.308602 risk differences separated real relationships from random-neighbor and degree-preserving shuffled-graph controls, respectively, across all 7,405 questions.In the evaluation-only opportunity population, the corresponding risk differences were 0.342532 and 0.637339, with both contrasts receiving WIN dispositions.
  • Interpretation and limits: The evaluation-only opportunity diagnostic strengthens conditional specificity but cannot represent expected deployment performance because it was selected using official support and real-graph reachability.The larger diagnostic effects should not replace the unconditional 0.237677 full-population estimate.
  • Sensitivity analyses: 0.221202 and 0.205537 paired risk differences remained positive for Sensitivity A and Sensitivity B against their own matched baselines.Both sensitivity comparisons received WIN dispositions.

5 Discussion

The discussion argues that context allocation is a consequential retrieval decision: under matched ceilings, bounded relational context improves complete support recovery over additional flat-ranked items. It also limits that conclusion, emphasizing heterogeneous effects, unresolved selection criteria, and the need for downstream and broader comparative evaluation.

  • Interpretation: 23.8-point paired difference favored relevance-selected seeds plus bounded graph-adjacent context over additional flat-ranked items under a matched allowance.The discussion argues that context allocation should not be treated as an implementation detail.
  • Interpretation: A controlled design held the common ranking stage and permitted context ceilings fixed while varying allocation, isolating allocation’s independent contribution without identifying the best production system.Complete support recovery measures evidence availability, not guaranteed answer correctness.
  • Heterogeneity and limits: The aggregate advantage was much larger for bridge questions, while comparison questions were practically inconclusive and included nearly as many harms as improvements.Relational expansion may help when initially relevant items connect to later support that flat ranking does not independently prioritize, but this mediator was not directly measured.
  • Control analyses: Real relationship allocation exceeded both random-neighbor and degree-preserving shuffled-graph controls under the frozen design.The gold-derived opportunity diagnostic strengthens the separation but cannot establish full causal decomposition or deployable opportunity recognition.
  • Implications: Retrieval evaluations should report context allocation explicitly, because systems sharing a nominal ceiling can expose downstream readers to different complete evidence sets.The study motivates measuring allocation choices rather than prescribing a universal graph method or judging untested systems.
  • Future work: Follow-up studies should test paired contexts with matched readers, compare modern retrieval alternatives under declared budgets, and evaluate selective policies by recoveries, displacement, harms, and cost.Reporting should separate ranking, context composition, support recovery, answer behavior, latency, and construction cost.

6 Limitations, Reproducibility, and Disclosure

The study’s evidence is limited to support-set recovery on one frozen HotpotQA FullWiki corpus and graph, with heterogeneous effects and no demonstrated downstream or general superiority. Independent sealed-result recomputation supports numerical consistency, but public materials do not permit full executable replication or operational-cost assessment.

  • Scope and external validity: Study A measures complete official support-set recovery, not answering correctness, citation, learning, belief revision, false-revision resistance, or discovery beyond the resident corpus.Any downstream benefit remains a hypothesis for a separately designed experiment.
  • Scope and external validity: The matched flat baseline isolates allocation after common ranking, but the study does not compare learned path retrieval, GraphRAG, agentic search, web retrieval, or production RAG.Results come from one frozen HotpotQA FullWiki design and one frozen relationship graph, so general superiority is outside the evidence.
  • Scope and external validity: HotpotQA’s constructed support annotations and fixed Wikipedia-derived corpus limit transfer to domains with different redundancy, relationships, document granularity, graph completeness, or unavailable evidence.The effect may change for temporal, adversarial, proprietary, or absent-from-corpus evidence.
  • Heterogeneity and selection: 192 questions were harmed by the Primary allocation, while comparison effects did not clear the adjusted practical-margin rule, precluding a monotonic-benefit interpretation.Sensitivity A and Sensitivity B retained positive aggregate effects but contained more harm cases than the Primary surface.
  • Heterogeneity and selection: Topology interactions cannot establish per-query routing: one hub-oriented regime was empty, the neighborhood-growth stratum contained only 10 questions, and aggregate effect modification requires prospective evaluation.The positive point estimate for the 10-question stratum was inconclusive.
  • Reproducibility and transparency: Independent sealed-result recomputation verified reported results and inferential surfaces, but missing source corpus, index, deterministic allocation details, and reconstruction fixtures prevent unconstrained public replication.Public materials support auditing result identity through authorized verification records, not full implementation stress-testing.
  • Operational and inferential limits: Budget matching covers frozen maximum object and token ceilings, not realized context use, latency, training, index or graph-construction cost, energy, maintenance, or production complexity.Graph identity and aggregate counts establish the tested result surface, not economic or operational superiority.
  • Operational and inferential limits: Bootstrap intervals, Holm adjustment, and the practical margin address prespecified frozen designs and families, not every uncertainty from benchmark construction, graph choice, resolver behavior, or transfer.Their numerical precision therefore answers narrower questions than it may suggest.

7 Conclusion

Under a fixed retrieval budget, complete supporting-evidence recovery depends on allocation as well as ranking. Study A found that bounded graph-adjacent context improved recovery over additional flat-ranked context, while the result remained specific to the tested design and did not establish downstream answer gains or universal graph benefits.

  • Conclusion: Allocation determines which ranked evidence and surrounding material enter the final context after ranking identifies promising evidence.Study A tested this allocation decision directly.
  • Conclusion: 23.8 percentage points: allocating the same maximum retrieval units and tokens to relevance-selected seeds plus bounded graph-adjacent context increased complete supporting-evidence recovery over additional flat-ranked context across 7,405 questions.The advantage was concentrated in bridge questions, while the comparison result was practically inconclusive.
  • Conclusion: Retrieval quality depends on how a finite context budget is allocated around material the system has already found relevant, not only on which items score highly.The study’s conclusion is about retrieval quality under bounded context, rather than downstream model performance.
  • Conclusion: The study does not show that downstream models answered better, that every graph expansion helps, or that the tested method outranks modern retrieval systems.These boundaries make the result specific to the tested retrieval design.
  • Conclusion: Once context is bounded, allocation becomes part of retrieval and should be designed, measured, and challenged rather than treated as neutral.A strong ranking can still leave the final context evidentially incomplete, whereas a modest relational step can recover evidence beyond the ranking boundary under the same finite allowance.

Declarations

The paper reports its data-access limits, independent verification scope, AI-assisted editorial support, and patent disclosure. It distinguishes sealed-data recomputation from a public source-to-result rerun and separates patent pendency from scientific validation.

  • Data availability: The paper cites the official HotpotQA acquisition, format, and license, reports sanitized hashes and aggregate graph counts, and withholds corpus bytes and reconstruction-capable materials.It does not distribute raw or processed corpus bytes, private retrieval traces, reconstruction-capable fixtures, or custody coordinates.
  • Verification: An independent verifier rehashed the immutable package and recomputed headline results and diagnostics from sealed evaluation data, but could not perform a public source-to-result rerun.The source corpus was not included.
  • Author contributions: AI-assisted tools supported drafting, source organization, review, copyediting, and assembly, while Thomson D. Nguy retained scientific conception, authorship decisions, and responsibility for the manuscript’s claims.The tools operated under the author’s direction.
  • Competing interests: The paper discloses its patent relationship and states that patent pendency is not scientific validation or evidence for scope, validity, infringement, licensing, or issued rights.The relationship is disclosed in Section 6.7.
Loading 2608.18448v1…