Source-linked AI summary

BoundaryMORPH: Budgeted Reranking via Active Set Selection for Diffuse Retrieval

Eylon Caplan, Shamik Roy, Shib Sankar Dasgupta, Yingfan Wang, Rashmi Gangadharaiah

arXiv:2609.27213v1cs.IR

TL;DR

Diffuse RAG queries require selecting many relevant documents for a finite context, while cross-encoder scoring is limited and may cover fewer documents than the context can hold. BoundaryMORPH uses a Gaussian Process with a MIPS prior to target uncertain top-k membership, and the authors report gains over baselines across benchmark configurations. The method is bounded by sequential execution, a discrete document-capacity assumption, and cubic scaling at extreme budgets.

  • Problem

    Diffuse queries need many relevant documents, but cross-encoder scoring is costly and its budget can be smaller than the context capacity.

    Source: §1 · p. 2
  • Method

    BoundaryMORPH uses a Gaussian Process with a MIPS prior to propagate CE information and target uncertain decisions at the top-k inclusion boundary.

    Source: §1 · p. 2, §3.2 · p. 5, §3.3 · p. 5
  • Results

    Across all dataset × B × k combinations, BoundaryMORPH outperforms baselines, with an average +3.8 pp nCG@100 over the second-best baseline at higher k values.

    Source: §5.1 · p. 7
  • Takeaways & Limitations

    By avoiding redundant measurements of securely included documents, BoundaryMORPH can capture diverse semantic peaks in multimodal queries.

    Source: §7 · pp. 9–10

    BoundaryMORPH currently relies on sequential execution, assumes a discrete document capacity k rather than strict token limits, and scales cubically with extreme budgets.

    Source: §7 · p. 10

Abstract

from arXiv · show

Open-ended queries in modern Retrieval-Augmented Generation (RAG) are increasingly "diffuse," requiring a large set of documents to be assembled into a finite LLM context window. To ensure retrieval quality, systems use fast dual-encoders and more expensive cross-encoders (CEs) to score candidates. However, the CE budget $B$ is strictly bounded by latency and is often smaller than the context window capacity $k$. This mismatch makes standard reranking structurally flawed: it wastes compute verifying obvious top candidates while ignoring relevant documents further down the initial ranking. To address this, we introduce BoundaryMORPH, a novel algorithm that allocates CE budget specifically for the LLM's context capacity $k$. Using a Gaussian Process, BoundaryMORPH treats the initial dual-encoder ranking as a structural prior and intelligently spends CE calls on resolving top-$k$ set membership at the boundary, rather than seeking a single most-relevant document. Information from each CE call propagates to unscored documents, maximizing the utility of the budget. We demonstrate that BoundaryMORPH achieves state-of-the-art set retrieval quality across multiple models and datasets with open-ended queries ($+5.4$ nCG@100 over the strongest baseline).

1 INTRODUCTION

BoundaryMORPH frames diffuse RAG retrieval as budgeted top-k set selection: a limited number of cross-encoder calls must inform which documents fill a larger context. It uses a Gaussian Process and MIPS ranking to focus calls on uncertain inclusion decisions, and the authors report gains across models, metrics, and multimodal queries.

  • Diffuse queries need many relevant documents gathered from across the ranking, rather than just one or a handful, for a strong response.The introduction gives “Effects of light pollution?” as an example expected to surface tens or hundreds of relevant documents.Source: §1 · p. 2
  • When the CE budget B is smaller than context capacity k, reranking only the top B candidates can miss relevant documents farther down and leaves other context slots unevaluated.The paper describes this as a resource-allocation problem: decide where to spend CE calls and which documents to return.Source: §1 · p. 2
  • BoundaryMORPH uses a Gaussian Process to estimate the CE relevance landscape, allowing each CE call to inform more documents than the number directly scored.The introduction states that information from each call propagates to unscored documents.Source: §1 · p. 2
  • The method preserves the MIPS ranking as a structural baseline and targets the top-k inclusion boundary instead of confirming documents already securely within the context.The paper argues that uncertainty about inclusion lies at the boundary, where a call can change which documents are delivered to the generator.Source: §1 · p. 2
  • BoundaryMORPH sets a GP prior mean from each document’s MIPS percentile rank and spends calls on contested boundary gaps.This lets the estimator fall back to the dual encoder in unscored regions while adjusting its estimates around observed CE scores.Source: §1 · p. 2
  • The paper frames RAG document selection as a budgeted top-k set-selection problem, rather than conventional reranking for a single most-relevant document.Its stated contributions include a parameter-free selector using a MIPS prior and standardized boundary gap.Source: §1 · pp. 2–3
  • BoundaryMORPH outperforms active-selection and single-stage reranking baselines at equal budgets across embedding–cross-encoder pairs, with gains on human relevance and cross-encoder fidelity.The authors also report that its advantage grows on highly multimodal queries, where it captures documents from secondary semantic peaks.Source: §1 · p. 3

2 PROBLEM SETTING

The problem setting treats diffuse RAG retrieval as selecting a fixed-size set from a candidate pool when true relevance is unknown and accurate cross-encoder scoring is costly. It therefore couples the choice of documents to score with the choice of documents to return.

  • Diffuse queries require more than a single or handful of relevant documents, and the system selects a fixed-size set S of k documents from candidate pool C.The setup distinguishes a cheap retriever that narrows the corpus from the more accurate but more expensive cross-encoder.Source: §2 · p. 3
  • Because true relevance is unknown and only B ≪ N cross-encoder calls are affordable, the system must choose which B documents to score so the returned k are as relevant as possible.If relevance were known, returning the k most relevant documents would be straightforward.Source: §2 · p. 3
  • The setting assumes a known document capacity k and treats retrieval as two decisions: where to spend B cross-encoder calls and which k documents to return.Arranging selected documents within the context is treated as a separate problem.Source: §2 · p. 3

3 METHOD

BoundaryMORPH uses a Gaussian process to preserve the dual-encoder ranking as a prior and spend cross-encoder calls resolving uncertain top-k membership rather than maximizing one score. Its posterior propagates information from scored documents while reverting to the prior far from observations.

3 METHOD

  • BoundaryMORPH specializes a GP for known-capacity selection by customizing its starting estimate and next-document acquisition rule.Figure 2 contrasts this boundary-focused approach with a standard GP that repeatedly scores obvious top documents.Source: §3 · p. 3, Figure 2 · p. 4

3.1 PRELIMINARIES: GAUSSIAN-PROCESS SURROGATES

  • A GP posterior updates means and uncertainty across documents from observations, with local effects and prior reversion far from evidence.The kernel determines how observations constrain other documents; scored-document uncertainty collapses and nearby uncertainty is reduced.Source: §3.1 · p. 4

3.2 THE MIPS PERCENTILE PRIOR

  • The GP models cross-encoder relevance residuals against each candidate’s MIPS percentile rank, falling back to that scale-free prior where unobserved.Calls locally bend the MIPS ranking rather than rebuild it; the residual is near zero across most of the pool.Source: §3.2 · p. 5

3.3 THE BOUNDARY ACQUISITION RULE

  • The selector partitions the posterior-mean top-k into incumbents and challengers, then targets the pair with the smallest standardized boundary gap.It spends the call on the pair member with higher uncertainty, focusing on the most-threatened incumbent or strongest challenger rather than secure inclusions or exclusions.Source: §3.3 · p. 5
  • Building the kernel costs O(N^2) and posterior updates cost O(B^3N), while cross-encoder inference at O(B) dominates measured method latency.The passage attributes the update cost to matrix inversion.Source: §3.3 · p. 5

4 EXPERIMENTAL SETUP

The evaluation uses filtered diffuse-query datasets and a fixed 10,000-document candidate pool, measuring selected-set relevance with human and cross-encoder metrics against tuned reranking and active-selection baselines.

4.1 DATASETS

  • The evaluation covers NeuCLIRBench-monolingual-eng, Robust04, and TravelDest, retaining queries with at least 20 relevant documents and the top 10,000 MIPS candidates.The retained query counts are 85, 187, and 99, with median relevant-document counts of 51, 62, and 462, respectively.Source: §4.1 · p. 6

4.2 METRICS

  • nCG@k measures human relevance in the selected k-set against the optimal set, while CE-nCG@k evaluates fidelity to CE scores; nDCG and MAP are considered unsuitable for context selection.nCG@k is the primary metric because relevant documents often exceed k and the generator receives a set of k documents.Source: §4.2 · p. 6

4.3 METHODS

  • Comparisons include BM25, single-stage top-B reranking, BAGEL’s zero-prior GP-UCB, and RGS’s kNN graph selection, with baseline hyperparameters tuned per budget.The stated dual-encoder and cross-encoder are multilingual-e5-large and zerank-2, respectively.Source: §4.3 · p. 6

5 RESULTS AND ANALYSIS

The evaluation compares set selection against existing selectors, tests the necessity and capacity sensitivity of BoundaryMORPH’s components, and investigates the sources of its gains.

5.1 MAIN RESULTS

  • +3.8 pp nCG@100 and +7.3 pp CE-nCG@200 over the second-best baseline at higher k; BoundaryMORPH wins every configuration, and CE gains carry over to human judgments.BAGEL is the clear second-best method.Source: §5.1 · p. 7, Table 1 · p. 7

5.2 ABLATIONS AND SENSITIVITY

  • Removing boundary acquisition lowers nCG@k by 1.5–2.6 percentage points, while removing the MIPS prior lowers it by 2.1–3.8 points.Both component removals reduce performance across per-dataset and CE-nCG cells as well.Source: §5.2 · p. 7, Table 2 · p. 7
  • For nearly all evaluated capacities, the run optimized for that same k performs best, showing BoundaryMORPH exploits its known context capacity.Figure 3(a) compares CE-nCG at evaluated capacities across runs optimized for different capacities.Source: §5.2 · p. 7, Figure 3 · p. 8

5.3 SOURCES OF IMPROVEMENT

  • BoundaryMORPH scores fewer documents it ultimately returns, redirecting calls from secure top-k members toward contested boundary gaps and secondary relevance modes.BAGEL spends more calls near the top, while the pure reranker can only return scored documents.Source: §5.3 · p. 8
  • On NeuCLIR, multimodal mass predicts per-query advantage over BAGEL (Spearman ρ = +0.59, p = 3.5×10−9); the illustrated multimodal query has five coherent peaks versus one for Amazon deforestation.The peaks correspond to distinct subtopics, and an LLM audit confirms their semantic coherence.Source: §5.3 · p. 8, Figure 4 · p. 9

6 RELATED WORK

Prior work covers adaptive budgeted reranking, top-k membership estimation, offline corpus optimization, and agentic retrieval; BoundaryMORPH is positioned as a budgeted selector within this landscape.

Budgeted reranking

  • Many adaptive retrieval methods expand neighborhoods or halt reranking over a fixed order, but are value-seeking and cannot estimate unscored documents; AcuRank updates beliefs independently, while BAGEL uses a pool GP.BAGEL differs through deterministic warm-starting, a zero prior mean, and maximum-seeking GP-UCB.Source: §6 · p. 9

Budgeted Selection Goals

  • Level-set estimation, active set ordering, and LUCB-style bandits address threshold or top-k membership decisions; the paper frames budgeted retrieval as active set ordering [Bryan et al., 2005; Gotovos et al., 2013; Nguyen et al., 2024; Kalyanakrishnan et al., 2012; Jiang et al., 2017; Pony et al., 2026].Source: §6 · p. 9

Indexing-time methods and agentic retrieval

7 CONCLUSION

The conclusion presents BoundaryMORPH as a budget-aware selector for diffuse RAG that targets the top-k boundary using a MIPS-prior Gaussian Process, and notes system and algorithmic limits.

Conclusion

  • BoundaryMORPH targets the top-k decision boundary with a Gaussian Process and MIPS prior, achieving state-of-the-art selection quality over maximum-seeking baselines.The conclusion links avoiding redundant primary-topic measurements with navigating multimodal queries to capture diverse semantic peaks.Source: §7 · pp. 9–10

Limitations

  • The method currently runs sequentially, assumes discrete document capacity k rather than strict token limits, and scales cubically with extreme budgets.Source: §7 · p. 10

Disclosure

  • Generative AI tools assisted with writing, literature discovery and summarization, and coding; the authors report manual review and responsibility for the final content.Source: AI USE STATEMENT · p. 10

Reproducibility

  • Appendix B provides method details and pseudocode, Appendix C provides experiment replication details, and an anonymized implementation is included.Source: REPRODUCIBILITY STATEMENT · p. 10

A LIMITATIONS

BoundaryMORPH’s limitations concern sequential execution, scaling at extreme budgets, its document-count capacity assumption, and its focus on diffuse-query set selection. The appendix describes residual-GP conditioning, the acquisition loop, and their computational costs.

A LIMITATIONS

  • Sequential posterior updates require one-at-a-time cross-encoder calls, which can limit hardware utilization in heavily parallelized production settings.The authors note that single-stage reranking can amortize latency through massive batching; selecting batches with the boundary rule is future work.Source: A LIMITATIONS · p. 15
  • The exact GP update scales as O(B^3N), remaining lightweight at B ≤200 but taking over 30 seconds at B = 500.If cross-encoder inference becomes much cheaper, GP overhead will account for a larger share of latency.Source: A LIMITATIONS · pp. 15–16
  • Capacity is modeled as a document count k, a practical proxy when chunk lengths are narrow, but variable-length chunks remain an adaptation challenge.Real context windows are token-bounded; the authors cite chunking practice as support for the discrete-count proxy.Source: A LIMITATIONS · p. 16
  • The method targets diffuse, multi-source queries; for single-document factoids with k = 1, its exploratory benefit does not apply and maximum-seeking suffices.The method selects which documents enter the context, not their within-context ordering; optimal arrangement remains an open question.Source: A LIMITATIONS · p. 16

B.1 ALGORITHM PSEUDOCODE

  • The posterior routine fits a GP to CE-score residuals against the MIPS prior, standardizes residuals before conditioning, then de-standardizes its predictions.The pseudocode returns a posterior mean and variance for the candidate pool.Source: §B.1 · p. 16

B.2 TIME COMPLEXITY

  • Kernel construction costs O(N^2); posterior updates total O(B^3N), while boundary acquisition costs O(Bk(N − k)) over the run.At the budgets studied, sequential CE inference, rather than GP bookkeeping, dominates wall-clock latency.Source: §B.2 · pp. 16–17
  • The outer loop repeatedly selects the less-certain document from the closest incumbent–challenger boundary pair, spends one CE call, and updates the posterior.After B calls, it returns the top-k candidates by posterior mean, using true CE scores for scored documents.Source: §B.2 · p. 17

B.4 KERNEL ABLATION

The kernel ablation finds no significant, consistent performance benefit from replacing vMF, while holding acquisition and prior fixed. Its alternatives use the same cosine geometry and bandwidth-selection procedure.

B.4 KERNEL ABLATION

  • The vMF, RBF, and Matérn-5/2 kernels share cosine-based geometry but assume different covariance-decay rates and smoothness of the CE surface.Their length scales use the same median heuristic, with the remaining GP settings held unchanged.Source: §B.4 · pp. 17–18
  • The ablation varies only the kernel for BoundaryMORPH and BAGEL, averaging each query’s results over B ∈{25, 50} before paired testing.BAGEL’s β0 is reselected for each kernel, and both nCG and CE-nCG are reported.Source: §B.4 · p. 18
  • Kernel choice has no significant effect: all 72 deltas are below 1.5 pp, with a 0.32 pp median, and only five corrected tests survive, all for BAGEL.The five surviving effects move BAGEL in both directions; none is large enough to matter, and none occurs for BoundaryMORPH.Source: §B.4 · p. 18, Table 3 · p. 18

C.1 DATASET STATISTICS

The appendix describes three diffuse-query testbeds and shared-pool baselines, then reports ablation and model-robustness results. BoundaryMORPH’s advantage persists across tested embedder–cross-encoder combinations, with larger relative gains for a weaker retriever.

C.1 DATASET STATISTICS

  • NeuCLIRBench covers open-ended information needs over translated newswire, Robust04 uses TREC topics over newswire and government documents, and TravelDest covers broad travel-intent topics.The datasets differ in relevance judgments and pool coverage; the reported qrels contain 5,400, 14,351, and 49,822 relevant in-pool documents, respectively.Source: §C.1 · p. 19

C.2 DETAILED BASELINES

  • Baselines share a fixed top-10,000 MIPS pool; single-stage reranking returns the MIPS top-k when k ≥ B, while BAGEL uses a zero-prior GP-UCB approach.Other tested comparators include BM25, graph-based RGS, and a multi-hop agentic retrieval loop.Source: §C.2 · p. 20
  • BoundaryMORPH reaches nCG@100 0.621 in 25.7 seconds at B = 100, while none of 29 agentic-loop configurations matches its quality or cost on NeuCLIR.The loop’s nCG@100 ranges from 0.554 to 0.618 and its latency from 57.6 to 1,922.8 seconds.Source: §C.2 · p. 21

D.1 FULL ABLATION RESULTS

  • Removing either the MIPS prior or boundary acquisition lowers every reported dataset-and-metric cell in the full ablation.Table 8 reports per-dataset changes from full BoundaryMORPH for nCG@k and CE-nCG@k.Source: §D.1 · p. 21, Table 8 · p. 22

D.2 ROBUSTNESS TO RETRIEVER AND CROSS-ENCODER

  • At B = 50 and k = 100, BoundaryMORPH maintains a gap over baselines on every dataset across all four embedder–CE combinations.The robustness evaluation uses Qwen3-Embedding-4B or multilingual-e5-large paired with Qwen3-Reranker-8B or zerank-2.Source: §D.2 · p. 21, Figure 7 · p. 22
  • Mean lift over single-stage reranking rises from +3.4 pp with a strong retriever and strong CE to +14.0 pp with a weaker retriever.Switching from the weaker e5 embedder to Qwen3 improves nCG@100 at B = 50 for all methods: +10.6 for SS, +8.6 for RGS, and +4.9 for BoundaryMORPH.Source: §D.2 · p. 21

D.3 PER-DATASET EMBEDDER × CE RESULTS

  • The per-dataset results report nCG@100 lift over single-stage reranking in percentage points at B = 50 for each of the four embedder–CE combinations.The supplied description specifies the comparison dimensions, but not the individual cell values.Source: §D.3 · p. 22

D.4 LATENCY ANALYSIS

BoundaryMORPH nearly Pareto-dominates in the latency comparison, while methods converge at high CE budgets. The analysis also reports component costs, batching constraints, and a hypothesis about why the agentic loop falls behind.

D.4 LATENCY ANALYSIS

  • On one GPU, BoundaryMORPH and BAGEL run sequentially while SS can be batched; the reported SS setting uses batch size 1 because padding increased latency at larger sizes.With enough GPUs and memory, SS latency could theoretically improve linearly with GPU count.Source: §D.4 · p. 23
  • The agentic loop is Pareto-dominated at every latency it reaches, with latency growing close to linearly in hops due to autoregressive decoding.The authors hypothesize that GP propagation is more efficient than passing relevance information through bounded natural-language notes.Source: §D.4 · p. 23
  • BoundaryMORPH nearly Pareto-dominates, but methods converge when B >> k because selection matters less once plausible documents can be scored.The authors note that results depend on document length and hardware.Source: Figure 8 · p. 23, §D.4 · p. 23

D.5 LATENCY SETUP

  • CE accounts for 74–95% of latency; its share is highest at B = 25 and B = 50 (95%) and falls to 74% at B = 500 as GP update cost grows.The cited table reports median per-query component latency on NeuCLIR, with N = 10,000, capacity 50, and n = 85.Source: §D.5 · p. 23, Table 9 · p. 23
  • The algorithm’s own cost is under 1.5 s for B ≤ 100 and reaches 32.86 s at B = 500, driven almost entirely by GP updates; per-call CE cost is ∼0.19 s.Absolute times are hardware-specific; the setup used one A10G, and the passage identifies ratios as the claim.Source: §D.5 · pp. 23–24

E QUERY MULTIMODALITY

  • The analysis distinguishes unimodal from multimodal diffuse queries and defines multimodal mass as a geometric scalar to measure multimodality.The authors expect multimodal queries to be especially difficult for max-seeking selectors, which concentrate calls on the primary mode.Source: E QUERY MULTIMODALITY · p. 24

E.1 PEAK EXTRACTION AND LLM AUDIT

The peak audit tests whether geometric peaks in CE relevance correspond to semantic distinctions. Across audited peak pairs, the LLM’s partitions generally align with the geometric peak labels.

E.1 PEAK EXTRACTION AND LLM AUDIT

  • Peaks are extracted from CE scores over a cosine kNN graph; threshold sweeps form connected components, and persistence-based merging yields a partition of the top 1,000 documents.The graph uses knn = 10 and the partition is simplified at τ = 0.20.Source: §E.1 · p. 24
  • For each eligible peak pair, the audit samples documents, asks an LLM judge to split them by relevance aspect, and scores the partition against peak-of-origin labels using adjusted Rand index.The protocol samples up to eight documents from each peak’s 0.5-core, falling back to the full basin when the core has fewer than three.Source: §E.1 · p. 24
  • 85.1% of audited peak pairs beat chance, 62.7% reach ARI ≥0.5, and half are separated perfectly by the LLM judge.The audit covers 496 pairs from 65 multi-peak NeuCLIR queries.Source: §E.1 · p. 25
Loading 2609.27213v1…