PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Marginal Viewpoint Utility for Budgeted Evidence Selection in RAG

Yiyang Duan, Ziqi Yu, Tiantong Wu, Yebo Wu, Wei Yang Bryan Lim

arXiv:2610.29234v1cs.IR

TL;DR

Budgeted RAG contexts may omit distinct answers or positions despite containing relevant evidence. LMVU learns to rank candidates by their marginal viewpoint coverage and selects them sequentially without runtime access to viewpoint labels. Across four benchmarks, it improves viewpoint coverage over baselines and reaches the exact breadth ceiling on two benchmarks.

  • Problem

    A limited RAG context may contain relevant evidence while leaving different answers or positions unrepresented, raising the question of how to cover viewpoints within the budget.

    Source: §1 · p. 1

  • Method

    LMVU trains a lightweight ranker on development-split viewpoint annotations to predict each candidate’s added coverage in the current selection, then selects budget-feasible candidates sequentially without labels at inference.

    Source: §1 · p. 2

  • Results

    Across four benchmarks, LMVU improves viewpoint coverage over MMR on three, reaches the exact coverage ceiling on WVS, and matches saturated ArgKP.

    Source: §6 · p. 11

  • Takeaways & Limitations

    Exact coverage oracles reveal recoverable gaps within the same evidence pools and budgets, identifying evidence selection as an opportunity to improve viewpoint coverage.

    Source: §1 · p. 2

    Performance depends on the embedding pipeline and viewpoint annotations, the full-pool extension is exploratory, and the learned scorer has no coverage approximation guarantee.

    Source: §6 · p. 11

Abstract

from arXiv · show

Retrieval-augmented generation over multi-viewpoint queries must preserve distinct answers and positions from the retrieved evidence while operating under a fixed context budget. We introduce LMVU, a lightweight ranker that learns to prioritize candidates by the marginal viewpoint coverage they add to the current selection. Training uses viewpoint annotations from each benchmark's development split to compare candidates within the same selection state. At inference, LMVU repeatedly scores budget-feasible candidates as the context grows, without accessing these annotations. We evaluate two forms of recovered viewpoint coverage: RVC$_1$ assigns each viewpoint full credit for at least one supporting source unit, while RVC$_2$ assigns half credit for one and full credit for two or more distinct source units. Exact oracles measure achievable coverage under the same evidence pool and budget. Across four viewpoint-annotated benchmarks at a 1,024-token budget, LMVU achieves gains of up to 0.047 and 0.052 points in RVC$_1$ and RVC$_2$, respectively, over the strongest non-oracle baseline for each benchmark and metric, and reaches the exact viewpoint-breadth ceiling on two benchmarks.

1 Introduction

The paper asks how to cover different viewpoints within a limited evidence budget and introduces LMVU, which ranks evidence by its predicted added viewpoint coverage. Across four benchmarks, it reports improved viewpoint coverage under a 1,024-token budget.

1 Introduction

  • A budget-limited context can contain relevant evidence yet omit viewpoints needed for a comprehensive response.The paper frames the question as how to cover different viewpoints within the limited evidence budget.Source: §1 · p. 1
  • LMVU learns a lightweight linear ranking rule from development-split viewpoint annotations, predicting each candidate’s added coverage given the selected context.At each step it adds the highest-scoring candidate that fits the remaining budget and updates scores after the addition.Source: §1 · p. 2
  • In a WVS example using the same pool and 1,024-token budget, MMR covers three viewpoints while LMVU covers all four.LMVU includes evidence for the corruption-free perception viewpoint that MMR leaves unrepresented.Source: §1 · p. 2
  • Across four benchmarks at 1,024 tokens, LMVU leads in RVC on WVS and ESS10, reaches 94.6% on Perspectrum, and matches full coverage on ArgKP.It also outperforms the strongest baseline on ESS10 under both reported metrics.Source: §1 · pp. 2–3
  • Exact coverage oracles reveal recoverable viewpoint gaps within the same candidate pools, making evidence selection an opportunity to improve coverage.The paper also reports that evaluation finds improvements in repeated support from distinct sources.Source: §1 · pp. 2–3

2 Related Work

Related work diversifies or selects evidence using relevance, similarity, proxy aspects, or learned ranking. LMVU instead learns state-conditioned rankings from marginal gains in annotated viewpoint coverage.

2 Related Work

  • Similarity-based and coverage-oriented selectors optimize relevance, similarity diversity, or proxy aspect coverage rather than supervised marginal gains in annotated viewpoint coverage.The paper notes that semantic diversity need not imply viewpoint coverage.Source: §2 · p. 3
  • Adaptive selectors vary redundancy penalties or use clusters, while neural sequential rankers use substantially larger models than LMVU’s linear scorer.The cited comparison says LMVU is lighter to train, but its frozen representations may not separate viewpoints adequately.Source: §2 · p. 3
  • Typical query-conditioned rerankers score candidates independently of the partial result set, whereas LMVU supervises the marginal viewpoint coverage added in the current state.Viewpoint labels train the scorer but are not used at runtime.Source: §2 · p. 3
  • The evaluation covers survey viewpoints, claim perspectives, and argument key points, focusing on which viewpoints enter the context under label-blind selection.The cited resources include WVS, ESS Round 10, Perspectrum, and ArgKP.Source: §2 · pp. 3–4

3 Problem Formulation

The formulation selects evidence from a query-specific pool to maximize recovered viewpoint coverage under a token budget. Gold annotations define coverage and evaluation targets, but are unavailable to the runtime selector.

3 Problem Formulation

  • Each evidence item has a token cost and supports a subset of the query’s annotated viewpoints; a selection covers the union of its items’ viewpoints.The basic coverage objective gives each viewpoint credit once, regardless of repeated support.Source: §3 · p. 4
  • The feasible context is a subset of the full evidence pool whose total token cost does not exceed the budget, with the objective of maximizing recovered viewpoint coverage.Source: §3 · p. 4
  • Gold viewpoint annotations support training and evaluation but are unavailable at inference, so the selector estimates gains from observable evidence features and the current state.A full-pool gold-access oracle measures the maximum coverage achievable under the same pool and budget; its gap to a reference allocator is headroom.Source: §3 · p. 4

4 Method: Learned Marginal Viewpoint Utility

LMVU learns a state-dependent ordering surrogate for marginal viewpoint gains, then uses it in a sequential, budget-feasible greedy allocator. Training uses within-state gain comparisons; inference is label-blind.

4.1 Marginal utility and training states

  • LMVU targets the marginal viewpoint gain of a candidate in the current selection state, where a candidate’s added coverage can shrink to zero as its viewpoints become covered.Development states and candidates are sampled from local pools, and unequal realized gains within a state provide training preferences.Source: §4.1 · pp. 4–5

4.2 State-conditioned scoring and learning

  • The linear scorer uses runtime-visible candidate and state features, and pairwise training compares unequal-gain candidates only within the same state.Differencing removes across-state gain-scale differences; the pairwise objective matches the allocator’s within-state ranking decision.Source: §4.2 · pp. 5–6

4.3 Sequential allocation

  • At inference, LMVU repeatedly scores budget-feasible candidates, adds the top-ranked item, updates the state, and stops when no item fits or the pool is exhausted.The loop guarantees budget feasibility but has no coverage approximation bound.Source: §4.3 · p. 6

5 Experiments and Analysis

Experiments compare LMVU with selection baselines and exact oracles across four viewpoint-annotated benchmarks, using coverage breadth and repeated support. LMVU leads on average, while its gains vary with state information, available headroom, and embedding pipeline.

5.1 Setup

  • The evaluation covers WVS, Perspectrum, ArgKP, and ESS10 at budgets of 512, 1,024, and 2,048 tokens, reporting breadth and repeated-support coverage against exact oracles.Training uses smaller frozen pools; selection uses the full evidence pools, and the protected WVS fold is not used for fitting.Source: §5.1 · pp. 6–7

5.2 Coverage under fixed budgets

  • Only the learned state-dependent utility improves over MMR on all three headroom-positive benchmarks; neural specialists also fail to lead across them.Heuristics are benchmark-dependent, and neither JPR nor RichRAG improves over MMR on all three benchmarks with positive headroom.Source: §5.2 · pp. 7, 9
  • 0.9588 RVC and 0.8795 RVC2 give LMVU the best four-benchmark averages among 16 non-oracle methods, ahead of MMR (0.9222) and FacLoc (0.8469).Table 1 reports the main results at a 1,024-token budget on the full per-query pools.Source: §5.2 · p. 7, Table 1 · p. 8

5.3 What does state conditioning contribute?

  • Against static M0, LMVU gains +0.202 on WVS and +0.061 on ESS10; shuffling state features collapses development ranking, implicating state information.The WVS and ArgKP development ranking also falls under the capacity-matched shuffled-state control.Source: §5.3 · p. 9

5.4 Cross-benchmark transfer

  • A model trained on the other three benchmarks improves over MMR on WVS (+0.150), Perspectrum (+0.044), and ESS10 (+0.037), and ties ArgKP at its ceiling.These leave-one-benchmark-out results use development folds only; WVS reaches full breadth (1.0000).Source: §5.4 · p. 10

5.5 Breadth and repeated support

  • LMVU’s gains over MMR in repeated-support RVC2 remain positive on all headroom-positive benchmarks (+0.119, +0.021, +0.052), but WVS retains a 0.0985 depth gap to the oracle.WVS reaches the pool optimum in breadth, while repeated support remains short of its depth oracle.Source: §5.5 · p. 10

5.6 Robustness across embedding models

  • With bge-base, LMVU beats MMR by 0.025 RVC on Perspectrum and 0.074 on ESS10, with intervals excluding zero; bge-m3 intervals cross zero.Each encoder evaluation rebuilds embeddings, pools, and features together, so it measures pipeline sensitivity rather than an encoder-only change.Source: §5.6 · p. 10, Table 2 · p. 10

5.7 Sensitivity and operating boundaries

  • At 1,024 tokens, realized gains track available headroom descriptively (Pearson r=0.574, n=4), and gains vanish where the ceiling is met.WVS gains narrow as budget rises, while ESS10 gains remain positive through 2,048 tokens.Source: §5.7 · p. 11, Figure 3 · p. 11

6 Discussion and Conclusion

The discussion concludes that LMVU’s state-dependent selection improves viewpoint coverage over MMR on three benchmarks and reaches the WVS ceiling, while its benefits vary by benchmark. The authors identify embedding and annotation dependence, exploratory evaluation, and the absence of an approximation guarantee as limitations.

6 Discussion and Conclusion

  • LMVU improves viewpoint coverage over MMR on three benchmarks, reaches the exact ceiling on WVS, and matches saturated ArgKP; state conditioning benefits WVS and ESS10, while candidate-only scoring is stronger on Perspectrum.The authors also report gains over MMR in repeated evidence support.Source: §6 · p. 11
  • Performance depends on the embedding pipeline and viewpoint annotations; the full-pool extension is exploratory, and the scorer has no coverage approximation guarantee.The authors call for downstream-generation evaluation and adaptation to domains requiring new viewpoint inventories.Source: §6 · p. 11

AI use statement

  • AI tools aided manuscript polishing and literature retrieval; the authors retain responsibility for verifying assisted text and cited sources.Source: AI use statement · p. 11

A Evaluation Protocol and Integrity

The protocol locks method choices before held-out evaluation and separates development training from gold-annotation access at runtime. Experiments use shared full selection pools, defined baseline rules, and paired comparisons with uncertainty reporting.

A Evaluation Protocol and Integrity

  • Method decisions were locked before held-out access; the protected WVS fold received a single-touch confirmation, and the full-pool extension is exploratory after prior held-out inspection.Runtime audits recorded zero gold-field accesses and zero external provider calls; the extension also refit learned rows and recomputed exact oracles on full pools.Source: A Evaluation Protocol and Integrity · pp. 14–15

B Experimental Setup

  • Selection uses each query’s full prequalified pool, while development training states come from smaller frozen MMR-oversampled pools; learned rankers train on within-state marginal-coverage comparisons.Splits are grouped by query, and the protected WVS evaluation fold is not used for fitting.Source: B Experimental Setup · p. 16

C Baseline Definitions

  • Baselines share the same full pools and token accounting but differ in per-step scoring, spanning relevance, diversity, allocation heuristics, and neural rerankers.The neural rankers score only the top-64 dense-relevant budget-feasible candidates at each step.Source: C Baseline Definitions · pp. 16–17

D Paired Statistics Protocol

  • LMVU–comparator differences use paired per-query RVC, bootstrap confidence intervals, win/tie/loss counts, and sign tests; WVS and ArgKP statistics are descriptive because each has only three independent groups.Perspectrum and ESS10 also receive Wilcoxon signed-rank checks.Source: D Paired Statistics Protocol · p. 17

E Formal Analysis

The appendix gives exact dynamic programs for budgeted viewpoint coverage and formalizes why runtime allocation cannot use true marginal gains. It also relates surrogate ranking quality to coverage under a unit-cost assumption and reports development-fold evidence.

E Formal Analysis

  • Coverage is monotone and submodular: adding evidence cannot reduce covered viewpoints, and a candidate's marginal gain cannot increase as the selection grows.Source: E Formal Analysis · pp. 17–18
  • The bitmask dynamic program returns exact budget-feasible full-pool coverage; with at most 512 states per query, the solve is negligible.Source: E Formal Analysis · p. 18
  • The repeated-support oracle is exact when pool items have distinct source units, using at most 19,683 states for the evaluated inventories.Source: E Formal Analysis · p. 18
  • Under a knapsack budget, raw-gain greedy has an unbounded approximation ratio, motivating a learned surrogate because runtime allocation cannot observe true gains.Source: E Formal Analysis · pp. 18–19
  • Under unit costs, the surrogate trajectory's final gap is bounded by OPTk/e + sum_t ηt; this induction does not cover the unequal experimental token costs.Source: E Formal Analysis · p. 19
  • LMVU's learned hypothesis class contains MMR as a fixed-coefficient case, while learning θ permits a state-dependent relevance–novelty trade-off.Source: E Formal Analysis · p. 19

F Development-Fold Evidence

  • M1 improves over MMR by +0.062 on Perspectrum and +0.033 on ESS10 in development; the M0-over-M1 Perspectrum inversion and small ArgKP gains also appear there.Source: F Development-Fold Evidence · p. 20

G Pre-Registered Learnability Probe

H Feature Ablations

The ablations show that state-dependent features drive ranking gains on WVS and ArgKP, while atomic-granularity features matter especially for WVS. Development-only sensitivity tests find the selected operating point is not finely tuned.

H Feature Ablations

  • State-dependent features drive ranking-quality gains on WVS and ArgKP, while M0 and M1 are near-identical on Perspectrum, foreshadowing the held-out M0-over-M1 inversion.Source: H Feature Ablations · p. 20
  • Removing six atomic-granularity features lowers WVS NDCG@5 from 0.6597 to 0.4418 and top-1 positive-gain rate from 0.5425 to 0.4088.Perspectrum and ArgKP remain unchanged within NDCG@5 deltas ≤0.0002; the WVS result highlights atomic structure when survey statements express multiple claims.Source: H Feature Ablations · p. 21

I Parameter Sensitivity (Development Only)

  • ESS10 RVC stays within [0.838, 0.842] across the tested pair-cap and regularization settings.Perspectrum varies mildly with pair cap (0.9406 at 5 pairs, 0.9625 at 10, 0.9583 at 20) and is insensitive to C ∈{0.1, 1, 10}; these runs used development folds only.Source: I Parameter Sensitivity (Development… · p. 21, Figure 4 · p. 21

J Embedding-Model Robustness

Embedding-model robustness is conditional on the representation: gains require available oracle headroom but are not assured when headroom exists. Development diagnostics locate many zero-gain actions in mid- and late-stage allocations, while case-study trajectories use gold labels only to label picks.

J Embedding-Model Robustness

  • 14 of 17 positive-headroom encoder–benchmark–budget cells have positive deltas, while all three negative cells occur under bge-m3 and have intervals including zero.Across held-out positive-headroom cells, deltas are positive in 10 of 13; four intervals exclude zero on the positive side and none on the negative side, all four under bge-base-en-v1.5.Source: J Embedding-Model Robustness · p. 21, Table 5 · p. 22
  • Across the 24 reported cells, none of the seven zero-headroom cells admits a positive delta; headroom is necessary but not sufficient.The robustness table reports exact-oracle headroom, M1 and MMR RVC, paired deltas with 95% bootstrap intervals, and win/tie/loss counts.Source: J Embedding-Model Robustness · p. 21, Table 5 · p. 22
  • The frozen encoder is part of the operating point: the original-protocol result replicates under the closest-family encoder but attenuates under the more distant representation.The paper claims no representation-free gains.Source: J Embedding-Model Robustness · p. 22

K Failure-Case Tally

  • M1 takes zero-gain actions despite available positive-gain actions in 200 WVS, 336 Perspectrum, and 2113 ArgKP development states, concentrated in mid/late steps.Failure mass is largest on ArgKP, where saturation makes ordering errors inconsequential, and smallest relative to opportunity on WVS.Source: K Failure-Case Tally · p. 22

L Case Studies

  • Case-study trajectories come from full-pool evaluation runs; gold viewpoint annotations label picks only.The case-study section presents cumulative-coverage trajectories and pick tables.Source: L Case Studies · p. 22

L.1 WVS protected fold: corruption query

The WVS corruption case shows LMVU covering a viewpoint MMR misses, while the ESS10 voting case shows LMVU spanning all three answers early. A development failure state and feature analysis clarify where ranking errors and runtime signals arise.

L.1 WVS protected fold: corruption query

  • LMVU reaches RVC 1.000 in 539 tokens on the WVS corruption query, while MMR ends at 0.750 after 891 tokens.MMR’s first pick covers three viewpoints, but its remaining picks revisit covered viewpoints; LMVU selects the missing corruption-free viewpoint at step 1 and adds c001 at step 2.Source: §L.1 · p. 22, Table 6 · p. 23, Figure 5 · p. 23
  • The 508-item pool has 223 c001 carriers, including 89 supporting only c001, but just sixteen c003 carriers; MMR spends 673 of 891 tokens revisiting covered viewpoints.Figure 6’s numbered rings show pick order, and its caption reports that LMVU reaches full coverage at step 2.Source: §L.1 · p. 23, Figure 6 · p. 24

L.2 ESS10 held-out: non-voting query

  • LMVU covers all three ESS10 vote viewpoints in its first three picks, reaching RVC 1.000 at 982 tokens; all 12 MMR picks support one answer, for RVC 0.333.The coverage-relevant first three LMVU picks are identical across the three training seeds.Source: §L.2 · p. 23, Table 7 · p. 24

L.3 A development failure state

  • At an empty selection state, M1 chooses an action worth 4 viewpoints versus MMR’s 7 and an oracle best of 9; state-dependent features provide no signal there.This development-only WVS political-participation failure is a pure scoring error.Source: §L.3 · p. 24

M Feature List

  • M1 uses 18 runtime-visible, label-blind features in four groups; incremental sufficient statistics avoid materializing a pairwise pool-similarity matrix.The groups are candidate descriptors, evidence–state geometry, atomic–state geometry, and shared state descriptors.Source: M Feature List · p. 24, Table 8 · p. 25
  • Evidence–state geometry has the largest-magnitude coefficient weights, while shared state descriptors have zero coefficients and cannot change within-state argmaxes.The budget binds through the feasibility filter, not the learned score.Source: M Feature List · p. 25, Figure 7 · p. 25

N Per-Benchmark Structure

Allocation trajectories show that LMVU’s coverage gains emerge at different points in the budget on WVS and ESS10. Across budgets, realized gains track available headroom descriptively, and the sweep does not indicate a universal win.

N Per-Benchmark Structure

  • Full-pool query sizes average 507, 9, 76, and 88 items for WVS, Perspectrum, ArgKP, and ESS10, respectively; MMR uses 0.85–0.97 of the 1024-token budget.M1’s positive-marginal-action frequencies are 0.321, 0.520, 0.110, and 0.404 in that benchmark order.Source: N Per-Benchmark Structure · p. 26

O Allocation Dynamics and Headroom

  • At 1024 tokens, LMVU reaches RVC 1.000 on WVS versus MMR’s 0.917, and 0.889 on ESS10 versus MMR’s 0.843.On WVS, MMR covers faster through the first half of the budget before LMVU overtakes it near 530 tokens; on ESS10, LMVU leads after roughly 250 tokens.Source: O Allocation Dynamics and Headroom · p. 26, Figure 8 · p. 26
  • ESS10 gains over MMR are +0.047 at 1024 and +0.040 at 2048; Perspectrum favors LMVU at all three budgets, while ArgKP’s only nonzero gap is +0.0095 at 512.The gain–headroom correlation is descriptive (Pearson r=0.574, n = 4).Source: O Allocation Dynamics and Headroom · p. 26

P Repeated-Support Coverage

The repeated-support study extends viewpoint breadth evaluation to ask whether viewpoints receive support from multiple distinct source units. Its full-pool comparisons, paired uncertainty analysis, and post-hoc target probe show that depth outcomes depend on benchmark, budget, and training target.

Measure and source units

  • RVC2 gives half credit for one and full credit for two or more distinct source units; units are respondents on WVS and ESS10, claim-scoped documents on Perspectrum, and arguments on ArgKP.Perspectrum’s documents trace to only three source websites, so document-level repeated support is not claimed to be site-independent corroboration.Source: P Repeated-Support Coverage · p. 27

Full-pool results

  • The strongest RVC2 allocator depends on benchmark: M0 leads Perspectrum, FacLoc edges LMVU on WVS at 1,024 tokens (0.9167 vs. 0.9015), and RichRAG exceeds LMVU on ESS10 at 1,024 and 2,048 tokens.On ESS10, LMVU leads the non-neural field throughout the budget grid; Table 11 reports full-pool RVC2 results.Source: P Repeated-Support Coverage · p. 28, Table 11 · p. 28
  • LMVU’s RVC2 advantage over MMR narrows from 0.0282 to 0.0140 on Perspectrum but grows from 0.0220 to 0.0565 on ESS10 as the budget increases.The remaining ESS10 Oracle2–LMVU gaps are 0.108, 0.154, and 0.066 at 512, 1,024, and 2,048 tokens.Source: P Repeated-Support Coverage · p. 28
  • At 1,024 tokens, WVS reaches full RVC but LMVU’s RVC2 is 0.9015 versus Oracle2’s 1.000; at 1,024 and 2,048 tokens, LMVU and MMR both reach ArgKP’s breadth and depth ceilings.At 512 tokens on ArgKP, LMVU captures part of the depth gap, reaching 0.931, +0.038 over MMR.Source: P Repeated-Support Coverage · p. 28

Paired uncertainty

  • WVS and ArgKP win/tie/loss counts cover six queries, while their descriptive sign tests use only three issue or topic groups; positive means can coexist with unchanged queries and regressions.Those small group counts preclude the confidence-interval reporting used on the other benchmarks.Source: P Repeated-Support Coverage · p. 29

Objective-alignment ablation

  • On Perspectrum, Δ2 training improves RVC2 by +0.029 at 512 and +0.022 at 1,024 tokens, but its +0.005 gain at 2,048 has an interval crossing zero.On ESS10, all three depth intervals cross zero; at 1,024 tokens, RVC and RVC2 means decline by −0.019 and −0.013, respectively.Source: P Repeated-Support Coverage · p. 30

Takeaway

  • The full-pool evaluation and separate target probe support reporting breadth and repeated support together, but neither study establishes improved end-to-end generation quality.The target probe does not show uniform depth improvement across datasets and budgets, and distinct source units need not provide independent corroboration.Source: P Repeated-Support Coverage · pp. 27, 30

Related papers

Loading 2610.29234v1…