LENS: In-Context Search via Latent Evidence Exploration over Dynamic Raw Documents

Xingjun Wang; Gongsheng Li; Qi Fan; Yunlin Mao; Luyan Su; Yingda Chen

Source-linked AI summary

LENS: In-Context Search via Latent Evidence Exploration over Dynamic Raw Documents

Xingjun Wang, Gongsheng Li, Qi Fan, Yunlin Mao, Luyan Su, Yingda Chen

arXiv:2608.16185v1Aug 17, 2026cs.CLcs.AI

TL;DR

Dynamic raw-document QA needs evidence localization that adapts to changing documents and query-dependent granularity rather than relying on pre-materialized indexes. LENS performs budgeted, index-free evidence exploration and achieves stronger supporting-fact localization and grounding than ReAct, while remaining query-ready after corpus changes.

Problem
Dynamic raw-document collections lack stable, query-independent evidence units, while pre-materialized retrieval fixes granularity and incurs setup, update, and staleness costs.
Method
LENS formulates search as budgeted evidence localization and uses index-free sequential exploration with low-cost priors, oracle-guided refinement, and source-grounded consolidation.
Results
28.0–28.8 pp EM losses occur for stale-index BM25-RAG and Hybrid-RAG on updated questions, while LENS localizes evidence more reliably than ReAct and maintains high grounding.
Takeaways & Limitations
LENS targets auditable search by preserving source traceability and immediate query readiness over changing raw documents.
Takeaways & Limitations
HotpotQA emphasizes lookup and comparison over encyclopedic text, leaving richer layouts, aggregation intents, table evidence, and warm-reuse behavior for future work.

Abstract

from arXiv · show

LLM agents increasingly answer questions over dynamic raw-document collections, where files may change before preprocessing, and relevant evidence (spans, sections, pages, or tables) is query-dependent. Existing retrieval-augmented approaches pre-materialize evidence via fixed chunking, embeddings, or persistent indexes: effective for lookup, yet costly, stale-prone, and committed to a granularity before the query is known. We formulate in-context search as Budgeted Evidence Localization over a latent evidence space induced by dynamic raw documents and propose LENS (Latent Evidence Exploration and Search), an index-free framework. Instead of pre-materializing the evidence space, LENS maintains a query-conditioned belief over candidate units, iteratively selecting candidates via complementary lexical, local, and exploratory proposal policies, updating the belief via an LLM relevance oracle, and narrowing toward high-posterior regions under a controllable budget. Evidence is consolidated into compact, source-grounded regions of interest and compressed into self-organizing knowledge clusters reused across related queries. On a controlled 500-question evaluation with matched corpus snapshots, LENS reaches 62.4% exact match and 84.8% evidence recall vs. 65.2% exact match but 50.4% evidence recall for a ReAct-style baseline. Across scales, LENS gives the strongest supporting-fact localization and answer grounding. On a fixed 150-question fullwiki subset over the raw Wikipedia dump with zero indexing, LENS and ReAct are nearly tied in official answer quality (43.3% vs. 42.7% EM), with LENS grounding more answers in retrieved evidence (84.0% vs. 70.7%). A no-retrieval Closed-Book reference highlights the contribution of model memory. LENS is query-ready after corpus changes, needs no preprocessing or persistent index, and preserves source-grounded evidence localization throughout.

1 Introduction

The paper frames question answering over evolving raw documents as Budgeted Evidence Localization in a latent, query-dependent evidence space. It introduces LENS, an index-free sequential exploration framework that uses low-cost priors, LLM-guided refinement, budget-aware stopping, and source-grounded consolidation.

Problem: Dynamic raw documents make relevant evidence query-dependent in both location and granularity, ranging from spans and tables to sections, pages, or cross-document chains.The evidence space is latent, variable-boundary, dynamic, and structured, so useful evidence is not fixed before the query.
Motivation: Pre-materialized chunks, embeddings, summaries, indexes, and graph-like memories support static lookup but can become stale or costly when document collections change.These approaches commit to evidence representations before the query and rely on preprocessing that may not remain aligned with dynamic raw documents.
Formulation: The paper formulates search as Budgeted Evidence Localization: infer a compact, source-grounded evidence set while accounting for token, latency, and relevance-oracle-call constraints.The formulation preserves query-conditioned evidence granularity and document freshness while making LLM-oracle cost explicit.
LENS: LENS builds a low-cost prior, sequentially updates beliefs with an LLM relevance oracle, and explores regions with higher expected utility before consolidating selected evidence.The framework is index-free and combines oracle-guided refinement with budget-aware stopping and source-grounded consolidation.
Evaluation: The evaluation uses nested question sets and matched raw-document snapshots to jointly assess answer quality, evidence localization, freshness under corpus growth, and query-time budget.Systems are compared on identical sampled questions and corpus boundaries, enabling paired and auditable analysis.

2 Background

Dynamic raw-document search must handle changing corpus contents and query-dependent evidence granularity, unlike static retrieval settings. Because prebuilt representations become fixed before queries and require updates after document changes, in-context search is framed as budgeted sequential evidence localization using costly LLM relevance judgments.

Dynamic raw documents: Dynamic collections may add, update, or remove files, while query-relevant evidence can occur in spans, tables, pages, or cross-document relations.Both corpus state and appropriate evidence granularity depend on the query.
Pre-materialized evidence: Retrieval-augmented methods preconstruct chunking, embedding, sparse, summary-tree, or graph representations before queries, requiring rebuilding or updating when documents change.These representations are useful for stable corpora but commit to a fixed representation before the query is known.
Budgeted evidence localization: LLM relevance judgments consume tokens, latency, and cost, so search must balance immediate relevance, information gain, source traceability, and budget.LENS therefore treats in-context search as sequential evidence localization rather than one-shot top-k lookup.

3 Budgeted Evidence Localization

LENS frames search over changing raw documents as budgeted localization in a latent, combinatorially large evidence space whose answer-bearing regions are unknown until querying. It localizes evidence per atomic fact through sequential LLM-oracle observations, returning a compact source-grounded evidence set alongside the synthesized answer.

Latent Evidence Space: The latent evidence space contains candidate windows (d, s, e) from current raw documents, with variable boundaries and no explicit enumeration by LENS.Its structure changes with the current corpus and induces non-uniform query-conditioned beliefs.
Query-Dependent Requirements: Evidence requirements are induced by query intent, with lookup queries requiring K = 1 fact and comparison, computation, or aggregation queries requiring K > 1 facts.Required facts may reside in different documents.
Per-Fact Evidence Targets: Localization targets a minimal sufficient evidence window for each atomic fact, and completion requires covering every requirement rather than finding one query-level window.This keeps inference tractable for multi-hop queries while allowing the returned evidence to be a set of windows.
Sequential Inference: LENS approximates posterior concentration through sequential observations because the evidence likelihood is unavailable in closed form and an LLM serves as a costly relevance oracle.The initial belief is query-conditioned and updated using oracle-observation history.
Budgeted Output: A budget B limits oracle calls, tokens, or wall-clock time, while the output pairs compact source-grounded evidence covering localized windows with a synthesized answer.Correct answer generation without traceable evidence is insufficient in this setting.

4 The LENS Algorithm

LENS performs budgeted evidence localization by forming a query-conditioned prior, iteratively exploring candidate regions with relevance-oracle updates, and consolidating localized windows into source-grounded evidence for answer synthesis. Its design adapts proposal and stopping decisions to evidence coverage while keeping online cost independent of corpus size under stated analysis conditions.

Prior formation: LENS fuses five corpus-independent signal families into a low-cost prior, separating document selection from precise within-document localization.The prior combines lexical anchors, document-path structure, compiled summaries, historical evidence, and lightweight scans, then factors into file-level and within-document components.
Sequential exploration: Each exploration iteration proposes a region, queries the LLM relevance oracle on raw text, updates per-fact beliefs, and adapts proposal weights and coverage estimates.A single oracle call serves all outstanding facts, avoiding multiplication of the oracle budget by the number of facts.
Sequential exploration: LENS combines lexical, local, and global proposal families, adapting their mixture weights online to balance exploitation of promising regions with discovery of semantically missed evidence.The proposal approximation targets the trade-off between immediate relevance and long-run information gain under an intractable ideal objective.
Stopping and coverage: LENS stops when the weakest requirement reaches the intent-modulated confidence threshold or when the budget is exhausted, requiring confirmed windows for every fact in comparison and computation queries.Lookup queries may stop with one compact region, whereas multi-fact intents require coverage of all requirements.
Consolidation and synthesis: After stopping, LENS merges, de-duplicates, and expands per-fact windows while preserving source traces, producing evidence set E∗ for grounded answer synthesis.The final output is the pair (E∗, a), with answers grounded in explicit evidence regions rather than only retrieved snippets.
Analysis: Under locally stable oracle relevance, successive observations concentrate belief with diminishing exploration returns, while online cost remains bounded independently of corpus size.These are analysis guides rather than tight guarantees; stricter oracle, position-level prior, and resampling analyses remain future work.

5 Related Work

Related work primarily addresses evidence access through persistent retrieval representations, long-context prompting, or iterative tool use. These approaches trade preprocessing and index maintenance against query-time cost and imperfect localization of specific evidence.

Retrieval-Augmented Generation: Retrieval-augmented generation retrieves external evidence before generation and is standard for knowledge-intensive tasks.Dense, sparse, and hybrid systems work effectively when stable corpora can be preprocessed into persistent representations.
Structured Retrieval: Hierarchical and graph-based retrieval organize documents into summaries, trees, or memory graphs.These systems extend retrieval beyond flat document representations.
Index-Free and Agentic Methods: Long-context models avoid persistent indexes by placing raw text directly into context, but increase query-time token consumption and may miss specific evidence.Tool-using agents instead search, read, and refine context over multiple steps.

6 Experiments

Experiments compare LENS with index-free and index-dependent systems on controlled matched snapshots and raw fullwiki Wikipedia. LENS is competitive on answer quality while consistently improving evidence localization, grounding, and readiness after corpus changes.

Controlled evaluation: On D500, ReAct leads answer quality at 65.2% EM and 78.9% F1, while LENS achieves 62.4% EM and 76.9% F1.The controlled evaluation uses 500 questions paired with a matched corpus snapshot containing gold supporting articles, distractors, and a background pool.
Controlled evaluation: LENS achieves 84.8% evidence recall and 96.8% grounded answers on D500, versus 50.4% and 71.8% for ReAct.Ev.Rec measures retrieval of gold supporting-fact documents, while Ground measures whether the final answer is traceable to retrieved evidence.
Open-domain fullwiki: On raw fullwiki with zero indexing, LENS and ReAct are nearly tied at 43.3% versus 42.7% EM, while LENS grounds 84.0% of answers versus 70.7%.The evaluation uses 150 fixed questions over 15,517 unprocessed JSON shards; Closed-Book reaches 38.7% EM, with retrieval gains of +4.6 pp for LENS and +4.0 pp for ReAct.
Scaling: Across controlled scales, LENS records 84.8–89.1% evidence recall and leads ReAct by 43.0 pp on D125, 38.0 pp on D250, and 34.4 pp on D500.The results indicate that locating candidate evidence and synthesizing the exact multi-hop answer are separable failure modes.
Lifecycle and efficiency: After expansion from D125 to D250, index-dependent systems lose 28.0 and 28.8 pp EM and 70.1 and 69.6 pp evidence recall, while LENS remains query-ready with 84.7% to 83.9% supporting-fact recall.LENS and ReAct are index-free and can query the expanded corpus immediately; on D500, LENS also uses 16.5K tokens per query versus ReAct’s 11.8K while improving evidence recall by 34.4 pp and grounding by 25.0 pp.

7 Discussion and Conclusions

LENS is presented as an index-free evidence-localization method rather than an EM-dominant answer generator, delivering competitive answer quality with substantially stronger source traceability and immediate readiness for changing raw documents. Its conclusions are strongest within HotpotQA-style lookup and comparison over encyclopedic text, while broader layouts, aggregation, tables, and warm reuse remain open challenges.

Comparative results: 34.4 pp evidence recall and 25.0 pp grounding offset LENS’s 2.8 pp EM deficit to ReAct on D500.On fullwiki dev-150, the systems are effectively tied in official EM/F1 while LENS retains stronger grounding.
Contribution: LENS’s central contribution is index-free evidence localization that makes answers more traceable to current raw sources.The method is explicitly distinguished from an EM-dominant answer generator.
Evaluation interpretation: 35.2% EM on D500 and 38.7% EM on fullwiki show that Closed-Book model memory can produce correct answers without retrieved evidence.The paper therefore interprets EM/F1 jointly with supporting-fact recall and grounding.
Lifecycle robustness: 28.0–28.8 pp EM losses and near-total evidence-recall collapse expose the stale-index lifecycle weakness under corpus expansion.BM25-RAG and Hybrid-RAG reuse a D125 index while answering newly added D250 questions, whereas index-free systems query the updated corpus directly.
Limitations and scope: HotpotQA’s emphasis on encyclopedic lookup and comparison bounds the evidence, leaving richer layouts, aggregation intents, table evidence, and warm-reuse behavior for future work.Within this scope, LENS demonstrates competitive answer quality, stronger source traceability, and immediate query readiness over changing raw documents.