PaperlayerView exact revision on arXiv

Source-linked AI summary

SEEDS: Preview-Guided Evidence Admission for Search Agents

Jiajin Liu, Chenxi Xu, Dongzhe Fan, Yuanfu Sun, Qiaoyu Tan

arXiv:2610.12932v1cs.CL

TL;DR

Search agents often retrieve and consume evidence through fixed or rank-based policies, although evidence utility depends on the agent’s current reasoning state. SEEDS uses lightweight previews to select which candidates and how many to inspect. Across nine knowledge-intensive QA benchmarks spanning local and live-web search, it improves search effectiveness while reducing full-evidence consumption.

  • Problem

    Search pipelines commonly admit fixed numbers of highly ranked results or ranking prefixes, although a candidate’s utility depends on the agent’s current reasoning state.

    Source: §1 · pp. 1–2

  • Method

    SEEDS previews retrieved candidates and lets the agent select an arbitrary subset and its size for full inspection, conditioned on its reasoning state.

    Source: §1 · p. 2

  • Results

    Across nine knowledge-intensive QA benchmarks, SEEDS improves search effectiveness while reducing full-evidence consumption under local-corpus and live-web search.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    Evidence admission is a complementary dimension of agentic search that separates retrieved information from information allowed to shape reasoning.

    Source: §5 · p. 10

Abstract

from arXiv · show

Search agents extend large reasoning models with multi-round access to external knowledge, yet most existing systems largely conflate retrieving evidence with consuming it. Retrieved documents are commonly incorporated according to fixed or rank-based policies, despite the fact that evidence utility is conditioned on the agent's current reasoning state. Consequently, a lower-ranked candidate may be critical for the current step, while a higher-ranked candidate may provide redundant, irrelevant, or misleading information. We formulate this overlooked problem as evidence admission, which asks, at each search step, which retrieved candidates should be allowed to enter the agent's evolving reasoning context. To enable such selective control, we introduce SEEDS (Selective Evidence Entry During Search), a preview-guided evidence admission framework for search agents. SEEDS first exposes lightweight previews of retrieved candidates and then lets the agent jointly determine which candidates and how many should be fully inspected, conditioned on its current reasoning state. Unlike fixed or adaptive prefix policies, SEEDS can select arbitrary subsets with flexible cardinality, and it can be instantiated directly with an existing reasoning agent or further optimized through learning. Across nine knowledge-intensive QA benchmarks under both local corpus and live web search, SEEDS consistently improves search effectiveness while substantially reducing full-evidence consumption. Experimental analyses show that SEEDS more effectively selects useful candidates beyond the corresponding retrieval prefix. These results establish evidence admission as a complementary dimension of agentic search, separating what information is retrieved from what information is ultimately allowed to shape reasoning.

1 INTRODUCTION

Search agents retrieve evidence iteratively, but retrieval alone does not determine which evidence should shape later reasoning. SEEDS uses lightweight previews to let an agent select evidence according to its current reasoning state.

1 INTRODUCTION

  • Evidence admitted into a reasoning trajectory can affect later reasoning, future search queries, and the final answer.Thus, deciding what to retrieve and deciding what to consume are distinct decisions.Source: §1 · p. 1
  • Evidence utility depends on the agent’s current state: lower-ranked candidates may be critical, while higher-ranked ones may be redundant or misleading.The optimal prefix size varies across datasets, and adaptive prefix sizing can still fall short of the best fixed setting.Source: §1 · p. 2, Figure 1 · p. 2
  • Previews provide a low-cost view of candidates before full inspection, using compact query-relevant excerpts for local corpora or search-result titles and snippets for live web.This offers an intermediate interface between retrieval and full evidence consumption.Source: §1 · p. 2
  • SEEDS previews retrieved candidates, then lets the agent choose which to inspect and how many, based on its current reasoning state.It supports arbitrary subsets rather than only fixed or adaptive retrieval prefixes.Source: §1 · p. 2

2 RELATED WORK

Related work covers adaptive evidence selection in RAG and multi-round retrieval integrated with reasoning. SEEDS adds an explicit evidence-admission decision conditioned on the agent’s evolving reasoning state.

Adaptive Retrieval in Retrieval-Augmented Generation

2 RELATED WORK

  • The related-work discussion frames evidence selection as complementary to adapting retrieved context and to improving search or inspection behavior.It distinguishes deciding which evidence enters reasoning from decisions about retrieval and inspection.Source: §2 · p. 3

Multi-Round Retrieval and Search Agents

3 METHODOLOGY

SEEDS separates candidate retrieval, preview, admission, and full evidence consumption within multi-round search. The agent uses its reasoning state to choose a potentially non-prefix subset for detailed inspection.

3.1 PROBLEM FORMALIZATION

  • The methodology formalizes agentic search as interleaved reasoning and external search, with previously incorporated evidence forming part of the evolving state.Evidence admission selects a useful subset from retrieved candidates according to the agent’s current information need.Source: §3.1 · pp. 3–4
  • Unlike fixed-k or adaptive-k prefix strategies, evidence admission allows both candidate identity and set size to depend on the current reasoning state.The admitted set can therefore be an arbitrary subset rather than a contiguous retrieval prefix.Source: §3.1 · p. 4

3.2 PREVIEW-GUIDED EVIDENCE ADMISSION

  • SEEDS first exposes lightweight previews so the agent can assess candidates without reading every candidate in full.Local-corpus previews use document identity and a query-relevant text window; live-web previews use search-result titles and snippets.Source: §3.2 · pp. 4–5
  • The admission policy uses previews and the current reasoning state to select candidates, and can admit arbitrary subsets, including none.Its composition and cardinality can change as the agent acquires information; the policy can be prompted directly or optimized with supervised or reinforcement learning.Source: §3.2 · p. 5
  • Expansion reveals the full contents of admitted documents and incorporates them into the evidence used for subsequent reasoning and search.Admission determines what is consumed; expansion accesses the full passage or web-page content.Source: §3.2 · p. 5

3.3 INFERENCE

  • At inference, the agent repeatedly reasons and either answers or searches, then retrieves candidates, observes previews, selects an admitted subset, and expands it.Only expanded evidence is appended to the reasoning history, and the loop continues until a final answer is produced.Source: §3.3 · pp. 5–6

4 EXPERIMENTS

Across nine benchmarks in local-corpus and live-web settings, the experiments test search effectiveness, evidence admission, efficiency, robustness, generalization, and learning.

4.1 EXPERIMENT SETUP

  • Evaluation spans nine QA benchmarks, local Wikipedia and live-web search, QwQ-32B with K=10 candidates, and retrieval-free, RAG, multi-round, and document-refinement baselines.The benchmarks include open-domain and search-intensive tasks; SEEDS is instantiated with ReAct and Search-o1.Source: §4 · p. 6, §4.1 · p. 6

4.2 MAIN RESULTS (RQ1)

  • SEEDS-ReAct leads all six local benchmarks (average EM +6.7 vs single-round MAIN-RAG; +2.9–6.3 vs multi-round baselines), while SEEDS-o1 raises live-web accuracy +2.5.Full-document use falls 60–79% against multi-round baselines; SEEDS-o1 fetches about 70% fewer web pages.Source: Table 1 · p. 7, Table 2 · p. 7, §4.2 · p. 7

4.3 EVIDENCE ADMISSION BEHAVIOR AND EFFICIENCY (RQ2)

  • SEEDS selects beyond rank prefixes and admits answer-bearing documents more reliably and densely than prefix baselines.Hit-rate gains average 13.8% for SEEDS-ReAct and 27.6% for SEEDS-o1; 44.1% and 39.6% of their admitted documents rank beyond top-|S| prefixes, with prefix departures in roughly 80% of rounds.Source: Figure 3 · p. 7, Table 3 · p. 8, §4.3 · p. 8
  • SEEDS-ReAct cuts document-token use by 58% versus ReAct dyn-k; SEEDS-o1 uses nearly the same number of searches as Search-o1 while reducing full-document reads and tokens by about 58%.The evaluation reports searches, documents, and document tokens per question.Source: Table 4 · p. 8, §4.3 · p. 8

4.4 ROBUSTNESS AND SENSITIVITY (RQ3)

  • Fewer than 8% of injected distractors are admitted by SEEDS-ReAct across settings; at ρ = 0.8, it is 1.3 EM above clean retrieval.Smaller-prefix baselines degrade as high-ranked candidates are replaced, while ReAct k=10 is more stable.Source: §4.4 · pp. 8–9, Table 5 · p. 9
  • Accuracy remains relatively stable as the preview pool grows from 5 to 20, though effects vary by task: larger pools slightly benefit NQ and hurt HotpotQA at N = 20.Larger pools generally increase documents admitted per question while requiring fewer search rounds.Source: Figure 5 · p. 9, §4.4 · p. 9

4.5 GENERALIZATION AND LEARNING (RQ4)

  • SEEDS-R1 raises average EM from 37.1 to 39.3, with mixed dataset-level improvements and particularly strong gains on 2Wiki.The learned policy uses SFT followed by PPO with final-answer and evidence-selection rewards; admission-specific training remains an open direction.Source: Figure 6 · p. 9, §4.5 · p. 9
  • With DeepSeek-V4.1-Flash, SEEDS gains 1.0 EM on HotpotQA and 1.2 on 2Wiki over Search-o1, matches HLE accuracy, and loses 1.4 EM on NQ while reducing full-document access.It reduces local-QA full-document access by 52–61% and fetched HLE pages by approximately 44%.Source: Table 6 · p. 9, §4.5 · p. 9

5 CONCLUSION

The conclusion presents SEEDS as a preview-guided way to select evidence before it shapes reasoning, reporting stronger search effectiveness with less full-evidence consumption across nine benchmarks.

5 CONCLUSION

  • Across nine QA benchmarks with local-corpus and live-web search, SEEDS improves search effectiveness while substantially reducing full-evidence consumption.The conclusion frames this as the paper’s overall result.Source: §5 · p. 10
  • Analyses find that SEEDS selects useful evidence beyond retrieval prefixes and more reliably recovers answer-bearing evidence.Source: §5 · p. 10
  • SEEDS remains effective across retrieval perturbations and reasoning backbones.Source: §5 · p. 10
  • The findings position evidence admission as a complementary dimension of agentic search, distinct from retrieving information.Source: §5 · p. 10

A.1 DATASETS

The evaluation spans local-corpus and live-web QA, compares retrieval and search-agent baselines, and specifies the retrieval setup and preview-based interaction protocol.

A.1 DATASETS

  • Six local-corpus benchmarks cover factual and multi-step QA, while three live-web benchmarks target demanding search-intensive questions.The local evaluation uses all 125 Bamboogle examples and 500 randomly selected examples from each other dataset; the live-web sets contain 1,266 BrowseComp, 289 BrowseComp-zh, and 435 HLE questions.Source: §A.1 · p. 13

A.2 BASELINES

  • Baselines include fixed-k RAG, ReAct with fixed or model-selected prefix lengths, MAIN-RAG, Search-o1, and SEEDS-ReAct or SEEDS-o1.Search-o1 and SEEDS-o1 use a back-off strategy when no final answer is produced.Source: §A.2 · p. 13

A.3 RETRIEVAL AND INFERENCE SETTINGS

  • Local retrieval uses QwQ-32B with a Wikipedia-2018 corpus of about 21 million 100-word passages; live-web search uses Serper and Jina Reader APIs.Local queries retrieve 10 candidates; live-web pages retain up to 3,000 characters around the matched snippet.Source: §A.3 · p. 13

A.4 PROMPT INSTRUCTIONS AND INTERACTION TEMPLATES

  • After retrieval, SEEDS presents candidate previews and asks the agent to expand helpful passages, skip irrelevant or redundant ones, or select none.Selected passages are returned as full evidence; the dynamic-k ReAct baseline instead reads a retrieval-rank prefix without previews.Source: §A.4 · pp. 13–14

B PREVIEW CONSTRUCTION AND ABLATION

The section describes compact preview construction and reports that preview representation and length do not have a uniformly best setting across datasets.

B PREVIEW CONSTRUCTION AND ABLATION

  • For local retrieval, previews combine a title with a 240-character query-aware text window and are capped at 60 tokens; live-web previews use search titles and snippets.The local preview uses the first 80 characters for the title and requires no extra model call or gold answers.Source: B PREVIEW CONSTRUCTION AND ABLATION · p. 14
  • No preview representation consistently performs best across datasets, and longer previews do not monotonically improve accuracy.Table 7 compares title-only, initial-token, query-aware-window, and one-line-summary representations; the authors use the query-aware window for compact, query-relevant information without an additional summarization model.Source: B PREVIEW CONSTRUCTION AND ABLATION · p. 14, Table 7 · p. 14

C EVIDENCE ADMISSION ANALYSIS AND EFFICIENCY

The analysis uses answer occurrence as a document-level proxy and interprets its hit rate conditionally on answer-bearing evidence being available in the candidate pool.

C.1 ANSWER-BEARING EVIDENCE PROXY

  • A document is counted as answer-bearing when its normalized full text contains a normalized gold-answer alias.Normalization lowercases text, removes diacritics and punctuation, and collapses whitespace.Source: §C.1 · p. 14
  • The proxy measures answer occurrence rather than evidential sufficiency, so it can miss bridge passages or count unrelated answer-string mentions.Source: §C.1 · p. 14
  • Figure 4(a)'s answer-bearing hit rate is interpreted only when such a document is available in the candidate pool.The non-prefix comparison marks a selected document as beyond the matched prefix when its rank exceeds the number admitted in that round.Source: §C.1 · p. 14

C.2 PER-DATASET EVIDENCE CONSUMPTION

SEEDS-ReAct inspects about three documents per question, while SEEDS-o1 uses fewer document reads and tokens at search frequencies close to Search-o1. The breakdown distinguishes retrieval calls from full-document consumption.

C.2 PER-DATASET EVIDENCE CONSUMPTION

  • Tables 8–10 decompose aggregate costs into search calls, full-document reads, and document tokens per question.The tables report macro-averages across six local benchmarks where specified in their captions.Source: §C.2 · p. 14, Table 8 · p. 15, Table 9 · p. 15
  • Document-token counts measure full-evidence consumption, excluding prompts, previews, reasoning, and generated output, and are macro-averaged across six local benchmarks.The count uses the model tokenizer and is not a measure of total inference tokens or latency.Source: §C.2 · p. 15
  • SEEDS-o1 has search frequency close to Search-o1 but lower document reads and document-token consumption on every dataset.The breakdown supports distinguishing less searching from less reading per search.Source: §C.2 · p. 15
  • SEEDS-ReAct inspects about three documents per question across datasets, while fixed-k agents read more as search frequency increases.Full-document reads count passages passed to the reasoning model or Reason-in-Documents reader.Source: §C.2 · pp. 14–15

D DETAILED LIVE-WEB RESULTS

The HLE subject-level breakdown descriptively favors SEEDS-o1. The appendix details SEEDS-R1’s interaction environment, supervised warm start, and reinforcement-learning setup.

D DETAILED LIVE-WEB RESULTS

  • SEEDS-o1 answers 60 of 435 text-only HLE questions correctly, compared with 45 for Search-o1, with higher accuracy in seven subjects and a tie in Engineering.The small subject subsets should be interpreted descriptively.Source: D DETAILED LIVE-WEB RESULTS · p. 15, Table 11 · p. 16

E TRAINING AND EVALUATION OF SEEDS-R1

  • SEEDS-R1 uses Qwen2.5-7B-Base, with supervised fine-tuning followed by PPO to train the search–preview–expand–answer protocol and optimize answer correctness and evidence selection.The training and evaluation configuration is summarized in Table 12.Source: E TRAINING AND EVALUATION OF SEEDS-R1 · p. 16, §E.3 · p. 17

E.1 INTERACTION ENVIRONMENT

  • The training environment uses Wikipedia and E5 with five candidates per query, query-aware previews, and an expand decision selecting zero to five passages.Episodes are limited to four retrieval rounds and ten policy actions; observations are capped at 1,200 tokens per step.Source: §E.1 · p. 16

E.2 STAGE 1: SUPERVISED WARM START

  • Development EM rises from 0.14 before SFT to 0.49 after one epoch and 0.53 after five epochs, when the checkpoint is selected despite lower loss at epoch one.The accepted dataset contains 1,000 trajectories: 900 training and 100 development; acceptance requires protocol compliance and either an exact-match answer or an observed gold answer.Source: §E.2 · p. 16

E.3 STAGE 2: REINFORCEMENT LEARNING

  • PPO combines final-answer exact-match reward with an auxiliary selection bonus that credits positive documents and penalizes opening negative documents.For HotpotQA, supporting-fact-title matches receive positive labels, including bridge documents even when they omit the final answer; missed visible evidence is penalized only when the final answer is incorrect.Source: §E.3 · p. 17
  • The PPO objective uses token-level rewards with generalized advantage estimation and a per-token KL penalty regularizing the actor toward the frozen SFT reference policy.The critic is updated alone for the first ten steps; training runs for 500 updates and reports the step-450 checkpoint.Source: §E.3 · p. 17

E.4 TRAINING PROCESS

During PPO training, selection rates and precision rise as document reads first increase and then decline. The appendices describe the SEEDS-O1, SEEDS-ReAct, and SEEDS-R1 interaction protocols.

E.4 TRAINING PROCESS

  • Over updates 426–450, positive-document open rates reach 74% on NQ and 88% on HotpotQA, precision reaches 0.43 and 0.46, and reads decrease to 2.12 documents per question.Across updates 91–110, open rates are 74% and 81%, while reads increase from 1.30 to 2.34 per question; the reported window averages indicate initial evidence acquisition followed by more selective reading.Source: §E.4 · p. 17, Figure 7 · p. 18

F.1 SEEDS-O1

  • SEEDS-O1’s max-information variant lets the agent expand a flexible number of candidates up to K and instructs it to select inclusively.Only selected passages are sent to the Search-o1 Reason-in-Documents module.Source: §F.1 · p. 18

F.2 SEEDS-REACT

  • SEEDS-ReAct requires one expand decision after each snapshot, selecting zero to ten candidates before the system returns their full passage contents.The prompt instructs the agent to omit only clearly irrelevant or fully redundant passages.Source: §F.2 · p. 19

F.3 SEEDS-R1

  • SEEDS-R1 extends the Search-R1 format with snapshot and expand actions, requiring an expand after every search and allowing at most five documents per expand and four search rounds.The same prompt is used for supervised warm-up, RL, and evaluation.Source: §F.3 · p. 20
Loading 2610.12932v1…