PaperlayerView exact revision on arXiv

Source-linked AI summary

Search, Inspect, Fetch: Exploiting Structure-Aware Boolean Retrieval for Deep-Research Agents

Shuai Wang, Haodong Chen, Yu Yin, Shengyao Zhuang, Bevan Koopman, Guido Zuccon

arXiv:2608.02751v2cs.IRcs.AIcs.CL

TL;DR

Deep-research agents typically retrieve whole webpages without using their titles, headings, sections, or metadata to constrain access. Sieve introduces search–inspect–fetch with Boolean field filtering and selective section retrieval, achieving higher accuracy on all three QA collections while using 20.7–50.6% fewer tokens than the strongest conventional Search–Visit configuration.

  • Problem

    Deep-research agents retrieve whole webpages without using webpage structure to control retrieval and content access.

  • Method

    SIEVE uses fielded Boolean candidate selection, interchangeable ranking, structure-rich result inspection, and selective section fetching.

  • Results

    SIEVE achieves the highest accuracy on all three QA collections, while its default uses 20.7–50.6% fewer tokens than the strongest conventional Search–Visit configuration.

  • Takeaways & Limitations

    The findings support preserving webpage organization as searchable and fetchable fields so agents can access evidence at its original granularity.

  • Takeaways & Limitations

    SIEVE jointly changes candidate selection, result cards, and content access, so the evidence supports the complete system rather than independent effects of each Boolean operator.

Abstract

from arXiv · show

Existing deep-research agents use a Search--Visit workflow that retrieves whole webpages without considering the structure they expose through titles, headings, sections, and metadata. This prevents agents from directly constraining retrieval to parts of a webpage and often carries irrelevant content into their context. We introduce \textsc{Sieve}, a search--inspect--fetch strategy driven by a Boolean Query Language (BQL): it searches webpage fields to filter candidates, uses an interchangeable ranker to order them, presents structure-rich result cards for inspection, and fetches only selected sections. Across three QA collections, \textsc{Sieve} is more accurate than the strongest conventional Search--Visit configuration on each collection while using $20.7$--$50.6\%$ fewer tokens. Boolean filtering improves every tested ranker, and the accuracy--context advantage persists across retriever choices and agent backbones. Our implementation is included in the SkimSearchAgent library at https://github.com/ielab/skim-search-agent.

1 Introduction

The introduction identifies a mismatch between structure-rich webpages and whole-webpage Search–Visit workflows, then proposes SIEVE’s search–inspect–fetch strategy to preserve structure throughout retrieval and evidence access. SIEVE combines Boolean field selection, interchangeable ranking, structure-rich inspection, and selective section fetching, improving accuracy while reducing token use.

  • Motivation: Whole-webpage visits discard actionable webpage structure, including titles, headings, sections, dates, and authors, while failing to let agents control retrieval and content access.The proposed workflow preserves structure throughout research rather than treating webpages as undifferentiated documents.
  • Method: Boolean constraints define eligible webpages, while lexical or dense ranking determines which eligible sources the agent inspects first.This division supports exact field requirements, date ranges, and exclusions while keeping the ranker interchangeable.
  • Method: SIEVE uses BQL for fielded candidate selection, exposes titles, headings, and query-focused snippets in result cards, and fetches named sections instead of complete webpages.Structure is preserved across search, inspection, and fetching.
  • Results: 1.6/3.1/0.7 accuracy points is SIEVE’s improvement over the most accurate Search–Visit configuration, while using 20.7–50.6% fewer tokens.The default configuration is BM25+Dense, and SIEVE achieves the highest accuracy across all three collections.
  • Contributions: SIEVE’s contributions are a structure-preserving search–inspect–fetch strategy, a modular system combining Boolean selection with ranking and selective fetching, and evidence of improved accuracy with fewer tokens.The evaluation uses paired flat and structured variants of BrowseComp-Plus, HotpotQA, and MuSiQue.

2 Related Work

SIEVE extends research-agent retrieval by preserving webpage structure from search through evidence access, combining fielded Boolean constraints with focused retrieval over webpage components. It builds on iterative agent workflows, Boolean search, and structure-aware retrieval while comparing against an Indri-style alternative under fixed result-card and fetching conditions.

  • Retrieval workflows for research agents: Research agents commonly use a ReAct-style search-and-visit loop, whereas SIEVE asks whether preserving webpage structure through evidence access improves efficiency without sacrificing effectiveness.The conventional loop retrieves a top-k list before webpage visits; SIEVE changes how structure is retained across retrieval and evidence access.
  • Boolean search and agent-generated queries: Boolean retrieval supports precise inclusion and exclusion through field restrictions, Boolean operators, phrases, wildcards, and ranges.Field restrictions distinguish terms appearing in titles from the same terms appearing in webpage bodies, refining the candidate set.
  • Structure and retrieval granularity: Focused retrieval has targeted passages or webpage components, while SIEVE uses existing collection structure to construct candidates with fielded constraints and order them with a separate ranker.Related systems also vary dense-indexing granularity or construct hierarchies over otherwise unstructured text.
  • Structure and retrieval granularity: Appendix C.4 compares SIEVE with an Indri-style structured-retrieval alternative while holding result cards and section fetching fixed.This comparison isolates the structured-retrieval executor under shared presentation and evidence-access conditions.

3 SIEVE: Search–Inspect–Fetch

SIEVE preserves webpage structure throughout retrieval and evidence access by separating candidate selection, ranking, inspection, and fetching. Its Boolean filtering, structure-rich result cards, and section-level fetches let agents constrain eligibility while limiting retrieved context.

  • Design principle: SIEVE keeps webpage structure actionable from retrieval through evidence access, rather than discarding headings during search or returning whole webpages during visits.The design separates which webpages qualify, how they are ranked, what the agent sees before content access, and what content it fetches.
  • Search: Boolean queries combine AND, OR, NOT, field restrictions, phrases, wildcards, and date ranges to control which webpages are eligible.Ranking remains a separate decision that orders acceptable sources for inspection.
  • Search: When filtering returns no webpages, SIEVE relaxes constraints through partial-match coverage tiers before falling back to ranking the full collection.This recovery mechanism lets agents test precise hypotheses, obtain evidence when queries are too restrictive, and reformulate subsequent queries.
  • Inspect: SIEVE presents the top k webpages as compact cards showing titles, section headings, and 25-token query-focused snippets for local relevance evidence.The headings become possible fetch targets without returning webpage bodies during inspection.
  • Fetch: Fetch returns one selected named section or field instead of the concatenated webpage, while preserving the same per-action token ceiling as visit.The complete strategy combines Boolean selection, modular ranking, structure-rich cards, and section-level fetching; its default ranker uses BM25+Dense.

4 Experimental Setup

The experimental setup evaluates structure-aware retrieval on paired flat and structured versions of three QA collections, using matched retrieval controls, multiple agent and retriever configurations, and collection-specific accuracy metrics. Baselines isolate iteration, selective content access, and BQL candidate selection under fixed retrieval and context budgets.

  • Collections: Paired flat and structured versions of HotpotQA, MuSiQue, and BROWSECOMP-PLUS hold questions, webpages, and emitted text identical while making sections and metadata addressable only in structured variants.The paired design measures the value of exposing webpage structure as addressable fields, although headings and recoverable boundaries remain in flat text.
  • Collections: HotpotQA, MuSiQue, and BCP-S increase in difficulty and source complexity, with exact match reported for the Wikipedia collections and LLM-judge accuracy for BCP-S.Accuracy is intended for comparison within, rather than across, collections; retained questions number 7,343/7,405 for HotpotQA and 2,409/2,417 for MuSiQue.
  • Baselines: Search–Visit, Search–Fetch, and SIEVE are run with BM25, dense, and BM25+Dense ranking, while matched Search–Fetch controls differ from SIEVE only through BQL candidate selection.Both use the same rankers, top-k result depth, and section fetching, separating BQL selection from ranker, result depth, and access granularity.
  • Agents and retrievers: The primary agent is Tongyi-DeepResearch-30B-A3B, with Qwen-AgentWorld-35B-A3B and OpenResearcher-30B-A3B used for the agent-backbone study; generations use temperature 0.6 and seed 42.The released ReAct scaffold is preserved while collection-access tools and condition-specific instructions vary.
  • Budgets and evaluation: Every search returns at most k=5 results, visits or fetches have a 12,000-token ceiling, and iterative conditions allow at most 100 agent steps.The ten-webpage DCI working set is a one-time staging depth separate from the per-call search depth.
  • Budgets and evaluation: Tok. measures mean distinct input and output tokens, LLM calls measures mean model invocations, and significance uses two-sided exact McNemar and paired t-tests at α=0.05 with Bonferroni correction.Step-summed tokens are additionally reported to reflect serving work when conversation history is repeatedly processed.

5 Results

SIEVE improves the accuracy–context trade-off over Search–Visit by combining iterative search with selective, section-level access. Results show that Boolean candidate selection complements ranking, while retrieval remains necessary for heterogeneous corpora.

  • Accuracy–context trade-off: SIEVE improves accuracy by 1.6, 3.1, and 0.7 points while using 30.4%, 50.6%, and 20.7% fewer tokens on HotpotQA, MuSiQue, and BCP-S.On BCP-S, judge accuracy and exact match improve by 2.5 and 3.4 points, respectively.
  • Accuracy–context trade-off: Section-level fetching reduces context by controlling what enters it, not by curtailing search, allowing evidence-chain following without repeatedly carrying complete webpages.SIEVE makes more model calls on Wikipedia collections and a similar number on BCP-S, so call count alone poorly measures efficiency.
  • Accuracy–context trade-off: On roughly two-thirds of BCP-S questions, SIEVE uses fewer tokens, with accuracy gains clustering among token-saving cases and losses becoming more common as consumption rises.The broader frontier likewise shows that more context does not systematically produce more correct answers.
  • Access and iteration: Effective deep research requires repeated search and selective access: Search–AutoRead adds full results to context without comparable accuracy gains, while one-shot baselines struggle on difficult questions.Iteration alone is insufficient when every retrieved result is added in full.
  • Retrieval and ranking: DCI is competitive on Wikipedia but weaker on heterogeneous BCP-S, where BM25 bounding recovers much of the gap, showing that corpus tools do not replace retrieval.Candidates still need organization into a manageable evidence space.
  • Retrieval and ranking: SIEVE improves accuracy in all nine Search–Fetch pairs, showing that BQL candidate selection complements rankers by determining what may be considered before ranking decides inspection order.The best ranker varies across collections and access strategies, so ranking quality must be assessed jointly with downstream access.

6 Ablation and Diagnostic Analysis

Ablations show that query-focused snippets make webpage structure actionable, while SIEVE’s accuracy–context benefits transfer across retrievers and agent backbones. Error analysis identifies collection-specific bottlenecks and confirms that agents use BQL’s field restrictions, Boolean combinations, and fallback retrieval.

  • Query-focused snippets: Removing 25-token query-focused snippets lowers accuracy by 2.9–6.8 points across all three collections and increases model calls without recovering the loss.The snippets provide local evidence for deciding which section to fetch next.
  • Retriever choice: 34.6–38.0 accuracy and 46k–48k context tokens across six dense encoders outperform the 68.1k-token Search–Visit context baseline, with Qwen3-Embedding-4B performing best.Accuracy is not monotonic with encoder scale: Qwen3-Embedding-8B trails 4B, and bge-large trails smaller bge models.
  • Agent backbone: Across nine backbone–collection pairs, SIEVE uses less context in eight and improves accuracy in seven; the other two accuracy differences are within 0.6 points.On BCP-S, all three backbones move toward higher accuracy and lower context use.
  • Where errors arise: SIEVE reduces selection failures on every collection, while retrieval failures comprise 42.7–46.5% of BCP-S questions and synthesis failures remain comparatively stable between systems.The analysis attributes remaining needs to better retrieval on BCP-S and better evidence extraction and synthesis on HotpotQA and MuSiQue.
  • How agents use BQL: Field restrictions appear in 42.8–69.1% of questions, Boolean combinations in 34.1–44.1%, and fallback retrieval handles 36.8–53.7% of search calls.Exact constraints and fallback retrieval are complementary: zero-hit constrained calls trigger ranked term retrieval.

7 Conclusion

SIEVE exploits webpage structure through fielded Boolean retrieval and a search–inspect–fetch workflow, achieving the highest observed accuracy across three QA collections while using fewer tokens than conventional Search–Visit. The findings support preserving source organization and using language-model agents to make precise Boolean search practical.

  • Empirical conclusion: 20.7–50.6% fewer tokens accompany SIEVE’s higher accuracy than the best conventional Search–Visit configuration across three QA collections.SIEVE also achieves the highest observed accuracy on each collection.
  • Implications: Source structure shapes what agents retrieve, inspect, and include by exposing webpage organization as searchable and fetchable fields.The paper argues that retrieval should preserve source organization instead of flattening webpages, enabling access at the original granularity.
  • Implications: Language-model agents can author and revise precise fielded Boolean queries during research while retaining explicit selection constraints.This makes capabilities historically used in specialist search practical for general deep-research systems.

Limitations

The evaluation is limited by narrow coverage, asymmetric experimental conditions, incomplete isolation of component effects, and potential dataset or structure-quality noise. Selective retrieval may also omit relevant context, so lower token use does not establish completeness or correctness.

  • Evaluation coverage: The main comparison uses one backbone and three QA collections, while transfer tests do not establish generalization to other agents, domains, or live web search.HotpotQA and MuSiQue use exact match as the primary metric because judge overlays are incomplete for Wikipedia conditions.
  • Experimental conditions: Experimental conditions are not fully symmetric because systems receive different manuals, final-answer recovery is incomplete in some cells, and early stopping was introduced part-way through.The appendices document these differences rather than assuming equivalent conditions.
  • Component attribution: The structured/flat controls diagnose field access rather than structure itself, while SIEVE jointly changes candidate selection, result cards, and content access.Controls do not fully cross every interaction, so the evidence supports the complete system rather than independent effects of each Boolean operator; zero-hit BQL queries also use approximate BM25.
  • Data and structure quality: Model-generated BCP-S section trees may contain imperfect headings or boundaries, and title-matched Wikipedia articles may contain drift from the newer snapshot.Wikipedia matching does not include a content check.
  • Deployment and ethics: Selective fetching can exclude relevant sources or surrounding context when metadata or section boundaries are wrong, so lower token use does not prove retrieved material is complete or correct.Deployed systems should retain provenance, permit full-webpage visits, and respect access, privacy, and licensing restrictions; the experiments used released QA benchmarks and collected no user data.

A Dataset Construction and Statistics … B.3.1 Sample of Issued Queries

The paper constructs structured and flat webpage twins for three QA collections, then defines SIEVE’s fielded Boolean search, structure-aware inspection, and section-level fetching workflow. It also documents corpus-specific instruction and indexing mismatches and illustrates issued BQL queries, including zero-hit fallback behavior.

  • A Dataset Construction and Statistics: 100,195 BCP-S webpages and all 830 questions are loaded without query-dependent corpus filtering, preserving released identifiers and relevance annotations.The flat and structured variants search the same complete webpage collection.
  • A Dataset Construction and Statistics: 94.9% of 20,000 audited BCP-S webpages have multiple sections, with a mean of 14.7 sections per webpage.Section headings and boundaries are proposed by gpt-5.5-nano and applied deterministically without rewriting webpage text.
  • A Dataset Construction and Statistics: Wikipedia collections use native sections, infoboxes, and abstracts, while structured and flat twins retain identical identifiers, questions, qrels, and emitted text.Fielded retrieval can scope over sections in the structured versions, whereas flat versions expose title and text only.
  • A Dataset Construction and Statistics: In HotpotQA, 89.3% of 59,833 webpages have multiple sections with a mean of 8.29; MuSiQue has 87.9% and 9.70 across 16,583 webpages.These counts include the leading introductory unit, and BCP-S section trees are model-generated, which the paper treats as a limitation.
  • B.1 Query Grammar and Tool Behavior: The BQL language supports field restrictions, Boolean operators, wildcards, phrases, and typed date ranges, with available fields varying by corpus.Wikipedia indexes title, section, and body; BCP-S indexes title, section, body, author, and date.
  • B.2.2 Complete Search–Fetch Instructions: SIEVE’s search exposes structure-rich result cards rather than document bodies, and fetch retrieves a selected named section or infobox slice.The instructions emphasize searching entity names, chaining hops through fetched slices, and answering from the shortest verbatim span of the correct fact type.
  • B.2.3 Condition-Specific Instructions: BCP-S instructions add author and date while omitting section and infobox from the field table, whereas whole-webpage controls add typed date ranges and zero-hit constraint feedback.Indri receives a separate, collection-independent manual for its own query language in an appendix-only comparison.
  • B.2.4 Instruction and Index Mismatches: The BrowseComp manual incorrectly denies addressable sections, while Wikipedia manuals advertise an infobox search field that post-hoc auditing found was not indexed.These converse instruction–index mismatches can alter which query operators agents use.

B.3.2 A Worked Interaction … C.2 Decoding, Serving, and Budgets

The worked interaction demonstrates SIEVE’s search–inspect–fetch cycle, including an initial fetch error, correction using advertised section names, and an exact-match answer. The experimental appendices document DCI reimplementation choices, shared serving settings, context and step budgets, and their comparability limitations.

  • B.3.2 A Worked Interaction: A four-step interaction shows SIEVE progressing from search to inspection, failed fetch, corrected fetch, and an exact-match answer.The interaction uses three tool calls before the final answer.
  • B.3.2 A Worked Interaction: The agent searches title and body fields for “Fort Santiago,” receiving five matches and structure-rich result cards.The query compiles to IN(title, "Fort Santiago") OR IN(body, "Fort Santiago").
  • B.3.2 A Worked Interaction: A generic body fetch fails because the records expose named sections, after which the agent fetches advertised sections successfully.The corrected requests target “Origins and construction” and “(intro),” returning the relevant text.
  • B.3.2 A Worked Interaction: The model checks all four historical constraints against the fetched text and outputs Fort Santiago as exact-match correct.The checks cover 16th-century construction, Manila’s historical rule, the 1574 attack, rebuilding, and mid-17th-century damage.
  • C Experimental Details and Baselines: The appendix covers baseline implementations, serving and budget settings, evaluation procedures, and retrieval-engine validation.These materials follow the experimental setup in the main paper.
  • C.1 DCI Reimplementation: DCI searches the raw corpus with only bash and read tools, while the BM25-bounded control stages ten retrieved webpages in a flat-file filesystem.The bounded control uses the Search–Visit BM25 implementation and DCI’s shell tools, enforcing the bound through staged files.
  • C.1 DCI Reimplementation: Omitting DCI’s five-step search prompt and limiting it to 100 steps may disadvantage or understate its performance relative to the published implementation.The original paper uses 300 turns and reports that DCI needs more tool calls than a retrieval agent to find an initial anchor webpage.
  • C.2 Decoding, Serving, and Budgets: All backbones use temperature 0.6, seed 42, local vLLM serving, and a fixed ReAct scaffold, while context windows range from 131,072 to 262,144 tokens.Search–Visit and SIEVE retain each backbone’s context window; iterative conditions use k=5 result cards and a 100-step maximum.

C.3 Evaluation and Answer Recovery … D.1 Complete System Results

The appendix details answer recovery, retrieval-engine controls, and complete system results, while documenting asymmetries and implementation divergences that affect comparisons. It then expands the main findings with collection-specific metrics for HotpotQA, MuSiQue, and BCP-S.

  • C.3 Evaluation and Answer Recovery: About 93% of empty runs are recovered by replaying terminal transcripts with answer-only continuation before judged comparisons.Recovered answers are stored in separate overlays and can change system rankings.
  • C.3 Evaluation and Answer Recovery: Recovery overlays cover all BCP-S cells and two Wikipedia Search–Visit baselines, but not Wikipedia SIEVE or other Search–Fetch conditions.This incomplete and asymmetric coverage limits comparisons across groups.
  • C.3 Evaluation and Answer Recovery: Agentic webpage visits and section fetches share a 12,000-token ceiling, while iterative conditions share 100 steps and five results per search call.The webpage comparison therefore changes returned content rather than the maximum available amount.
  • C.4 Retrieval Engine Controls: Zero-hit SIEVE queries first use partial matches by constraint coverage, then fall back to an approximate BM25 ranking when no partial match exists.Single-constraint zero-hit queries use the fallback directly.
  • C.4 Retrieval Engine Controls: 0.546 is the top-5 Jaccard overlap between approximate and Lucene BM25 on BCP-S, while zero-hit fallbacks comprise 36.8–53.7% of SIEVE search calls.Reported results use Lucene for Boolean filtering and ordinary sparse ranking, reserving approximate BM25 for fallback paths.
  • C.4 Retrieval Engine Controls: An Indri-style executor is compared with SIEVE under fixed result cards, section-fetch behavior, result-pool size, and content-access budget.The comparison uses structured-query operators and is excluded from the comprehensive main system comparison.
  • D Full Results and Additional Analyses: The appendix presents complete system results first, followed by component and robustness analyses and failure diagnostics.This ordering follows the main findings.
  • D.1 Complete System Results: The expanded tables report recall, token usage, accumulated prompt tokens, and model-call counts for HotpotQA, MuSiQue, and BCP-S.Recall denotes surfaced gold webpages, or the retrieved set for one-shot systems; the tables follow the main-text collection order.

D.2 Component and Robustness Analyses

The section analyzes SIEVE’s component choices and robustness through snippet ablations, retriever and agent-backbone sensitivity, BQL usage, and additional controls. These experiments compare accuracy, cost, and structured-retrieval behavior under matched configurations.

  • Query-focused snippets: A matched snippet ablation tests the effect of including a 25-token query-focused snippet in each result card.Table 9 reports the corresponding accuracy and cost measurements.
  • Retriever sensitivity: Retriever sensitivity varies only the dense encoder across rows while measuring accuracy and cost.Table 10 provides the complete comparison summarized in Figure 6.
  • Agent-backbone transfer: Agent-backbone transfer pairs BM25 Search–Visit with SIEVE for each backbone and collection under the same serving configuration.Accuracy is reported as the judge verdict on BCP-S and exact match elsewhere, with tokens measured per question.
  • BQL use and additional controls: BQL usage is analyzed by reporting the share of questions in which each operator class appears at least once.Rows overlap because a question may use several operator classes, while no field/Boolean/date excludes those three classes but may include phrases or wildcards.
  • BQL use and additional controls: Additional controls test zero-hit fallback, snippets, and structured-field exposure, including comparisons between Search–Visit over structured and flat corpora.The controls assess whether SIEVE remains effective without ranked fallback and whether exposing structured corpus fields changes Search–Visit.

D.3 Hand-Traced Failure Examples

Hand-traced examples illustrate three operational failure types: retrieval, selection, and synthesis. They distinguish missing gold webpages, surfaced-but-unfetched evidence, and errors after accessing relevant evidence.

  • Failure definitions: The examples are diagnostic rather than representative and support Figure 8’s distinctions between retrieval, selection, and synthesis.Retrieval asks whether a gold webpage appears; selection asks whether its content is accessed; synthesis covers errors after access.
  • Retrieval failures: Retrieval failures occur when no gold webpage identifier appears in any search observation, regardless of later result choice or reasoning.Instances __775, __770, and __774 produce incorrect answers despite never surfacing a gold webpage identifier.
  • Selection failure: Selection failure occurs when gold evidence is surfaced but the agent fetches sections from non-gold webpages instead.For __788, three of four gold webpage identifiers appear, but both fetches target a “largest mosque list” webpage.
Loading 2608.02751v2…