PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Retrieving Interaction Spaces for Agentic Search

Shengyao Zhuang, Yuansheng Ni, Hengxin Fun, Jimmy Lin, Xueguang Ma

arXiv:2606.06880v1cs.IR

TL;DR

Agentic search raises the question of how retrieval can support scalable corpus exploration beyond selecting documents for a context window. RISE constructs a BM25-bounded interaction space with documents processed for shell-based navigation. On BrowseComp-Plus, it matches pure-shell DCI at 78% accuracy with gpt-5.4-mini at one quarter of the per-query cost.

  • Problem

    Agentic search raises the question of how retrieval can support scalable corpus exploration beyond selecting documents that fit the context window.

    Source: §1 · p. 2

  • Method

    RISE constructs a BM25-bounded interaction space and augments its documents offline with structural metadata for shell-based navigation.

    Source: §3.2 · p. 4, §7 · p. 9

  • Results

    78% accuracy on BrowseComp-Plus: RISE matches pure-shell DCI with gpt-5.4-mini at one quarter of the per-query cost.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Agentic corpus interaction is a design problem involving retrieval outputs and document structure, not only retriever quality or agent prompting.

    Source: §7 · p. 9

    Document processing is evaluated only at 100k documents, so the 1M-document scaling result primarily supports the retrieval-boundary design.

    Source: §8 · p. 9

Abstract

from arXiv · show

Retrieval for search agents is still inherited from non-agentic information retrieval: a retriever ranks the corpus and the agent reads a small set of returned documents. Recent direct corpus interaction (DCI) work shows that agents can instead interact with the raw corpus through shell tools such as grep and file reads. But unbounded interaction does not scale: every broad shell command is a scan over the whole corpus, and latency degrades sharply as the corpus grows. We argue that the role of retrieval for agentic search is not just to select documents that fit in the LLM context window, but to construct an interaction space: a bounded subset of the corpus the agent can explore with associated tools. Two design consequences follow. The space needs a boundary supplied by retrieval, and the objects within it should be processed for interaction. As a proof of concept, we propose RISE (Retrieving Interaction SpacE): we use BM25 to construct the interaction space; meanwhile, its documents are processed during indexing for shell-style navigation. On BrowseComp-Plus, RISE matches the pure-shell DCI baseline at 78% accuracy with gpt-5.4-mini at roughly one quarter of the per-query cost. At 1M documents, RISE-BM25 reaches 81% on gpt-5.4-mini, whereas DCI on gpt-5.4-nano degrades to 60% with 33 of 100 wall-clock failures.

1 Introduction

RISE reframes retrieval as constructing a bounded corpus subset that agents explore with tools, with documents processed for shell-style navigation. On BrowseComp-Plus, it matches pure-shell DCI at 78% accuracy with gpt-5.4-mini at one quarter of the per-query cost.

1 Introduction

  • Corpus exploration replaces fixed-context reading through DCI [Li et al., 2026b], file-based processing [Cao et al., 2026], and iterative lexical search [Sen et al., 2026].Traditional retrieval instead ranks the corpus and returns a small set of documents or snippets for the agent’s current prompt.Source: §1 · p. 2
  • 37.5% accuracy at 400k documents and 20 of 100 queries exhausting the tool budget expose DCI’s scaling limits [Li et al., 2026b].Expanding from 100k to 200k documents increases average tool calls from 38.5 to 86.9, more than doubles latency and cost, and reduces accuracy by 13.6 points. Broad grep commands scan the whole corpus, consuming budget on output triage.Source: §1 · p. 2
  • RISE uses BM25 to construct an explicit, persistent interaction space that agents explore with tools outside the context window.The boundary is neither unrestricted full-corpus access, prompt-sized evidence, nor merely a ranked list that cannot be re-traversed with shell tools.Source: §1 · p. 2
  • RISE adds in-place structural metadata during indexing so shell tools can jump to relevant spans without document-scale reads or losing verbatim grounding.Unlike retrieval-time chunks or summaries, this processing does not predetermine visible evidence or discard its original wording. Removing processing at 100k documents lowers accuracy by 1–4 points across model tiers.Source: §1 · p. 2
  • 78% accuracy on BrowseComp-Plus matches pure-shell DCI using gpt-5.4-mini, while RISE costs one quarter as much per query.This comparison evaluates RISE with both retrieval-defined boundaries and document processing.Source: §1 · p. 2
  • 1M documents leave RISE-BM25 stable, while DCI falls to 60% accuracy with 33 of 100 wall-clock failures.This expands the corpus 10×; the scaling result concerns the boundary-only RISE-BM25 variant, not full RISE.Source: §1 · p. 2

2 Related Work

Related work develops agentic search loops, agent-aware ranking, and corpus interfaces. RISE focuses on returning a bounded, shell-navigable workspace rather than context-window snippets or unrestricted access to the raw corpus.

Retrieval-augmented and deep research agents

  • RAG coupled models with retrieval [Lewis et al., 2020]; WebGPT and ReAct established explicit action–observation loops [Nakano et al., 2021, Yao et al., 2023].Deep research systems extend these loops across repeated searches, evidence inspection, and synthesis. Search-R1 learns multi-turn search queries during reasoning [Jin et al., 2025], while WebThinker interleaves reasoning, web navigation, and report drafting [Li et al., 2026a].Source: §2 · p. 2
  • BrowseComp and BrowseComp-Plus evaluate difficult browsing tasks, with BrowseComp-Plus using a fixed corpus to make retrieval choices comparable [Wei et al., 2025, Chen et al., 2025].RISE uses this agentic search setting but asks what corpus interface agents should receive after retrieval selects an initial evidence set.Source: §2 · pp. 2–3

Retrieval and ranking for agentic search

  • Agent-aware retrieval research improves or diagnoses ranking; RISE instead retains simple BM25 and changes retrieval’s output into an externally navigable candidate space.AgentIR embeds reasoning traces with queries [Chen et al., 2026], while Agentic-R trains using passage relevance and answer correctness [Liu et al., 2026]. Meng et al. [2026] compare ranking configurations, while SAGE finds keyword-oriented agent queries can make lexical baselines competitive with dense retrieval [Hu et al., 2026].Source: §2 · p. 3

Corpus interfaces for agentic search

  • RISE returns a bounded filesystem workspace with structural metadata for shell navigation, unlike PI-SERINI’s [Hsu et al., 2026] cached BM25 ranking.Other interface classes deliver ranked snippets into context or expose the entire corpus to shell tools, as in DCI [Li et al., 2026b] and Cao et al. [2026]. Sen et al. [2026] show that retrieval success within agent loops depends strongly on the harness and information delivery.Source: §2 · p. 3

3 Method

RISE retrieves a persistent interaction space outside the model’s context window and processes its documents for shell-based navigation. BM25 supplies the workspace boundary, while offline table-of-contents processing enables targeted reads without discarding original text.

3 Method

  • RISE combines a persistent, retrieval-bounded workspace with documents processed for shell navigation; RISE-BM25 implements only the workspace boundary.The candidate documents remain outside the model’s context window and can be repeatedly traversed with shell tools.Source: §3 · p. 3

3.1 Bounding the Interaction Space

  • The workspace boundary is an explicit, re-explorable candidate set—not full-corpus shell access, a context-window limit, or a one-read ranking.RISE-BM25 implements this as a per-query filesystem directory. Dense, late-interaction, or hybrid retrievers could also supply the boundary; BM25 isolates the interface from retriever engineering.Source: §3.1 · pp. 3–4
  • Multi-query BM25 imports each sub-query’s top-K documents into a persistent working directory, where agents alternate shell inspection and targeted reads.Only search can import documents: it hardlinks their union, preserves corpus-relative paths, and grows the workspace monotonically. Each sub-query returns a top-10 preview, but all top-K matches remain accessible through bash and line-numbered read calls.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 Processing Objects for Interaction

  • In-place structural metadata enables direct access to sub-document evidence while preserving verbatim text, unlike raw files, retrieval-time chunks, snippets, or summaries.Raw files require end-to-end scanning; chunks and snippets restrict re-exploration to selected evidence, while summaries lose the verbatim text needed for verification.Source: §3.2 · p. 4
  • An offline LLM proposes sections and verbatim anchors; deterministic validation adds headings and a table of contents with 1-indexed line ranges.gpt-5.4-nano proposes the structure, and anchors are validated by exact substring matching or a short whitespace-normalized fallback; processing costs about $0.0014 per document at 100k scale. Original body text remains unchanged, and agents must verify committed facts against it rather than relying on the TOC.Source: §3.2 · pp. 4–5
  • TOCs support broad browsing and precise line-range reads; section graphs, paragraph anchors, or learned navigation hints could satisfy the same interaction requirement.The authors choose TOCs as the cheapest in-place metadata supporting both navigation modes; alternatives must likewise expose sub-document evidence without document-scale reads.Source: §3.2 · p. 5

4 Experimental Setup

Experiments compare RISE, its boundary-only ablation, and two baselines on 100 BrowseComp-Plus [Chen et al., 2025] questions using 100k and 1M corpora. Full RISE is evaluated only at 100k, while DCI receives larger model-call and wall-clock budgets.

Dataset and corpora

  • 100 fixed BrowseComp-Plus [Chen et al., 2025] queries are evaluated at both corpus scales, preserving the same questions and gold documents.The 100k setting uses the released corpus; the 1M setting adds 900,000 fixed-seed FineWeb-Edu distractors [Penedo et al., 2024]. The sample size reflects the API and wall-clock expense of full interactive trajectories followed by LLM judging.Source: §4 · p. 5

Agent models and judging

  • Three agents—mimo-v2.5-pro, gpt-5.4-mini with medium reasoning, and gpt-5.4-nano with high reasoning—are evaluated using gpt-5.1 judging.Judging follows the official BrowseComp-Plus Appendix F prompt.Source: §4 · p. 5
  • Evaluation reports judged accuracy, no-answer counts, mean tool calls, mean per-query USD cost, and two gold-document coverage diagnostics.gold_R measures whether gold documents appear in the BM25 candidate union; cov_mean measures whether reads or shell output surface them to the agent. cov_mean is not directly comparable across all systems because their interfaces expose evidence differently.Source: §4 · p. 5

Compared systems

  • RISE combines BM25 bounding with offline TOC processing; RISE-BM25 isolates bounding by retaining original documents.Both use bm25s [Lù, 2024] defaults (k1=1.5, b=0.75), retrieve K=1000 documents per sub-query, provide top-10 previews, and expose accumulated workspaces through bash and line-range reads. Working sets usually contain roughly 10,000 unique files or fewer; full RISE is restricted to 100k because preprocessing the additional distractors was not budgeted, not because of an interface limitation.Source: §4 · p. 5
  • The retrieval-agent uses BM25 document retrieval, while DCI explores the full corpus through bash and read without a retriever.The retrieval-agent reimplements BrowseComp-Plus Appendix E with k=5, 512-token snippets, and get_document returning full document text. DCI uses its original prompt and DCI-Agent-Lite level3 context management with truncation and compaction; its bash and read outputs have DCI-style limits, whereas RISE variants use 4000-character bash truncation, 60-second subprocess caps, and line-range reads with a 2000-line default.Source: §4 · pp. 5–6

Budgets and stopping conditions

  • 300 model calls and 1.5 hours per query are allowed for DCI, versus 100 calls and one hour for RISE, RISE-BM25, and retrieval-agent.These deliberately DCI-favouring limits provide 3× the model calls and 1.5× the wall-clock budget, following its original configuration. Headline runs disable forced final-answer coercion: exhausting either budget without committing an answer produces a no-answer case labelled by stopping condition.Source: §4 · p. 5

5 Results

RISE reaches 78% accuracy with mimo-v2.5-pro and gpt-5.4-mini at lower cost than both baselines, while boundary-only RISE-BM25 remains robust at 1M documents. Larger interaction spaces do not monotonically improve accuracy.

5.1 Headline Results

  • 78% accuracy with mimo-v2.5-pro and gpt-5.4-mini makes RISE best or tied-best on these tiers, at lower cost than both baselines.On gpt-5.4-mini, DCI matches RISE’s accuracy but costs 4× more despite receiving 3× the model-call budget and 1.5× the wall-clock cap. DCI leads the nano tier, but on mimo-v2.5-pro it reaches only 60% accuracy with 18 of 100 wall-clock failures.Source: §5.1 · p. 6
  • RISE-BM25’s close performance suggests bounding the interaction space supplies most gains, while indexing-time document processing adds a smaller improvement.The retrieval-agent remains less accurate than RISE on larger models despite similar BM25 gold-document recall, suggesting insufficient document exposure rather than candidate retrieval is its main limitation.Source: §5.1 · p. 6
  • 82% accuracy makes RISE with full gpt-5.4 at medium reasoning effort the most accurate configuration, but costs 4.4× more than RISE on gpt-5.4-mini.Gold-document context coverage reaches cov_mean=92.4%, versus 82.4% for the strongest smaller-tier configuration. The four-point accuracy gain costs $1.25 versus $0.28 per query and includes eight turn-cap no-answers, leaving the accuracy–cost frontier favoring smaller tiers.Source: §5.1 · pp. 6–7

5.2 Corpus Scaling

  • 75% to 83% (mimo-v2.5-pro), 77% to 81% (gpt-5.4-mini), and 64% to 65% (gpt-5.4-nano) are RISE-BM25’s accuracy changes from 100k to 1M documents.Shell exploration remains confined to the retrieved working set. The authors emphasize robustness rather than improvement: changed BM25 idf statistics or relevant but unannotated added documents could explain the gains.Source: Table 2 · p. 7, §5.2 · p. 7
  • 71% to 60% accuracy and 7 to 33 wall-clock failures mark nano DCI’s deterioration from 100k to 1M documents.The snippet retrieval-agent drops on mimo and nano and remains below RISE-BM25 in every 1M comparison. DCI’s lower reported API cost is not an efficiency gain: full-corpus shell waits exhaust wall-clock budgets after fewer billed model interactions.Source: Table 2 · p. 7, §5.2 · p. 7
  • 1M-document experiments test only boundary-only RISE-BM25, not document processing, because preprocessing was not extended to the 900k added FineWeb-Edu distractors.This is a preprocessing-cost constraint rather than an interface constraint. DCI was evaluated only on nano at 1M because larger models already faced substantial wall-clock pressure at 100k; even the nano run took roughly two days.Source: §5.2 · p. 7, §5.1 · p. 7

5.3 BM25 Top-K Ablation

  • K=1000 per sub-query is best for gpt-5.4-mini and within one accuracy point of the best settings on the other tiers.It produces roughly 7.6k–10.4k-file workspaces; K=100 is competitive and more compact, whereas K=10000 produces 42k–50k files without clear accuracy gains. Because hits become local files rather than context, K mainly trades recall against navigation overhead, with little direct API-cost effect.Source: §5.3 · pp. 7–8, Table 3 · p. 8

6 Case Studies

Two correct agent trajectories illustrate RISE’s intended workflow: retrieval shapes a workspace for verification, while indexing-time processing makes long documents navigable.

6 Case Studies

  • Two correct gpt-5.4-mini trajectories on the 100k corpus answer obfuscated, multi-clue BrowseComp-Plus questions whose answer entities are never named.Together, they illustrate retrieval shaping a workspace for in-place verification and indexing-time processing providing navigable document structure.Source: §6 · pp. 8–9
  • Fifteen paraphrased sub-queries across five search calls construct the RISE-BM25 workspace through BM25 unions before the agent switches to verification with rg and read.Because the agent does not know the target entity, each search targets a different clue cluster rather than retrieving the entity directly.Source: §6 · pp. 8–9
  • Two of four reads in the RISE trajectory use non-zero offsets selected from an LLM-generated, line-numbered table of contents rather than reading the full document.The agent maps a clue to the Acknowledgements entry, reads that range to confirm the paper, then reads the title header for the journal name.Source: §6 · p. 9

7 Conclusion

Retrieval for agentic search should return a bounded, interaction-ready space rather than merely extract context. RISE instantiates this design and matches pure-shell DCI accuracy at lower per-query cost on BrowseComp-Plus.

7 Conclusion

  • Retrieval should construct bounded interaction spaces whose objects are processed for exploration, rather than simply extract context.RISE is a proof of concept: BM25 bounds the space, and documents are offline-augmented with line-numbered tables of contents for shell navigation.Source: §7 · p. 9
  • RISE matches pure-shell DCI accuracy on BrowseComp-Plus at a fraction of its per-query cost.The snippet retrieval-agent remains weaker on the larger model tiers.Source: §7 · p. 9
  • 100k to 1M documents: RISE-BM25 remains stable as the corpus expands, while pure-shell DCI degrades sharply with wall-clock failures.Source: §7 · p. 9
  • Interaction spaces bridge ranked snippets and unrestricted filesystems, letting agents re-explore candidates with shell tools and cheaply reach sub-document evidence.Corpus interaction becomes a design problem involving retrieval outputs and object structure, not only retriever quality or agent prompting. Existing retrieval benchmarks were not designed to assess interaction-space quality.Source: §7 · p. 9

8 Limitations

The experiments test only BM25 boundaries and line-numbered TOCs, with evaluation limited to one benchmark family, 100 queries, closed-weight agents, and one judge. The 1M scaling result primarily tests the boundary mechanism rather than indexing-time document processing.

8 Limitations

  • 1M scaling results primarily support the boundary consequence, because indexing-time processing was evaluated only on the 100k corpus.Rerunning the offline TOC pipeline on the 900k FineWeb-Edu distractors used at 1M exceeded the budget.Source: §8 · p. 9
  • Each design consequence has only one tested implementation: BM25 for boundaries and line-numbered TOCs for indexing-time document processing.Dense, late-interaction, and hybrid retrievers, alongside section graphs and paragraph-level anchors, remain follow-ups for testing the framing’s broader applicability.Source: §8 · p. 9
  • A missing fourth configuration—snippet retrieval over the TOC-augmented corpus—would better separate the filesystem-workspace interface’s contribution from BM25 prefiltering.The authors identify this fully crossed design as a useful next step.Source: §8 · p. 9
  • 100 queries from BrowseComp-Plus, closed-weight agents, and a single gpt-5.1 judge limit evaluation scope, making few-point per-tier accuracy gaps directional rather than precise.The judge shares a model family with two of three agents and the structured-corpus generator.Source: §8 · p. 9

A RISE-BM25 System Prompt

RISE-BM25’s system prompt couples bag-of-words search with an accumulating working directory inspected through bash and read. Agents gather evidence iteratively, then return an explanation, exact answer, and confidence.

A RISE-BM25 System Prompt

  • Agents answer research questions over a corpus they cannot see directly, alternating search and inspection until confident before stopping tool calls.The required final format contains Explanation, Exact Answer, and Confidence, with confidence expressed as 0–100%.Source: A RISE-BM25 System Prompt · p. 12
  • Search queries should use natural-language descriptions with distinctive terms, batching complementary queries for broader coverage because matching treats each query as a bag of words.The search tool accepts one or more queries in a single call.Source: A RISE-BM25 System Prompt · p. 12
  • Hundreds to thousands of matching documents per query accumulate in the working directory across turns, while search previews show only the top 10.Previews include file paths and short snippets; agents must explore beyond them using bash or read.Source: A RISE-BM25 System Prompt · p. 12
  • Bash inspects retrieved plain-text files within the working directory using relative paths, while read opens a file by its full path.The prompt recommends running ls first and using rg or grep to find phrases across documents. Bash cannot access outside the working directory, and absolute paths will not work.Source: A RISE-BM25 System Prompt · p. 12

B Structured Documents: Prompt and Example

RISE retains RISE-BM25’s search and shell interface while adding TOC-augmented documents and prompt instructions to verify facts in document bodies rather than TOC summaries.

B Structured Documents: Prompt and Example

  • 94.5% of documents received validated TOC entries, and 99.3% of proposed section anchors were located in the 100k-corpus processing run.All 100,195 documents were processed without API or postprocessing failures; 802,177 of 808,062 anchors matched exactly or after whitespace normalization. Section proposal used gpt-5.4-nano at low reasoning effort; 22 documents with entirely invalid anchors and four with unparseable output retained original text with empty TOCs.Source: B Structured Documents: Prompt and… · p. 12

B.1 RISE System Prompt (with Document Processing)

  • Search accepts complementary bag-of-words queries, returns per-query top-10 previews, and adds full match sets to a working directory that accumulates across turns.Queries should use natural-language descriptions with distinctive terms; full match sets contain hundreds to thousands of documents, extending exploration beyond the preview.Source: §B.1 · p. 12
  • Agents inspect retrieved files using shell commands confined to the working directory and a read tool supporting line-based slices.Shell paths must be relative; absolute paths cannot access files outside the working directory. The read tool uses zero-indexed offsets and line limits, defaulting to lines 0..2000.Source: §B.1 · pp. 12–13

B.2 Example Structured Document

  • The Bath Spa example illustrates structured files containing YAML metadata, a TOC with one-indexed section ranges and descriptions, a body sentinel, and headed body sections.The abridged example comes from the structured 100k corpus and explicitly marks omitted body text.Source: §B.1 · p. 13, §B.2 · p. 13

B.1 RISE System Prompt (with Document Processing)

  • Reading starts with the document header, expands until the body sentinel appears, and broadens to other sections when expected evidence is missing.The suggested initial read requests 60 lines, increasing to 200 if necessary. Agents inspect multiple plausible sections and read documents without sections normally.Source: §B.1 · p. 13
  • The prompt treats TOC summaries as navigation aids, requiring agents to read relevant sections fully and verify specific facts in the body.TOC summaries can omit details, lose nuance, or be slightly inaccurate; they must not supply specific numbers, names, dates, or quotes.Source: B Structured Documents: Prompt and… · p. 12, §B.1 · p. 13

C Structured Corpus Generation Prompt

The corpus-generation prompt proposes navigable document sections without rewriting the source text. Exact anchors support downstream heading insertion and a line-numbered table of contents, while sectioning rules distinguish prose, catalogs, and procedures.

C Structured Corpus Generation Prompt

  • The offline prompt proposes section boundaries, headings, and descriptions without rewriting documents; a downstream script inserts headings and builds a line-numbered table of contents.The script preserves the original text and includes section descriptions in the table of contents. The prompt requires JSON output without commentary or Markdown fences.Source: C Structured Corpus Generation Prompt · p. 14
  • 2-7-word headings should describe specific content, while 10-25-word descriptions should concretely identify the section's focus.Descriptions should mention key names, dates, places, or numbers when central to the section. Do not section YAML frontmatter or trailing boilerplate unless the latter contains substantial content.Source: C Structured Corpus Generation Prompt · pp. 14–15
  • 6-12-word anchors must reproduce section openings exactly, remain unique within the document, and follow document order; sections without valid anchors must be omitted.Preserve punctuation, capitalization, and internal whitespace without normalization; the first anchor starts at the document body, skipping YAML frontmatter. For list openings, use only the first item's opening words, including any leading list marker.Source: C Structured Corpus Generation Prompt · p. 15
  • ~500-2000 words per prose section is the default cadence, with boundaries at natural topic shifts rather than excessive fragmentation.A 3000-word narrative should yield 2-6 sections, and documents exceeding approximately 1500 characters almost certainly warrant at least two. Empty section lists are reserved for genuinely short single-paragraph documents under approximately 1000 body characters or pure metadata/boilerplate.Source: C Structured Corpus Generation Prompt · p. 15
  • Catalogs warrant aggressive sectioning, often one section per independent item, whereas sequential procedural steps belong together under one umbrella section.The distinction rests on catalog items being independent and procedural steps being dependent and sequential.Source: C Structured Corpus Generation Prompt · p. 15

Related papers

Loading 2606.06880v1…