GrepSeek: Training Search Agents for Direct Corpus Interaction

1 INTRODUCTION

• Direct corpus interaction enables retrieval at arbitrary text granularity through executable search operations, supporting exact matching, lexical filtering, symbolic pattern search, and multi-step evidence composition.Unlike index-based approaches, agents can inspect intermediate results and refine constraints across steps.Source: §1 · p. 2, Figure 1 · p. 2
• GrepSeek makes direct corpus interaction a learned capability of a compact LLM that searches, filters, and composes evidence using shell commands.Contemporary DCI approaches use large proprietary models prompted at inference time; some can take an hour or more per query.Source: §1 · p. 2
• GrepSeek’s two-stage training first builds verified, causally grounded search trajectories with an answer-aware Tutor and answer-blind Planner, then refines the policy with GRPO.The Tutor constructs a backward evidence chain; the Planner turns it into a forward trajectory based on observable history, aligned to verified commands and evidence.Source: §1 · pp. 2–3
• A semantics-preserving sharded-parallel engine distributes compatible shell pipelines across corpus shards, reducing retrieval latency while preserving byte-exact equivalence with sequential execution.The engine addresses I/O and processing bottlenecks when shell pipelines run over multi-gigabyte collections.Source: §1 · p. 3
• GrepSeek achieves the strongest overall performance, with minor degradation on surface-form variation; sharded execution accelerates retrieval up to 7.6×, from 5.39 to 0.71 seconds on average.Average end-to-end latency is approximately 8.6 seconds per query on a single NVIDIA A100 GPU with 80GB VRAM, 32 CPU cores, and 32GB of system RAM.Source: §1 · p. 3

2 OPTIMIZING DIRECT CORPUS INTERACTION SEARCH AGENTS

• The ReAct agent searches a logical corpus file through shell commands, appending command outputs to its interaction history before deciding whether to search again or answer.The agent conditions each reasoning trace and action on the question and prior interactions, and operates for at most T steps.Source: §2 · p. 4, §2.1.1 · p. 4

2.1 TRAINING DCI SEARCH AGENT

• Direct RL on large corpora is unstable because the agent struggles to form effective commands and may retrieve excessive text, so training first uses cold-start data before RL.The paper reports VRAM and host RAM out-of-memory failures even on systems with up to 1024 GB of RAM.Source: §2.1 · p. 4, §2.1.1 · p. 4

2.1.1 COLD-START DATA GENERATION

• Cold-start generation works backward from the gold answer to verify evidence and intermediate bridges, while masking the target answer and its aliases from retrieval commands.The Tutor can refine commands up to M times, executing and checking each one for supporting evidence.Source: §2.1.1 · pp. 4–5
• The verified evidence chain is reversed into forward order, then an answer-blind Planner drafts reasoning from observable history and the Tutor aligns it to justify the verified command.This combines forward causal reasoning with backward-verified evidence construction.Source: §2.1.1 · pp. 5–6
• Cold-start trajectories are retained only when the generated answer has non-zero token-level F1 overlap with the gold answer and a Tutor judge approves causal and logical consistency.The judge rejects reasoning or commands that reveal entities or facts not yet observable in the interaction history.Source: §2.1.1 · p. 6

2.1.2 OPTIMIZATION OF DCI SEARCH AGENT

• SFT initializes structured retrieval behavior before GRPO, whose answer-F1 reward is zeroed for trajectories that violate the required interaction format.GRPO samples groups of five trajectories per query and normalizes rewards within each group; the SFT initialization is intended to improve optimization stability.Source: §2.1.2 · p. 6

2.2 EFFICIENT CORPUS INTERACTION

• The execution engine parallelizes compatible shard-independent pipelines while preserving byte-exact equivalence with sequential execution, and falls back to sequential processing when a command is unsafe to shard.It reconstructs results using strategy-specific reductions, including ordered concatenation, top-N selection, count aggregation, or deterministic k-way merging for sorted output.Source: §2.2 · p. 7
• A persistent search daemon keeps the corpus in memory and reuses long-lived workers across tool calls, avoiding repeated process startup and corpus loading.Commands use memory-mapped search primitives, and retrieval performance is primarily limited by memory bandwidth and data access patterns.Source: §2.2 · p. 7

3.1 EXPERIMENTAL SETUP

• Evaluation covers seven QA datasets using a 21M-document Wikipedia corpus; training uses NQ and HotpotQA, with the other datasets testing out-of-distribution generalization.Token-level F1 is the primary metric, and EM is reported in the appendix.Source: §3.1 · pp. 7–8

3.2 MAIN FINDINGS

• GrepSeek achieves the strongest overall micro-average F1 (0.5691) and leads on 4 of 7 benchmarks, with significant gains on 3 and its only significant drop on PopQA.The strongest baseline is Search-R1; GrepSeek’s largest gains occur on multi-hop benchmarks, while TriviaQA and Bamboogle differences are not statistically significant.Source: §3.2 · pp. 8–9, Table 1 · p. 9
• GrepSeek significantly outperforms variants without SFT or RL across datasets; removing SFT causes the largest overall drop, while removing RL especially harms multi-hop performance.Even 2.5k SFT trajectories improve over the base model across benchmarks; 5k and 10k improve further, with micro-average gains beginning to plateau beyond 5k.Source: Table 2 · p. 9, §3.2 · pp. 10–11, Figure 4 · p. 10
• 8.67 s per query for GrepSeek exceeds E5 (4.77 s) and Qwen3-4B (6.07 s), but it uses 14 GB memory and avoids embedding precomputation.Tool interaction takes 0.81 s; E5 and Qwen3-4B use 70 GB and 221 GB, respectively, and require 3.2 and 62.4 A100-hours for indexing, versus about 1 minute of setup for GrepSeek.Source: Figure 3 · p. 10, §3.2 · p. 10
• Command latency falls from 5.39 s at one shard to 1.22 s at 8 shards and 0.71 s at 32, with gains plateauing as hardware constraints bind.The tested shard counts are 1, 2, 4, 8, 16, and 32.Source: Figure 3 · p. 10, §3.2 · p. 10
• During GRPO, GrepSeek earns higher average rewards than retrieval baselines but generates longer responses and gradually uses fewer search commands, composing more expressive pipelines.Across training, commands per trajectory decrease from 3.06 to 2.56, extracted context grows (about 5 to 10 lines), and reasoning tokens rise from 4,251 to 6,409; SFT-established pipeline structure remains largely stable.Source: Figure 5 · p. 11, §3.2 · p. 11, Table 3 · p. 11

3.3 ANALYSIS

• Across benchmarks, GrepSeek commonly uses output truncation, fixed-string matching, and cascaded filtering, issuing more retrieval commands on multi-hop than single-hop datasets.About 70% of commands use cascaded filtering; the agent averages 2.6–3.4 commands on multi-hop datasets versus 2.0–2.4 on single-hop datasets.Source: §3.3 · p. 11
• Shell search handles rare symbols, exact names, and multi-hop entity linking precisely, but exact matching and file-order results make it brittle to lexical variation and ambiguous keywords.The paper reports a significant PopQA drop and notes dense retrievers can better handle surface-form variation; GrepSeek also sometimes misses evidence with spelling or diacritic variation or returns less-informative earlier matches.Source: §3.2 · p. 9, §3.3 · p. 12

Retrieval-Augmented Agentic Search

• RAG retrieves evidence from an external corpus to condition generation, commonly through indexed sparse retrieval such as BM25 or dense retrieval such as E5 (Wang et al., 2022).The section frames retrieval as providing access to non-parametric knowledge when model memory is missing, outdated, or unreliable (Mallen et al., 2023).Source: §4 · p. 12

Direct Interaction with Corpus

• GrepSeek trains a compact open-weight model to use deterministic shell-based operations directly on the corpus rather than to use a fixed retriever.DCI lets the agent control how evidence is matched, filtered, and composed as part of multi-step reasoning.Source: §4 · pp. 12–13

Retrieval-Augmented Agentic Search

• Agentic search ranges from prompting systems such as Search-O1 (Li et al., 2025), to retriever-fixed training such as Search-R1 (Jin et al., 2025), to joint optimization such as CoSearch (Zeng et al., 2026).Multi-turn methods interleave reasoning and retrieval, as in IRCoT (Trivedi et al., 2023), and may refine operations over structured data, as in STARQA (Maddela et al., 2025).Source: §4 · p. 12
• The evaluation follows prior search-agent work such as Search-R1 (Jin et al., 2025) on single-hop and multi-hop QA, while broader deep-search benchmarks remain future work.The section names BrowseComp (Wei et al., 2025) and Total Recall QA (Rafiee et al., 2026) as broader deep-research settings not evaluated here.Source: §4 · pp. 12–13

Direct Interaction with Corpus

• Prior DCI work spans code and repository search (Di Grazia & Pradel, 2023; Wang et al., 2026), open-domain retrieval agents (Sen et al., 2026; Subramanian et al., 2025), and efficient text-search systems.The paper characterizes the latter systems as efficiency-oriented predecessors, focusing instead on whether an LLM can learn to use direct corpus operations for multi-step reasoning.Source: §4 · p. 13

Conclusion

• GrepSeek bypasses precomputed indexes with shell-based corpus search and combines cold-start SFT with GRPO to learn interpretable, lexically precise retrieval programs.The authors report strong performance on challenging multi-hop reasoning by isolating symbolic patterns and enforcing entity-level constraints, including cases where dense embedding models fail through semantic conflation; its sharded-parallel engine reduces runtime memory requirements and eliminates offline indexing.Source: §5 · p. 13
• Purely lexical retrieval remains sensitive to surface-form variation, such as diacritics, and lacks semantic relevance ranking.Source: §5 · p. 13

Future Work

• Future work will investigate hybrid retrieval, richer shell matching primitives, and more efficient inference through compact trajectories and improved context management.The goal is more efficient deployment in high-throughput settings.Source: §5 · p. 13
• The authors also plan to evaluate GrepSeek on long-form question answering, ad hoc document retrieval, and retrieval from unseen corpora.Source: §5 · p. 13

A DATASETS

• Following prior work (Jin et al., 2025), the evaluation uses seven benchmarks from FlashRAG, selected to cover single-step fact retrieval and complex multi-step reasoning.All datasets are obtained from FlashRAG to standardize formatting and evaluation protocol.Source: A DATASETS · p. 18

Single-Hop Datasets

• Single-hop tasks test retrieval of one relevant fact or document, using open-domain Natural Questions, compositional TriviaQA (Joshi et al., 2017), and rare-entity PopQA (Mallen et al., 2023).PopQA specifically probes long-tail knowledge where LLM parametric knowledge typically fails.Source: §A.1 · p. 18

Multi-Hop Datasets

• Multi-hop tasks require interdependent queries; HotpotQA (Yang et al., 2018) requires evidence across two or more Wikipedia articles, while 2WikiMultihopQA tests explicit reasoning chains across documents.HotpotQA requires synthesizing information from multiple sources; 2WikiMultihopQA uses compositional questions generated from Wikidata properties.Source: §A.1 · p. 18
• MuSiQue (Trivedi et al., 2022) chains single-hop questions to discourage shortcut reasoning, while Bamboogle (Press et al., 2023) requires deep evidence gathering beyond surface-level snippets or simple entity linking.MuSiQue is filtered to prevent answers from being guessed through lexical overlap or single-document retrieval; Bamboogle questions were manually authored to defeat standard search engines.Source: §A.1 · pp. 18–19

Evaluation Corpus

• Evaluation uses the 2018 Wikipedia dump of 21M documents as its corpus.The dump citation is Karpukhin et al. (2020).Source: §A.1 · p. 19

A.2 DATA SPLITS AND TRAINING PROTOCOL

• Training uses 79, 168 NQ and 90, 447 HotpotQA examples, totaling 169, 615 examples.The combined training data exposes the agent to single-hop retrieval and multi-hop reasoning.Source: §A.2 · pp. 19–20
• Evaluation covers seven datasets and 51, 713 total queries, using official test splits where publicly available.The evaluation tests held-out in-domain performance and generalization to unseen datasets.Source: §A.2 · p. 20

• EM measures the percentage of predictions matching the gold answer exactly after lowercasing, punctuation removal, and article stripping.It serves as a measure of final answer correctness.Source: §A.3 · p. 20
• Token-level F1 measures the harmonic mean of precision and recall over normalized answer tokens, taking the maximum score across multiple references when available.Source: §A.3 · p. 20
• The main paper reports F1, while EM is additionally provided in the appendix.Source: §A.3 · p. 20

B GR E PSE E K’S IMPLEMENTATION DETAILS

• This appendix provides implementation details and settings used for GrepSeek.Source: B GR E PSE E K’S IMPLEMENTATION DETAILS · p. 21

B.1 PROMPTS

• An answer-aware Tutor constructs and validates backward multi-hop retrieval paths, then an answer-blind Planner forms forward, history-grounded trajectories that a coherence judge checks for leakage.Tutor edits align Planner reasoning with verified commands while keeping it grounded in observable interaction history.Source: §B.1 · p. 21

B.2 EFFICIENT CORPUS INTERACTION

• The execution engine follows a correctness-first principle, using a single-file fallback whenever byte-identical parallel execution cannot be guaranteed.This system-level approach is described as an optimization of the DCI agent’s command-execution engine.Source: §B.2 · p. 22

B.2.1 CORPUS SHARDING AND PARALLEL FAN-OUT

• Line-aligned sharding keeps JSON records intact and reconstructs the original corpus unchanged; concurrent shard scans reduce latency roughly in proportion to shard count, subject to I/O and memory bandwidth.At inference, independent shard commands run concurrently using a thread pool.Source: §B.2.1 · p. 24

B.2.2 PIPELINE CLASSIFICATION AND MERGE STRATEGIES

• Supported pipelines use shard-order concatenation, local-plus-global head processing, summed shard counts, or sorted merges; unsafe or stateful pipelines fall back to sequential execution.The fallback protects byte-exact equivalence when parallel execution is not safe.Source: §B.2.2 · pp. 24, 27

B.2.3 I/O AND SYSTEM-LEVEL OPTIMIZATIONS

• When memory permits, the engine stages shards in RAM and warms the page cache; deterministic tool flags, disabled user configuration, and LC_ALL=C support reproducible, efficient literal searches.Memory mapping reduces system-call overhead, while LC_ALL=C enables bytewise matching without changing literal search semantics.Source: §B.2.3 · p. 27

B.2.4 PERSISTENT DAEMON ARCHITECTURE AND TELEMETRY

• The persistent ShardedSearchEngine daemon communicates over a length-prefixed JSON Unix-socket protocol, reducing per-invocation latency by approximately 1–3 milliseconds.Keeping the engine running avoids repeated Python-wrapper initialization and process startup within a trajectory.Source: §B.2.4 · p. 28
• Each invocation logs its parsed command, merge strategy, shard configuration, and fallback decision, enabling analysis of fast-path use during inference.The engine also preserves Unix failure semantics and deduplicates standard-error streams.Source: §B.2.4 · p. 28

B.3 REWARD FUNCTION

• Answer correctness uses normalized token-level F1: text is lowercased, punctuation and articles are removed, and tokens are whitespace-separated; empty predictions or references score zero.Precision and recall use multiset token overlap; normalization follows Rajpurkar et al. (2016).Source: §B.3 · p. 28
• The answer reward takes the maximum F1 across gold references, providing dense partial credit rather than a sparse binary exact-match signal.This accommodates multiple valid answer surface forms.Source: §B.3 · pp. 28–29
• A rollout earns format validity only when its special tags are balanced and non-overlapping, its content follows the prescribed state transitions, and it ends with </answer>.No text may appear outside designated blocks.Source: §B.3 · p. 29
• The format indicator mirrors cold-start SFT constraints and gates learning: only structurally valid trajectories receive nonzero reward, preventing malformed outputs from exploiting it.The indicator isolates and penalizes formatting violations introduced during reinforcement-learning exploration.Source: §B.3 · p. 29

SFT Training Phase

• 27B Qwen3.5 Tutor and Planner models generated 10,000 cold-start trajectories, with a dynamic Tutor temperature schedule and up to five backward-refinement steps.The Tutor temperature was 0.4 + 0.1 × iter; Planner assembly and the coherence judge used fixed temperatures of 0.7 and 0.6, respectively, with top-p = 1.0.Source: §B.4 · pp. 29–30
• The 9B Qwen3.5 policy was fine-tuned for one epoch with AdamW at a peak learning rate of 5 × 10^-6 and a global batch size of 32.Training used a 16,384-token sequence limit, 5% linear warmup followed by a constant schedule, bfloat16, and four A100 80GB GPUs.Source: §B.4 · p. 30

GRPO Training Phase

• GRPO refined the SFT checkpoint for 200 global steps, sampling five trajectories per query and rewarding token-level F1 gated by valid formatting.Group-centered, standard-deviation-normalized rewards avoided an explicit critic; training used a symmetric 0.2 PPO clip ratio, no KL penalty, and one proximal epoch per step.Source: §B.4 · p. 30
• Rollouts used temperature 1.0 and top-p 1.0, with trajectories capped at 16,384 tokens and six assistant turns.Updates used AdamW at 5 × 10^-6 after 5% warmup, a global batch of 256 questions, and bfloat16 Ulysses parallelism of size 2; runs took approximately four days on four A100 80GB GPUs.Source: §B.4 · p. 31

Inference Setting

• Inference used temperature 0.6, top-p 1.0, a 2,048-token shell-output limit, and a six-turn trajectory cap.The configuration allowed 16,384-token sequences and 256 concurrent sequences, using two A100 GPUs with tensor parallel size 2.Source: §B.4 · p. 31

• GrepSeek achieves the highest overall micro-average EM, 0.4948, significantly outperforming the best dense retrieval baseline (p < 0.05).The passage says the conclusions from the F1 analysis remain consistent under EM.Source: C ADDITIONAL RESULTS · p. 32
• GrepSeek leads EM on four of seven benchmarks—significantly on NQ, HotpotQA, and 2Wiki—but drops significantly on PopQA and trails Search-R1’s strongest Qwen3-4B dense-retriever configurations on TriviaQA and Bamboogle.The passage attributes this limitation to exact string-level matching, which is sensitive to surface-form variation and semantic paraphrasing; it describes direct corpus interaction as particularly effective for precise multi-hop reasoning, iterative evidence aggregation, and exact lexical matching.Source: C ADDITIONAL RESULTS · p. 32
• Removing GRPO reduces micro-average EM from 0.4948 to 0.3569, while removing SFT initialization lowers it to 0.2836; the full model outperforms both ablations across all datasets.The passage links the no-SFT degradation to instability when RL is applied without structured trajectory bootstrapping.Source: C ADDITIONAL RESULTS · p. 32, Table 9 · p. 32
• With cold-start SFT, 2.5k trajectories substantially improve over the base model, and gains remain consistent but diminish as the dataset grows, plateauing beyond 5k examples.The EM scaling trends closely match those reported for F1; evaluated sizes are 0, 2.5k, 5k, and 10k trajectories.Source: C ADDITIONAL RESULTS · p. 32

Case-study overview

• GrepSeek’s shell commands support exact lexical matching and iterative evidence filtering, especially for multi-hop reasoning, entity disambiguation, and cases where small lexical details determine correctness.The paper contrasts this with dense retrievers’ tendency to smooth over distinctions through semantic compression, sometimes causing incorrect generalization or entity confusion.Source: D CASE STUDIES · p. 33

Limitations of Ranking and Surface Forms (Examples 6, 7, and 8)

• Without relevance ranking, GrepSeek can bury correct evidence in file order, and strict surface matching can fail on spelling or diacritic variations where dense retrieval generalizes.These examples expose a trade-off between lexical control and semantic robustness.Source: D CASE STUDIES · p. 33

Symbolic and Rare-Token Matching (Example 1)

• An exact-string search demonstrates GrepSeek’s ability to locate a specific chemical formula that the dense retriever fails to identify.The case study presents rare symbolic strings as a setting where vector-space retrieval can return chemistry-adjacent material without the exact formula.Source: D CASE STUDIES · p. 33

Entity Precision and Disambiguation (Examples 2 and 3)

• GrepSeek distinguishes a subsidiary studio from its parent brand and uses a school’s full name to avoid a name-collision error that misleads dense retrieval.In the school example, the wrong entity leads the dense-retrieval reasoning toward an incorrect city.Source: D CASE STUDIES · p. 33

Multi-Hop Evidence Composition (Example 4)

• By filtering first for a band and then for “singer,” GrepSeek isolates a person and composes evidence across multiple steps to answer a distinctive “highest [noun]” query.The case frames the corpus as a structured database that can be searched iteratively.Source: D CASE STUDIES · p. 33

Temporal and Stale Information (Example 5)

• For a current record-holder question, GrepSeek retrieves the document naming the current holder, while dense retrieval is misled by stale information about the CN Tower.The case study contrasts Tokyo Skytree’s current-record context with the older CN Tower information.Source: D CASE STUDIES · p. 33

Citibank case

• Grep follows a misleading 1863 reference to Abraham Lincoln, while dense retrieval surfaces Citibank’s 1812 founding and correctly answers James Madison.The case attributes the difference to grep’s file-order output burying the authoritative entity page, while dense retrieval ranks it first and resolves the two-hop question.Source: D CASE STUDIES · p. 39

Walter W. Arndt case

• Grep finds Walter W. Arndt’s birthplace as Istanbul, while dense retrieval conflates him with a similarly named Walter Arndt and returns no answer.The case notes that an exact full-name match identifies the right entity, but the agent continues searching until it reaches its turn limit.Source: D CASE STUDIES · p. 40

Édouard Vaillant case

• A grep search for Édouard Vaillant misses the diacritic-sensitive name and yields the wrong answer, Paris; dense retrieval finds the correct birthplace, Vierzon.The case describes this as lexical brittleness when the page is not reachable by the surface form, leading the agent to fall back on parametric memory.Source: D CASE STUDIES · p. 41

E EXAMPLES OF GENERATED SYNTHETIC TRAJECTORIES

• The three trajectories illustrate step-by-step reasoning that uses shell commands to retrieve and cross-check evidence.They exemplify a geographical intersection, multi-hop compositional reasoning, and cross-domain bridging.Source: E EXAMPLES OF GENERATED SYNTHETIC… · p. 42

SFT Example 1. MULTI-HOP INTERSECTION (3 HOPS)

• The intersection example cross-checks separate and joint mentions of Fetteresso Castle and Cowie Castle, concluding that both are in Scotland.The agent searches for each castle, then filters for passages mentioning both before answering.Source: E EXAMPLES OF GENERATED SYNTHETIC… · p. 42

SFT Example 2. COMPOSITIONAL (2 HOP)

• The compositional example identifies Dwight D. Eisenhower as the 34th U.S. president, then answers that R Adams Cowley developed his prototype pacemaker.The trajectory searches first for the president and then for passages linking Eisenhower and pacemakers.Source: E EXAMPLES OF GENERATED SYNTHETIC… · p. 42

SFT Example 3. TWO-ROLE BRIDGE (3 HOPS)

• The two-role bridge example links Iris Carr in The Lady Vanishes and Riley Blue in Sense8 to the same actress, Tuppence Middleton.The trajectory searches for each role and then broadens the Iris Carr search to check for conflicting information.Source: E EXAMPLES OF GENERATED SYNTHETIC… · p. 43