PaperlayerView exact revision on arXiv

Source-linked AI summary

GrepSeek: Training Search Agents for Direct Corpus Interaction

Alireza Salemi, Chang Zeng, Atharva Nijasure, Jui-Hui Chung, Razieh Rahimi, Fernando Diaz, Hamed Zamani

arXiv:2605.29307v1cs.CLcs.AIcs.IRcs.LG

TL;DR

Existing search agents typically rely on indexed document representations, limiting direct, fine-grained interaction with raw corpora. GrepSeek trains a compact agent for executable shell-based corpus search and achieves the strongest overall performance across seven question-answering benchmarks, while remaining challenged by limited lexical overlap.

  • Problem

    Existing retrieval systems rely on indexed document representations and ranked documents, motivating finer-grained search directly over unstructured corpora.

  • Method

    GrepSeek trains a compact direct-corpus-interaction agent with verified Tutor-Planner trajectories, GRPO refinement, and semantics-preserving parallel shell execution.

  • Results

    GrepSeek achieves the best performance on 4 of 7 benchmarks, with statistically significant improvements on 3, and especially strong gains on multi-hop tasks.

  • Takeaways & Limitations

    Direct corpus interaction provides a competitive search-agent approach for precise lexical retrieval and compositional reasoning, particularly on multi-hop questions.

  • Takeaways & Limitations

    GrepSeek is less effective on datasets with limited lexical overlap or semantically broad and ambiguous phrasing.

Abstract

from arXiv · show

Large Language Model (LLM) search agents have shown strong promise for knowledge-intensive language tasks through multiple rounds of reasoning and information retrieval. Most existing systems access information using a retriever that takes a keyword or natural language query and returns a ranked list of documents using an index of pre-computed document representations. In this work, we explore a complementary perspective in which the search agent treats the corpus itself as the search environment and finds evidence by issuing executable shell commands. We introduce GrepSeek, an optimized direct corpus interaction (DCI) search agent that trains a compact search agent to find, filter, and compose evidence from large text corpora. To address the instability of learning behavior directly with reinforcement learning on large corpora, we propose a two-stage training pipeline. First, we construct a cold-start dataset using an answer-aware Tutor and answer-blind Planner to generate verified, causally grounded search trajectories. Second, we refine the initialized policy with Group Relative Policy Optimization (GRPO), allowing the agent to improve its task-oriented search behavior through direct interaction with the corpus. To make DCI practical at scale, we further use a semantics-preserving sharded-parallel execution engine that accelerates shell-based retrieval by up to $7.6\times$ while preserving byte-exact equivalence with sequential execution of the shell command. Experiments across seven open-domain question answering benchmarks show that GrepSeek achieves the strongest overall token-level $F_1$ and Exact Match. Our analysis also highlights the limitations of purely lexical interaction on queries with substantial surface-form variation, suggesting DCI as a practical and competitive method for search agents that can complement existing retrieval paradigms in the real world.

1 INTRODUCTION

GrepSeek reframes search as direct corpus interaction, letting a compact agent retrieve text at arbitrary granularity through executable shell commands rather than pre-computed document indices. It combines verified Tutor–Planner trajectories with semantics-preserving sharded parallelism to support efficient, fine-grained evidence discovery over large corpora.

  • Motivation: Direct corpus interaction retrieves text at any granularity, enabling more surgical retrieval than fixed document chunks, representations, and relevance scores.The approach avoids restricting retrieval to predetermined text units.
  • GrepSeek: GrepSeek trains a compact LLM to search, filter, and compose evidence over large text corpora through executable shell commands.This shifts direct corpus interaction beyond inference-time prompting toward a trained search agent.
  • Training: The two-stage pipeline uses an answer-aware Tutor to construct verified backward command chains and an answer-blind Planner to convert them into forward, causally valid trajectories.Backward construction supports complex and multi-hop questions by identifying evidence one hop at a time while preserving an explicit answer-to-question chain.
  • Scalability: A semantics-preserving sharded-parallel execution engine addresses the I/O and processing bottlenecks of sequential shell pipelines over multi-gigabyte corpora.The engine is designed to keep retrieval latency manageable for interactive agents operating over corpora containing millions of documents.
  • Search behavior: Exact string-matching pipelines preserve fine-grained lexical distinctions and isolate rare symbolic patterns, exact entity names, and bridge entities needed for compositional reasoning.The gains are especially pronounced on multi-hop reasoning tasks, where retrievers can introduce semantic conflation and entity ambiguity.

2 OPTIMIZING DIRECT CORPUS INTERACTION SEARCH AGENTS

GrepSeek trains a ReAct search agent to interact directly with a corpus through executable shell commands. Its two-stage pipeline uses verified cold-start trajectories before supervised fine-tuning and GRPO refinement to stabilize corpus search.

  • DCI Search Agent: The DCI agent operates in ReAct, repeatedly generating reasoning traces and shell-tool actions from the question and prior interaction history.The corpus is exposed as a unified logical file whose lines correspond to documents.
  • Training Challenges: Direct reinforcement-learning optimization is unstable because the agent produces ineffective commands and retrieves excessively large corpus segments.Such behavior increases context length and can trigger VRAM or host-RAM failures, even with up to 1024 GB of RAM.
  • Cold-Start Trajectory Generation: The cold-start pipeline combines an answer-aware Tutor’s backward evidence verification with an answer-blind Planner’s forward reasoning trajectories.Target masking prevents backward retrieval from using the answer or its aliases, while Tutor alignment grounds Planner reasoning in observable interaction history.
  • Cold-Start Trajectory Generation: The Tutor proposes and executes shell commands iteratively, retaining documents only after verifying that they support the current target answer.Up to M refinement attempts improve retrieval robustness and filter brittle or spurious trajectories.
  • Two-Stage Optimization: Supervised fine-tuning initializes stable retrieval and reasoning behavior, after which GRPO optimizes policies from sampled interaction trajectories using answer rewards based on token-level F1.For each query, GRPO samples a group of n = 5 trajectories.

3 EXPERIMENTS

Experiments evaluate GrepSeek on seven open-domain QA benchmarks using a 21M-document Wikipedia corpus and compare it with direct, retrieval-augmented, and agentic-search baselines. GrepSeek shows strong overall performance and benefits from both cold-start SFT and GRPO, while shell-based interaction yields interpretable lexical retrieval with higher end-to-end latency but low execution cost.

  • Datasets & Evaluation: Evaluation spans seven single-hop and multi-hop QA benchmarks, using a 21M-document 2018 Wikipedia dump and token-level F1 as the primary metric.Training uses only NQ and HotpotQA, with the remaining datasets serving as out-of-distribution tests.
  • Comparison of Performance with Baselines: GrepSeek substantially outperforms direct inference, standard RAG, untrained agentic methods, and rejection sampling across the evaluated settings.Search-R1 is identified as the strongest competitor among the baselines.
  • Efficiency: 8.67 s is GrepSeek’s end-to-end inference latency per query, versus 4.77 s for E5 and 6.07 s for Qwen3-4B, while tool interaction requires 0.81 s.The higher latency is primarily attributed to longer reasoning trajectories and increased LLM decoding time, which takes 7.86 s.
  • Efficiency: 5.39 s at one shard falls to 1.22 s at 8 shards and 0.71 s at 32 shards, with near-linear speedups at smaller shard counts and plateauing gains later.The measurements concern command execution latency as corpus shard parallelism increases from 1 to 32 shards.
  • Ablations of GrepSeek: Both synthetic cold-start SFT and subsequent RL optimization are critical, as GrepSeek significantly outperforms variants that omit either stage across all datasets.Cold-start scale experiments show further gains from 2.5k to 5k and 10k trajectories, with progressively smaller improvements and micro-average plateauing beyond 5k.
  • Retrieval Behavior: GrepSeek’s shell interaction uses structured lexical operations, including exact-string matching, cascaded filtering, and | head truncation, while RL mainly shapes higher-level search behavior.Low-level pipeline properties remain largely stable during RL, indicating that retrieval primitives are established during SFT.

4 RELATED WORK

Related work frames search agents as systems that retrieve external evidence through multi-turn reasoning, while DCI instead lets agents operate directly on raw corpora. Question answering, especially multi-hop QA, is a primary evaluation setting, whereas broader deep-search benchmarks remain future work.

  • Retrieval-Augmented Agentic Search: Retrieval-augmented methods address missing or unreliable model knowledge by retrieving external evidence and conditioning generation on it.RAG connects language models to external corpora through retrieved context.
  • Retrieval-Augmented Agentic Search: Multi-turn retrieval methods iteratively identify entities, issue follow-up queries, retrieve additional evidence, and compose information across documents.This treats retrieval as part of reasoning rather than only as preprocessing.
  • Retrieval-Augmented Agentic Search: Agentic search systems vary in what they optimize: prompting and orchestration, language-model query generation with a fixed retriever, or other pipeline components.Search-O1 exemplifies black-box orchestration, while Search-R1 trains the language model and keeps the retriever fixed.
  • Evaluation: Multi-hop question answering is widely used to evaluate search systems because it tests iterative evidence retrieval and synthesis across steps.Broader deep-search benchmarks such as BrowseComp and Total Recall QA are acknowledged but left for future evaluation.
  • Direct Interaction with Corpus: Direct Corpus Interaction lets agents issue explicit operations over raw corpora instead of relying on a retriever to rank passages.DCI controls how evidence is matched, filtered, and composed, extending prior direct textual search work in code and repository settings.

5 CONCLUSION & FUTURE WORK

GrepSeek is presented as a Direct Corpus Interaction search agent that operates on raw text with Unix shell commands and is trained through cold-start SFT followed by GRPO-based reinforcement learning. Future work targets hybrid retrieval, richer shell matching, and improved inference efficiency.

  • Conclusion: GrepSeek bypasses pre-computed search indexes by operating directly over raw text corpora with standard Unix shell commands.The approach is described as effective, interpretable, and lexically precise.
  • Conclusion: Its two-stage training pipeline combines synthetically generated cold-start SFT with reinforcement learning using GRPO.This pipeline enables search agents to learn shell-based retrieval programs.
  • Future Work: Future work will investigate hybrid architectures combining direct corpus interaction with index-based retrieval models.This direction is intended to address the approach’s identified issues.
  • Future Work: The authors plan to improve the shell interface with richer matching primitives, including fuzzy matching and advanced regular-expression operators.These additions aim to enhance expressiveness and robustness.
  • Future Work: They will also pursue greater inference efficiency by reducing decoding overhead.The supplied passage introduces this direction but is truncated before specifying further details.

A DATASETS

The method is evaluated on seven knowledge-intensive benchmark datasets covering both single-step fact retrieval and complex multi-step reasoning. All datasets come from the FlashRAG repository to standardize formatting and evaluation.

  • A DATASETS: The evaluation uses seven knowledge-intensive benchmark datasets.The suite follows prior work by Jin et al. (2025).
  • A DATASETS: The datasets assess both single-step fact retrieval and complex multi-step reasoning.
  • A DATASETS: All datasets are obtained from the FlashRAG repository to standardize formatting and evaluation protocols.

A.1 EVALUATION BENCHMARKS

The evaluation suite separates single-hop retrieval from multi-hop corpus exploration across seven open-domain QA datasets. All evaluations use a 21M-document dump of Wikipedia as the corpus.

  • Evaluation structure: The suite divides benchmarks into single-hop tasks for targeted retrieval and multi-hop tasks for iterative, interdependent corpus exploration.Single-hop tasks require one highly relevant fact or document, whereas multi-hop tasks gather partial information across successive searches.
  • Single-Hop Datasets: Single-hop evaluation uses Natural Questions, TriviaQA, and PopQA, covering user queries, complex trivia, and rare-entity knowledge requiring external retrieval.Natural Questions uses open-domain questions answered from Wikipedia passages; TriviaQA answers typically come from one document; PopQA targets long-tail entities from Wikidata triples.
  • Multi-Hop Datasets: Multi-hop evaluation uses HotpotQA, 2WikiMultihopQA, MuSiQue, and Bamboogle, requiring evidence synthesis across documents and resistance to shortcut reasoning.These datasets require reasoning chains, multiple-source synthesis, filtered compositional questions, or manually authored questions designed to defeat standard search.
  • Evaluation corpus: 21M documents from the 2018 Wikipedia dump form the corpus used for evaluation.The dump is attributed to Karpukhin et al. (2020).

A.2 DATA SPLITS AND TRAINING PROTOCOL

GrepSeek trains on a strict in-distribution split and evaluates on out-of-distribution datasets to measure generalization. The protocol combines NQ and HotpotQA training examples, then evaluates across all seven datasets using official test splits where available.

  • Training data: 169,615 training examples combine NQ and HotpotQA training splits, exposing the agent to single-hop retrieval and multi-hop reasoning.The component counts are 79,168 NQ examples and 90,447 HotpotQA examples.
  • Corpus setup: The shared corpus is corpus.jsonl, treated as a large text file with one Wikipedia passage per line.The corpus contains 21 million lines, and output is capped with head for narrow searches or broader article scans.
  • Evaluation protocol: 51,713 total queries comprise the final evaluation across all seven datasets, including held-out in-domain and unseen-dataset performance.Official test splits are used where publicly available.

A.3 EVALUATION METRICS

The evaluation uses Exact Match and token-level F1 to assess open-domain question-answering responses. EM measures exact normalized-answer correctness, while F1 captures partial token overlap; the paper reports F1 in the main text and EM in the appendix.

  • Exact Match (EM): Exact Match (EM) measures the percentage of predictions matching the gold answer exactly after lowercasing, punctuation removal, and article stripping.It serves as a measure of final answer correctness.
  • F1 Score: Token-level F1 computes the harmonic mean of precision and recall over normalized predicted and reference-answer tokens.When multiple references exist, the maximum F1 across references is reported.
  • Reporting: The main paper reports F1, while Exact Match is additionally provided in the appendix.Both are standard metrics used across the benchmark datasets.

B GR E PSE E K’S IMPLEMENTATION DETAILS … B.2.3 I/O AND SYSTEM-LEVEL OPTIMIZATIONS

GrepSeek’s implementation combines causally constrained prompt-based trajectory generation with a correctness-first shell execution engine. Its corpus interaction stack uses sharding, deterministic pipeline-specific merging, and I/O optimizations to accelerate retrieval while preserving sequential behavior when required.

  • B GR E PSE E K’S IMPLEMENTATION DETAILS: The implementation details define GrepSeek’s agent prompts, corpus interaction mechanisms, and system-level execution settings.These details span prompt design, efficient corpus interaction, pipeline execution, and I/O optimization.
  • B.1 PROMPTS: The cold-start pipeline uses an Answer-Aware Tutor for backward, verified retrieval and an Answer-Blind Planner for forward trajectory assembly.A final coherence judge rejects trajectories that reveal target entities, retrieval terms, or unobserved facts before they become available.
  • B.2 EFFICIENT CORPUS INTERACTION: A correctness-first execution engine falls back to single-file execution whenever parallel behavior cannot be guaranteed byte-identical to sequential execution.This conservative policy governs the system-level optimization design.
  • B.2 EFFICIENT CORPUS INTERACTION: The DCI agent searches a Wikipedia corpus by issuing shell commands and reasoning over multi-hop interaction histories.The Planner prompts specify shell-command retrieval, while the final answer is conditioned on the accumulated trajectory.
  • B.2.1 CORPUS SHARDING AND PARALLEL FAN-OUT: The engine shards large corpora along line boundaries and fans shell pipelines out concurrently across disjoint shards.Ordered shard concatenation reconstructs the original corpus without byte-level modification, while thread-level execution reduces full-corpus latency.
  • B.2.2 PIPELINE CLASSIFICATION AND MERGE STRATEGIES: A conservative parser classifies pipelines into CONCAT, HEAD, COUNT, SORTHEAD, or SEQUENTIAL strategies based on statelessness and global state.Unsupported primitives and stateful operations, including contextual windows and in-place transformations, are routed away from unsafe parallel execution.
  • B.2.2 PIPELINE CLASSIFICATION AND MERGE STRATEGIES: The merge strategies preserve command semantics by concatenating shard outputs, applying global truncation or counts, or performing deterministic k-way merges.Unrecognized, unparseable, or globally stateful pipelines execute sequentially against the unified corpus.
  • B.2.3 I/O AND SYSTEM-LEVEL OPTIMIZATIONS: The I/O stack places the corpus in RAM when possible, warms the page cache, and uses deterministic tool flags with memory-mapped I/O and LC_ALL=C.These measures target memory bandwidth, filesystem latency, cold-start penalties, reproducibility, and locale-independent literal matching.

B.2.4 PERSISTENT DAEMON ARCHITECTURE AND TELEMETRY · B.3 REWARD FUNCTION

The execution engine uses a persistent daemon to reduce repeated tool-call overhead, while reinforcement learning combines answer correctness with strict trajectory-format validation. Correctness uses normalized token-level F1 against reference answers, and formatting acts as a gate against malformed outputs.

  • B.2.4 PERSISTENT DAEMON ARCHITECTURE AND TELEMETRY: A long-running persistent daemon communicates with the ShardedSearchEngine over a length-prefixed JSON protocol on a Unix socket.This amortizes Python wrapper initialization and process startup across tool calls within a trajectory.
  • B.2.4 PERSISTENT DAEMON ARCHITECTURE AND TELEMETRY: 1–3 milliseconds per invocation is the approximate latency reduction from amortizing per-call overhead through the persistent engine.
  • B.3 REWARD FUNCTION: The correctness reward extracts the final answer from the last <answer> block and compares it with a set of gold reference answers.
  • B.3 REWARD FUNCTION: Predictions and references are normalized by lowercasing, removing punctuation, dropping articles, and whitespace tokenization before token-level F1 computation.
  • B.3 REWARD FUNCTION: Token-level F1 is computed from multiset overlap using precision and recall, with a score of 0 when either the prediction or reference is empty.
  • B.3 REWARD FUNCTION: The final answer reward takes the maximum F1 score across reference answers, providing partial credit for partially correct answers.This supports multiple valid surface forms rather than relying only on binary exact-match scoring.
  • B.3 REWARD FUNCTION: A trajectory receives format indicator ϕ = 1 only when tags are balanced and non-overlapping and the prescribed reasoning-to-answer transition structure is followed.The format constraint mirrors the strictly formatted interaction protocol used during cold-start SFT.
  • B.3 REWARD FUNCTION: Only structurally valid trajectories receive a non-zero learning signal, preventing reward exploitation through malformed or out-of-format outputs.

B.4 EXPERIMENTAL SETTINGS & HYPERPARAMETERS

The experimental setup uses a two-stage SFT–GRPO pipeline with Qwen3.5-based synthetic trajectory generation, followed by controlled rollout and inference settings for deep corpus interaction. Training uses long context windows, AdamW optimization, and staged sampling configurations to support exploration and stable tool use.

  • SFT Training Phase: The 27B Qwen3.5 Tutor and Planner generate 10,000 cold-start trajectories, using dynamic Tutor temperature up to 5 refinement steps and fixed Planner and judge temperatures.The Tutor uses temperature 0.4 + 0.1 × iter with Nucleus Sampling; the Planner and coherence judge use temperatures 0.7 and 0.6, respectively, with top-p = 1.0.
  • SFT Training Phase: The 9B Qwen3.5 policy is SFT-trained for one epoch with AdamW, a peak learning rate of 5 × 10^-6, global batch size 32, and maximum sequence length 16,384.Training uses 5% linear warmup followed by a constant learning-rate schedule in verl22 with FSDP.
  • GRPO Training Phase: GRPO initializes from the SFT checkpoint, trains for 200 global steps, and samples n = 5 independent trajectories per query for relative-advantage computation.The configuration is described as Group Relative Policy Optimization inside the verl framework.
  • GRPO Training Phase: During vLLM rollouts, temperature 1.0 and top-p 1.0 encourage diverse tool-use exploration, while trajectories are capped at 16,384 tokens and 6 assistant turns.These settings support deep, multi-turn corpus interactions.

C ADDITIONAL RESULTS

Under the stricter Exact Match metric, GrepSeek’s main findings remain consistent, with leading performance on four of seven benchmarks. Ablations show that both GRPO optimization and SFT initialization are important for reliable performance.

  • Exact Match Results: Exact Match confirms that GrepSeek’s improvements are genuine rather than artifacts of partial token overlap.EM provides a stricter assessment of answer correctness than the main text’s token-level F1 metric.
  • Dataset-Level Results: Four of seven benchmarks—NQ, HotpotQA, 2Wiki, and MuSiQue—show GrepSeek achieving the best EM scores.Gains are statistically significant on NQ, HotpotQA, and 2Wiki.
  • Ablation Results: 0.4948 to 0.3569: removing GRPO reduces micro-average EM, while removing SFT causes EM to collapse to 0.2836.The full model outperforms both ablated variants across all datasets, indicating the importance of policy optimization and SFT initialization.

D CASE STUDIES

The case studies show that GrepSeek’s direct corpus interaction is strongest for exact lexical matching, entity disambiguation, temporal evidence, and iterative multi-hop composition. They also expose DCI’s dependence on file order and exact surface forms, which limits ranking and semantic robustness.

  • Limitations of Ranking and Surface Forms: DCI’s lack of learned ranking can bury the correct answer in file order, while strict surface-form matching fails on spelling, diacritic, and name variations.These examples establish a trade-off between lexical control and semantic robustness.
  • Symbolic and Rare-Token Matching: GrepSeek isolates rare symbolic strings with exact shell matching, succeeding on chemical-formula queries where dense retrieval returns semantically related but incorrect passages.The agent uses rg -F to match the formula literally, and the example attributes the answer to lexical grep rather than semantic embedding.
  • Entity Precision and Disambiguation: Lexical precision helps GrepSeek distinguish closely related entities, such as a subsidiary from its parent brand and one school from another with a similar name.The school example shows dense retrieval selecting a different institution, causing the reasoning to drift toward an incorrect geography.
  • Multi-Hop Evidence Composition: GrepSeek composes multi-hop evidence by iteratively filtering results, including searches that identify a band, isolate its singer, and parse a “highest [noun]” construction.This treats the corpus as a structured database and supports complex, multi-step queries.
  • Temporal and Stale Information: GrepSeek retrieves current record-holder information when dense retrieval is misled by stale documentation about a previous record holder.The case study emphasizes that precise record-keeping is important for temporal questions.

E EXAMPLES OF GENERATED SYNTHETIC TRAJECTORIES

The appendix presents synthetic trajectories demonstrating coherent multi-turn reasoning with shell-based search. The examples cover geographical intersection, compositional multi-hop reasoning, and cross-domain role bridging.

  • Overview: The data-construction pipeline generates trajectories that supervise and train the DCI agent to combine reasoning with shell-based search operations.The examples illustrate coherent multi-turn reasoning during corpus interaction.
  • SFT Example 1: Multi-hop intersection: In the geographical intersection example, the agent independently searches for Fetteresso Castle and Cowie Castle before answering Scotland.It verifies each castle’s location separately after recognizing that an incidental mention does not directly establish the shared country.
  • SFT Example 2: Compositional reasoning: In the compositional example, the agent identifies Dwight D. Eisenhower as the 34th U.S. president and finds R Adams Cowley as the prototype pacemaker’s developer.The trajectory uses successive searches linking the president to the pacemaker and its creator.
  • SFT Example 3: Two-role bridge: In the two-role bridge example, the agent connects Iris Carr in The Lady Vanishes and Riley Blue in Sense8 to actress Tuppence Middleton.It first identifies Middleton in the film and then verifies her role in the series.
Loading 2605.29307v1…