PaperlayerView exact revision on arXiv

Source-linked AI summary

Let Agents Design How They Search: Automatic Interface Optimization for Agentic Search

Sangjune An, Jaeyoon Jung, Yumin Choi, Jinheon Baek, Junmo Cho, Sung Ju Hwang, Hae Beom Lee

arXiv:2610.29250v1cs.CL

TL;DR

Existing search interfaces are generally fixed across corpora, though effective designs may depend on the corpus and agent. SearchWeaver jointly optimizes interface artifacts, tools, and instructions using agent execution feedback. With GPT-5.4-nano, it improves accuracy and retrieval performance over DCI on BrowseComp-Plus and four BRIGHT subsets.

  • Problem

    Existing agentic-search interfaces are generally fixed across corpora, although effective designs may depend on the corpus, query types, and language model.

    Source: §1 · p. 1

  • Method

    SearchWeaver iteratively uses agent trajectories and task rewards to revise corpus artifacts, executable tools, and usage instructions together.

    Source: §1 · p. 1, §2.2 · p. 2

  • Results

    71.6% to 78.4% answer accuracy on BrowseComp-Plus and 45.7 to 55.0 average nDCG@10 across four BRIGHT subsets over DCI.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results support treating a search interface as a reusable corpus resource that can be optimized through agent execution.

    Source: §1 · p. 2

    The experiments use fixed corpora; updates to artifacts and tools as documents change, including whether updates can be incremental, remain unstudied.

    Source: Limitations · p. 5

Abstract

from arXiv · show

Agentic search depends on the interface through which a language-model agent accesses and explores a corpus. Existing interface designs are fixed by hand and applied unchanged to every corpus, although effective designs may depend on the corpus. We introduce SearchWeaver, a framework for automatic search interface optimization guided by downstream search performance. A proposer uses execution trajectories and task rewards to jointly revise corpus artifacts, executable tools, and usage instructions. This pro cess optimizes what information is prepared in advance, which operations tools perform, and which decisions remain with the search agent. With GPT-5.4-nano, the optimized interfaces improve answer accuracy from 71.6% to 78.4% on BrowseComp-Plus and average nDCG@10 from 45.7 to 55.0 across four BRIGHT subsets over Direct Corpus Interaction. These findings highlight the value of jointly optimizing corpus artifacts, executable tools, and usage instructions to develop effective search interfaces.

1 Introduction

Search interfaces shape what evidence language-model agents can access and how they act on it, yet existing designs are generally fixed across corpora. SearchWeaver jointly revises interface components using search trajectories and task rewards, improving performance over DCI on both benchmarks.

1 Introduction

  • Existing interfaces are generally fixed in advance and applied unchanged across corpora, although effective designs may depend on corpus, queries, and the search model.The paper names retrieval-augmented generation, DCI, DR-DCI, and RISE as examples of existing approaches.Source: §1 · p. 1
  • An effective interface aligns prepared information, available operations, and agent guidance with how the agent searches and uses evidence.The paper’s example is a title index paired with tools and instructions for inspecting documents and verifying query clues.Source: §1 · p. 1
  • SearchWeaver uses agent trajectories and task rewards to iteratively revise corpus artifacts, executable tools, and usage instructions together.The proposer analyzes failures and bottlenecks, and each revised interface is evaluated to produce feedback for subsequent revisions.Source: §1 · pp. 1–2
  • 71.6% to 78.4% BCP answer accuracy and 45.7 to 55.0 average BRIGHT nDCG@10 over DCI, with mean BRIGHT tool time reduced by 42%.These results use GPT-5.4-nano.Source: §1 · p. 2

2 Method

SearchWeaver formulates interface design as choosing a reusable search interface for a fixed corpus and agent to improve task reward across queries. It represents interfaces through artifacts, tools, and instructions, revises them using execution feedback, and selects the best candidate for test evaluation.

2.1 Problem Definition

  • For a fixed corpus and agent, the objective is to choose one interface that improves task success across optimization queries while keeping the model parameters fixed.Task success is measured by answer accuracy or nDCG@10 in the experiments.Source: §2.1 · p. 2

2.2 SearchWeaver: Automatic Search Interface Optimization

  • SearchWeaver represents an interface with reusable corpus artifacts, executable tools, and instructions that guide their use.Tools may use BM25 or dense retrieval, restrict exploration, or combine retrieval rankings with agent document judgments; raw corpus paths are retained for source inspection.Source: §2.2 · p. 2
  • A proposer revises artifacts, tools, and instructions using per-query rewards and execution trajectories that expose agent actions, tool outputs, and failures.Gold document identifiers for optimization queries provide additional diagnostic feedback.Source: §2.2 · p. 2
  • After a fixed number of iterations, SearchWeaver selects the candidate with the highest mean reward on optimization queries and evaluates it on held-out test queries.Candidate artifacts and tools are prepared before the fixed agent runs; every evaluated candidate contributes feedback, whether or not its score improves.Source: §2.2 · pp. 2–3

3 Experiments

Experiments show that SearchWeaver improves over DCI on BrowseComp-Plus and BRIGHT, while using less tool execution time. The optimized interfaces use different corpus-specific strategies, and their gains also persist in the reported ablation and transfer evaluations.

3.1 Experimental Setup

  • Evaluation covers BrowseComp-Plus question answering and four BRIGHT reasoning-intensive retrieval subsets, using 30 BCP optimization queries and 800 test queries, plus 213 and 210 BRIGHT queries.The agent uses GPT-5.4-nano; the interface proposer evaluates one candidate per iteration for ten iterations, starting from DCI.Source: §3.1 · p. 3

3.2 Main Results and Analyses

  • Compared with DCI, SearchWeaver raises BCP accuracy from 71.6% to 78.4% and average BRIGHT nDCG@10 from 45.7 to 55.0, and is Pareto-optimal among evaluated methods on both benchmarks.It improves over DCI on all four BRIGHT subsets.Source: §3.2 · pp. 3–4, Figure 2 · p. 4
  • Optimized interfaces use corpus-specific strategies, including a BCP title index for candidate retrieval and different retrieval or agent-judgment strategies across BRIGHT subsets.Biology restricts retrieval to a topic directory; Earth Science introduces retrieval when constructing the final ranking; Economics and Robotics use agent judgments in document ordering and relevance grades.Source: §3.2 · p. 4
  • Mean tool execution time falls from 42.4 s to 32.9 s on BCP and from 63.2 s to 36.6 s on BRIGHT, reductions of 22% and 42%.Tool execution time excludes model inference, and the reduction holds across all four BRIGHT subsets.Source: §3.2 · p. 4
  • Removing generated artifacts, tools, and their usage instructions reduces performance on both benchmarks; SearchWeaver also exceeds prompt-only optimization on BCP and BRIGHT.Prompt Optimization uses the same optimization loop but limits candidates to instructions.Source: §3.2 · p. 4
  • Interfaces optimized with GPT-5.4-nano improve BCP accuracy and average BRIGHT nDCG@10 over DCI when transferred to Gemini 3.5 Flash-Lite without re-optimization.Gains hold across all four BRIGHT subsets, while mean tool execution time on BRIGHT also decreases.Source: §3.2 · p. 4, Table 3 · p. 4

Limitations

The experiments evaluate interfaces optimized for fixed corpora, not how they should be maintained as corpora change.

Limitations

  • Because experiments use fixed corpora, the paper leaves open when revised documents require artifact and tool updates and whether those updates can avoid repeating full optimization.For evolving corpora, the authors identify maintenance of corpus artifacts and checks on continued interface effectiveness as needed extensions.Source: Limitations · p. 5

A SearchWeaver Algorithm

Algorithm 1 describes SearchWeaver’s optimization loop from corpus and queries to an optimized interface.

A SearchWeaver Algorithm

  • The algorithm takes a corpus and queries as input and outputs an optimized interface.It specifies a fixed agent, proposer, initial interface, and iteration count.Source: A SearchWeaver Algorithm · p. 7
  • The algorithm names the resulting interface ϕ⋆.The initial interface is ϕ0.Source: A SearchWeaver Algorithm · p. 7
  • The loop runs for k = 1, . . . , K, using per-query rewards and trajectories as feedback.The supplied algorithm excerpt identifies these as r and T.Source: A SearchWeaver Algorithm · p. 7

B Additional Implementation Details

Additional implementation details specify the corpora, agent runtime, optimization safeguards, retrieval resources, and baseline configurations.

B Additional Implementation Details

  • The evaluation uses the DCI-Agent-Lite file-system corpus, with 30 BCP optimization queries and 800 test queries, and 213 BRIGHT optimization and 210 test queries.BRIGHT covers Biology, Earth Science, Economics, and Robotics.Source: B Additional Implementation Details · p. 7
  • GPT-5.4-nano runs with high reasoning effort, no web access, and limits of 300 turns and 30 minutes per query.The proposer can inspect the corpus and run commands, but test queries and labels are unreadable during optimization and candidates containing query identifiers or gold labels are rejected.Source: B Additional Implementation Details · p. 7
  • Retrieval resources differ by benchmark: BCP provides BM25 and searchable dense embeddings, while BRIGHT’s dense resource only returns released rankings for original questions.Baseline interfaces vary in tools and indexes, including DCI, RISE, DR-DCI, and Agentic Search configurations.Source: B Additional Implementation Details · p. 7

C Additional Related Work

Prior work develops iterative retrieval and direct corpus exploration, and studies how search interfaces organize evidence. SearchWeaver applies feedback-driven optimization to corpus-specific combinations of artifacts, tools, and instructions.

Agentic Search

Search Interface Design

Automatic Optimization of Agent Systems

  • SearchWeaver optimizes corpus-specific combinations of artifacts, executable tools, and usage instructions using trajectories and task rewards.This applies feedback-driven optimization—which prior work uses for prompts, pipelines, and harness code—to search interfaces.Source: C Additional Related Work · p. 8

D BRIGHT Tool Time by Subset

Table 4 reports BRIGHT tool time per test query by subset, with aggregate timings covering 210 test queries.

D BRIGHT Tool Time by Subset

E BRIGHT Transfer Results by Subset

Table 5 reports BRIGHT nDCG@10 by subset for interfaces optimized with GPT-5.4-nano and transferred to Gemini 3.5 Flash-Lite.

E BRIGHT Transfer Results by Subset

F BCP Agent Trajectory

Figure 3 presents the tool calls and final answer for the test query shown in Figure 1.

F BCP Agent Trajectory

  • Figure 3 presents all 13 tool calls and the final answer for Figure 1’s test query.Outputs and paths are abbreviated for readability.Source: F BCP Agent Trajectory · p. 8

G Optimized Interfaces on BRIGHT

The optimized BRIGHT interfaces differ in how they combine agent judgments with retrieval, and removing generated artifacts and tools hurts most on Biology and Earth Science, where the gains are largest.

G Optimized Interfaces on BRIGHT

  • All four interfaces retain shell-based search and reading, but differ in where retrieval enters and how agent judgments shape the submitted list.Earth Science uses retrieval only in the final tool; Biology confines search to the question’s topic directory, while Economics and Robotics start from dense candidates.Source: G Optimized Interfaces on BRIGHT · p. 9
  • Biology and Earth Science improve over DCI by 13.0 and 13.1 points, then lose 9.7 and 11.5 points without generated artifacts and tools.These are the subsets where removing the artifacts and tools hurts most.Source: G Optimized Interfaces on BRIGHT · p. 9
  • On Economics, performance improves over DCI by 7.0 points; it is 38.8 with artifacts and tools versus 38.3 without them, while Prompt Optimization scores highest at 41.8.Source: G Optimized Interfaces on BRIGHT · p. 9
  • Figures 4–7 show each subset’s tools and instructions alongside a test-question trace; example (b) scores are 90.6 for Biology, 61.9 for Earth Science, and 100.0 for Economics and Robotics.Panel (b) traces tool calls to the submitted list, with gold documents highlighted.Source: G Optimized Interfaces on BRIGHT · p. 9, Figure 4 · p. 11, Figure 5 · p. 11

Related papers

Loading 2610.29250v1…