PaperlayerView exact revision on arXiv

Source-linked AI summary

ITER: Interaction-Aware Retrieval for Agentic Search

Haodong Chen, Shuai Wang, Yu Yin, Shengyao Zhuang, Guido Zuccon, Teerapong Leelanupab

arXiv:2608.27912v1cs.IR

TL;DR

Existing agentic retrievers underuse information accumulated from earlier search interactions. ITER combines the main question, current and previous sub-queries with trajectory-relative supervision, consistently outperforming LRAT and transferring better across unseen agents. The paper concludes that retrieval should target useful next evidence while leaving full modeling of an agent’s evolving information state open.

  • Problem

    Existing retriever training largely uses the current sub-query and search results, underusing previous sub-queries and visited documents in multi-step search.

  • Method

    ITER uses a history-conditioned query containing the main question, current sub-query, and previous sub-queries, with trajectory-relative supervision from agent interactions.

  • Results

    Across two benchmarks and six agent backbones, ITER consistently outperforms LRAT and achieves higher task success than AgentIR on all five unseen backbones.

  • Takeaways & Limitations

    The main question and previous sub-queries provide robust retrieval context, while document visits work better as supervision than as query content.

  • Takeaways & Limitations

    Fully capturing the agent’s evolving information state remains an open problem, including how to represent learned information and transfer signals across agents.

Abstract

from arXiv · show

Deep-research agents answer complex user questions through an iterative sequence of search steps, where the agent autonomously formulates sub-queries to retrieve the evidence needed at each stage. However, existing retriever training typically relies only on the sub-query and its corresponding search results at the current step as training signals, leaving the information accumulated from previous interactions largely underutilized. We introduce iter, an agent interaction-aware dense retriever trained using agent trajectory learning signals. iter represents each query by incorporating not only the current sub-query, but also the main question and preceding sub-queries, and is trained using trajectory-relative learning signals derived from the agent's interactions. Across six agent backbones from three model families, iter consistently outperforms the existing agent-trajectory-trained dense retriever, LRAT, achieving an average improvement of 7.5% on InfoSeek-Eval and 13.5% on BrowseComp-Plus. iter also demonstrates stronger cross-agent robustness than AgentIR, a deep-research retriever that relies on external LLM-judge signals and the agent's pre-search reasoning. Ablations further show that the main question and previous sub-queries provide the most robust query representation, while previously visited and useful documents, used as redundancy negatives in subsequent searches, provide the strongest trajectory-relative supervision. Code is available at https://github.com/ielab/ITER.

1 Introduction

ITER addresses the underuse of interaction history in agentic retrieval by representing search context beyond the current sub-query and learning from trajectory-relative signals. It consistently outperforms LRAT and shows stronger cross-agent robustness than AgentIR.

  • Motivation: Existing agentic retriever training often treats search steps in isolation, underusing previous sub-queries and visited documents.Previous sub-queries reveal explored directions, while visited documents indicate which information needs may already be satisfied.
  • Method: ITER combines the main question, current sub-query, and previous sub-queries into a history-conditioned query representation.The main question preserves the overall task, the current sub-query specifies the immediate need, and previous sub-queries indicate explored directions.
  • Results: 80.0 versus 72.7 task success on InfoSeek-Eval and 46.6 versus 43.4 on BrowseComp-Plus, comparing ITER with LRAT.The matched-agent evaluation uses Tongyi-DeepResearch-30B trajectories for training.
  • Results: ITER outperforms LRAT in all 12 backbone–benchmark comparisons and achieves higher task success than AgentIR on five unseen backbones across both benchmarks.Ablations identify the main question and previous sub-queries as robust query context, while redundancy negatives provide the strongest trajectory-relative supervision.
  • Method: ITER uses trajectory-relative supervision with step-specific positives and tiered negatives derived from agent trajectories.This supervision reflects document utility relative to the evidence already collected during the search trajectory.

2 Related Work

Deep-research retrieval extends search from one-shot query matching to iterative evidence gathering, while related methods adapt retrieval context, agent access, or supervision to intermediate search needs.

  • Deep-research retrieval: Deep-research agents issue sub-queries, accumulate evidence across retrieval steps, and synthesize answers or reports.This setting studies the retriever supporting successive search requests.
  • Trajectory-trained retrieval: Related approaches train retrievers with intermediate-stage feedback, local query–passage relevance, global answer correctness, or agent interaction signals.NExT-Search, Agentic-R, and LRAT represent different uses of intermediate or trajectory-level supervision.
  • Retrieval workflow: Some systems change corpus access by giving agents shell commands, bounded workspaces, or structured inspection and selective section fetching.Direct Corpus Interaction, RISE, and SIEVE optimize how agents access retrieved evidence.
  • Context-conditioned retrieval: Context-conditioned retrieval uses task instructions, conversational history, or generated hypothetical content in addition to the current query.Examples include task-aware retrievers, conversational rewriting or history encoding, and HyDE.
  • Deep-research context: AgentIR uses pre-search reasoning as retrieval context, whereas ITER anchors retrieval to the agent’s current search state and what it has already explored.The distinction is between expanding the current sub-query and representing the evolving interaction state.

3 Observations from Agent Search

Agent trajectories reveal that query reformulation does not prevent repeated results: visited documents often resurface near the top, motivating retrieval conditioned on interaction history and marginal information gain.

  • Trajectory observations: Figure 3 analyzes sub-query similarity, result novelty, and visited-document recurrence across 26,482 agent trajectories.The trajectories were collected with Tongyi-DeepResearch across four retrieval backends using standard top-10 retrieval.
  • Finding 1: Query reformulation alone does not prevent repeated results.Adjacent sub-queries remain semantically close, and later rankings contain an increasing share of previously returned documents.
  • Finding 1: 52.1% of top-10 results from the second search onward had appeared earlier, while only 3.9 of 10 results were first-time documents from the eighth search onward.Adjacent and non-adjacent sub-queries also remained more similar within trajectories than across random trajectories.
  • Finding 2: 54.9% of visited documents reappeared in later searches; 44.3% of these reappearances were at rank 1 and 70.1% were within the top three.Visited relevant documents can therefore be repeatedly ranked highly by a retriever using only the current sub-query.
  • Interaction-aware retrieval: A current-sub-query-only retriever may rank previously examined documents above documents containing new evidence.The mismatch arises because the retriever does not observe earlier queries or document visits as the agent’s information state changes.
  • Interaction-aware retrieval: Interaction-aware retrieval defines a document’s utility by its marginal gain given the current sub-query and interaction history.A document’s utility may decrease after it has been visited even when it remains topically relevant.
  • Interaction-aware retrieval: The interaction history includes the main question, previous sub-queries, previously retrieved and visited documents, and document-specific interpretations.These interaction signals motivate history-conditioned retrieval and trajectory-relative training supervision.

4 ITER: Query Representation and Training

ITER implements interaction-aware retrieval by conditioning queries on search history and deriving supervision from document interactions recorded in agent trajectories.

  • ITER design: ITER conditions retrieval on interaction history and derives trajectory-relative supervision from document interactions.These are the two design choices identified for modeling history-dependent document utility.

4.1 History-Conditioned Query Representation

ITER serializes the main question, current sub-query, and prior sub-queries to represent the agent’s current search state. A reasoning-augmented variant additionally includes pre-search reasoning and its instruction text.

  • ITER serializes selected interaction-history components together with the current sub-query as query-encoder input.
  • The default representation combines the main question, current sub-query, and previous sub-queries.The main question preserves the task, the current sub-query specifies the immediate need, and previous sub-queries indicate explored directions.
  • The reasoning-augmented representation adds pre-search reasoning and corresponding instruction text to the default representation.The effects of visited documents, document interpretations, and other history combinations are examined separately in the query-representation ablation.

4.2 Constructing Training Signals from Agent Trajectories

ITER constructs trajectory-derived training signals from de-duplicated search interactions, distinguishing useful delayed visits from different kinds of negative evidence. This preserves candidate coverage while retaining access to earlier results.

  • The de-duplicated search setting presents more unseen candidates while preserving access to earlier results.It removes previously returned documents from the displayed unseen results and places them in a separate returned-earlier section accessible through get_document.
  • 26.7% of 67,934 positive pairs involve delayed visits to documents returned by an earlier search.These delayed visits show that not visiting a document immediately does not establish that it is unhelpful.
  • ITER pairs each history-conditioned query with documents visited after that search and before the next search, including delayed visits.
  • Only documents judged Relevant by post-visit reasoning are retained as positives.A visit alone is insufficient because full-document examination may show that the document was unhelpful.
  • Negative documents are divided into redundancy, hard, and weak tiers according to prior visits and relevance judgments.Redundancy negatives were previously useful, hard negatives were visited but irrelevant, and weak negatives were returned but never visited.
  • Documents returned earlier but never visited remain unlabeled because delayed visits show that immediate non-visit does not establish unhelpfulness.
  • Each training group contains one positive and nine negatives, with up to three negatives sampled from each of the redundancy and hard tiers.Remaining slots are filled with weak negatives.

4.3 Trajectory-Relative Training

ITER uses a weighted contrastive objective that combines instance-level weights from positive-document reasoning with tier-specific weights for trajectory-derived negatives.

  • ITER’s weighted contrastive objective applies instance-level and tier-specific weights.Instance weights derive from reasoning associated with positive documents, while tier weights apply to sampled negatives.
  • Positive-instance weights use a length-to-weight mapping in which longer post-visit reasoning traces receive a saturating utility signal.The raw score reaches half its asymptotic value at the median positive reasoning length β and is normalized to mean one.
  • Redundancy negatives receive the largest weight, hard negatives the standard weight, and weak negatives a discounted weight.The weighting reflects stronger evidence for ranking previously useful but redundant documents below the current positive.
  • The loss uses cosine similarity with softmax temperature τ=0.02, while documents from other training instances serve as in-batch negatives.
  • Negative weights control within-instance competition with the positive, and instance weights control each training instance’s contribution to the final objective.
  • Training fully fine-tunes Qwen3-Embedding-0.6B for two epochs with AdamW and a learning rate of 10^-6.Documents are truncated to 512 tokens and query inputs to 8,192 tokens.

5 Experimental Setup

The evaluation compares ITER with sparse and dense retrievers on matched-agent and cross-agent deep-research settings using task success, retrieval recall, and execution-efficiency metrics. InfoSeek-Eval is in-domain, while BrowseComp-Plus is out-of-domain.

  • Evaluation benchmarks: ITER is evaluated on two fixed-corpus deep-research benchmarks under the same in-domain and out-of-domain setup as LRAT.
  • Evaluation benchmarks: InfoSeek-Eval contains 300 disjoint multi-hop questions over an 11.2-million-document Wiki-25-Dump1 corpus and is treated as in-domain.
  • Evaluation benchmarks: BrowseComp-Plus contains 830 complex questions over 100,195 documents and is treated as out-of-domain because neither its questions nor corpus were used for training.
  • Evaluation metrics: Task success rate measures correctly answered questions, using exact string matching for InfoSeek-Eval and Qwen3-30B-A3B-Thinking-2507 as BrowseComp-Plus judge.
  • Evaluation metrics: BrowseComp-Plus retrieval quality uses evidence search recall and evidence visit recall, while efficiency uses average tool calls per question.Both recall metrics are macroaveraged over questions.
  • Compared retrievers: Comparisons include BM25, Base, LRAT, AgentIR, and ITER with default, current-sub-query-only, and reasoning-augmented representations.The default ITER representation combines the main question, current sub-query, and previous sub-queries.
  • Evaluation protocol: Cross-agent transfer is tested on five unseen backbones from two model families, ranging from 4B to 120B parameters.
  • Evaluation protocol: Every retriever returns top-10 documents with 64-token snippets, and agents are limited to 50 tool-calling turns per question.

6 Main Results

ITER improves retrieval and task outcomes over LRAT across matched-agent and cross-agent evaluations, while history conditioning remains beneficial and pre-search reasoning transfers less reliably.

  • Matched-Agent Evaluation: Trajectory-relative training alone raises InfoSeek-Eval task success from 72.7 to 76.7 while reducing average tool calls from 19.6 to 18.3 with identical current-sub-query input.On BrowseComp-Plus, evidence search recall rises from .602 to .619 and task success from 43.4 to 43.7.
  • Matched-Agent Evaluation: Adding the main question and previous sub-queries raises InfoSeek-Eval task success from 76.7 to 80.0 and BrowseComp-Plus task success from 43.7 to 46.6.BrowseComp-Plus evidence search recall also increases from .619 to .636.
  • Cross-Agent Evaluation: ITER improves task success over LRAT in all 12 backbone–benchmark comparisons, averaging +5.4 points on InfoSeek-Eval and +4.3 points on BrowseComp-Plus.Seven gains are statistically significant, and BrowseComp-Plus evidence search recall improves for all six backbones by 4.8 percentage points on average.
  • Cross-Agent Evaluation: Pre-search reasoning performs worse than the default representation on InfoSeek-Eval for all five unseen backbones and on BrowseComp-Plus for four of five.It can help the matched agent but transfers less consistently across agents.
  • Cross-Agent Evaluation: AgentIR often has higher evidence search recall, but ITER achieves higher task success on all five unseen backbones in the size-matched comparison.Against smaller 0.6B ITER, ITER also has higher visit recall on four of five backbones and higher task success on all five.

7 Ablation Analysis

Ablations identify structured history and redundancy negatives as ITER’s strongest design choices, while visited text and free-form reasoning are less reliable inputs.

  • Query Representation: Adding the main question and previous sub-queries provides complementary gains, reaching 80.0 InfoSeek-Eval task success and 46.6 BrowseComp-Plus task success.The main question preserves the overall task, while previous sub-queries indicate explored search directions.
  • Query Representation: Explicitly adding visited documents, post-visit interpretations, or both lowers BrowseComp-Plus task success to 41.1–42.7 and evidence search recall to .541–.579.All three variants perform worse than the default representation.
  • Negative Supervision: Removing redundancy negatives reduces BrowseComp-Plus task success from 46.6 to 40.8 and evidence search recall from .636 to .561.InfoSeek-Eval task success decreases by only 0.7 points, indicating benchmark-dependent impact.
  • Negative Supervision: Removing hard negatives reduces task success from 80.0 to 79.3 on InfoSeek-Eval and from 46.6 to 45.8 on BrowseComp-Plus.Hard negatives complement the history-dependent redundancy signal by providing conventional relevance supervision.
  • Negative Supervision: Uniform negative weighting reduces BrowseComp-Plus task success from 46.6 to 43.9 and evidence search recall from .636 to .608.The default weighting (3, 1, 0.3) emphasizes redundancy negatives while discounting uncertain unvisited negatives.

8 Conclusion

ITER models cumulative search progress through history-conditioned representations and trajectory-relative supervision. Across benchmarks and agent backbones, it outperforms LRAT and transfers more strongly across agents than AgentIR, while several representation and transfer questions remain open.

  • Conclusion: ITER models each document’s marginal gain using a history-conditioned query representation and trajectory-relative supervision.This frames retrieval around cumulative search rather than isolated current-sub-query relevance.
  • Conclusion: ITER achieves higher task success than LRAT for all five unseen backbones on both benchmarks and shows stronger cross-agent transfer than AgentIR.The conclusion also identifies the main question and previous sub-queries as robust context, while document visits work better as supervision than query content.
  • Conclusion: Capturing the agent’s evolving information state remains an open problem involving learned representations, cross-agent transfer, and joint retrieval–search optimization.The paper suggests optimizing trajectory progress rather than relevance at an isolated step.
Loading 2608.27912v1…