PaperlayerView exact revision on arXiv

Source-linked AI summary

IterResearch: Rethinking Long-Horizon Agents with Interaction Scaling

Guoxin Chen, Zile Qiao, Xuanzhong Chen, Donglei Yu, Haotian Xu, Wayne Xin Zhao, Ruihua Song, Wenbiao Yin, Huifeng Yin, Liwen Zhang, Kuan Li, Minpeng Liao, Yong Jiang, Pengjun Xie, Fei Huang, Jingren Zhou

arXiv:2511.07327v2cs.AIcs.CL

TL;DR

Long-horizon research agents are limited by mono-contextual accumulation, which produces context suffocation and noise contamination. IterResearch reconstructs workspaces around evolving report memory and trains the paradigm with EAPO; it improves open-source agents across six benchmarks and scales to 2048 interactions.

  • Problem

    Mono-contextual agents accumulate retrieved information and reasoning in one expanding context, leading to context suffocation and noise contamination during long-horizon tasks.

  • Method

    IterResearch uses iterative workspace reconstruction with an evolving report memory, while EAPO applies efficiency-aware geometric rewards and adaptive downsampling.

  • Results

    IterResearch improves existing open-source agents by an average of 14.5 percentage points across six benchmarks and scales from 3.5% to 42.5% as interactions increase to 2048.

  • Takeaways & Limitations

    The paradigm functions both as a trained long-horizon agent and as a prompting strategy that improves frontier models.

Abstract

from arXiv · show

Recent advances in deep-research agents have shown promise for autonomous knowledge construction through dynamic reasoning over external sources. However, existing approaches rely on a mono-contextual paradigm that accumulates all information in a single, expanding context window, leading to context suffocation and noise contamination that limit their effectiveness on long-horizon tasks. We introduce \textbf{IterResearch}, a novel iterative deep-research paradigm that revisits long-horizon research through the lens of Interaction Scaling. Instead of relying on linear context accumulation, we adopt an MDP-inspired architecture with strategic workspace reconstruction. By maintaining an evolving report as memory and periodically synthesizing insights, our approach preserves consistent reasoning capacity across arbitrary exploration depths. To effectively train this paradigm, we employ Efficiency-Aware Policy Optimization (EAPO), a training strategy that adapts geometric reward discounting to incentivize efficient exploration and utilizes adaptive downsampling for stable distributed training. Extensive experiments demonstrate that IterResearch achieves substantial improvements over existing open-source agents with average +14.5pp across six benchmarks and narrows the gap with frontier proprietary systems. Remarkably, our paradigm exhibits unprecedented interaction scaling, extending to 2048 interactions with dramatic performance gains (from 3.5\% to 42.5\%), and serves as an effective prompting strategy, improving frontier models by up to 19.2pp over ReAct on long-horizon tasks. These findings position IterResearch as a versatile solution for long-horizon reasoning, effective both as a trained agent and as a prompting paradigm for frontier models.

1 INTRODUCTION

IterResearch replaces mono-contextual accumulation with iterative workspace reconstruction and report-based memory, while EAPO trains efficient exploration. Experiments report stronger benchmark performance, interaction scaling, and broader transfer and prompting benefits.

  • 1 INTRODUCTION: The mono-contextual paradigm appends retrieved information and reasoning to one expanding context, causing context suffocation and persistent noise contamination.These effects constrain later reasoning and dilute signal quality through the research process.
  • 1 INTRODUCTION: IterResearch reconstructs each workspace around the question, evolving report, and immediate context instead of retaining the entire interaction history.This design uses periodic synthesis and strategic forgetting to preserve reasoning capacity across exploration depths.
  • 1 INTRODUCTION: EAPO combines trajectory-length geometric discounting with adaptive downsampling for efficient exploration and stable distributed training.Adaptive downsampling preserves over 99% of training data.
  • 1 INTRODUCTION: IterResearch outperforms existing open-source agents by an average of 14.5 percentage points across six challenging benchmarks.It also narrows the gap with frontier proprietary systems and surpasses some on these benchmarks.
  • 1 INTRODUCTION: IterResearch scales to 2048 interactions, with performance increasing from 3.5% to 42.5% as maximum interactions grow from 2 to 2048.The contribution list also reports cross-paradigm knowledge transfer and model-agnostic prompting improvements for frontier models.

2 RELATED WORK

Related work frames deep research as an extension beyond static RAG, using autonomous agents to navigate complex web environments and synthesize diverse sources. Existing open-source methods nevertheless predominantly use mono-contextual accumulation.

  • 2 RELATED WORK: Traditional RAG typically uses static retrieval environments with limited exploration, making it inadequate for complex long-horizon reasoning requiring dynamic information gathering.
  • 2 RELATED WORK: Deep-research agents extend beyond RAG by navigating complex web environments and synthesizing information from diverse sources.
  • 2 RELATED WORK: Existing open-source deep-research methods predominantly append information and reasoning within a mono-contextual paradigm.

3 METHODOLOGY

IterResearch structures long-horizon research around iterative workspace reconstruction rather than linear context accumulation, combining an MDP-inspired state design with EAPO training. The workspace preserves an evolving report and immediate context while EAPO encourages efficient exploration and stabilizes training over multi-round trajectories.

  • 3.1 MDP-INSPIRED FORMULATION: The MDP-inspired formulation separates internal thought updates from external interactions and defines deterministic state transitions from the current state, decision, and environment response.Each decision combines thought, report update, and action; the environment returns the immediate response used in the next workspace.
  • 3.1 MDP-INSPIRED FORMULATION: IterResearch reconstructs each workspace from the question, evolving report, and immediate context instead of appending the full research history.The transition preserves synthesized knowledge while discarding the historical trajectory, maintaining bounded workspace complexity.
  • 3.1 MDP-INSPIRED FORMULATION: IterResearch maintains O(1) workspace complexity as exploration grows, contrasting with the O(t) context growth of mono-contextual approaches.The design supports theoretically unbounded exploration steps while preserving consistent reasoning capacity through selective compression and synthesis.
  • 3.2 EFFICIENCY-AWARE POLICY OPTIMIZATION: EAPO geometrically discounts terminal rewards so earlier task completion receives higher rewards, incentivizing more direct exploration.The method addresses terminal-only binary rewards that otherwise treat successful trajectories equally despite differing computational costs.
  • 3.2 EFFICIENCY-AWARE POLICY OPTIMIZATION: Multi-round trajectories yield one training sample per round, and adaptive downsampling preserves uniform distributed sampling with typically less than 1% data loss.EAPO combines efficiency-aware rewards with adaptive downsampling for stable training on variable-length trajectories.

4 EXPERIMENTS

Experiments evaluate IterResearch across six deep-research benchmarks, compare training and paradigm variants, and test interaction scaling and prompting transfer. IterResearch outperforms open-source agents and ReAct, with gains especially pronounced on long-horizon information-seeking tasks.

  • 4.2 MAIN RESULTS: IterResearch outperforms existing open-source agents by an average of 14.5 percentage points across six benchmarks and surpasses OpenAI’s DeepResearch on HLE and BrowseComp-zh.It achieves comparable results with OpenAI’s DeepResearch on BrowseComp and GAIA.
  • 4.3 ABLATION STUDY: EAPO reduces average interactions by 5.7% versus GSPO while maintaining or improving accuracy, requiring 18.04 turns compared with GSPO’s 19.13.The ablation attributes this efficiency to geometric discounted rewards that favor focused exploration.
  • 4.3 ABLATION STUDY: IterResearch outperforms the mono-contextual baseline by 12.6 percentage points on average across benchmarks, despite using a 40K-token context versus the baseline’s 64K.The largest reported gains are +11.8% on BrowseComp and +10.6% on BrowseComp-zh.
  • 4.3 ABLATION STUDY: Augmenting Mono-Agent with iterative-paradigm trajectories improves performance by an average of 5.4 percentage points across most benchmarks without increasing total data volume.This result supports transfer of exploration behavior between paradigms.
  • 4.5 ITERRESEARCH AS A EFFECTIVE PROMPTING STRATEGY IN LONG-HORIZON TASKS: As a prompting strategy, IterResearch consistently outperforms ReAct, with BrowseComp gains of +12.7pp for o3 and +19.2pp for DeepSeek.The comparison uses frontier models without additional training.

5 CONCLUSION

The conclusion presents IterResearch as an iterative alternative to mono-contextual research, combining workspace reconstruction with EAPO. It reports benchmark gains, scaling to 2048 interactions, prompting improvements, and transferable exploration behavior.

  • 5 CONCLUSION: IterResearch addresses context suffocation and noise contamination through iterative workspace reconstruction and Efficiency-Aware Policy Optimization.The paradigm uses strategic synthesis rather than linear accumulation.
  • 5 CONCLUSION: IterResearch improves performance by an average of 14.5pp across six benchmarks and improves frontier-model prompting by up to 19.2pp.The conclusion also reports interaction scaling to 2048 interactions with gains from 3.5% to 42.5%.
  • 5 CONCLUSION: The reported findings position iterative synthesis as a framework applicable to trained agents, frontier-model prompting, and cross-paradigm transfer.The conclusion describes these as complementary uses of the paradigm.
  • 5 CONCLUSION: IterResearch’s related-work context connects its evolving-report architecture to iterative reasoning systems such as AlphaEvolve.The cited comparison concerns a shared conceptual architecture across different domains.

B MORE ANALYSIS

The analysis explains how geometric discounting makes shorter successful research trajectories more rewarding and discourages redundant or unfocused exploration. The chosen discount factor balances exploration with efficiency.

  • B.1 THEORETICAL MOTIVATION: EFFICIENCY THROUGH DISCOUNTING: With γ = 0.995, geometric discounting gives earlier steps in shorter successful trajectories higher rewards than corresponding steps in longer trajectories.The formulation compares trajectories requiring 5 versus 20 steps.
  • B.1 THEORETICAL MOTIVATION: EFFICIENCY THROUGH DISCOUNTING: A 7.8% reward difference at the same step position provides a gradient toward more efficient research strategies.The analysis presents this difference as a multiplicative advantage for shorter trajectories.
  • B.1 THEORETICAL MOTIVATION: EFFICIENCY THROUGH DISCOUNTING: The discounting scheme penalizes redundant exploration, circular reasoning, and unfocused browsing by reducing future rewards for additional steps.These penalties arise from geometric discounting rather than an explicit length objective.
  • B.1 THEORETICAL MOTIVATION: EFFICIENCY THROUGH DISCOUNTING: The discount factor γ controls the exploration–efficiency trade-off, with γ = 0.995 associated with a 5.7% reduction in average trajectory length.Values closer to 1 permit more exploration, while smaller values create stronger pressure for direct problem-solving.

B.2 COMPUTATIONAL COMPLEXITY ANALYSIS.

IterResearch bounds its workspace through report synthesis instead of accumulating all prior tool responses. This keeps context and attention costs stable with depth and supports theoretically unbounded exploration when the workspace fits within the context limit.

  • B.2 COMPUTATIONAL COMPLEXITY ANALYSIS: IterResearch maintains workspace size O(|M| + |TR|), while mono-contextual context grows as O(t · |TR|) with the number of rounds.|M| is the bounded report size and |TR| is the average tool-response size.
  • B.2 COMPUTATIONAL COMPLEXITY ANALYSIS: Workspace reconstruction keeps attention computation at O((|M| + |TR|)^2) per round instead of mono-contextual growth at O((t · |TR|)^2).The iterative cost is independent of trajectory length when the report remains bounded.
  • B.2 COMPUTATIONAL COMPLEXITY ANALYSIS: IterResearch preserves an effective reasoning window of L − |M| − |TR| across rounds, whereas mono-contextual reasoning shrinks as max(0, L − t·|TR|).The comparison describes how accumulated history eventually exhausts the context limit.
  • B.2 COMPUTATIONAL COMPLEXITY ANALYSIS: Mono-contextual approaches face approximately L/|TR| maximum rounds, while IterResearch can theoretically continue indefinitely when |M| + |TR| < L.The condition is maintained through report synthesis.
  • B.2 COMPUTATIONAL COMPLEXITY ANALYSIS: Scaling experiments extend IterResearch to 2048 interactions using a 40K context length, demonstrating the practical relevance of its bounded workspace.The analysis contrasts this with the structural context limits of mono-contextual approaches.

B.3 EXTRAPOLATION BEYOND TRAINING HORIZON

IterResearch extrapolates beyond its 32-round training horizon by reconstructing bounded, position-agnostic workspaces around an evolving report. On BrowseComp, extending inference to 2048 rounds raises accuracy from 15.2% to 42.5%.

  • Empirical extrapolation: A 64× extrapolation factor lets the learned agent operate at Tmax = 2048 or higher despite training with Tmax = 32.The paper describes this extension as structurally infeasible for mono-contextual approaches.
  • Design basis: The workspace depends on the current question, evolving report, and latest interaction rather than absolute position or full trajectory history.This position-agnostic state design keeps decision-making consistent across research rounds.
  • Training setup: Training is deliberately limited to Tmax = 32 to control computation and encourage concise exploration strategies.The paper argues that learning to maximize information gain within limited rounds supports later extrapolation.
  • Empirical extrapolation: 42.5% BrowseComp accuracy at Tmax = 2048 versus 15.2% at Tmax = 32 demonstrates strong extrapolation beyond the training horizon.The agent is trained with Tmax = 32 but can use substantially longer inference horizons.
  • Comparison with mono-contextual methods: Mono-contextual methods face context saturation, position-embedding overflow, and attention-pattern collapse when extrapolated to extreme horizons.Accumulating 2048 rounds can exceed model context limits and cause hard failures.
  • Theoretical basis: The MDP formulation supports horizon-independent policies, while report synthesis maintains bounded complexity and stabilizes the state distribution.This combination provides a practical route to deploying agents on tasks whose exploration depth is unknown.

B.4 TRAINING DYNAMICS OF EFFICIENCY-AWARE POLICY OPTIMIZATION

Figure 5 shows stable EAPO training and improving benchmark accuracy across optimization. Rewards rise smoothly, while BrowseComp and BrowseComp-zh improve through distinct learning patterns.

  • Reward dynamics: Training rewards increase from 0.55 to approximately 0.72, a 30.9% improvement, with stable convergence across 150 optimization steps.The smooth EMA curve exhibits only minor oscillations.
  • Accuracy dynamics: BrowseComp accuracy jumps from 32% to 39% at step 50 before stabilizing, indicating rapid acquisition of effective search strategies.The passage associates the pattern with query reformulation or result-filtering strategies.
  • Accuracy dynamics: BrowseComp-zh accuracy improves monotonically from 40% to 45%, reflecting smoother incremental optimization.The passage suggests this may reflect differences in Chinese web-content organization.
  • Efficiency interpretation: Reward growth exceeds accuracy gains, with 30.9% reward improvement versus 18.8% on BrowseComp and 12.5% on BrowseComp-zh.The authors interpret this pattern as learning to solve tasks more efficiently, consistent with EAPO ablations.

C MORE IMPLEMENTATION DETAILS

IterResearch runs research in discrete rounds that generate a report and action, execute tools, and reconstruct a bounded workspace. The evolving report selectively preserves findings while termination can be natural or forced.

  • Iterative procedure: Each round generates reasoning, an updated report, and an action; non-terminal actions are executed by the environment before workspace reconstruction.The algorithm initializes an empty report and returns a final answer after natural or forced termination.
  • Workspace reconstruction: Workspace reconstruction retains only the question, updated report, and latest action-response pair instead of appending the full history.This bounded workspace is the central distinction from ever-growing contextual accumulation.
  • Report memory: The report evolves by incorporating new tool findings while preserving essential earlier insights and filtering redundant information.Selective retention maintains bounded complexity across trajectory length.
  • Termination: The algorithm terminates either when the agent outputs an answer or when the round counter reaches Tmax.These conditions support autonomous stopping while preventing infinite loops.
  • Tool environment: The environment supplies Google Search, Google Scholar, a goal-oriented web browser, and a sandboxed Python interpreter.These tools cover web retrieval, academic search, focused document extraction, and computation.

C.3 IMPLEMENTATION DETAILS

The implementation uses supervised warm-up followed by reinforcement learning, with curated and difficulty-calibrated data. Training uses short maximum horizons and LLM-based correctness rewards, while baseline scores come from published results.

  • Reproducibility: The implementation provides code and training data for reproducing IterResearch.Additional implementation details are supplied in the supplementary materials.
  • Supervised warm-up: The supervised phase curates 30K high-quality question-answer pairs filtered for answer quality, factual accuracy, and research complexity.These data are prepared to require genuine multi-step research.
  • Reinforcement learning: The reinforcement-learning phase selects 4,096 questions with 20%-60% success rates to target the model’s learning zone.Each question is evaluated with five independent trials before selection.
  • Maximum-round settings: Tmax = 32 is used during SFT and RL to encourage concise research strategies, while inference settings adapt to benchmark characteristics.The configuration balances training efficiency with inference flexibility.
  • Reward design: Rewards use an LLM judge that assigns 1.0 for a correct answer and 0.0 otherwise.Qwen3-235B-A22B evaluates answer correctness.
  • Baseline methodology: Prompting baselines are taken from published SOTA results because prior works often do not release their optimized prompts.This choice is intended to compare against the reported state of the art rather than weaker reimplementations.

C.4 AVERAGE TOKEN BUDGETS

The section measures IterResearch-30B-A3B’s internal generation-token budget across six benchmarks, excluding tool-returned content. BrowseComp requires substantially more reasoning tokens than the other tasks, while the reported gains are characterized as not resulting from uncontrolled token expenditure.

  • Approximately 376k tokens make BrowseComp the most reasoning-intensive benchmark, about an order of magnitude above the others.The passage links this budget to extensive iteration, thinking, and report synthesis in a complex web environment.
  • Performance continually improves as more interaction steps and generation tokens are permitted in the scaling experiments.
  • Approximately 81k tokens place BrowseComp-zh second in reasoning cost among the evaluated benchmarks.
  • Approximately 28k–33k tokens cover GAIA, HLE, Seal-0, and Xbench-DS, indicating lower iterative-synthesis requirements than the BrowseComp tasks.
  • The authors report that performance gains do not stem from uncontrolled token expenditure but from IterResearch operating within a manageable, task-dependent reasoning budget.The token accounting excludes tool responses and measures only the agent’s internal Think and Report steps.

D CASE STUDY OF ITERRESEARCH

The case study follows IterResearch solving a Micromalthidae biology question through planning, targeted literature search, and iterative report synthesis. It concludes that adult males do not feed after becoming adults, so their only food was their mother during the larval stage.

  • D CASE STUDY OF ITERRESEARCH: The example is presented as demonstrating evolving report synthesis, efficient information gathering, and autonomous termination decisions.
  • D CASE STUDY OF ITERRESEARCH: The agent’s planning identifies adult male feeding behavior as the key unresolved fact and flags assumptions about universal adult feeding as a cognitive blind spot.It then directs a Google Scholar search toward authoritative sources on the life cycle and feeding ecology.
  • D CASE STUDY OF ITERRESEARCH: The case study’s answer is that an adult male Micromalthidae fed only on its mother during the larval stage.The conclusion is attributed to male-larval matriphagy combined with non-feeding adulthood.
  • D CASE STUDY OF ITERRESEARCH: Male Micromalthus debilis larvae practice obligate matriphagy by consuming their mothers before pupating into adult males.
  • D CASE STUDY OF ITERRESEARCH: Adult males are described as non-feeding, with vestigial feeding structures and non-functional digestive systems during their brief adult stage.The reported morphology indicates that feeding is completed during larval development.
Loading 2511.07327v2…