Source-linked AI summary
Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window
Qiaoyu Tang, Hao Xiang, Le Yu, Bowen Yu, Yaojie Lu, Xianpei Han, Le Sun, WenJuan Zhang, Pengbo Wang, Shixuan Liu, Zhenru Zhang, Jianhong Tu, Hongyu Lin, Junyang Lin
TL;DR
Long-horizon search agents need harder tasks and better context management to sustain deep reasoning beyond short multi-turn interactions. DeepMiner constructs verifiable multi-source web questions and trains agents with a dynamic sliding window that preserves reasoning while compressing distant tool outputs. DeepMiner-32B improves across multiple benchmarks, reaching 33.5% on BrowseComp-en and nearly 100 turns within 32k context.
Problem
Existing search-agent tasks often permit shallow retrieval, while accumulated tool responses rapidly exhaust context and limit long-horizon reasoning.
Method
DeepMiner combines reverse construction of difficult QA pairs from authentic web sources with dynamic sliding-window context management for training and inference.
Results
DeepMiner-32B reaches 33.5% accuracy on BrowseComp-en, nearly doubles prior open-source performance, and improves consistently across BrowseComp-zh, XBench-DeepSearch, and GAIA.
Takeaways & Limitations
Dynamic context management supports sustained long-horizon interactions while preserving assistant reasoning traces and avoiding external summarization models.
Abstract
from arXiv · showhide
While recent advances in reasoning models have demonstrated cognitive behaviors through reinforcement learning, existing approaches struggle to invoke deep reasoning capabilities in multi-turn agents with long-horizon interactions. We propose DeepMiner, a novel framework that elicits such abilities by introducing high-difficulty training tasks and dynamic context window. DeepMiner presents a reverse construction method to generate complex but verifiable question-answer pairs from authentic web sources, which ensures the challenge and reliability of training data while injecting cognitive capabilities into multi-turn reasoning scenarios. We further design an elegant yet effective dynamic context management strategy for both training and inference, utilizing sliding window mechanisms while eliminating the dependency on external summarization models, thereby efficiently empowering the model to handle continuously expanding long-horizon contexts. Through reinforcement learning on Qwen3-32B, we develop DeepMiner-32B, which achieves substantial performance improvements across multiple search agent benchmarks. DeepMiner attains 33.5% accuracy on BrowseComp-en, surpassing the previous best open-source agent by almost 20 percentage points, and demonstrates consistent improvements on BrowseComp-zh, XBench-DeepSearch, and GAIA. Notably, our dynamic context management enables sustained interactions of nearly 100 turns within standard 32k context length, effectively addressing the context limitations that constrain existing multi-turn interaction systems.
1 INTRODUCTION
DeepMiner targets two barriers to deep reasoning in long-horizon search agents: insufficiently demanding tasks and rapidly exhausting context. It combines difficult, verifiable web-based training data with dynamic context management and improves performance across deep-research benchmarks.
- DeepMiner addresses inadequate task complexity and context-management limits that hinder long-horizon multi-turn search agents.Existing tasks can be solved through shallow retrieval, while 32k contexts typically support only 10–15 effective turns.
- DeepMiner generates challenging QA pairs from multiple authentic web sources and manages expanding contexts with a dynamic sliding window.The strategy selectively compresses distant tool responses while preserving assistant reasoning traces and avoiding external summarization models.
- 33.5% accuracy on BrowseComp-en surpasses the previous best open-source agent by nearly 20 percentage points.The model also improves consistently on BrowseComp-zh, XBench-DeepSearch, and GAIA.
- Dynamic context management enables sustained interactions of nearly 100 tool calls within standard 32k context length.This extends the operational horizon beyond the approximately 10–15 turns typically supported under context limits.
2 COMPLEX QUESTION CONSTRUCTION
DeepMiner constructs difficult, verifiable questions by combining information from multiple authentic web sources. Entity-driven collection, deliberate cross-source synthesis, obfuscation, and multi-stage filtering produce training tasks requiring extended reasoning.
- The pipeline comprises entity-driven information collection, multi-source question generation, and strict quality filtering.Figure 1 presents this overall construction process.
- Sources undergo entity correspondence, information complementarity, and credibility checks before question generation.These filters remove entity confusion, redundant information, and unreliable websites.
- Questions synthesize information from at least four distinct sources and exclude Wikipedia pages to force cross-document inference.Secondary obfuscation further increases the reasoning demands of generated questions.
- Difficulty and quality filtering retain QA pairs that require tool-assisted multi-step exploration and meet reliability requirements.Questions solvable through direct search or zero-shot reasoning are eliminated.
- The reverse-construction pipeline creates complex QA pairs requiring multi-source reasoning rather than shallow retrieval.Questions are grounded in authentic web information and designed to demand extended exploration and strategic planning.
3 REINFORCEMENT LEARNING WITH DYNAMIC CONTEXT WINDOW
DeepMiner trains agents to operate under dynamically managed contexts by replacing distant tool outputs while preserving assistant reasoning. It aligns training with inference and propagates trajectory-level reinforcement-learning signals across context-specific training sequences.
- 3.1 DYNAMIC CONTEXT MANAGEMENT STRATEGY: Without sliding windows, tool responses grow exponentially and squeeze assistant context; with sliding windows, tool length stays constant while assistant content grows normally.Figure 2 compares these context-length dynamics on BrowseComp preliminary experiments.
- 3.1 DYNAMIC CONTEXT MANAGEMENT STRATEGY: Dynamic context management replaces early tool responses with placeholders while retaining recent tool outputs and all assistant reasoning outputs.Sliding occurs when accumulated tool responses reach window size W, using slide step S to define the replacement boundary.
- 3.1 DYNAMIC CONTEXT MANAGEMENT STRATEGY: Training sequences reproduce the context states encountered during inference, preventing mismatch between training and deployment.Each trajectory is decomposed into multiple sequences with progressively different sliding-window contexts.
- 3.1 DYNAMIC CONTEXT MANAGEMENT STRATEGY: Each assistant response is trained exactly once across decomposed sequences through careful masking.This prevents optimization conflicts while preserving the dynamic context conditions used at inference.
- 3.3 REINFORCEMENT LEARNING TRAINING: The strategy enables models to learn under the same dynamic context conditions used during inference while scaling to arbitrarily long interaction sequences.A supervised fine-tuning cold start supplies high-quality tool-use trajectories before reinforcement learning.
- 3.3 REINFORCEMENT LEARNING TRAINING: Trajectory-level advantages are propagated to every sequence derived from the same rollout, preserving group-relative policy optimization under sliding-window training.Rewards are computed from final answers across grouped trajectory rollouts, while each derived sequence receives the shared trajectory advantage.
4 EXPERIMENTS
DeepMiner is evaluated across deep research benchmarks, training stages, context-management strategies, and scaling settings. Results show strong benchmark performance, gains from reinforcement learning, and efficient long-horizon interaction within constrained context lengths.
- 4.2 MAIN RESULTS: +12.3 percentage points on BrowseComp-en, +12.1 on BrowseComp-zh, +9.0 on XBench-DeepSearch, and +4.3 on GAIA are obtained by reinforcement learning over the SFT model.The largest gains occur on BrowseComp-en and BrowseComp-zh.
- 4.3 CONTEXT MANAGEMENT EFFICIENCY ANALYSIS: 33.3% BrowseComp accuracy is achieved with 32k context, exceeding alternative methods requiring 128k context.The comparison uses GPT-OSS-120B and evaluates vanilla, external summarization, and sliding-window strategies.
- 4.4 DETAILED ANALYSIS: Training rewards rise from 0.45 to 0.60 while BrowseComp performance improves from 22% to 33.5% during reinforcement learning.Trajectory length steadily increases while remaining within the 40k trajectory-length limit.
- 4.4 DETAILED ANALYSIS: At 100 tool calls, performance reaches 33.5, while 32k context yields nearly 33.0 and supports close to 100 rounds of tool interactions.Performance increases with larger tool-call budgets and approaches leading proprietary systems at 100 calls.
- 4.4 DETAILED ANALYSIS: 21.2% BrowseComp accuracy for the DeepMiner SFT model exceeds the 15.6% achieved by a HotpotQA-trained model.The comparison supports the effectiveness of deliberately challenging, reverse-constructed training data.
5 RELATED WORK
The paper situates DeepMiner among reinforcement-learning approaches for LLMs and deep research agents, emphasizing web-based multi-step reasoning and the limitations of closed architectures.
- Table 3 compares supervised fine-tuning models trained on HotpotQA and DeepMiner data.
- Reinforcement Learning with Verifiable Rewards has become a standard approach for training large language models on complex tasks.
- Deep research agents autonomously combine search engines, web browsing, and tools to solve complex research tasks through multi-step reasoning.
- Proprietary deep research systems demonstrate strong capabilities, but their closed architectures and inaccessible training details limit reproducibility.
6 CONCLUSION
DeepMiner targets sustained long-horizon web interactions by combining difficult verifiable training tasks with dynamic context management, and reports strong benchmark performance and extended interaction capacity.
- DeepMiner addresses insufficient task complexity and context explosion through verifiable QA construction from authentic web sources and dynamic context management without external summarization models.
- DeepMiner preserves complete assistant reasoning traces while compressing distant tool responses, enabling interactions exceeding 100 turns within 40k contexts.
- 33.5% on BrowseComp was achieved by DeepMiner-32B, nearly doubling prior state-of-the-art open-source agent performance.
- DeepMiner is presented as a shift from context-limited, shallow reasoning toward unbounded, deep exploration.
B ETHICAL CONSIDERATIONS
The paper acknowledges privacy and misuse risks from web-sourced training data and proposes filtering, anonymization, controlled access, and review procedures.
- Publicly available training sources may inadvertently contain personal information, creating a privacy risk.
- The authors restrict collection to public webpages, filter irregular websites and social media, and plan comprehensive anonymization before release.
- Dataset access will require formal academic approval, while model-weight requests will undergo review for legitimate research and educational use.
C ENHANCED TOOL SUITE
The enhanced tool suite supports fine-grained web exploration through search, paginated fetching, and in-page finding, preserving information access across extended interactions.
- The tool suite combines Web Search, Fetch, and Find for fine-grained web exploration.
- Paginated Fetch lets the model inspect initial webpage content and decide whether to continue browsing or exit instead of relying on truncation or external summarization.
- Find locates relevant sections and surrounding context within lengthy webpages before detailed examination.
- By preserving complete information access and navigation flexibility, the suite provides fine-grained control over information gathering across extended interaction sequences.
D TEMPLATES
The templates define a strict judge that extracts a final answer, compares it with the correct answer, explains only meaningful differences, and returns a binary correctness decision.
- The final judgment is “yes” for an exact or numerically near-equivalent match and “no” for inconsistency, ambiguity, or incorrectness.
- The judge extracts the response’s exact final answer, returning “None” when no exact final answer is available.
- The reasoning evaluates only whether the extracted answer matches the correct answer, without solving the problem or discussing background.
E QUESTION EXAMPLES
The section presents representative complex questions together with their core entities and answers, spanning journalism, academia, and basketball.
- The examples pair each question with its core entity and answer for inspection.
- One question identifies journalist Michael Crowley and asks for his birth month and day, answered as April 1.
- Other examples concern an academic and basketball player Ben Vander Plas, whose birth date is answered as September 19.
E.1 CASE STUDY
The section introduces a case trajectory showing DeepMiner operating on BrowseComp.
- A BrowseComp case trajectory is presented as a concrete example of DeepMiner’s operation.
A Case Trajectory on BrowseComp
The trajectory shows iterative search and hypothesis revision before converging on Ahsan Manzil as the answer to a multi-constraint historical-place question.
- The search considers multiple capital cities and candidate sites while repeatedly revising hypotheses when evidence appears weak or uncertain.
- The trajectory includes explicit web-search calls and tool responses, with the responses omitted from the recorded case.
- The eventual evidence combines Ahsan Manzil’s Dhaka location, river setting, construction dates, wall thickness, damage history, and 1985 government acquisition.
- The final trajectory identifies Ahsan Manzil in Dhaka as satisfying the stated historical-place criteria.