PaperlayerView exact revision on arXiv

Source-linked AI summary

DR-Venus: Towards Frontier Edge-Scale Deep Research Agents with Only 10K Open Data

Venus Team, Sunhao Dai, Yong Deng, Jinzhen Lin, Yusheng Song, Guoqing Wang, Xiaofeng Wu, Yuqi Zhou, Shuo Yang, Zhenzhe Ying, Zhanwei Zhang, Changhua Meng, Weiqiang Wang

arXiv:2604.19859v1cs.LGcs.AIcs.CLcs.IR

TL;DR

Training strong small deep-research agents with limited open data remains an underexplored challenge. DR-Venus combines cleaned, resampled supervised fine-tuning with turn-level IGPO reinforcement learning, and its 4B model outperforms prior sub-9B agents on most benchmarks while approaching a 30B system on one benchmark.

  • Problem

    How to train strong small deep-research agents under limited open-data supervision remains an underexplored question, despite the deployment appeal of small models.

    Source: §1 · p. 2

  • Method

    DR-Venus uses cleaned, resampled trajectory SFT followed by IGPO-based agentic RL with turn-level information-gain rewards and format penalties.

    Source: §1 · p. 2

  • Results

    DR-Venus-4B outperforms prior agentic systems under 9B parameters on most benchmarks and narrows the gap to 30B-class systems; on xBench-DS-2505, it reaches 74.7 versus Tongyi-DR-30B’s 75.0.

    Source: §1 · p. 2, §3.2 · p. 10

  • Takeaways & Limitations

    The results show that strong deep-research capability is attainable in a 4B model under limited open-data supervision.

    Source: §4 · p. 13

    The current RL training data is entirely English, which may partly explain weaker large-K performance on BrowseComp-ZH.

    Source: §3.4 · p. 12

Abstract

from arXiv · show

Edge-scale deep research agents based on small language models are attractive for real-world deployment due to their advantages in cost, latency, and privacy. In this work, we study how to train a strong small deep research agent under limited open-data by improving both data quality and data utilization. We present DR-Venus, a frontier 4B deep research agent for edge-scale deployment, built entirely on open data. Our training recipe consists of two stages. In the first stage, we use agentic supervised fine-tuning (SFT) to establish basic agentic capability, combining strict data cleaning with resampling of long-horizon trajectories to improve data quality and utilization. In the second stage, we apply agentic reinforcement learning (RL) to further improve execution reliability on long-horizon deep research tasks. To make RL effective for small agents in this setting, we build on IGPO and design turn-level rewards based on information gain and format-aware regularization, thereby enhancing supervision density and turn-level credit assignment. Built entirely on roughly 10K open-data, DR-Venus-4B significantly outperforms prior agentic models under 9B parameters on multiple deep research benchmarks, while also narrowing the gap to much larger 30B-class systems. Our further analysis shows that 4B agents already possess surprisingly strong performance potential, highlighting both the deployment promise of small models and the value of test-time scaling in this setting. We release our models, code, and key recipes to support reproducible research on edge-scale deep research agents.

1 Introduction

The paper asks how to train a capable edge-scale deep research agent from limited open data, treating data quality and utilization as central and proposing DR-Venus.

  • Small models offer cost, latency, and privacy advantages, but the frontier for open-data small deep research agents remains underexplored.Source: §1 · p. 2
  • Small models are sensitive to noisy trajectories, while RL rollout groups may contain no successes, causing advantage collapse and making effective supervision allocation crucial.Source: §1 · p. 2
  • DR-Venus combines cleaned, resampled REDSearcher SFT with IGPO-based RL using turn-level information-gain rewards and format penalties.These rewards target supervision density, credit assignment, and behavioral stability in long-horizon tasks.Source: §1 · p. 2
  • On roughly 10K open data, DR-Venus-4B-SFT outperforms prior agentic systems under 9B on most benchmarks; DR-Venus-4B-RL improves on SFT and narrows the gap to 30B-class systems.Source: §1 · p. 2
  • Analyses show long-horizon resampling improves limited-data SFT and turn-level IGPO RL outperforms sparse trajectory-level optimization.Pass@K suggests a high capability ceiling for small agents; successful trajectories browse more than failed ones, and RL calibrates tool use toward evidence acquisition.Source: §1 · pp. 2–3
  • The authors release DR-Venus models, code, and key recipes to support transparent and reproducible research.Source: §1 · p. 3

2 Methodology

DR-Venus trains a deep research agent to reason, use tools, gather external evidence, and synthesize answers through a two-stage pipeline: cleaned, resampled agentic SFT followed by agentic RL with turn-level credit assignment.

2.1 Problem Formulation

  • Deep research is framed as long-horizon interaction in which an agent reasons, searches and browses for evidence, then produces a final answer.The agent must gather enough external evidence to support a correct answer rather than rely solely on parametric knowledge.Source: §2.1 · p. 3

2.2.1 Data Filtering and Construction

  • Turn-aware resampling increases the final SFT set from 9,365 to 18,745 instances and raises trajectories over 50 turns from 60.28% to 80.15%, and over 100 turns from 13.29% to 33.21%.Source: §2.2.1 · p. 4

2.2.2 Agentic Supervised Fine-Tuning

  • Agentic SFT initializes structured tool use and long-horizon interaction by training on assistant-generated reasoning, tool calls, and answers while masking environment observations.The training pipeline converts REDSearcher (Chu et al., 2026) trajectories to the inference format and masks non-assistant tokens in the loss.Source: §2.2 · p. 4, §2.2.2 · p. 5

2.3 Pushing Toward Frontier Performance with Reinforcement Learning

  • Agentic RL addresses residual formatting errors, redundant reasoning, and inefficient tool use with dense turn-level rewards to improve data efficiency despite scarce, costly training data.The approach adopts IGPO (Wang et al., 2026), which uses a GRPO-style group of rollouts for each query.Source: §2.3 · p. 5

2.3.1 Turn-Level Reward Design for Long-Horizon Agentic Tasks

  • Information-gain rewards score how much each turn increases the policy’s probability of the ground-truth answer; optional browse-aware assignment places rewards on browse turns and preceding searches.The answer turn instead retains an outcome reward based on correctness.Source: §2.3.1 · pp. 5–6
  • Turn-level format penalties target malformed outputs, while optional IG-Scale rebalances information-gain and outcome signals and discounted cumulative rewards incorporate future turn rewards.IG-Scale is intended for ultra-long tasks where outcome supervision may be weak; discounted cumulative rewards are assigned to each token in the turn output.Source: §2.3.1 · pp. 6–7

2.3.2 Policy Optimization via IGPO

  • IGPO assigns turn-level rewards to policy-generated tokens and optimizes them with a GRPO-style objective, using clipping and a KL penalty.The KL coefficient controls penalty strength.Source: §2.3.2 · pp. 7–8

3 Experiments

Experiments show that DR-Venus-4B performs strongly across deep-research benchmarks: SFT establishes a competitive small-model baseline, and agentic RL improves results on most benchmarks. Resampling, IGPO, test-time scaling, and browse use further illuminate how the training and inference setup affects performance.

3.2 Main Results

  • DR-Venus-4B-SFT gains over AgentCPM-Explore-4B on five benchmarks, including +2.7 on BrowseComp and +6.6 on BrowseComp-ZH.It also gains +1.5 on GAIA, +1.3 on xBench-DS-2510, and +4.9 on DeepSearchQA; the paper reports that SFT surpasses OpenResearcher-30B-A3B on all reported benchmarks.Source: §3.2 · p. 9, Table 1 · p. 10
  • DR-Venus-4B-RL sets a new small-agent state of the art and improves over SFT on five of six benchmarks, including +2.3 on BrowseComp and +5.7 on xBench-DS-2505.It also gains +2.0 on BrowseComp-ZH, +5.4 on xBench-DS-2510, and +1.9 on DeepSearchQA; the authors associate improvements with better formatting accuracy, more stable tool use, and stronger execution reliability.Source: §3.2 · p. 9, Table 1 · p. 10
  • On xBench-DS-2505, DR-Venus-4B-RL scores 74.7, approaching Tongyi-DR-30B at 75.0 despite the scale difference.The authors report that SFT already matches or exceeds some 30B-scale agents on individual benchmarks, and RL further narrows the gap.Source: §3.2 · pp. 9–10

3.3 Ablation Study

  • Resampling long-horizon trajectories improves DR-Venus-4B-SFT by +4.0 on BrowseComp and +1.8 on BrowseComp-ZH over the no-resampling variant.Both variants use the same REDSearcher trajectory source; resampling upweights long-horizon trajectories during SFT.Source: §3.3 · pp. 10–11, Table 2 · p. 11
  • On BrowseComp and BrowseComp-ZH, IGPO improves over the resampled SFT baseline by +2.3 and +2.0, while GRPO changes scores by -1.5 and -0.1, respectively.The paper attributes IGPO’s stronger results to dense, well-aligned turn-level supervision for long-horizon tasks, where sparse trajectory-level optimization was less effective.Source: §3.3 · p. 11

3.4 Analysis of Capability Boundary

  • Pass@K results show that RL improves BrowseComp performance across K, while on BrowseComp-ZH its largest gains occur at low K and SFT leads at larger K.BrowseComp Pass@1 and Pass@16 rise from 26.8 to 29.1 and 61.7 to 63.7. On BrowseComp-ZH, Pass@1 and Pass@2 rise from 35.7 to 37.7 and 52.9 to 53.3, but SFT scores 74.0 at Pass@8 and 78.5 at Pass@16 versus RL’s 73.0 and 76.5; English-only RL training may contribute to the large-K mismatch.Source: §3.4 · pp. 11–12, Figure 2 · p. 12
  • On BrowseComp-ZH, SFT reaches 78.5 at Pass@16, exceeding Tongyi-DR-30B at 46.7, Gemini-3-Pro at 66.8, and GPT-5 High at 65.0.The authors argue that Pass@1 alone may underestimate small agents’ latent capability and that test-time scaling can reveal a stronger performance frontier.Source: §3.4 · p. 12, Figure 2 · p. 12

3.5 Analysis of Tool Use

  • Across six benchmarks, correct trajectories generally browse more than wrong ones, and RL raises overall browse ratio from 17.49% to 22.46%.Correct-trajectory browse ratio rises from 23.71% to 28.96%; on xBench-DS-2510, RL reverses SFT’s wrong-over-correct pattern (22.99% versus 17.50%).Source: §3.5 · pp. 12–13, Figure 3 · p. 12

4 Conclusion

DR-Venus is a 4B edge-scale deep research agent trained entirely on open data with a two-stage SFT-and-RL recipe. It outperforms prior similarly scaled agents, narrows the gap to 30B-class systems, and the authors release its models, code, and recipes.

  • DR-Venus-4B significantly outperforms prior similarly scaled agentic models and narrows the gap to substantially larger 30B-class systems.Source: §4 · p. 13
  • The two-stage recipe combines agentic SFT with strict trajectory cleaning and long-horizon resampling, followed by agentic RL using IGPO.Source: §4 · p. 13
  • Under limited open-data supervision, improving data quality and utilization unlocks strong small-model research capability, while RL stabilizes formatting, tool use, and long-horizon execution.Source: §4 · p. 13
  • The authors release DR-Venus models, code, and recipes as a practical starting point for research on edge-scale deep research agents.Source: §4 · p. 13

A System Prompt

DR-Venus’s system prompt adapts REDSearcher (Chu et al., 2026) and Tongyi DeepResearch (Team et al., 2025b), refining their prompts to support its reasoning and interaction patterns. It directs the agent to investigate topics across sources and use structured tool calls.

A System Prompt

  • The prompt is adapted from REDSearcher (Chu et al., 2026) and Tongyi DeepResearch (Team et al., 2025b), with refinements for DR-Venus’s reasoning and interaction patterns.The section presents the complete system prompts used in DR-Venus.Source: A System Prompt · p. 16
  • The assistant is instructed to investigate broad or specialized topics using credible, diverse sources, cross-check information, and deliver a comprehensive, accurate, structured, objective response.It must enclose its definitive final response in <answer></answer> tags.Source: A System Prompt · p. 16
  • The prompt permits one or more function calls and specifies search and webpage-visit tools, including query, URL, and visit-goal arguments.Search accepts multiple queries, while visit accepts one or more URLs and a specific information goal.Source: A System Prompt · p. 16
  • Each function call must be formatted as a JSON object containing the function name and arguments, enclosed in <tool_call></tool_call> tags.The prompt supplies the function signatures within <tools></tools> tags.Source: A System Prompt · p. 16

Related papers

Loading 2604.19859v1…