Source-linked AI summary
DR-Venus: Towards Frontier Edge-Scale Deep Research Agents with Only 10K Open Data
Venus Team, Sunhao Dai, Yong Deng, Jinzhen Lin, Yusheng Song, Guoqing Wang, Xiaofeng Wu, Yuqi Zhou, Shuo Yang, Zhenzhe Ying, Zhanwei Zhang, Changhua Meng, Weiqiang Wang
TL;DR
Small deep research agents promise lower cost, latency, and privacy, but strong performance under limited open-data supervision remains difficult. DR-Venus addresses this with a two-stage, data-focused SFT and agentic RL recipe for a 4B model, using cleaned and resampled trajectories plus turn-level rewards. DR-Venus-4B outperforms prior similarly sized agents across most benchmarks, approaches a 30B-class system on one benchmark, and shows strong potential for test-time scaling.
Problem
The paper asks how to train a strong small deep research agent under limited open-data supervision, where noisy trajectories and unsuccessful RL rollouts limit effective learning.
Method
DR-Venus combines cleaned and resampled agentic SFT with agentic RL using IGPO-based information-gain rewards and format-aware regularization.
Results
DR-Venus-4B consistently outperforms prior 4B–9B agentic systems on most benchmarks, while DR-Venus-4B-RL reaches 74.7 on xBench-DS-2505 versus 75.0 for Tongyi-DR-30B.
Takeaways & Limitations
Strong deep research capability can emerge in a 4B model through careful open-data quality and utilization, with analysis also indicating value in test-time scaling.
Takeaways & Limitations
The RL training data is entirely English, which may contribute to weaker large-K performance on BrowseComp-ZH.
Abstract
from arXiv · showhide
Edge-scale deep research agents based on small language models are attractive for real-world deployment due to their advantages in cost, latency, and privacy. In this work, we study how to train a strong small deep research agent under limited open-data by improving both data quality and data utilization. We present DR-Venus, a frontier 4B deep research agent for edge-scale deployment, built entirely on open data. Our training recipe consists of two stages. In the first stage, we use agentic supervised fine-tuning (SFT) to establish basic agentic capability, combining strict data cleaning with resampling of long-horizon trajectories to improve data quality and utilization. In the second stage, we apply agentic reinforcement learning (RL) to further improve execution reliability on long-horizon deep research tasks. To make RL effective for small agents in this setting, we build on IGPO and design turn-level rewards based on information gain and format-aware regularization, thereby enhancing supervision density and turn-level credit assignment. Built entirely on roughly 10K open-data, DR-Venus-4B significantly outperforms prior agentic models under 9B parameters on multiple deep research benchmarks, while also narrowing the gap to much larger 30B-class systems. Our further analysis shows that 4B agents already possess surprisingly strong performance potential, highlighting both the deployment promise of small models and the value of test-time scaling in this setting. We release our models, code, and key recipes to support reproducible research on edge-scale deep research agents.
1 Introduction
DR-Venus addresses the underexplored challenge of building strong small deep research agents with limited open data by improving data quality, data utilization, and long-horizon training. Its 4B model achieves strong results while analysis highlights the roles of turn-level reinforcement learning, test-time scaling, and tool-use calibration.
- Motivation: Small deep research agents are attractive for cost, latency, and privacy, but their open-data frontier remains insufficiently explored.Existing systems generally use substantially larger models and often depend on closed data or complex training pipelines.
- Motivation: Limited open-data training is constrained by noisy trajectories and rollout groups with no successful paths, making supervision allocation critical for long-horizon search.Small models are especially sensitive to formatting artifacts and imperfect tool-use traces, while agentic RL can suffer advantage collapse.
- Approach: DR-Venus is a 4B deep research agent trained entirely on open data through agentic SFT followed by agentic RL with information-gain and format-aware rewards.The two-stage recipe targets basic agentic capability first and execution reliability second, while improving supervision density and credit assignment.
- Results: Built on roughly 10K open-data, DR-Venus-4B-SFT outperforms prior agentic systems under 9B parameters on most benchmarks, while RL further improves the SFT baseline and narrows the gap to 30B-class systems.The results support careful data quality and utilization improvements as a way to compensate for part of the scale gap.
- Analysis: Analysis finds that long-horizon resampling strengthens limited-data SFT, turn-level IGPO RL outperforms sparse trajectory-level optimization, and Pass@K exposes high small-agent capability ceilings.Successful trajectories also browse more than failed ones, while RL calibrates tool use toward more effective evidence acquisition.
- Reproducibility: The authors release models, code, and key recipes to support reproducible research on edge-scale deep research agents.The release is presented as a practical starting point for future work using small models and open data.
2 Methodology
DR-Venus formulates deep research as long-horizon reasoning and acting over external environments, then trains the agent with cleaned, resampled SFT data followed by agentic RL. The methodology targets both data quality and utilization while improving execution reliability through turn-level information-gain rewards.
- Overall Training Recipe: The two-stage pipeline first builds basic agentic capability with SFT, then improves long-horizon execution reliability with agentic RL.SFT learns from trajectories containing reasoning, actions, and observations; RL uses turn-level and trajectory-level rewards.
- Problem Formulation: Deep research agents iteratively reason, invoke tools, gather external evidence, and synthesize answers over long-horizon interactions.The formulation includes search, browsing, evidence collection, and a final answer action.
- Agentic SFT: SFT data construction aligns trajectories with the runtime protocol, prunes disallowed and duplicate tool calls, filters incorrect answers, and resamples longer trajectories.Correctness filtering retains 9,365 trajectories, while turn-aware resampling expands the final set to 18,745 instances with greater emphasis on long interactions.
- Agentic SFT: The data pipeline improves SFT quality and utilization by removing structurally invalid or low-value supervision and emphasizing long-horizon interactions.This design is intended to make limited open-data supervision more relevant to deep research.
- Agentic SFT: Agentic SFT trains on assistant-generated reasoning, tool invocations, and final answers while masking environment observations from the loss.The resulting initialization supports structured tool use and long-horizon interaction.
- Agentic RL: Information-gain rewards evaluate how much each turn increases the policy’s probability of generating the ground-truth answer.For ultra-long tasks, optional IG-Scale adjusts normalized information-gain rewards using outcome-reward scale to improve reward balance.
3 Experiments
Experiments show that DR-Venus is a strong 4B deep research agent: cleaned and resampled SFT already outperforms prior small agents, while IGPO-based agentic RL improves most benchmark results and execution reliability. Pass@K and browse-ratio analyses further indicate high latent capability, stronger low-budget reliability after RL, and important effects from data utilization and reward design.
- Main Results: SFT alone outperforms AgentCPM-Explore-4B by +2.7 on BrowseComp, +6.6 on BrowseComp-ZH, +1.5 on GAIA, +1.3 on xBench-DS-2510, and +4.9 on DeepSearchQA.The result establishes a strong 4B baseline across most evaluated benchmarks.
- Main Results: Agentic RL improves over SFT on five of six benchmarks, including +2.3 on BrowseComp, +2.0 on BrowseComp-ZH, +5.7 on xBench-DS-2505, and +5.4 on xBench-DS-2510.The RL model establishes a new state of the art among small deep research agents.
- Main Results: RL improvements are associated with better formatting accuracy, more stable tool use, and stronger execution reliability over long interaction trajectories.These analyses connect the benchmark gains with improved behavior during agent execution rather than only final-answer optimization.
- Main Results: DR-Venus-4B-SFT surpasses OpenResearcher-30B-A3B on all reported benchmarks, while DR-Venus-4B-RL reaches 74.7 on xBench-DS-2505 versus 75.0 for Tongyi-DR-30B.The results narrow the performance gap between the 4B models and substantially larger systems.
- Ablation Study: Long-horizon resampling improves BrowseComp by +4.0 and BrowseComp-ZH by +1.8 when comparing SFT with and without resampling.Both models use the same underlying trajectory source; the difference is whether long-horizon trajectories are upweighted during SFT.
- Ablation Study: IGPO improves over the resampled SFT baseline by +2.3 on BrowseComp and +2.0 on BrowseComp-ZH, whereas GRPO decreases performance by -1.5 and is nearly flat at -0.1.The ablation attributes consistent long-horizon RL gains to denser, better-aligned turn-level supervision.
- Analysis of Capability Boundary: Pass@K results show that RL improves BrowseComp from 26.8 to 29.1 at Pass@1 and from 61.7 to 63.7 at Pass@16, while SFT reaches 78.5 at Pass@16 on BrowseComp-ZH.RL mainly improves reliability under limited sampling on BrowseComp-ZH, where SFT is already strong at larger K.
4 Conclusion
DR-Venus is a frontier 4B edge-scale deep research agent built entirely on open data. Its two-stage recipe combines agentic SFT with cleaned, resampled trajectories and agentic RL with IGPO to achieve strong small-model performance.
- DR-Venus is a frontier 4B edge-scale deep research agent built entirely on open data.
- The method uses agentic SFT with strict trajectory cleaning and long-horizon resampling, followed by agentic RL with IGPO.
- DR-Venus-4B outperforms prior similarly sized agentic models and narrows the gap to substantially larger 30B-class systems.
- The results indicate that improving training-data quality and utilization can unlock strong deep research capability in small models.
- The authors release models, code, and recipes as a practical starting point for research on edge-scale deep research agents.
5 Contributions
The listed contributors are presented alphabetically by last name. The section records the contributor names and identifies dagger-marked entries.
- Contributors are listed alphabetically by their last names.
- The contributor list includes Sunhao Dai†, Yong Deng, Jinzhen Lin, Yusheng Song, Guoqing Wang, Xiaofeng Wu, and Yuqi Zhou.
- Additional contributors are Shuo Yang, Zhenzhe Ying, and Zhanwei Zhang.
- Changhua Meng† and Weiqiang Wang are also listed as contributors.
A System Prompt
The system prompt defines DR-Venus as a multi-source deep research assistant that investigates questions with search and webpage visits. It also specifies tool-call and final-answer formatting.
- The prompts are adapted from REDSearcher and Tongyi DeepResearch and refined for DR-Venus’s reasoning and interaction patterns.
- DR-Venus is instructed to conduct thorough, multi-source investigations across broad and specialized queries.
- The assistant must cross-check credible, diverse sources and integrate findings into comprehensive, accurate, structured, objective responses.
- The available tools include a search function for multiple Google queries and a visit function for webpages with a specified information goal.
- Function calls must be returned as JSON objects containing the function name and arguments inside tool-call XML tags.