PaperlayerView exact revision on arXiv

Source-linked AI summary

Mind DeepResearch Technical Report

MindDR Team, Li Auto Inc

arXiv:2604.14518v2cs.AI

TL;DR

Deep research agents remain costly because they often require massive models and inefficient long-horizon tool use. MindDR addresses this with a three-agent architecture and four-stage, agent-specialized training pipeline using ~30B-parameter models, achieving strong results across deep-search and deep-research evaluations and introducing a multidimensional 500-query benchmark. The framework reaches state-of-the-art performance on MindDR Bench while demonstrating strong open-source benchmark results.

  • Problem

    Deep research agents face prohibitive training and inference costs, creating a challenge of achieving leading performance and user experience with smaller models and low-cost optimization.

  • Method

    MindDR combines Planning, DeepSearch, and Report Agents with a staged curriculum of SFT, Search-RL, Report-RL, and preference alignment.

  • Results

    MindDR-v1.5-30B-A3B achieves the best results among open-source agent-style systems across BrowseComp-ZH, BrowseComp, xbench-DS, and GAIA-DS, while MindDR Bench reaches a state-of-the-art RACE score of 51.8.

  • Takeaways & Limitations

    MindDR demonstrates that ~30B-parameter models can match or surpass the deep research capabilities of much larger foundation models while supporting multidimensional evaluation of report quality.

  • Takeaways & Limitations

    Scaling beyond 128K context for extremely complex research tasks remains an open challenge, and automated evaluation misses methodological soundness, novelty, and uncertainty calibration.

Abstract

from arXiv · show

We present Mind DeepResearch (MindDR), an efficient multi-agent deep research framework that achieves leading performance with only ~30B-parameter models through a meticulously designed data synthesis and multi-stage training pipeline. The core innovation of MindDR lies in a collaborative three-agent architecture (Planning Agent, DeepSearch Agent, and Report Agent) and a four-stage agent-specialized training pipeline comprising SFT cold-start, Search-RL, Report-RL and preference alignment. With this regime, MindDR demonstrates competitive performance even with ~30B-scale models. Specifically, MindDR achieves 45.7% on BrowseComp-ZH, 42.8% on BrowseComp, 46.5% on WideSearch, 75.0% on xbench-DS, and 52.5 on DeepResearch Bench, outperforming comparable-scale open-source agent systems and rivaling larger-scale models. MindDR has been deployed as an online product in Li Auto. Furthermore, we introduce MindDR Bench, a curated benchmark of 500 real-world Chinese queries from our internal product user interactions, evaluated through a comprehensive multi-dimensional rubric system rather than relying on a single RACE metric. On MindDR Bench, MindDR achieves a state-of-the-art score of 51.8.

1 Introduction

MindDR targets the high training and inference costs of deep research agents with a ~30B-parameter framework that combines specialized agents and staged optimization. It also introduces a 500-query Chinese benchmark with multidimensional evaluation aligned to practical user experience.

  • Motivation: Deep research agents face prohibitive training and inference costs, especially from massive models, long-horizon reasoning, and inefficient tool exploration.The paper frames reducing cost while preserving performance and user experience as the central challenge.
  • Approach: MindDR addresses this challenge with specialized inference subtasks and a targeted multi-stage training pipeline using ~30B-parameter models.Its design combines inference-stage decomposition with training-stage optimization rather than expensive mid-training or monolithic end-to-end reinforcement learning.
  • Approach: The inference architecture uses Planning, DeepSearch, and Report Agents to decompose queries, search in parallel, and synthesize reports.The DeepSearch Agent handles multi-step retrieval, while the Report Agent focuses on conflict resolution and human-aligned content.
  • Training: The four-stage pipeline comprises SFT cold-start, Search-RL, Report-RL, and preference alignment for heterogeneous agent capabilities.Search-RL improves BrowseComp-ZH accuracy while reducing context and tool-call consumption relative to the SFT baseline, and Report-RL improves report RACE scores.
  • Evaluation: MindDR Bench contains 500 Chinese deep research queries from real-world Livis interactions and evaluates search, content quality, and presentation format.The benchmark is designed to provide practical, fine-grained insights beyond a single aggregate RACE score.

2 Related Works

Related work positions deep research as an agentic extension of retrieval and reasoning, while identifying unresolved challenges in realistic data, fine-grained search supervision, and report evaluation.

  • Deep Research Agents: Deep research agents extend RAG with open-domain retrieval, evidence verification, long-horizon reasoning, tool use, and structured report generation.Proprietary systems demonstrate strong performance, but their closed-source nature limits reproducibility and systematic analysis.
  • Data Synthesis: Knowledge-graph synthesis produces logically consistent retrieval-reasoning data but is limited by graph coverage and dynamic open-domain scenarios.Agent-simulation methods improve realism and task alignment but incur high computational costs.
  • Data Synthesis: Existing synthesis methods often lack fine-grained per-step retrieval contributions, hindering differentiated signals for critical and peripheral search steps.This limitation makes targeted training supervision difficult.
  • Search Reinforcement Learning: Trajectory-level RL assigns uniform rewards across steps, whereas step-level methods provide stronger supervision but may require costly critics or complex sampling.MindDR proposes lightweight step-level credit assignment without critic overhead or exponential sampling complexity.
  • Report Reinforcement Learning: Report-RL research targets factual consistency, long-form structure, information conflict resolution, and human-aligned generation through staged rewards, reflection, revision, and self-correction.These approaches support report quality optimization beyond retrieval performance.
  • Evaluation and Reward Design: RACE evaluates comprehensiveness, insight, instruction following, and readability, while other rubric frameworks add dimensions such as factual accuracy and structural coherence.Such multidimensional rubrics can supply reward signals for report-generation RL.

3 MindDR Framework

MindDR combines a three-agent inference pipeline with a four-phase capability curriculum, coordinating search and report generation through shared reasoning memory. Its training stages use capability-specific objectives and rewards rather than a single end-to-end optimization target.

  • 3.1 Inference Pipeline: The inference pipeline contains Planning, DeepSearch, and Report Agents coordinated through shared memory and Extended Chain-of-Thought.It decomposes queries, retrieves and synthesizes heterogeneous evidence, and assembles research reports.
  • 3.1 Inference Pipeline: The Planning Agent analyzes intent and decomposes a query into subtasks that independent DeepSearch Agent instances execute in parallel.Parallel execution supports separate searches for the resulting subtasks.
  • 3.1 Inference Pipeline: Each DeepSearch Agent uses a ReAct-style loop for multi-source retrieval, evidence integration, and intermediate reasoning until sufficient information is gathered.The Report Agent then generates an outline and organizes sub-reports into a coherent Markdown report.
  • 3.1 Inference Pipeline: Extended Chain-of-Thought memory carries reasoning traces across multiple agent interactions and tool calls, while tool memory supports coordination.DeepSearch and Report Agents append to shared reasoning context across the pipeline.
  • 3.2 Training Pipeline Overview: MindDR decomposes training into four phases because heterogeneous objectives require tractable rewards, capability dependencies, and capability-specific data.The curriculum avoids a single sparse, noisy reward spanning tool correctness, reasoning, report coherence, and preferences.
  • 3.2 Training Pipeline Overview: SFT establishes tool-use and ReAct-format behavior, Search-RL optimizes long-horizon search decisions, and Report-RL improves long-form quality using RACE and citation or format rewards.Search-RL uses real tool execution and progressively scheduled rewards; Report-RL uses LLM-as-Judge evaluation.
  • 3.2 Training Pipeline Overview: Preference alignment uses on-policy self-improvement with DPO and Self-SFT to address temporal correctness and table-format errors without catastrophic forgetting.This phase targets residual user-experience issues in generated long-form reports.

4 Data Synthesis

MindDR’s data synthesis combines structured multi-hop reasoning data with real user queries to support controllable yet ecologically valid training and evaluation. Its benchmark construction adds expert filtering and modular evaluation to align assessment with practical research-system development.

  • Multi-hop Query Synthesis: The synthesis pipeline constructs multi-hop questions through graph sampling, implicit QA generation, condition obfuscation, and multi-stage quality filtering.Filtering removes directly answerable, contradictory, implausible, or semantically incoherent samples before training use.
  • Query Source Diversification and Mixing: Knowledge-graph queries provide complete reasoning chains and ground-truth answers, while real user queries compensate for corpus-coverage limits and distributional shift.The two sources are mixed at calibrated proportions to balance controllability with ecological validity.
  • MindDR Bench: MindDR Bench contains 500 Chinese deep research queries mined from Li Auto assistant logs and filtered by LLM prescreening, expert annotation, and review.The queries span 16 domains, including automotive, travel, technology, and finance.
  • MindDR Bench: The benchmark evaluates four critical pipeline stages with modular process indicators alongside content-focused RACE metrics.This design captures intermediate behaviors and final report presentation rather than relying solely on holistic scoring.
  • SFT Data Synthesis: SFT data synthesis produces ReAct-style multi-turn trajectories containing thoughts, actions, and observations for supervised fine-tuning and Report-RL seed data.The corpus combines approximately 12K trajectories from knowledge-graph, real-world scenario, and human-annotated edge-case sources.
  • SFT Data Synthesis: Trajectory diversity is controlled across reasoning complexity, length, and tool-use patterns, while progressive context augmentation raises 128K-context format correctness from 72% to 94%.The augmentation phases cover average 8K-token data, 32K/64K extensions, and 128K extreme-length trajectories.

5 Training Pipeline

MindDR uses a four-stage, agent-specialized training pipeline that progressively builds tool use, search reasoning, report quality, and preference alignment. The pipeline calibrates SFT for later RL exploration, stabilizes Search-RL for long-horizon search, and uses dedicated rewards and objectives for search and report generation.

  • SFT cold-start: SFT cold-start training establishes instruction following, tool invocation, format adherence, and multi-turn reasoning from ReAct-style expert trajectories.Training uses pre-recorded tool observations rather than executing tools, and behavior cloning initializes a policy for subsequent RL.
  • SFT cold-start: Approximately 15K SFT trajectories use a curriculum that introduces longer and harder contexts, with 32K–64K data at about 30% and 128K data at about 15%.Shorter samples dominate early training before longer contexts are progressively added.
  • Search-RL: Search-RL uses GRPO to progressively optimize tool invocation, intermediate reasoning, information consistency, and long-horizon search through dynamic reward scheduling.The reward combines answer correctness, entity coverage, tool success, and format compliance; stronger negative penalties for format and tool failures accelerate error avoidance.
  • Search-RL: GSPO replaces token-level importance ratios with length-normalized sequence-level ratios, reducing MoE routing-induced variance and preventing search capability degradation.The sequence-level ratio averages local routing changes so clipping remains stable under sparse expert activation.
  • Report-RL: Report-RL trains long-form report generation with LLM-based RACE evaluation, citation and format rewards, and DAPO’s token-level, asymmetrically clipped, KL-free objective.DAPO also filters groups with uniformly correct or incorrect samples to preserve non-trivial advantage estimates.

6 Main Results

MindDR is evaluated across deep-search and deep-research benchmarks against large foundation models and comparable-scale agent systems. Its final variants achieve broad competitive performance, while staged refinement improves report quality and maintains search and inference efficiency.

  • DeepSearch benchmark performance: MindDR-v1.5-30B-A3B achieves the best results on BrowseComp-ZH, BrowseComp, xbench-DS, and GAIA-DS, while MindDR-v1.5-32B leads WideSearch.These results indicate that gains generalize across backbones and that final MindDR models outperform comparable-scale open agent systems.
  • DeepResearch benchmark performance: MindDR-v1.5 leads overall RACE and the main report-quality dimensions on MindDR Bench, including comprehensiveness, insight, instruction following, and readability.Citation accuracy remains a distinct objective, with MindDR-v1.0 slightly stronger on that metric.
  • Detailed analysis: Report-RL improves report quality, while GSPO and DAPO preserve more prior search capability than GRPO during refinement.The comparison frames report optimization as a balance between improving report quality and limiting deep-search regression.
  • Detailed analysis: 50.60 RACE is achieved when short-form data is mixed with long-form data in Report-RL, compared with 48.82 using long-form-only training.Comprehensiveness increases from 49.10 to 51.08, and Insight increases from 47.68 to 50.96.
  • Detailed analysis: DPO reduces structured factual and formatting errors, while Self-SFT further improves expression and coherence and adds a benchmark gain.Together, the two refinement stages improve report correctness and presentation without sacrificing the reported overall performance.
  • Detailed analysis: 2.0% temporal error is achieved by MindDR-v1.5, the lowest rate among compared systems on MindDR Bench.The result is reported as an improvement over MindDR-v1.0 and all external baselines.
  • Inference efficiency: MindDR-v1.5-30B-A3B occupies the Accurate & Efficient region while exhibiting a favorable accuracy–resource trade-off across tool calls and context tokens.It consistently matches or surpasses compared models under context limits from 16k to 128k and tool-call limits from 8 to 64.
  • Training and test-time efficiency: 1.03B training-related tokens and about 6K GPU card-hours are used by MindDR 1.5, versus approximately 3.6B tokens and about 15K GPU card-hours for MindDR 1.0.MindDR 1.5 replaces a large intermediate training phase with staged SFT, Search-RL, Report-RL, and preference alignment.

7 Discussion and Conclusion

MindDR combines multi-agent inference decomposition with targeted training to achieve strong deep-research performance at roughly 30B parameters. The discussion reports broad benchmark strength and identifies context management and evaluation coverage as remaining limitations.

  • Limitations: Scaling beyond 128K context remains an open challenge despite 94% format correctness at 128K context.Future directions include hierarchical memory, selective context compression, and adaptive attention mechanisms.
  • Limitations: Automated evaluation struggles to capture methodological soundness, argument novelty, and appropriate hedging of uncertain claims.The paper identifies more nuanced human evaluation protocols as a needed direction.
  • Conclusion: ~30B-parameter models can match or surpass much larger foundation models through MindDR’s cost-effective multi-agent framework.The approach avoids computationally expensive continual pre-training and targets both training and inference efficiency.
  • Conclusion: MindDR coordinates Planning, DeepSearch, and Report Agents through Extended Chain-of-Thought to parallelize search and isolate contexts.The staged training pipeline progresses from SFT to Search-RL and further agent-specific optimization.
  • Empirical Results: MindDR-v1.5-30B-A3B achieves the best results among open-source agent-style systems across BrowseComp-ZH, BrowseComp, xbench-DS, and GAIA-DS.The advantage remains under varying context-length and tool-call constraints.
  • Empirical Results: 51.8 RACE score on MindDR Bench leads across comprehensiveness, insight, instruction following, and readability.MindDR Bench evaluates content quality and presentation format with a multi-dimensional system.

8 Appendix

The appendix documents RACE rubric construction and scoring prompts, alongside an automated pipeline for detecting temporal-tense errors. These components operationalize report evaluation and fact-checking through explicit criteria, structured outputs, and contextual verification.

  • RACE Rubrics: RACE rubrics evaluate comprehensiveness, insight, instruction following, and readability with fine-grained criteria and importance weights.Per-dimension scores provide reward signals during Report-RL training.
  • Report Prompts: The report-generation prompts require complete, structured, readable reports and can condition synthesis on specified evaluation dimensions.The templates request sectional conclusions, final summaries, and tables for fact or statistic listings.
  • Scoring Model: The scoring model compares generated and reference articles criterion by criterion on a 0–10 scale across four dimensions.Its output records analyses and separate scores for both articles, using defined performance bands from very poor to excellent.
  • Temporal Verification: Temporal-tense detection first extracts future-tense predictions whose dates precede the current date, then verifies forecasting institutions with contextual LLM analysis.A sentence is classified as an error only when it predicts a past time point and lacks an identified forecasting institution nearby.

Contributions

The paper lists its core contributors, additional contributors, and sponsors as separate groups. These acknowledgments identify the personnel associated with the work without assigning specific roles or contributions.

  • Core Contributors: Core Contributors are Biao Wang, Haozhi Xie, Heyang Xu, Liping Wang, Shirui Zhang, Shuai Wang, Sirui Miao, Tiankuo Xu, Xuefeng Hao, Ying Liu, Yingjie Feng, Yuchen Liu, Yuhang Wu, Zhengxin Yu, and Zhuo Liu.
  • Contributors: Additional Contributors are Bin Huang, Dong Wu, Handong Cui, He Cao, Jiabang He, Jiajun Yang, Jialu Chen, Jiqing Zhan, Li Gong, Lian Wen, Qingfeng Cai, Xiaobo Liu, Yuan Xue, and Yun Zhu.
  • Sponsors: Sponsors are Xiaofei Gou and Wei Chen.
Loading 2604.14518v2…