Source-linked AI summary
DuMate-DeepResearch: An Auditable Multi-Agent System with Recursive Search and Rubric-Grounded Reasoning
Lingyong Yan, Can Xu, Yukun Zhao, Wenxuan Li, Qingyang Chen, Jiulong Wu, Wenli Song, Xiangnan Li, Weixian Shi, Yiqun Chen, Xuchen Ma, Yuchen Li, Jiashu Zhao, Shuaiqiang Wang, Jianmin Wu, Dawei Yin
TL;DR
Deep research systems still face challenges in long-horizon planning, task decomposition, synthesis reliability, and process auditability. DuMate-DeepResearch addresses these with an auditable multi-agent architecture, dynamic recursive planning, and rubric-guided reasoning, achieving the best overall scores on two benchmarks.
Problem
Current deep research systems have unresolved challenges in long-horizon planning, task decomposition, synthesis reliability, and process auditability.
Method
DuMate-DeepResearch decouples reasoning from tools and combines graph-based dynamic planning, two-level recursive search, and rubric-guided test-time reasoning.
Results
DuMate-DeepResearch achieves the best overall scores on both DeepResearch Bench and DeepResearch Bench II, leading the latter in information recall and analysis.
Takeaways & Limitations
The results support combining auditable multi-agent infrastructure with adaptive planning and rubric-guided reasoning for deep research.
Takeaways & Limitations
Evaluation uses only benchmark queries and excludes reference reports, expert reports, and hidden evaluation rubrics during report generation.
Abstract
from arXiv · showhide
Deep Research (DR) has emerged as a new agentic paradigm to tackle complex, open-ended research tasks, demanding systems that can iteratively frame problems, acquire evidence, verify sources, and synthesize long-form reports. In practice, however, current DR systems are constrained by four interrelated limitations: long-horizon planning over an underspecified scope, the bottleneck of decomposing and scheduling such tasks within a single agent, hallucination risk in long-form synthesis, and limited process auditability. This technical report presents DuMate-DeepResearch, a multi-agent DR framework built on the Qianfan Agent Foundry. The framework decouples the Agent Core, which handles task understanding, planning, and scheduling, from an extensible Tool Ecosystem for retrieval, evidence acquisition, and report rendering, making every intermediate decision and tool invocation explicitly traceable. Building on this infrastructure, DuMate-DeepResearch further introduces three mechanisms: (i) a graph-based dynamic planning strategy expands the research roadmap coarse-to-fine and continuously revises it through reflection, re-planning, backtracking, and parallel branching; (ii) a recursive two-level execution design delegates each complex search sub-task to an inner Search Agent that runs its own planning loop, isolating noisy retrieval and stabilizing long-horizon execution; (iii) a rubric-based test-time optimization mechanism dynamically generates task-specific quality criteria and uses them as live reasoning scaffolds for evidence-grounded synthesis and adaptive stopping. Across two deep research benchmarks, DuMate-DeepResearch establishes new state-of-the-art results: the best overall score (58.03%) on DeepResearch Bench, and the best overall score (61.95%) on DeepResearch Bench II while ranking first in information recall and analysis.
1 Introduction
DuMate-DeepResearch addresses the reliability, scalability, factual-grounding, and auditability challenges of deep research with a decoupled, recursive multi-agent framework. Its graph-based planning, nested search agents, and rubric-guided inference support adaptive execution and evidence-grounded synthesis, achieving state-of-the-art results on two benchmarks.
- Key Challenges: Deep research systems must overcome long-horizon planning, complex task decomposition, hallucination mitigation, and process auditability challenges.These challenges arise from underspecified scopes, stochastic multi-step retrieval, long-form synthesis over dynamic evidence, and the need to expose reasoning and tool use.
- Rubric-Grounded Reasoning: Dynamically generated rubrics serve as inference-time scaffolds that calibrate synthesis against retrieved evidence and define an adaptive stopping criterion.The mechanism supports factual grounding while bounding exploration according to evidence sufficiency rather than fixed exploration budgets.
- System Architecture: DuMate-DeepResearch decouples the Agent Core from the Tool Ecosystem, exposing planning decisions and tool invocations as inspectable artifacts.Built on the Qianfan Agent Foundry, the architecture separates reasoning from execution while preserving an auditable understanding–planning–execution cycle.
- Experimental Results: DuMate-DeepResearch achieves the best overall score on both DeepResearch Bench and DeepResearch Bench II, leading DeepResearch Bench II in information recall and analysis.The results are reported against strong commercial and open baselines, with DeepResearch Bench II using fine-grained expert-derived rubrics.
- Dynamic Planning: A graph-based dynamic planning algorithm expands research roadmaps coarse-to-fine and revises them through reflection, re-planning, backtracking, and parallel branching.The dynamic directed acyclic graph maintains a global trajectory view and adapts scope as evidence accumulates, unlike myopic one-action-at-a-time reasoning.
- Recursive Execution: The recursive execution framework assigns complex search subtasks to inner search agents with their own planning–execution cycles.This nesting isolates noisy, multi-step retrieval from high-level strategy, preventing individual search failures from destabilizing the global trajectory.
2 DuMate-DeepResearch Framework
DuMate-DeepResearch is an auditable, evidence-grounded multi-agent framework built on Qianfan Agent Foundry, combining modular control with traceable research-state transitions. Its graph planning, recursive search, and dynamic rubric mechanisms support revisable long-horizon research and guided synthesis.
- Research Loop: Each research session is an auditable state-transition process in which the Router specifies the task, the Planner evolves its plan, Execution gathers evidence, and guidance steers stopping and writing.The Writer synthesizes the final report after STOP holds, such as when the plan is fully explored or no evidence gap remains.
- Foundational Infrastructure: The framework separates the Agent Core, which orchestrates reasoning, planning, and scheduling, from the extensible Tool Ecosystem for evidence acquisition and deliverable rendering.This decoupling provides cognitive control alongside extensible execution capabilities.
- Research Loop: The Router, Planner, and Execution Module expose structured task representations, decision traces, decompositions, tool invocations, and retrieved evidence for inspection beyond the final report.This makes intermediate reasoning and action paths explicitly inspectable.
- Core Mechanisms: Three mechanisms shape long-horizon research: graph-based dynamic planning revises a global roadmap, recursive execution delegates searches, and rubric optimization guides synthesis.Planning expands coarse-to-fine as evidence accumulates, while rubrics are updated iteratively as the information state changes.
- Recursive Two-Level Execution: The outer Research Agent delegates open-ended retrieval to inner Search Agents, creating nested planning loops that isolate stalled searches and preserve the global plan.Inner agents use their own Router, Planner, and Execution Module and return evidence lists and summaries to the outer state.
3 Experiments and Evaluation
DuMate-DeepResearch was evaluated on two deep research benchmarks under their official protocols. The experiments also specify a multi-level execution setup with bounded planning and retrieval, concurrent execution, and Baidu Search as the primary backend.
- Benchmarks: Two benchmarks evaluate DuMate-DeepResearch: DeepResearch Bench and its extension, DeepResearch Bench II.DeepResearch Bench contains 100 tasks across 22 Chinese and English domains and uses reference-based and adaptive criteria-driven LLM-as-a-judge evaluation.
- Implementation Details: 15 outer planning iterations, 10 retrieval rounds per inner Search Agent, 3 sub-queries per round, and 3 results per query bound execution.Independent ready-frontier sub-tasks execute concurrently through fan-out parallel execution.
- Implementation Details: Baidu Search serves as the primary retrieval backend.
- Evaluation Protocol: Official benchmark scores and leaderboards provide baselines, while evaluation follows each benchmark’s official protocol without reference reports, expert reports, or evaluation rubrics.This restriction is especially important for DeepResearch Bench II because its evaluation rubrics derive from expert reports.
3.1 Overall Performance
DuMate-DeepResearch achieves the best overall performance on both DeepResearch Bench and DeepResearch Bench II. It leads key dimensions including Comprehensiveness, Insight, Information Recall, and Analysis, while remaining competitive in Instruction Following and Readability.
- DeepResearch Bench: 59.48% and 61.48% are DuMate-DeepResearch’s first-place scores in Comprehensiveness and Insight, improving over second-best systems by 0.90% and 1.34%.It ranks second in Instruction Following at 53.87% and remains within 0.2–0.7% of top systems in Instruction Following and Readability.
- DeepResearch Bench II: DeepResearch Bench II evaluates Information Recall, Analysis, and Presentation using fine-grained binary rubrics derived from expert reports.The dimensions assess retrieval of key facts, correctness of reasoning and synthesis, and report structure and readability.
- DeepResearch Bench II: 61.95% is DuMate-DeepResearch’s best overall score on DeepResearch Bench II, exceeding iFlow-Researcher by 2.04%.It ranks first in Information Recall at 57.58% and Analysis at 71.70%, improving over second-best systems by 2.59% and 1.80%.
3.2 Detailed Analysis
The ablations show that rubric guidance modestly improves report quality, while the report-stage model has the largest effect. Model substitutions particularly affect comprehensiveness in long-context, multi-source synthesis.
- Ablation Study: Ablation studies on DeepResearch Bench examine rubric-guided generation and report-stage model choice, reporting averages from 3 runs.The study evaluates key design choices in DuMate-DeepResearch.
- Effect of Rubric Guidance: 58.03→57.61 overall, −0.42, after removing rubric guidance only from report generation, with largest declines in Insight (−0.75) and Comprehensiveness (−0.47).Removing rubric guidance from planning and research produces only a further −0.08 overall decline, reaching 57.53.
- Effect of Report-Stage Model: −2.82 and −2.48 overall degradations occur with MiniMax-M3 and Qwen-3.7 Max, respectively, exceeding the quality differences from rubric removal.These models show broad declines across all dimensions and struggle with long-context, multi-source synthesis.
- Effect of Report-Stage Model: Information coverage is particularly sensitive to report-stage model capability, as substitutions preserve Insight more robustly than Comprehensiveness.The analysis suggests that assembling relevant evidence into a coherent narrative benefits most from model scale.
3.3 Qualitative Case Study
The qualitative cases show DuMate-DeepResearch expanding plans coarse-to-fine, evaluating evidence before action, recursively refining retrieval, and propagating rubrics into evidence-grounded synthesis. These mechanisms produce adaptive depth, quantitative modeling, cross-validation, and auditable citation trails.
- Case A: Coarse-to-Fine Planning: Case A expands an ambiguous query from two parallel exploratory searches and an outline into 14 targeted subtasks across three depth layers.The coarse phase establishes research boundaries before fine-grained investigation commits computational resources.
- Case B: Reflective Scheduling: Case B uses reflection-before-action to verify evidence, inject searches for detected gaps, prune unnecessary branches, and assign integration or cross-validation roles to llm tasks.One evaluated enterprise-case task was missing Hongqi/Lantu/Toyota-Mirai sub-segment data, which was deferred to T-17.
- Case C: Recursive Retrieval: Case C’s Search Agent runs up to 10 internal iterations, with one task completing 6 rounds and 40+ queries through expansion, progressive specificity, and tool diversification.The retrieval sequence moves from broad domain queries to manufacturing processes, OEM or supplier names, and ISO standards.
- Case A: Rubric-Grounded Synthesis: Case A’s chapter rubrics guide Search Agents toward multi-source evidence and constrain the Writer to produce conditional, source-calibrated conclusions.The rubric compares vendor claims, third-party research, and hands-on testing across 5–7 mainstream platforms.
- Observed Quality and Benchmark Alignment: 59.48% Comprehensiveness and 61.48% Insight lead DeepResearch Bench, while the cases demonstrate cross-validation, a 13-indicator quantitative framework, adaptive depth, and citation trails.Case C scales plan iterations to 11 and total subtasks to 27 in response to retrieval difficulty while maintaining report quality.
4 Background and Related Work
Deep research builds on retrieval-augmented generation and agentic search, which improve evidence acquisition through retrieval, query planning, adaptive timing, and tool-augmented reasoning. It extends these foundations to long-form, evidence-grounded reports requiring broader exploration, long-horizon planning, verification, memory, and structured synthesis.
- Retrieval-Augmented Generation: RAG connects LLMs with external knowledge by retrieving relevant passages and conditioning generation on them for concise answers.Early RAG systems showed that non-parametric retrieval can improve knowledge-intensive generation.
- Query Planning and Rewriting: Query rewriting, decomposition, and planning address conventional RAG’s dependence on input-query quality before retrieval.Rewrite-Retrieve-Read uses reinforcement learning to train a rewriter whose rewritten query improves downstream answer accuracy.
- Adaptive Retrieval: Adaptive retrieval methods let models decide when additional evidence is needed, reducing inefficient or misleading fixed retrieval.IR-CoT interleaves retrieval with reasoning, while FLARE triggers retrieval based on generation uncertainty.
- Deep Research Systems: Deep research shifts the objective from localized answer accuracy to long-form, report-level synthesis with broader tool orchestration and persistent workflows.Its requirements include broader information exploration, longer-horizon planning, iterative reflection, source-level verification, and structured report writing.
- Deep Research Systems: Existing deep research systems pursue complementary improvements in tool-augmented exploration, verification, iterative refinement, training data, reproducibility, and persistent memory.The surveyed directions include dedicated verifiers, scalable reinforcement learning, open training pipelines, capability-specific data, and external workspaces.
- Deep Research Systems: Deep research is a broader agentic workflow coupling search, planning, verification, memory, and long-form synthesis rather than merely a longer version of RAG.This distinction summarizes the field’s movement from short-answer retrieval toward coordinated end-to-end research.
5 Conclusions
DuMate-DeepResearch is an auditable multi-agent framework that separates core research orchestration from an extensible tool ecosystem, making planning and tool use inspectable. Experiments show it achieves the best overall scores on both DeepResearch Bench and DeepResearch Bench II.
- Framework: DuMate-DeepResearch decouples task understanding, planning, and scheduling from retrieval, evidence acquisition, and report rendering.The Agent Core handles orchestration, while an extensible Tool Ecosystem supports research operations.
- Auditability: The framework exposes every planning decision and tool invocation as an inspectable artifact, addressing transparency and auditability in agentic deep research.This design directly targets the transparency and auditability challenge identified for deep research systems.
- Evaluation: DuMate-DeepResearch achieves the best overall scores on both DeepResearch Bench and DeepResearch Bench II.The reported gains are consistent across the benchmarks’ complementary evaluation protocols.
- Evaluation: The results support combining auditable multi-agent infrastructure with adaptive planning and rubric-guided reasoning for high-quality deep research.The conclusion attributes the effectiveness to this combination of infrastructure and reasoning mechanisms.
Contributions and Acknowledgments
The section identifies the project’s contributors and acknowledges the organizational, internal evaluation, and broader research-community support that shaped the system and its development.
- Contributors: The paper lists Lingyong Yan, Can Xu, Yukun Zhao, Wenxuan Li, Qingyang Chen, Jiulong Wu, Wenli Song, Xiangnan Li, Weixian Shi, Yiqun Chen, Xuchen Ma, Yuchen Li, Jiashu Zhao, Shuaiqiang Wang, Jianmin Wu, and Dawei Yin as contributors.
- Acknowledgments: The authors thank Baidu colleagues, internal evaluators, and the open-source and deep research communities for support, feedback, benchmarks, baselines, and prior work.Internal evaluations and feedback helped shape the design and improve system quality.
A Prompt Templates
The appendix releases desensitized prompt excerpts that preserve the framework’s reasoning logic and control structure while omitting sensitive engineering details. The prompts operationalize dynamic planning, rubric generation, and adaptive retrieval across orchestration and search levels.
- A Prompt Templates: The appendix retains reasoning logic and control structure but omits full schemas, field definitions, tool lists, and other sensitive engineering details.Omitted material is marked with bracketed ellipses.
- A.1 Planner Prompt: The planner maintains a complete executable research DAG, evaluates prior results, checks coverage and dependencies, and emits the next parallel actions.It uses the user request, report outline, previous plan graph, and execution results as inputs.
- A.1 Planner Prompt: Planner decisions follow persistent and single-step rubrics, requiring new tasks and follow-up validation to address unmet standards and align with rubric dimensions.The specific injected rubric contents are omitted.
- A.1 Planner Prompt: The planner enforces a legal DAG, bounded depth, tool-constrained task types, parallel independence at equal depth, and one final-stage report task.Subtasks must serve the request and outline, while relative time expressions become explicit ranges.
- A.1 Planner Prompt: Re-planning occurs only for failed, insufficient, uncovered, conflicting, newly valuable, or user-changed conditions; otherwise, existing unfinished tasks are preserved.The strategy explicitly avoids expanding the plan merely to appear more comprehensive.
- A.2 Rubric-Generation Prompts: The rubric generator operates after each outer planning–execution cycle and after every inner Search Agent tool response to guide integration, stopping, and subsequent retrieval.The orchestration level assesses cross-subtask quality, while the search level steers the next retrieval step.
- A.2 Rubric-Generation Prompts: Orchestration rubrics assess coverage, consistency, balanced depth, evidence support, and instruction completion, while single-step standards identify coverage gaps, shallow sections, and conflicts.Guidance must be executable reasoning instructions, and sufficient evidence should trigger an explicit “无需继续检索” stopping signal.
- A.2 Rubric-Generation Prompts: Search-level rubrics judge relevance, specificity, source reliability, coverage gaps, and conflicts, then classify each response by missing information, insufficient depth, or extension leads.Each recommendation specifies the current state, search target, and suggested action, prioritizing the next step’s greatest impact.