Source-linked AI summary
Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification
Yuxuan Wan, Tianqing Fang, Zaitang Li, Yintong Huo, Wenxuan Wang, Haitao Mi, Dong Yu, Michael R. Lyu
TL;DR
Deep Research Agents remain unreliable on long-horizon tasks, motivating scalable test-time improvement beyond post-training. DeepVerifier uses failure-taxonomy-guided, asymmetry-based verification to generate corrective feedback and iterative refinements, achieving 12–48% higher meta-evaluation F1 and 8–11% accuracy gains on challenging GAIA subsets, while DeepVerifier-4K extends verification support to open models.
Problem
DRAs produce unreliable outputs, and generating effective feedback for iterative test-time improvement requires sophisticated reasoning capability.
Method
DeepVerifier decomposes verification into simpler sub-tasks and uses taxonomy-derived rubrics to provide structured feedback for iterative self-evolution without additional training.
Results
12–48% higher meta-evaluation F1 scores and 8–11% accuracy improvements on challenging GAIA subsets were reported with capable closed-source LLMs.
Takeaways & Limitations
The framework provides a scalable approach to DRA verification and supports open-source development through a 4,646-example verification dataset.
Takeaways & Limitations
DeepVerifier depends on models that can follow rubrics, cross-check carefully, and use tools; iterative verification also increases inference cost and latency.
Abstract
from arXiv · showhide
Recent advances in Deep Research Agents (DRAs) are transforming automated knowledge discovery and problem-solving. While the majority of existing efforts focus on enhancing policy capabilities via post-training, we propose an alternative paradigm: self-evolving the agent's ability by iteratively verifying the policy model's outputs, guided by meticulously crafted rubrics. This approach gives rise to the inference-time scaling of verification, wherein an agent self-improves by evaluating its generated answers to produce iterative feedback and refinements. We derive the rubrics based on an automatically constructed DRA Failure Taxonomy, which systematically classifies agent failures into five major categories and thirteen sub-categories. We present DeepVerifier, a rubrics-based outcome reward verifier that leverages the asymmetry of verification and outperforms vanilla agent-as-judge and LLM judge baselines by 12%-48% in meta-evaluation F1 score. To enable practical self-evolution, DeepVerifier integrates as a plug-and-play module during test-time inference. The verifier produces detailed rubric-based feedback, which is fed back to the agent for iterative bootstrapping, refining responses without additional training. This test-time scaling delivers 8%-11% accuracy gains on challenging subsets of GAIA and XBench-DeepSearch when powered by capable closed-source LLMs. Finally, to support open-source advancement, we release DeepVerifier-4K, a curated supervised fine-tuning dataset of 4,646 high-quality agent steps focused on DRA verification. These examples emphasize reflection and self-critique, enabling open models to develop robust verification capabilities.
1 Introduction
DeepVerifier addresses unreliable DRA outputs and infeasible human supervision by enabling test-time self-evolution through verification, rubric-guided feedback, and iterative refinement. It combines a failure taxonomy, asymmetry-based verification, and scalable evaluation to improve agent performance without additional training.
- Motivation: DRAs remain vulnerable to incorrect actions, API failures, hallucinations, and other errors, while long-horizon tasks make online human supervision infeasible.These limitations motivate scalable automated reliability methods at test time.
- Motivation: Prior inference-time methods scale tokens, parallel sampling, selection, or aggregation, but generating reliable textual feedback remains difficult.The difficulty arises because feedback generation requires sophisticated reasoning capability.
- Approach: DeepVerifier decomposes complex verification into simpler sub-tasks and uses rubrics derived from an automatically constructed taxonomy of five major classes and thirteen subclasses.The approach exploits the asymmetry that checking correctness is often easier than generating an answer.
- Results: 12–48% higher meta-evaluation F1 scores were achieved than vanilla agent-as-judge and LLM judge baselines.The reported comparison evaluates DeepVerifier's verification performance.
- Results: 8–11% accuracy improvements on challenging GAIA subsets and 3–6% on XBench-DeepSearch were obtained with capable closed-source LLMs.These gains came from integrating DeepVerifier into test-time scaling.
- Open-source extension: DeepVerifier-4K contains 4,646 supervised fine-tuning prompt-response pairs focused on DRA verification, reflection, and self-critique.The dataset extends the framework beyond test-time inference for open-source advancement.
2 Related Work
Related work improves agents through test-time scaling, judging, and self-evolution, but existing approaches do not adequately address recurring DRA failures or provide automated DRA-specific verification.
- Test-time scaling: Test-time scaling methods use Best-of-N selection, majority voting, and related strategies to improve agent responses.Different rollouts can still reproduce the same failures, limiting reliability.
- Agent judging: Existing LLM-as-judge and agent-as-judge studies target web navigation, general reasoning, or software development rather than DRA responses.The passage explicitly identifies the lack of DRA-focused study.
- Self-evolving models: Self-evolving LLM methods explore code-as-task self-play, self-aware reinforcement learning, and test-time reinforcement learning, but do not address DRAs.Prior DRA failure analyses also lack an automated framework for detection and improvement.
- Positioning: DeepVerifier differs by constructing an agent failure taxonomy and introducing verification based on the asymmetry between generation and checking.The related-work passage positions these as the paper's distinguishing elements.
3 DRA Failure Taxonomy
The DRA Failure Taxonomy is built from agent trajectories and localized error annotations, then refined into a structured classification of recurring failure modes. Its analysis identifies source-finding as the dominant failure point, followed by reasoning errors and cascading operational failures.
- Construction: The taxonomy is constructed from trajectories on WebAggregatorQA and evaluated separately on GAIA, BrowseComp, and XBench-DeepSearch.This design aims to avoid data leakage or contamination while testing generalization.
- Trajectory collection: The collected corpus contains 2,997 agent actions across 90 tasks, with trajectories ranging from 2 to 156 steps and a correct/incorrect ratio of 0.96.Trajectories were generated with CK-Pro using Claude-3.7-Sonnet on WebAggregatorQA.
- Error annotation: For incorrect trajectories, two annotators identify concrete localized error points using human reference solution traces as grounding.The annotations focus on deviations from reference reasoning and evidence gathering.
- Taxonomy construction: The taxonomy is iteratively refined by clustering, merging similar labels, removing inadequate categories, and clarifying definitions.The resulting classification is illustrated in Figure 3, where wider branches indicate more frequent subclasses.
- Failure analysis: Finding Sources is the dominant failure class, especially wrong evidence and generic searches, followed by reasoning failures involving premature conclusions, misinterpretation, and hallucinated claims.Problem Understanding, Action Errors, and Max Step Reached account for the remaining failures and often cascade from early mistakes.
4 DeepVerifier
DeepVerifier uses a multi-module verification pipeline that summarizes trajectories, localizes taxonomy-linked vulnerabilities, formulates evidence-checking questions, and judges the resulting answer. This converts holistic re-solving into targeted verification and feedback.
- Framework: DeepVerifier uses decomposition, verification, and judge agents in a three-stage multi-module framework.The modules respectively prepare targeted checks, retrieve evidence, and score the unverified answer.
- Decomposition Module: The decomposition agent uses prior trajectories and the failure taxonomy to break complex verification into smaller questions targeting specific vulnerabilities.Examples ask whether a source supports a claim or provide an exact figure from a report.
- Decomposition Module: Trajectory summarization produces a compact, step-indexed record of visited sources and retrieved facts, numbers, or quotes.The summary is descriptive rather than interpretive, avoiding the need to reload the full trace.
- Decomposition Module: The decomposition agent pairs suspected behaviors with taxonomy labels and potential errors, then drafts follow-up questions answerable through external evidence.These questions are designed to confirm or refute risky claims.
- Verification: The verification agent answers follow-up questions sequentially using web search, screenshotting, and code execution.The implementation uses CK-Pro as the verification agent.
- Judge: The judge evaluates the summary, potential errors, follow-up questions, and answers, then assigns a score from 1 to 4.The scale ranges from entirely incorrect to entirely correct.
5 Enhancing Deep Research Agents with Scalable Verification
The framework extends DeepVerifier beyond verification by using it for reflective test-time scaling and by training open-source models for reflection. It constructs DeepVerifier-4K from filtered verification trajectories to support this capability.
- Reflective test-time scaling: DeepVerifier reviews prior agent actions and provides actionable retry instructions or available answers until success or a retry limit.The modified judge prompt supports iterative reflection and feedback.
- Training reflection ability: The dataset is designed to improve reflection and test-time scaling in open-source language models that lack reflection fine-tuning.The motivation is limited test-time scaling capability in many open-source models.
- Training reflection ability: DeepVerifier-4K contains 4,646 high-quality prompt-response pairs created from correctly verified base trajectories.The data are filtered for true-positive and true-negative verifications, balanced, and converted into supervised examples.
6 Experiment Setup
The experiments evaluate DeepVerifier across models and DRA benchmarks, with open-source fine-tuning and verification metrics defined explicitly. Baselines isolate decomposition and verification components.
- Models and benchmarks: Evaluations use Claude-3.7-Sonnet primarily, with GPT-4.1 and Qwen3-8B for generalization across GAIA, XBench-DeepSearch, and BrowseComp.GAIA-Web is the primary benchmark subset, while the additional datasets test broader browsing, search, tool-use, and difficult retrieval abilities.
- Training configurations: Qwen3-8B is fine-tuned on a mixture including DeepVerifier-4K to train reflection while preserving foundational capabilities.The supplied setup passage introduces this open-source training configuration.
- Baselines and metrics: Verification is measured with precision, recall, accuracy, and F1, defining rejection of wrong answers as true positive and acceptance of correct answers as true negative.The scaling experiment treats scores ≤2 as incorrect and scores ≥3 as correct, stopping when the verifier judges an answer correct.
1. RQ1: Is DeepVerifier effective in verification?
The section frames two evaluation questions: whether DeepVerifier improves DRA performance through test-time scaling and whether DeepVerifier-4K improves reflection in open-source models.
- RQ2 asks whether DeepVerifier improves DRA performance through test-time scaling.
- RQ3 asks whether DeepVerifier-4K improves the reflection ability of open-source models.
7 Results & Analysis
Experiments find that DeepVerifier improves verification and DRA accuracy through iterative feedback, with gains across datasets and models, while imperfect transitions produce an early performance peak. Fine-tuning reflection data also improves open-model performance.
- RQ1: Verification effectiveness: Removing verification or decomposition can preserve high precision but leaves recall and accuracy unsatisfactory because subtler reasoning and factual errors are often accepted.The decomposition-only and vanilla agent-as-judge variants are effective mainly at catching obvious execution failures.
- RQ1: Verification effectiveness: DeepVerifier achieves a 12%–48% improvement in F1 score and the highest accuracy compared with ablated versions.The result is reported for DRA verification with a balanced precision–recall tradeoff.
- RQ2: Reflective test-time scaling: Accuracy improves with additional feedback rounds and peaks at the fourth round, indicating that iterative reflection helps refine reasoning and correct previous errors.The experiment integrates DeepVerifier into CK-Pro with Claude-3.7-Sonnet on GAIA.
- RQ2: Reflective test-time scaling: GAIA-Full rises from approximately 52% to a 60.1% peak, while GAIA-Web rises from 52% to a 63.5% peak.The reported best gains are 8% on GAIA-Full and above 10% on the web-focused subset.
- RQ2: Reflective test-time scaling: XBench-DeepSearch improves from 41.0 to 47.0, and BrowseComp improves from 5.0 to 10.0 at their respective best feedback rounds.The scaling effect persists across multilingual search/tool-use and extremely difficult retrieval tasks.
- RQ2: Scaling trend analysis: Performance peaks early because incorrect-to-correct transitions initially outweigh persistent correct-to-incorrect regressions as feedback rounds continue.The interaction of these transition types explains the observed peak around the fourth round.
- Inference cost: Verification uses at most three targeted follow-up questions and supports early stopping around rounds 3–4, yielding a favorable accuracy–cost tradeoff without additional training.The loop also terminates when the verifier accepts the answer.
- RQ3: Open-source reflection: DeepVerifier-8B reaches 32.2% accuracy after reflection, a 5.5% improvement over its non-reflective result.The model is trained with CK-Pro and DeepVerifier-4K data; CK-Pro-8B gains 2.6 points, while Qwen3-8B improves minimally.
8 Conclusion
The paper addresses repeated DRA failures by leveraging verification asymmetry, localizing vulnerabilities, and improving outputs across models and datasets. It also releases DeepVerifier-4K to support development of more trustworthy agents.
- The framework uses verification asymmetry to address silent and repeated failures in Deep Research Agents.
- It combines a human-annotated failure taxonomy with vulnerability localization to turn holistic re-solving into targeted evidence checking.
- The authors report consistent improvements across models and datasets.
- DeepVerifier-4K is released to help the open community build more trustworthy agents.
9 Limitations
DeepVerifier depends on capable models for precise rubric following, careful cross-checking, structured feedback, and tool use, while iterative verification increases inference cost and latency.
- DeepVerifier performance depends on models precisely following rubrics, cross-checking carefully, and producing structured feedback.Weak models or insufficient tool-use ability can degrade feedback quality and produce noisy test-time gains.
- DeepVerifier-4K SFT can help alleviate capability limitations for open-sourced models.
- Iterative verification adds inference steps and often tool calls, increasing runtime and token usage.The method can reduce redundant problem-solving steps, but verification still introduces additional computation.
A Annotation Instructions
The annotation and verification pipeline compares agent executions with human references, decomposes trajectories, identifies taxonomy-linked errors, and checks targeted evidence using structured rubrics.
- A Annotation Instructions: Human annotators compare LLM agent executions with ground-truth human executions to summarize concrete error points.The comparison covers source finding, information location, observations, and problem understanding.
- B.1 Decomposition Module: The decomposition module summarizes each trajectory step with visited sources and information retrieved from them.Its required format repeats indexed step blocks containing source and information fields.
- B.1 Decomposition Module: The decomposition module maps suspicious behaviors to one potential error from the failure taxonomy.It returns paired suspicious-behavior and potential-error entries, or a no-error response when appropriate.
- B.1 Decomposition Module: The module proposes up to three source-question pairs that can verify the answer using the task, trajectory summary, and potential errors.Each question is framed as a yes-no check based on an additional source.
- B.2 Verification & Judge Module: The verification agent answers each source-question pair with a brief explanation and concise answer based on the source.
- B.2 Verification & Judge Module: The judge scores an unverified answer from 1 to 4 after considering the task, answer, trajectory summary, and additional research answers.The scale ranges from completely incorrect to completely correct and requires an explanation before the score.
- B.2 Verification & Judge Module: Corrective feedback provides a brief reflection followed by no more than three instructions for retrying a wrong answer.
- C Verification Rubrics: The rubrics evaluate reasoning, problem understanding, source quality, action execution, and trajectory efficiency across four performance levels.The levels range from Excellent to Poor and cover unsupported claims, task misalignment, execution failures, and inefficient or failed trajectories.