Source-linked AI summary
AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents
Shengda Fan, Xuyan Ye, Yupeng Huo, Zhi-Yuan Chen, Yiju Guo, Shenzhi Yang, Wenkai Yang, Shuqi Ye, Jingwen Chen, Haotian Chen, Xin Cong, Yankai Lin
TL;DR
Tool-using agents require step-level verification because long-horizon failures can have irreversible consequences, while existing benchmarks mainly cover mathematical reasoning or final outcomes. AgentProcessBench provides human-annotated ternary step supervision with error propagation across diverse trajectories, and its experiments show that process signals complement outcome supervision for improved Best-of-N performance.
Problem
Existing process-level benchmarks largely focus on closed-world mathematics, leaving dynamic and open-ended tool-use trajectories without dense step-level verification.
Method
AgentProcessBench evaluates every assistant step with correct, neutral, or incorrect labels and applies error propagation to dependent downstream actions.
Results
Process-derived signals provide complementary value to outcome supervision and improve Best-of-N performance.
Takeaways & Limitations
The benchmark offers a testbed for developing process reward models for tool-using agents.
Takeaways & Limitations
AgentProcessBench may still be affected by annotation noise and inherent human subjectivity.
Abstract
from arXiv · showhide
While Large Language Models (LLMs) have evolved into tool-using agents, they remain brittle in long-horizon interactions. Unlike mathematical reasoning where errors are often rectifiable via backtracking, tool-use failures frequently induce irreversible side effects, making accurate step-level verification critical. However, existing process-level benchmarks are predominantly confined to closed-world mathematical domains, failing to capture the dynamic and open-ended nature of tool execution. To bridge this gap, we introduce AgentProcessBench, the first benchmark dedicated to evaluating step-level effectiveness in realistic, tool-augmented trajectories. The benchmark comprises 1,000 diverse trajectories and 8,509 human-labeled step annotations with 89.1% inter-annotator agreement. It features a ternary labeling scheme to capture exploration and an error propagation rule to reduce labeling ambiguity. Extensive experiments reveal key insights: (1) weaker policy models exhibit inflated ratios of correct steps due to early termination; (2) distinguishing neutral and erroneous actions remains a significant challenge for current models; and (3) process-derived signals provide complementary value to outcome supervision, significantly enhancing test-time scaling. We hope AgentProcessBench can foster future research in reward models and pave the way toward general agents. The code and data are available at https://github.com/RUCBM/AgentProcessBench.
1 Introduction
AgentProcessBench addresses the need for step-level verification in open-ended tool use, where failures can have irreversible side effects and existing benchmarks largely focus on mathematics or final outcomes. It introduces dense ternary supervision and examines model behavior, reward-model capability, and process signals.
- Motivation: Tool-using agents remain brittle because inappropriate, repetitive, or hallucinated actions can cause irreversible side effects.Examples include sending erroneous emails or deleting essential files.
- Research gap: Existing verification benchmarks largely target closed-world mathematical reasoning, while standard agent benchmarks report end-to-end success without step-level signals.Tool-use failures additionally involve dynamic observations, ambiguous intent, and policy constraints.
- Benchmark: AgentProcessBench evaluates intermediate assistant actions in tool-using trajectories with labels for correct, neutral or exploratory, and incorrect or harmful steps.The benchmark is designed to support step-wise credit assignment in long-horizon tool use.
- Benchmark: 1,000 agent trajectories and 8,509 annotated actions span multi-hop reasoning, deep research, multi-turn tool execution, and long-horizon conversation, with 89.1% inter-annotator agreement.Trajectories come from four agent benchmarks and five source models.
- Findings: Weaker policy models can show higher correct-step ratios by terminating early, motivating First-Error Accuracy for fairer comparison.The benchmark also finds positive-label bias, difficulty separating neutral exploration from errors, and complementary process signals for Best-of-N evaluation.
- Benchmark: The benchmark proposes a neutral label and error-propagation rule to distinguish exploration and reduce ambiguity in cascading long-horizon failures.These design choices are part of the benchmark’s step-level evaluation protocol.
2 Related Work
Prior reward benchmarks mainly evaluate mathematical reasoning or provide trajectory-level preferences, leaving a gap in absolute, step-level supervision for interactive tool-using agents. AgentProcessBench fills this gap with human-annotated effectiveness labels across diverse environments.
- AgentProcessBench: AgentProcessBench is introduced as a human-annotated benchmark for measuring intermediate-step quality in tool-using agent trajectories.Its stated purpose is to support more effective process reward models for tool-using agents.
- Existing benchmarks: Prior process-supervision benchmarks primarily evaluate step correctness in mathematical reasoning tasks.The related work contrasts these closed-world settings with interactive agent environments.
- Existing benchmarks: Existing agent benchmarks often provide trajectory-level rubrics or preference signals rather than absolute effectiveness labels for every assistant action.This limits their direct use for step-wise process-reward evaluation.
3 Benchmark Construction
AgentProcessBench combines diverse benchmark tasks and model-generated trajectories with expert step-level annotation using ternary labels and error propagation. Its dataset contains 1,000 trajectories, and its statistics show that failures accumulate local mistakes and generally require longer interactions.
- 3.1 Evaluation Protocol: The evaluation protocol labels each assistant message in a trajectory as effective, neutral, or harmful with respect to overall task progress.The task maps a description and interleaved interaction trajectory to a sequence of labels in {-1, 0, +1}.
- 3.1 Evaluation Protocol: Correct steps advance task completion, neutral steps have negligible impact, and incorrect steps are counterproductive or factually wrong.The neutral category includes reasonable exploration or ambiguous outcomes without clear benefit or harm.
- 3.1 Evaluation Protocol: Once an error occurs, dependent downstream steps remain labeled −1 until correction or transition to an independent subtask.This rule prevents spurious downstream credit assignment and promotes consistent long-horizon supervision.
- 3.2 Data Collection: Tasks are drawn from HotpotQA, GAIA, BFCL, and τ2-Bench, covering multi-hop reasoning, deep retrieval, and complex tool use.Equal sampling across datasets is used to mitigate imbalance, while five heterogeneous source models provide diverse trajectories.
- 3.2 Data Collection: Expert annotation uses trained annotators, auxiliary references, and independent verification, with agreement between human annotations and reference models ranging from 66.9% to 72.1%.The reported discrepancy is interpreted as evidence that experts retained independent judgment.
- 3.3 Statistics: 1,000 trajectories cover 200 unique tasks evenly sampled across four benchmark subsets.The resulting dataset includes both successful and unsuccessful trajectories for analysis.
- 3.3 Statistics: Unsuccessful trajectories contain more incorrect steps and are generally longer than successful trajectories, indicating accumulated local mistakes rather than one isolated error.The length pattern holds across datasets except BFCL, whose termination criteria differ.
4 Evaluation
AgentProcessBench evaluates step-level process diagnosis across 20 models using StepAcc and FirstErrAcc, revealing systematic differences in labeling quality, error localization, and process-based trajectory selection.
- Evaluation Setup: 20 models are evaluated with StepAcc for global labeling reliability and FirstErrAcc for locating the earliest critical failure.StepAcc pools matched step labels, while FirstErrAcc compares the first predicted and human-labeled error index.
- Main Results: 81.6% average StepAcc is achieved by Gemini-3-Flash-Preview-Thinking, compared with 68.5% for Qwen3-30B-A3B-Thinking-2507.The reported gap indicates that the strongest open-source model trails the cited proprietary model on overall step-level accuracy.
- Detailed Analysis: Models over-predict positive labels and struggle most with neutral steps, whose utility often depends on downstream context rather than the local action alone.Neutral-step judgments can depend on later evidence use, uncertainty reduction, or whether alternative actions would have sufficed.
- Detailed Analysis: Step-level accuracy correlates strongly with final trajectory accuracy (Pearson r=0.814, p=1.2 × 10−5), while retaining variation beyond outcome performance.Models with similar final accuracy can have markedly different StepAcc, supporting process-level analysis as a distinct signal.
- Detailed Analysis: Combining outcome- and process-level signals in a two-stage Best-of-N selector consistently improves performance across generators.Simple positivity statistics also often outperform outcome-only selection for weaker generators such as Qwen3-30B-A3B.
5 Conclusion
The paper introduces AgentProcessBench for human-annotated evaluation of intermediate steps in tool-using agents and reports persistent positive-label bias alongside difficulty judging neutral steps. Its experiments also show that process-derived signals complement outcome supervision for Best-of-N selection.
- AgentProcessBench is a human-annotated benchmark for evaluating intermediate-step effectiveness in tool-using agents.
- Models show positive-label bias and difficulty distinguishing neutral steps.
- Process-derived signals provide complementary value to outcome supervision and improve Best-of-N performance.
A Limitations
AgentProcessBench has two stated limitations: annotation noise and human subjectivity, plus restricted coverage of text-only tool use. Multimodal behaviors such as GUI interaction and navigation remain outside its current scope.
- Annotation noise and human subjectivity may affect AgentProcessBench despite the authors’ mitigation efforts.
- The benchmark is restricted to text-only tool use and excludes multimodal behaviors such as GUI interaction and navigation.
- Extending AgentProcessBench to complex multimodal settings is identified as an important direction for future work.
B PRM Evaluation Prompt
The evaluation prompt instructs models to act as strict but fair trajectory annotators using a granular three-tier scoring schema for tool-use steps.
- The prompt evaluates tool-use trajectories with three labels: +1 for effective progress, 0 for neutral exploration, and −1 for incorrect or harmful steps.
C Analysis on Long-Horizon Trajectories
Longer trajectories substantially degrade model performance, with first-error localization worsening more than overall step accuracy. This indicates that identifying the earliest error is especially sensitive to trajectory length.
- StepAcc decreases by 3.8–13.49 points and FirstErrAcc decreases by 10.79–21.67 points on the longest trajectories.
- FirstErrAcc drops more substantially than StepAcc across models, indicating greater sensitivity of first-error localization to long trajectories.
D Qualitative Error Analysis
The qualitative analysis identifies failures in detecting informational, reasoning, and tool-invocation errors, alongside overthinking correct steps and ambiguity around exploration.
- Models may miss factual inaccuracies or policy violations, especially when errors are small or hidden in long messages.
- Models struggle to identify flawed reasoning, including repeated failed actions without strategic adjustment and causal inconsistencies.
- Models have difficulty detecting nuanced tool-invocation problems such as improper selection, malformed syntax, or indirect misuse.
- Overthinking can cause models to mislabel correct steps by imposing imaginary constraints and producing false negatives.
- Models struggle to distinguish acceptable exploration, inefficient redundant attempts, and essential exhaustive search.
E Ethical Statement
AgentProcessBench was constructed using public datasets and LLM-generated trajectories, with expert annotation procedures designed to support reliable step-level evaluation. The authors also disclose manuscript language assistance and emphasize responsibility for the paper’s content.
- Data and privacy: The benchmark draws tasks from HotpotQA, GAIA, BFCL, and 𝜏2-bench, with trajectories generated by LLMs.The authors state that this construction avoids private or sensitive personal information from real human-to-human interactions.
- Human annotation: Annotators had undergraduate-level computer-science training, at least one year of LLM experience, and completed proficiency testing and specialized tutorials.The training covered the ternary labeling scheme and error-propagation rules.
- Human annotation: Annotators were compensated at a competitive rate exceeding local standards for professional technical work.
- Broader impact: The benchmark targets reliable diagnosis of individual steps because tool-use failures can produce irreversible side effects.The authors identify this capability as critical for developing and evaluating process reward models for tool-augmented agents.
- Disclosure: The authors used LLMs to improve manuscript language and grammar, then reviewed and edited the resulting content.They take responsibility for the accuracy and integrity of the paper.