Source-linked AI summary
Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?
Jiale Liu, Huajun Xi, Shaokun Zhang, Yifan Zeng, Tianwei Yue, Chi Wang, Jian Kang, Qingyun Wu, Huazheng Wang
TL;DR
As agents become more capable, their failures grow subtler, while existing failure-attribution benchmarks have limited scale and modality coverage. WHO&WHEN PRO uses controlled error injection to build a large multimodal benchmark, and its analyses find systematic variation in attribution across modalities, protocols, and model families.
Problem
More capable agents can fail in subtler ways, while prior WHO&WHEN covers only text and has 184 instances, limiting benchmark scale and modality coverage.
Source: §1 · p. 1Method
The authors inject an error into a successful agent trajectory and warm-start execution from that step, creating 12,326 labeled traces across 26 benchmarks and three modalities.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
Full-trajectory attribution outperforms incremental protocols, and open-weight models offer practical attribution backbones with favorable cost-performance trade-offs.
Source: §1 · p. 2The 18-mode taxonomy covers dominant observed patterns but is not exhaustive, and new domains may require additional failure categories.
Source: §A.1 · p. 15
Abstract
from arXiv · showhide
Automated failure attribution uses LLMs to identify where and why agentic systems fail. As agents become more capable, their failures become subtler, making automated attribution increasingly important. We introduce Who&When Pro, a large-scale benchmark for automated failure attribution in agentic systems. Using a strictly controlled pipeline that injects a failure only after exactly replaying a successful prefix, we construct 12,326 failed trajectories with golden labels across 3 modalities and 26 benchmarks covering various scenarios. Beyond benchmarking, we conduct extensive experiments and analyses, revealing systematic patterns in how models attribute failures across modalities, protocols, and model families, and providing empirical guidance for future automated failure attribution systems.
1. Introduction
As agent failures become subtler, automated attribution matters for turning them into actionable feedback, but existing benchmarks are limited in scale and modality. WHO&WHEN PRO introduces a controlled data-construction pipeline and a benchmark spanning multiple modalities, with analyses revealing systematic differences in attribution performance.
1. Introduction
- Prior WHO&WHEN contains 184 traces from 127 systems but is limited to text, while obtaining high-quality annotations becomes harder beyond text tasks.Subtler failures in more capable agents are harder to detect, although they provide useful signals for improvement.Source: §1 · p. 1
- A controlled pipeline injects an error into a successfully completed trajectory, then warm-starts execution from that step to produce golden labels for the decisive agent and step.Because the injected error is the only controlled change turning success into failure, the resulting failure can be attributed to the injected agent and step.Source: §1 · p. 1
- WHO&WHEN PRO contains 12,326 failure traces from 26 benchmarks across nine task categories and text, image, and video modalities.Its examples span tasks including QA, data science, GUI interaction, and video understanding.Source: §1 · p. 2, Figure 1 · p. 2
- Attribution behavior varies across modalities, protocols, and model families: video challenges step localization, multimodal traces aid failure-mode diagnosis, and full-trajectory attribution outperforms incremental protocols.Open-weight models also offer practical attribution backbones with favorable cost-performance trade-offs.Source: §1 · p. 2
2. Related Work
Prior failure-attribution work includes curated and automatically generated benchmarks, but benchmark coverage and label quality remain constrained. WHO&WHEN PRO spans more modalities and agent regimes, while offering step-level labels relevant to self-evolution feedback.
2.1. Agents Failure Attribution
- Earlier benchmarks are text-only, isolate either single-agent or multi-agent settings, and trade off label quality against scale; WHO&WHEN PRO spans text, image, and video, covers both regimes, and reports higher-fidelity labels than other automatic generation pipelines.Table 1 compares existing failure-attribution benchmarks; its topology field distinguishes single-agent from multi-agent traces.Source: §2.1 · pp. 2–3, Table 1 · p. 3
2.2. Self-Evolving Agents
- WHO&WHEN PRO provides large-scale, externally verified, step-level failure labels across modalities, a feedback signal suited to self-evolution loops rather than terminal-only or self-graded feedback.Prior self-evolution approaches include reflection, self-rewarding judgment, refinement tuning, and bootstrapped multi-agent improvement.Source: §2.2 · p. 3
3. WHO&WHEN PRO
WHO&WHEN PRO defines the decisive step as the earliest action whose correction would turn failure into success, then constructs labels through controlled error injection into successful trajectories. The benchmark spans 12,326 failure traces across 26 benchmarks, 9 task categories, 3 modalities, and 15 agent frameworks.
3. WHO&WHEN PRO
- The decisive step is the earliest action whose correction would turn a failed trajectory into a successful one.In multi-agent traces, the agent that produced that action is the decisive agent.Source: §3 · p. 3
3.1. Source Trajectories
- The study collects trajectories from 15 agentic frameworks and 26 benchmarks across text, image, and video, spanning single-agent ReAct loops, multi-agent pipelines, code-as-action, and tool-calling.Successful trajectories provide action steps known to be sufficient for task completion; failed trajectories calibrate which error modes are realistic for each agent-benchmark setting.Source: §3.1 · pp. 3–4
3.3. Scalable Failure Trace Generation
- Warm-start replay avoids pre-injection drift; valid failure-mode-conditioned steps receive on-policy errors, and only failed continuations are retained, making the injected step decisive.Source: Figure 2 · p. 4, §3.3 · pp. 4–5
3.2. Failure Taxonomy
- Reviewers consolidate naturally occurring errors into six families and use setting-specific profiles to constrain which failures are generated.The families are perception, reasoning, planning, action, verification, and coordination.Source: §3.2 · p. 4
3.4. Benchmark Statistics
- WHO&WHEN PRO contains 12,326 failure traces spanning 26 benchmarks, 9 task categories, 3 modalities, and 15 agent frameworks.Source: §3.4 · p. 5
3.5. Human Review
- Across 100 stratified traces, annotators accept 94% of step labels and 90% of agent and error-family labels, with Fleiss’ κ=0.73.A majority marked only 2% of traces as lacking a clear decisive error.Source: §3.5 · p. 5
4. Experiments
Experiments compare frontier LLMs and attribution protocols across modalities, finding persistent attribution difficulties and systematic differences by modality, trace length, protocol, and access to ground-truth answers.
4.1. Experiment Setup
- Ten frontier LLMs are evaluated using all-at-once attribution by default, with agent, step, error-mode, and joint metrics across supported modalities.The three compared protocols are all-at-once, step-by-step, and binary search; all traces fit within each model’s native context window.Source: §4.1 · pp. 5–6, Table 4 · p. 6
4.2. Main Results
- The best model reaches 73.9% step-level attribution accuracy on text but only 22.2% error-mode F1, and no model dominates across metrics or modalities.GLM-5 has the highest Joint accuracy on text, GPT-5.4 leads on step identification, and Qwen3.5-122B leads on agent localization.Source: Table 4 · p. 6, §4.2 · p. 6
- Step accuracy is highest on text (69%) but drops on image and video, while mode classification rises from 17% on text to 37% on video.Visual and behavioral cues provide diagnostic signal, but longer multimodal context makes step localization harder.Source: §4.2 · p. 6, Figure 4 · p. 7
- Planning, verification, and coordination errors are often mislabeled as reasoning errors because their symptoms resemble reasoning mistakes later in the trace.Perception, reasoning, and action errors are more reliably recognized; more than half of other errors on multimodal traces are mislabeled.Source: §4.2 · p. 6, Figure 4 · p. 7
- Step accuracy falls from 94% on traces under 3K tokens to 50% on traces exceeding 12K tokens, consistently across all ten models.The sharpest decline occurs between 3–6K and 6–12K tokens, as traces transition toward multi-step sequences with interleaved observations.Source: Figure 4 · p. 7, §4.2 · p. 7
4.3. Comparison between Attribution Methods
- All-at-once achieves the highest Joint accuracy for eight of ten models and improves Step accuracy over step-by-step by 7.6 percentage points.Step-by-step tends to flag errors before they occur, while binary search is directionally unbiased but imprecise when slicing removes relevant context.Source: §4.3 · p. 7, Figure 5 · p. 7
4.4. Cost-Performance Trade-off
- All-at-once dominates the low-cost Pareto frontier across all three modalities, and open-weight models match or surpass closed-source alternatives at a fraction of the cost.Qwen3.5 achieves the highest text accuracy at seven times cheaper than GPT-5.4, while Gemma 4 leads on video.Source: §4.4 · pp. 7–8, Figure 6 · p. 8
4.5. Impact of Ground Truth
- Providing the ground-truth answer improves perception-error attribution but degrades or leaves flat reasoning-error attribution across task domains and frameworks.The authors attribute this split to the reference helping verify perception while encouraging answer-comparison shortcuts for reasoning errors.Source: Figure 7 · p. 8, §4.4 · p. 8, §4.5 · p. 8
5. Conclusion
The conclusion presents WHO&WHEN PRO as a broad benchmark built with warm-started error injection and golden labels. Evaluation finds persistent attribution challenges and systematic variation across modalities, protocols, and model families.
5. Conclusion
- WHO&WHEN PRO contains 12,326 traces across 26 benchmarks, spanning text, image, and video and covering single- and multiagent settings.A warm-started error-injection pipeline provides golden labels for responsible agent, decisive step, and failure mode.Source: §5 · p. 8
A.1. Limitations
The paper limits its 18-mode taxonomy to dominant observed failure patterns and describes it as extensible. It also presents the benchmark’s broader purpose and two case studies showing how ground truth can shift judges away from the underlying task misunderstanding.
A.1. Limitations
- The 18-mode taxonomy covers dominant observed patterns, not every possible agent failure, and may need additions as domains expand.The evaluation framework supports adding modes without changing existing annotations.Source: §A.1 · p. 15
A.2. Broader Impacts
- WHO&WHEN PRO is intended to support diagnosis of systematic weaknesses through fine-grained annotations and reproducible attribution evaluation.The authors also release traces, labels, and evaluation code for open research.Source: §A.2 · p. 15
- The authors acknowledge that the error-injection method could in principle inform adversarial attacks, while arguing that its benefits outweigh the marginal risk.They state that the studied weaknesses are already observable in production settings.Source: §A.2 · p. 15
B.1. Case 1: Ground Truth Shifts Diagnosis from Process to Outcome Verification
- In the MMSearch case, the agent retrieved the correct release date but answered a different question by interpreting “arrive” as anniversary content.The ground-truth error mode is task misunderstanding.Source: §B.1 · p. 15
- Supplying the gold date shifted judges from tracing the reasoning chain to matching the answer value, obscuring the semantic failure.Both judges instead classified the error as misapplied evidence, although the agent had answered a different question.Source: §B.1 · p. 15
B.2. Case 2: Ground Truth Causes the Judge to Skip the Root-Cause Error
The second case shows that a gold answer can draw judges toward a downstream error and away from the first decisive failure. Without the gold date, judges identify the agent’s fabricated product identification as the root-cause hallucination.
B.2. Case 2: Ground Truth Causes the Judge to Skip the Root-Cause Error
- The agent first fabricated a Ray-Ban Meta identification despite its prior observation, then selected the wrong Gentle Monster collection and propagated that error to its answer.Source: §B.2 · p. 16
- Without the gold date, both judges identified the Step 2 fabrication as the hallucination that first derailed the trajectory.Source: §B.2 · p. 16
- With the gold date, both judges skipped Step 2 and reclassified the downstream collection confusion as a reasoning error.The gold date made it easier to verify which date was wrong but drew attention away from where reasoning first broke.Source: §B.2 · p. 16
C. Error Taxonomy
The benchmark organizes 18 failure modes into six families, and the supplied results describe model confusions, human relabeling, and relaxed step-localization patterns. The full ablation comparison is reported across methods and models, while the available passage does not provide its cell values.
C. Error Taxonomy
- The taxonomy defines 18 modes across six families, spanning perception, reasoning, planning, verification, and multi-agent coordination in the supplied definitions.Examples distinguish hallucination from misapplying observed information, and distinguish task misunderstanding from planning or verification failures.Source: C. Error Taxonomy · pp. 16, 18, Table 5 · p. 17
- The taxonomy also distinguishes coordination failures such as withholding critical information from over-reliance on another agent that replaces sound independent reasoning.Source: C. Error Taxonomy · p. 18
D.1. Full results on Ablation Set
- Table 6 compares attribution methods across models on the ablation set, with each metric averaged over three modalities.The caption notes that text-only models are not directly comparable with the other multimodal models.Source: §D.1 · p. 18, Table 6 · p. 19
D.2. Fine-Grained Mode-Level Confusion Matrix
- On multimodal traces, visual misidentification, hallucination, and premature termination have clear signatures, while planning, verification, and coordination errors are often labeled as reasoning.Source: §D.2 · p. 19
D.3. Human Review Correction Matrix
- Human reviewers retain most pipeline-generated labels, with disagreements concentrated in adjacent categories such as planning or verification relabeled as reasoning.The transfer matrix pools three independent annotators.Source: §D.3 · p. 19
D.4. Relaxed Step Localization (Step@k)
- Step@1 improves over Step@0 by 13.9 pp on text, 16.9 pp on image, and 28.1 pp on video; gains from Step@1 to Step@2 are smaller.The largest adjacent-step effect occurs on video, where dense interleaved steps make neighboring-step ambiguity inherent to the trace structure.Source: §D.4 · p. 19
D.5. Detailed Benchmark Statistics
The appendix reports where injected failures occur in traces and identifies the scope and encoding of related benchmark figures. Most injections fall in the middle portion of a trace, while a separate figure summarizes framework contributions.
D.5. Detailed Benchmark Statistics
- 64% of injections fall between 40% and 80% of trace length, where partial evidence has often accumulated before the failure becomes plausible.Early injections are less common and associated with planning; late injections are less common and associated with verification.Source: §D.5 · pp. 19–20
- Figure 10 presents a row-normalized mode-level confusion matrix for image and video traces averaged across seven vision-capable models.Labels are colored by error category.Source: Figure 10 · p. 20
- Figure 16 reports the percentage of traces contributed by each of the 15 agent frameworks.Source: §D.5 · p. 20
E.1. Failure Attribution: Benchmarks and Methods
The section surveys failure-attribution benchmarks and methods, self-evolving agent systems, and synthetic or counterfactual trajectory generation. WHO&WHEN PRO extends controlled injection by labeling the injected step as the failure and validating retained traces with filters and human review.
E.1. Failure Attribution: Benchmarks and Methods
- Existing datasets range from expert-curated traces to roughly ten-thousand-trace LLM-injected AEGIS and controlled-injection, counterfactual-replay AgenTracer.TRAIL and AgentErrorBench densely annotate smaller corpora to expose long-context localization difficulties.Source: §E.1 · p. 20
- Attribution methods include hierarchical error analysis, suspicious-triple localization, iterative Judge–Evaluator passes, causal interventions, pattern matching, forecasting, and safety auditing.These approaches treat attribution as inference over execution traces.Source: §E.1 · p. 20
E.2. Self-Evolving Agentic Systems
- Self-evolving-agent systems combine reflection, self-rewarding judgment, refinement tuning, and bootstrapped multi-agent improvement, alongside newer closed-loop approaches.Examples include revising reasoning trajectories, producing executable improvement suggestions, and jointly training agents and data.Source: §E.2 · p. 21
E.3. Synthetic and Counterfactual Trajectories for Agents
- Synthetic-trajectory work spans tutorial- and simulator-based data, multi-agent sandboxes, computer-use engines, unified protocols, and web-environment synthesis.WHO&WHEN PRO positions its controlled-injection pipeline within this broader trajectory-generation line.Source: §E.3 · pp. 21–22
- WHO&WHEN PRO makes the injected step the labeled failure and validates retained traces with automatic filters and Ph.D.-level human review.The pipeline produces downstream behavior from a real agent reacting to a deliberately corrupted decision.Source: §E.3 · p. 22
F.1. Agent–Benchmark Coverage
The benchmark maps agent frameworks to modality-specific benchmarks and uses frontier language models as agent backbones. Warm-start replay addresses environmental drift through cached observations or fidelity-checked reconstruction of browser and executor state.
F.2. Base Agent Model
- GPT-4.1 and Gemini 3 Flash are the primary agent backbones, selected for interpretable reasoning chains and cost-efficient large-scale collection, respectively.The backbone varies by agent–benchmark pair according to modality support and framework compatibility.Source: §F.2 · p. 23
F.3. Details on Warm Starting
- Warm-starting replays a successful prefix to resume at the injection step, but environmental change can make the replayed state diverge from the recorded trajectory.The pipeline uses three complementary strategies to mitigate this drift.Source: §F.3 · p. 23
F.1. Agent–Benchmark Coverage
- The framework-to-benchmark mapping groups tasks by text, image, video, and GUI modalities; single-agent systems target specific modalities or task families, while multi-agent systems cover narrower complementary benchmark sets.Table 7 lists 15 agentic systems and their evaluated benchmarks.Source: §F.1 · p. 22, Table 7 · p. 24
F.3. Details on Warm Starting
- For smolagents, a synchronous SQLite cache replays recorded factual tool responses verbatim, making pre-injection observations byte-identical while leaving the agent’s reasoning and actions live.The cache covers environment-state calls; for web-visit summarization, it stores raw page content rather than the secondary LLM summary.Source: §F.3 · p. 24
- For Magentic-One, the pipeline rebuilds browser and code-executor state by replaying recorded actions with fidelity checks, aborting and retrying or skipping injections when replay diverges.A URL-construction fallback mitigates Bing DOM rotation, reported as the most common replay failure source for input_text actions.Source: §F.3 · p. 24
G. Seed Dataset for Benchmark Generation
This section describes the seed trajectories, agent frameworks, evaluation prompts, and protocols used to generate benchmark data and attribute failures across text, image, and video datasets, including methods for locating failure steps and templates for injecting controlled failures.
G.1. Text Datasets
- Seed trajectories span text benchmarks for embodied interaction, code and data-science execution, tool use, and expert reasoning.Successful base trajectories are collected across 26 datasets spanning text, image, and video, with retained traces linked to applicable error modes.Source: G. Seed Dataset for Benchmark Generation · p. 24, §G.1 · p. 25
G.2. Image Datasets
- Image datasets cover visual chart understanding, multimodal search and reasoning, and interactive computer or web tasks.Examples include vision-required questions, searches requiring fine-grained visual cues, and GUI workflows across desktop applications and live websites.Source: §G.2 · pp. 26–27
G.3. Video Datasets
- LVBench supplies the video seed set, targeting long-range temporal coherence, key-frame retrieval, and causal or intentional reasoning over long videos.Its publicly sourced videos include TV series, sports broadcasts, and surveillance footage.Source: §G.3 · p. 27
H. Agentic Systems
- The evaluated systems span single-agent and multi-agent topologies across text, image, and video modalities.Examples range from lightweight executable-code agents to specialized visual teams and searchable-video agents.Source: H. Agentic Systems · pp. 27–29
I. LLM Usage Declaration
- LLMs served only as auxiliary tools for data-analysis debugging and sanity checks, code implementation support, and wording polish; the core research contributions were the authors’ original work.Source: §I · p. 29
J. Evaluation Prompt Templates
- Evaluation prompts are identical across models, use the same error-mode taxonomy and output schema, and support open-book or closed-book attribution.Three protocols vary how much of the trajectory is presented at once and the localization strategy.Source: J. Evaluation Prompt Templates · pp. 29–30, Table 8 · p. 29
J.1. All-at-Once Protocol
- The all-at-once protocol presents the complete transcript in one prompt and asks for the first decisive error; it is the primary setting for the main results.The requested attribution identifies the agent, step coordinate, and matching error mode, with a brief rationale.Source: §J.1 · pp. 30–31
J.2. Step-by-Step Protocol
- The step-by-step protocol reveals the transcript incrementally and flags the first step judged to contain a decisive error.The model sees history through the current step, and the runner iterates until it flags one.Source: §J.2 · p. 31
J.3. Binary Search Protocol
- Binary search recursively bisects a transcript range, asking whether the error is more likely in the upper or lower half until the location converges.Each call presents a contiguous slice of steps.Source: §J.3 · p. 32
K. Injection Prompt Templates
- Failure injection uses a frontier model to generate an adaptive prompt conditioned on the target failure mode, then patches it into the base agent’s call so it produces the corrupted action.The base agent generates the action in its own style.Source: K. Injection Prompt Templates · pp. 32–33
K.1. R.1 – Hallucination
- For hallucination, the frontier model fabricates a specific, topical false claim, which anchors a prompt steering the base agent to treat it as background knowledge and search based on it.The agent must not acknowledge contradictory observations and must search for supporting evidence rather than conclude at that step.Source: §K.1 · pp. 33–34
K.2. R.2 – Reasoning Error
The reasoning-error injection procedure first identifies a concrete flaw grounded in the agent’s observations, then uses it to steer the base agent into a flawed continuation. The prompted agent must keep investigating rather than conclude immediately.
K.2. R.2 – Reasoning Error
- Stage 1 asks a frontier model to identify a specific reasoning flaw grounded in the agent’s observations and reference answer.The flaw is framed as a confusion point pinpointing which facts are misconnected and what logical error occurs.Source: §K.2 · p. 34
- The flaw prompt requires concrete confused facts, a specified logical error, and an explanation of how the error produces a plausible wrong answer.Examples include mixing similar entities or time periods, reversing comparisons, and misattribution.Source: §K.2 · p. 35
- The confusion point anchors Stage 2, where the base agent engages with its real observations but connects them incorrectly.The injected agent continues investigating through a search or visit action rather than concluding immediately.Source: §K.2 · pp. 35–36
- The injection template makes the agent reference its observations while following the flawed logic, search for supporting details, and avoid calling final_answer at that step.The requested output is formatted as Thought followed by code or an action.Source: §K.2 · p. 36