PaperlayerView exact revision on arXiv

Source-linked AI summary

Tracing Agentic Failure from the Flow of Success

Samuel Yeh, Yiwen Zhu, Shaleen Deep, Sharon Li

arXiv:2607.12747v1cs.AIcs.CL

TL;DR

Failure attribution in LLM-based agentic systems is difficult and costly, motivating methods that avoid step-level failure annotations. OAT learns successful-trajectory dynamics with neural CDEs and outperforms prompting-based approaches in-domain and out-of-distribution while running 200–5000× faster.

  • Problem

    Failure attribution must identify error steps in failed agentic trajectories, but existing methods require costly step-level annotations or computationally expensive prompting.

  • Method

    OAT uses one-class learning with neural controlled differential equations to model successful-trajectory dynamics and detect anomalous error steps at inference.

  • Results

    +20% F1 in-domain and +7% F1 out-of-distribution, while OAT runs 200–5000× faster than prompting-based approaches.

  • Takeaways & Limitations

    OAT demonstrates a label-efficient and computationally efficient direction for diagnosing agentic system failures.

  • Takeaways & Limitations

    The fixed-size detection strategy requires prior knowledge of the typical number of failure-contributing steps, which may vary across tasks and systems.

Abstract

from arXiv · show

Failure attribution for LLM-based agentic systems, i.e., identifying which steps in a failure trajectory caused the task to fail, is critical for debugging and improving these systems. Existing approaches either rely on prompting-based pipelines, which are computationally expensive, or require post-training on failure trajectories with step-level error annotations, which are costly to collect and difficult to scale. We argue that a practical failure attribution model should be lightweight and trainable without step-level supervision on failure data. To this end, we address unsupervised failure attribution, i.e., training exclusively on successful trajectories and identifying error steps at inference time given a failure trajectory. We propose OAT, which casts this problem as one-class learning with neural controlled differential equations, modeling the dynamical pattern of successful trajectories in latent space. At inference time, each step in a failure trajectory is assigned an anomaly score based on its deviation from the dynamics learned on successful trajectories, which is then used to form a set of error steps. With training on only 100 successful trajectories, experiments show that OAT is 200--5000 $\times$ faster than prompting-based baselines, and, at the same time, consistently outperforms them in both in-domain and out-of-distribution datasets with +20% and +7% F1 scores, respectively, demonstrating that OAT is a promising and efficient direction for diagnosing agentic system failures.

1 Introduction

The paper frames failure attribution as identifying error steps in long-horizon agentic trajectories, where existing methods are inaccurate, costly, or annotation-intensive. It proposes OAT, an unsupervised one-class model trained only on successful trajectories to detect failures efficiently from deviations in latent-space dynamics.

  • Motivation: Failure attribution identifies the step(s) responsible for an agentic system’s failure, yet state-of-the-art reasoning models achieve below 15% accuracy.The task is difficult because trajectories can span hundreds of actions across interacting agents and tools, while later steps may obscure earlier mistakes.
  • Limitations: Prompting-based failure attribution requires frontier LLM inference, creating token costs and latency that preclude practical deployment.Other approaches post-train LLMs on failure trajectories with step-level annotations, which are costly to collect and difficult to scale.
  • Problem Setting: Unsupervised failure attribution trains exclusively on successful trajectories and identifies error steps in failure trajectories without step-level failure annotations.This formulation is intended to eliminate the annotation bottleneck while keeping the model lightweight and deployable without frontier LLM inference.
  • Method: OAT models successful-trajectory dynamics in latent representation space with neural controlled differential equations and scores failure steps by deviation from the learned normal flow.Each trajectory step is represented as a latent vector extracted from an LLM; anomaly scores compare actual representations with the expected successful path.
  • Results: 200–5000× faster than prompting-based approaches, OAT uses zero token cost at inference and consistently outperforms prompting frontier LLMs, including GPT-5.It achieves these results with a lightweight 3-layer MLP trained on only 100 successful trajectories, using no failure data or step annotations.

2 Related Work

Related work spans failure-attribution taxonomies for LLM-based multi-agent systems and one-class anomaly detection, whose standard formulations operate at the sample level rather than the agent-step level.

  • Failure attribution for LLM agents: Failure-attribution research has characterized LLM multi-agent failures through taxonomies covering system design, inter-agent misalignment, task verification, reasoning, and planning.Cemri et al. define 14 failure categories, while Deshpande et al. introduce a finer-grained taxonomy.
  • Anomaly detection: One-class learning trains exclusively on normal data and identifies anomalies as out-of-distribution samples at test time.This approach is part of the broader machine-learning literature on detecting deviations from learned normal behavior.
  • Anomaly detection: Standard anomaly detection typically determines whether an entire sample is anomalous, whereas agent settings require finer-grained diagnosis.The passage contrasts sample-level anomaly detection with the step-oriented nature of agent failure analysis.

3 Problem Statement

The paper formalizes failure attribution as identifying the steps that meaningfully degrade a failed agentic trajectory. Existing supervised approaches require costly step-level annotations on failure trajectories.

  • Agentic system: An agentic system uses multiple LLM-based agents with tool-calling and inter-agent communication to solve complex, long-horizon tasks.At each time step, the system observes an environment state and selects an agent to execute an action.
  • Trajectory and outcome: A trajectory τ records the query, alternating actions and states, and terminal step T_τ.The trajectory is written as τ = (Q, a_1, s_1, a_2, …, s_T_τ−1, a_T_τ).
  • Trajectory and outcome: The evaluation function Z(τ) ∈ {0, 1} labels whether a trajectory successfully completes the task.Z(τ) = 1 denotes success, whereas Z(τ) = 0 denotes failure.
  • Failure attribution: Failure attribution predicts a set of error steps y(τ′) for any failure trajectory τ′ with Z(τ′) = 0.The target set is drawn from the trajectory’s step indices, {1, …, T_τ′}.
  • Limitations of supervision: Supervised failure attribution requires training on step-annotated failure trajectories, making step-level annotation costly to collect.The passage identifies annotation cost as a limitation of existing supervised approaches.
  • Failure attribution: Failure-contributing steps meaningfully degrade a trajectory toward failure, even when intervening at one such step alone cannot ensure recovery.This definition distinguishes contribution to failure from individual-step recoverability.

4 Methodology

OAT frames unsupervised failure attribution as one-class learning over latent representations, learning successful-trajectory dynamics and flagging failure steps that deviate from them. Neural CDEs model irregular, variable-length trajectories, while gating improves robustness to out-of-distribution control signals and conformal prediction adaptively thresholds anomalies.

  • Latent representations: OAT represents each agent step using aggregated layer-ℓ token representations and arranges them into a sequence alongside the query representation.The resulting sequence H(τ) captures the latent trajectory used for dynamics modeling and failure-step detection.
  • Trajectory dynamics modeling: Neural CDEs model successful trajectories as continuous latent paths conditioned on interpolated observations, capturing dynamics that discrete-time models may miss.The control path is constructed from step representations using cubic-spline interpolation, and its derivative drives the latent dynamics.
  • Trajectory dynamics modeling: Neural CDE conditioning makes latent states sensitive to local trajectory behavior, enabling detection of localized deviations while supporting multiple successful action sequences.Unlike Neural ODEs, the dynamics are continuously modulated by observed changes rather than determined solely by the initial condition.
  • Gated control path: A gated control path suppresses atypically large or unfamiliar test-time derivatives, preserving in-domain expressiveness while improving robustness under distributional shift.The gate passes control signals at full strength in-distribution and attenuates misleading signals out-of-distribution.
  • Failure-step detection: Failure steps receive reconstruction-error anomaly scores, which can be detected either by selecting top-k steps or by conformal prediction with a threshold calibrated from successful trajectories.Conformal prediction adaptively determines the number of flagged steps and controls the in-distribution false positive rate at level α.

5 Experiments

Experiments evaluate OAT on in-domain MCP-Atlas and out-of-distribution Who&When trajectories against prompting-based and simple baselines. OAT achieves stronger attribution performance while requiring substantially less computation and remaining effective in qualitative failure cases.

  • Datasets: The evaluation uses 103 successful and 88 failure trajectories from MCP-Atlas, with Who&When reserved for out-of-distribution testing.Successful MCP-Atlas trajectories have no claim-level errors, while failure trajectories are manually annotated for contribution steps.
  • Baselines: OAT is compared with GPT-4o and GPT-5 prompting approaches, plus random and first-step baselines that select one step per trajectory.The prompting methods identify failure-contributing steps by asking an LLM to predict a set of steps.
  • In-domain evaluation: +20% F1 score and +10% AUPRC: OAT consistently outperforms all baselines in the MCP-Atlas in-domain evaluation.The model is trained on fewer than 100 successful trajectories yet surpasses frontier LLMs for identifying failure-contributing steps.
  • OOD evaluation: +7% F1 score and +5% AUPRC: OAT outperforms prompting-based approaches on Who&When despite substantial distributional differences between benchmarks.MCP-Atlas contains single-agent tool-calling trajectories, whereas Who&When evaluates multi-agent collaboration with fundamentally different trajectory structures.
  • Computational efficiency: 0 token cost and 200–5000× faster: OAT is more efficient than prompting-based approaches and supports real-time failure attribution.OAT requires less than 1 GB VRAM for deployment, unlike prompting approaches that require frontier LLMs unavailable on a local server.
  • Qualitative case studies: OAT assigns high anomaly scores to hallucinations, unfaithful assumptions, and code bugs while keeping scores low on non-contributing steps in successful cases.A case study also shows correct identification of a hallucination at step 7 and its propagated error at step 9.

6 Ablation Studies

The ablations show that continuous control, gated dynamics, and later-layer representations each improve OAT’s failure-step identification, while OAT outperforms an RNN-based one-class learner. Gating particularly improves out-of-distribution performance, with a small in-domain trade-off.

  • Neural ODE vs. CDE: Neural CDE consistently outperforms Neural ODE across all metrics, highlighting the importance of continuously injecting trajectory observations through the control path.The control path continuously injects trajectory observations into latent dynamics to model agent trajectories more accurately.
  • Impact of gated control path: +0.172 AUROC is the gated control path’s OOD improvement, alongside a −0.028 in-domain performance trade-off.This asymmetric effect suggests that gating improves the generalizability of learned trajectory dynamics.
  • Comparison with RNN-based approach: OAT consistently outperforms the RNN-based one-class learning approach across metrics, indicating that continuous modeling characterizes trajectory dynamics more accurately.The RNN predicts the next latent representation from h_Q, h_1, ..., h_{t−1}, with anomaly score based on prediction error.
  • Representations at different layers: Later-layer representations consistently achieve higher precision, recall, and F1 score than earlier-layer representations across evaluated Qwen3.5-27B layers.Representations were evaluated at layers {0, 8, 16, 24, 32, 40, 48, 56, 64}, where layer 0 is the embedding-layer output.
  • Additional ablations: Additional ablations examine representation aggregation strategies, proxy-model choices, and the precision–recall trade-off.Further results are provided in Appendix G.

7 Conclusion · APPENDIX

The paper formulates unsupervised failure attribution for LLM-based agentic systems, removing the need for step-level annotations on failure trajectories. OAT trains only on successful trajectories and identifies error steps by detecting deviations from learned normal dynamics.

  • 7 Conclusion: OAT introduces unsupervised failure attribution for diagnosing failures in LLM-based agentic systems.This formulation avoids relying on labeled failure trajectories.
  • 7 Conclusion: The approach eliminates the need for step-level annotation of failure trajectories.Training does not require step-level labels on failures.
  • 7 Conclusion: OAT uses one-class learning to model normal agent behavior rather than directly learning from labeled failures.Its learning setup is based exclusively on successful trajectories.
  • 7 Conclusion: The method is based on Neural CDEs for representing the dynamics of successful trajectories.These dynamics define the reference pattern for normal behavior.
  • 7 Conclusion: OAT trains exclusively on successful trajectories.No failure-trajectory annotations are required during training.
  • 7 Conclusion: At inference time, OAT identifies error steps by detecting deviations from the learned dynamics of normal behavior.Failure steps are found through their divergence from the successful-trajectory dynamics.

A Limitations and future work · B Societal Impact

The paper identifies extensions to its successful-trajectory modeling approach as future work while presenting automated failure attribution as a transparency and reliability aid for high-stakes agentic systems. It also cautions that such tools require responsible deployment and continued human oversight.

  • A Limitations and future work: The work demonstrates that Neural CDEs are effective for continuous-time modeling of agent trajectories.It explores the feasibility of modeling successful trajectories with neural differential equations.
  • A Limitations and future work: Future work could extend vanilla NDEs, including through Continuous Normalizing Flows for exact likelihood computation.The cited formulation uses the instantaneous change-of-variables formula.
  • B Societal Impact: The approach aims to improve transparency and reliability by automatically identifying error-contributing steps without human annotation.This reduces the annotation burden for failure attribution.
  • B Societal Impact: Failure identification and explanation are especially relevant as agentic systems enter software engineering, scientific research, and automated decision making.The passage frames these as high-stakes domains where safer systems are needed.
  • B Societal Impact: The authors do not foresee direct negative societal impacts from this work.They nevertheless discuss potential misuse and deployment safeguards.
  • B Societal Impact: Failure attribution tools could be used to optimize agents for evading detection rather than for genuine improvement.This is identified as a potential misuse of automated failure attribution.
  • B Societal Impact: Automated failure attribution should support human oversight rather than replace it.The passage presents this as a responsible deployment practice.

C Reproducibility Statement

The paper provides comprehensive implementation and experimental details to support reproducing OAT, including model construction, hyperparameters, datasets, baselines, and evaluation metrics.

  • Implementation details: OAT’s implementation is detailed through representation extraction, neural CDE vector-field structure, CDE solver, control-path construction, and corresponding hyperparameters.These details appear in Section 5.1 and Appendix F.
  • Experimental settings: Experimental settings, datasets, step-level error annotations, baselines, and evaluation metrics are documented in Sections 5.1 and Appendices D–E.The paper presents these reports as support for future studies.

D Details of Datasets · E Annotation of Failure Contributing Steps

The paper evaluates failure attribution on MCP-Atlas and Who&When, then constructs MCP-Atlas labels through filtered, step-level manual annotation guided by claim-level judgments. Annotation covers reasoning and tool-use errors while distinguishing informative agent failures from infrastructure faults and acceptable search exploration.

  • D Details of Datasets: MCP-Atlas covers 36 MCP servers, 220 tools, and 1,000 realistic multi-step tasks for evaluating tool-use agents.Experiments use Qwen3.5-27B on 203 publicly available tasks with free access to the required MCP server APIs, using the same model as the LLM-as-a-Judge.
  • D Details of Datasets: Who&When benchmarks failure attribution in multi-agent systems using GAIA and AssistantBench tasks spanning multimodal processing, web browsing, coding, and cross-topic website interaction.Trajectories were generated by CaptainAgent and Magne...
  • D Details of Datasets: The annotation tool visualizes complete trajectories alongside claim-level judging results to support error tracing.Figure 7 presents the visualization tool developed for trajectory annotation.
  • E Annotation of Failure Contributing Steps: Each MCP-Atlas failure trajectory receives a manually annotated set of failure-contributing steps through trajectory filtering followed by step-level labeling.The contributing-step set is denoted C(τ).
  • E Annotation of Failure Contributing Steps: Failures caused purely by infrastructure errors, including a search API that consistently times out, are excluded because they do not reflect agent-level faults.Such failures have trivially identifiable contributing steps and are uninformative for evaluating failure attribution methods.
  • E Annotation of Failure Contributing Steps: Annotators use claim-level judging results and full step-by-step trajectory review to label steps exhibiting hallucinated reasoning, incorrect tool-call arguments, or mishandled tool-return errors.These error types respectively concern fabricated reasoning, malformed or semantically inappropriate arguments, and explicit tool errors or exceptions that lead to downstream failure.
  • E Annotation of Failure Contributing Steps: Failed exploratory search attempts are not penalized when the agent tries different queries, but repeated identical or semantically equivalent searches after receiving...Alternative phrasings are treated as a reasonable and expected exploration strategy.
  • E Annotation of Failure Contributing Steps: Authors independently review every trajectory, discuss ambiguous attributions until consensus, and retain labels reflecting their best judgment of steps that meaningfully degraded the trajectory toward failure.Ambiguities can involve reasoning errors versus uninformative tool returns.

F Additional Implementation Details & Hyperparameters · G Additional Experimental Results · G.1 Precision-Recall Trade-off

The appendix details implementation choices, distribution-shift mitigation, and detection hyperparameters, then reports how detection settings, representation aggregation, and proxy-LLM usage affect performance. OAT’s precision–recall balance changes with k and α, while F1 remains stable.

  • F Additional Implementation Details & Hyperparameters: The experiments summarize hyperparameters and examine representation extraction, Neural CDE gating, top-k detection, and conformal prediction detection through ablations.These design choices are discussed across the implementation and experimental appendices.
  • F Additional Implementation Details & Hyperparameters: CORAL is applied to mitigate distributional shift by aligning second-order statistics between in-domain and OOD representations.The mismatch arises because PCA is fitted solely on in-domain MCP-Atlas training data, while OOD WHO&WHEN trajectories may have different covariance.
  • G.1 Precision-Recall Trade-off: Increasing k in top-k detection or α in conformal prediction detection trades precision for recall, while F1 score remains stable.The ablation studies use k ∈ {1, 3, 5} and α ∈ {0.05, 0.1, 0.2}.
  • G.1 Precision-Recall Trade-off: The detection hyperparameters k and α control the precision–recall trade-off and can be selected according to the desired operating point.The provided passage introduces these parameters as controls for balancing precision and recall.
  • G Additional Experimental Results: Mean-pooling aggregation outperforms using the last-token representation.This comparison is reported as an additional experimental result.
  • G Additional Experimental Results: Performance remains stable when using a proxy LLM.The result is reported in an additional experimental figure.

G.2 Ablations on Step Representations · H Case Studies

The ablations show that mean-pooled step representations are important, while OAT remains effective when representations come from proxy LLMs. Case studies examine successful and failed detections, including repeated useless actions, inconsistent reasoning, and an unfaithful assumption.

  • G.2 Ablations on Step Representations: The ablations evaluate representation aggregation and proxy LLM choices on MCP-Atlas conformal prediction detection.These studies target design choices in step representation extraction.
  • G.2 Ablations on Step Representations: Replacing mean pooling with the last-token representation significantly degrades performance.The result suggests that the last token alone is insufficient to encode an action step’s complexity.
  • G.2 Ablations on Step Representations: Intermediate tokens can carry error signals in agent behavior, supporting aggregation across tokens rather than relying only on the last token.This motivates the mean-pooling design for step representations.
  • G.2 Ablations on Step Representations: OAT works well with proxy representations extracted by Llama-4-Scout, Gemma-4-31B, or GPT-oss-120B, with only a small performance degradation.The proxy setup uses an LLM different from the model that generated the trajectories.
  • H Case Studies: The case study samples three MCP-Atlas trajectories where OAT identifies at least one annotated contributing step and three where it identifies none.All case study outputs use conformal prediction detection.
  • H Case Studies: In a successful case, OAT identifies repeated useless actions at step 9, inconsistent reasoning at steps 8 and 11, and an unfaithful assumption at step 15.These findings are reported in Figure 11.

H.1 Successful Cases … NeurIPS Paper Checklist

Across successful cases, OAT localizes error origins, attenuates scores along propagation chains, and surfaces latent precursors, while failure cases reveal misses on implicit early errors and fixed-threshold limitations. The appendices provide prompting templates and checklist responses covering reproducibility, ethics, societal impacts, assets, human subjects, and LLM use.

  • H.1 Successful Cases: A significantly high score marks the hallucinated step 7, while the contingent final step receives a more moderate score.This reflects sensitivity to error origins and attenuation for downstream steps that inherit errors.
  • H.1 Successful Cases: A significantly high score at step 7 decays progressively across propagated errors at steps 8 and 10.The monotonic decay indicates that inherited errors are less anomalous than the originating bug.
  • H.1 Successful Cases: Across three successful cases, OAT scores hallucinations, unfaithful assumptions, and logical bugs highly while keeping benign-step scores low and surfacing unannotated precursors.Downstream inherited errors receive gradually lower anomaly scores than the steps that introduce them.
  • H.2 Failure Cases: Only step 6 is flagged although steps 5, 6, and 7 receive high anomaly scores, because conformal prediction uses a hard threshold.The root reasoning error at step 5 falls just below the detection threshold despite contributing to the failure.
  • I Prompt Templates: The prompting-based attribution prompt serializes trajectories and requests one or more candidate error steps and agents in exactly one JSON object.It specifies a valid step range, candidate-agent list, 0-based indexing, and that candidates need not be earliest mistakes.
  • Template for trajectory serialization.: Trajectory serialization records each turn’s role, agent, reasoning content, content, and tool arguments using repeated structured fields.The template repeats these fields across steps 1, 2, and T.
  • NeurIPS Paper Checklist: The NeurIPS checklist reports accurate claims, no theoretical results, experimental details in Section 5.1 and Appendix F, and ethics, societal-impact, asset, and release disclosures.It states that code and data guidance is addressed in Appendix C, existing and new assets in Appendix D, societal impacts in Appendix B, and annotations are author-produced.
Loading 2607.12747v1…