PaperlayerView exact revision on arXiv

Source-linked AI summary

Conformal Agent Error Attribution

Naihe Feng, Yi Sui, Shiyi Hou, Ga Wu, Jesse C. Cresswell

arXiv:2605.06788v1cs.LGcs.MA

TL;DR

Multi-agent system failures are difficult to attribute because existing approaches often identify a single responsible step without principled uncertainty. This paper develops contiguous conformal prediction sets for sequential agent trajectories, showing they can isolate decisive errors and support automated rollbacks.

  • Problem

    Existing MAS error-attribution methods typically predict one responsible step, offering practitioners limited actionable insight because they lack principled uncertainty quantification.

  • Method

    The paper introduces filtration-based conformal prediction algorithms that produce contiguous trajectory regions with finite-sample, distribution-free coverage guarantees.

  • Results

    Across real-world and synthetic MAS traces, conformal algorithms precisely isolated decisive errors when the algorithm matched the dataset’s error distribution.

  • Takeaways & Limitations

    Conformal prediction sets support human debugging and automated MAS rollbacks that restart trajectories before likely decisive errors.

  • Takeaways & Limitations

    One-direction filtration can fail to isolate decisive errors near the trajectory’s middle, motivating two-way filtration.

Abstract

from arXiv · show

When multi-agent systems (MAS) fail, identifying where the decisive error occurred is the first step for automated recovery to an earlier state. Error attribution remains a fundamental challenge due to the long interaction traces that large language model-based MAS generate. This paper presents a framework for error attribution based on conformal prediction (CP) which provides finite-sample, distribution-free coverage guarantees. We introduce new algorithms for filtration-based CP designed for sequential data such as agent trajectories. Unlike existing CP algorithms, our approach predicts sets that are contiguous sequences to enable efficient recovery and debugging. We verify our theoretical guarantees on a variety of agents and datasets, show that errors can be precisely isolated, then use prediction sets to rollback MAS to correct their own errors. Our overall approach is model-agnostic, and offers a principled uncertainty layer for MAS error attribution. We release code at https://github.com/layer6ai-labs/conformal-agent-error-attribution.

1 Introduction

Multi-agent systems can fail through incorrect intermediate decisions and miscoordination, while existing attribution methods typically make uncertain point predictions. The paper proposes conformal prediction methods that identify localized trace regions containing decisive errors with finite-sample, distribution-free guarantees.

  • Motivation: MAS complexity and rich agent interactions make systems prone to errors from incorrect intermediate decisions and miscoordination.These systems are increasingly used for complex tasks involving decomposition, coordination, and tool use.
  • Problem: Existing attribution approaches produce a single responsible step, limiting actionable insight because they lack principled uncertainty quantification.The approaches include LLM-as-a-judge methods, structured reasoning pipelines, and fine-tuned attribution models.
  • Background: Conformal prediction addresses this limitation by generating prediction sets with statistical guarantees for decision-making under uncertainty.The introduction presents CP as a promising direction for reliable attribution rather than committing to one step.
  • Contribution: The proposed framework identifies a localized execution-trace region guaranteed to contain the decisive error at a user-specified confidence level.Its guarantees are finite-sample and distribution-free, and the methods are adapted to sequential agent trajectories through filtration-based CP.

2 Background & Related Work

Prior MAS error-attribution methods use increasingly sophisticated LLM judges to localize a single decisive error in execution traces. Conformal prediction provides finite-sample, distribution-free coverage, while this work adapts it to sequential trajectories and contiguous prediction sets for rollback.

  • MAS Error Attribution: Existing MAS error-attribution methods progress from naive LLM judges to context-engineered and multi-LLM pipelines for post-hoc trace localization.Examples include hierarchical trace contexts and aggregation across multiple judgments.
  • MAS Error Attribution: Experiments compare naive, context-engineered, and fine-tuned LLM judges, while retaining the decisive-error setting because labeled datasets for accumulated errors are lacking.Prior work generally assumes one decisive error, although practical MAS failures may arise from accumulated smaller errors.
  • Conformal Prediction: Conformal prediction calibrates a threshold on held-out data and outputs prediction sets containing the ground truth with probability at least 1−α under a distribution-free, finite-sample guarantee.The user chooses the error tolerance α.
  • Conformal Prediction: For exchangeable data, conformal prediction forms sets by including labels whose conformal scores are at most the calibrated threshold, with exchangeability holding automatically for i.i.d. data.The paper argues this assumption is reasonable for agent error attribution in its experiments.
  • Sequential Conformal Prediction: For agent trajectories, the ground truth is a single decisive error, but arbitrary conformal subsets are unsuitable for rollback, motivating novel algorithms that produce contiguous sequences.The target set is represented as consecutive trajectory elements (cj, ..., ck).

3 Conformal Agent Error Attribution

The framework applies conformal prediction to identify the decisive error in failed agent trajectories while favoring small, contiguous prediction sets. Its filtration algorithms provide coverage-aware error localization that supports debugging and automated rollback.

  • Framework objective: Each failed trajectory contains a labeled decisive error, and the goal is a small prediction set covering that error with valid coverage.The framework treats each step as potentially containing environment state, action, and response information.
  • Framework objective: Conformal error attribution separates prediction-set generation from a scoring function that estimates whether the decisive error belongs to a candidate set.The two components are designed independently so different algorithms and scoring functions can be combined.
  • Baselines: VCP ignores sequence order, requires ℓ evaluations, and produces noncontiguous sets, while CRSVP produces contiguous tree nodes but has no upper coverage bound and inflexible splits.These methods serve as experimental baselines whose limitations motivate the filtration algorithms.
  • Filtration algorithms: Left and right filtration return suffixes or prefixes with conformal coverage, but one-directional filtering can include the entire trajectory or fail to isolate middle errors.Right filtration is advantageous when agents tend to fail earlier, whereas left filtration returns a suffix by progressively removing steps from the left.
  • Filtration algorithms: Two-Way Filtration combines bidirectional filtering and tends to produce a short subinterval when both directional scores narrow on the same steps, while retaining the coverage guarantee.This addresses the localization weaknesses of one-directional filtration regardless of the decisive error’s position.
  • Applications: For automated recovery, the MAS rolls back to the first step in the conformal set, using coverage to avoid rolling back too little and contiguity to avoid rolling back excessively far.Contiguous sets also make manual debugging easier because users can focus on consecutive steps rather than scattered locations.

4 Experiments

The experiments evaluate conformal prediction algorithms, scoring functions, and downstream rollback performance on real-world and synthetic multi-agent traces. They use step-level error-attribution metrics and assess rollback success, coverage, and cost against a Top-1 baseline.

  • Datasets: The evaluation combines 184 Who&When real-world traces with synthetic failed trajectories generated by controlled prompt-based error injection.Each Who&When example contains a full execution trace annotated with its decisive error step; synthetic trajectories vary agent and task conditions.
  • Datasets: Synthetic variants alter decisive-error distributions by conditioning on early, middle, or late thirds of normalized trajectory length.This design probes how nonuniform error locations affect attribution performance, reflecting the tendency of Who&When errors to occur early.
  • Metrics: Scoring functions are evaluated as classifiers using AUROC, AUPRC, and accuracy, with low baseline metric levels when the number of possible steps is large.The scoring function predicts the decisive error location in an ℓ-way task.
  • Metrics: Conformal attribution is assessed by Empirical Coverage and Removal Rate, targeting coverage of at least 1 −α while favoring smaller prediction sets.Removal Rate measures the fraction of trajectory steps filtered out and therefore the precision of error localization at equal coverage.
  • Rollback: Rollback performance measures Success Rate, Coverage, and Cost against restarting from the single most likely predicted step, or Top-1.These metrics respectively capture task completion, rollback reaching at least the decisive error, and the fraction of steps requiring redo.

5 Results

The results validate conformal coverage guarantees and show that error-attribution performance depends on matching the conformal algorithm to the dataset’s error distribution. Filtering methods are comparatively robust to scoring-function complexity and can substantially reduce inference cost when their directional bias matches the data.

  • Coverage guarantees: Empirical coverage stayed above the lower bound for all methods, while VCP, RF, and TWF respected their respective upper bounds.Coverage was averaged over 1000 random calibration/test splits, with shaded regions showing one standard deviation.
  • Scoring functions: Fine-tuning the scoring function for error classification substantially improved discriminatory power, whereas naive prompting was barely better than random guessing.Prompt engineering produced only small improvements on the combined GSM8k and MATH test sets.
  • Error-distribution matching: Error-attribution efficacy depended on error distribution: RF was strongest on early-error Who&When data, LF on right-dense synthetic data, and TWF on its matched distribution.The comparison used removal rate at 80% target coverage over 1000 random calibration/test splits.
  • Scoring-function robustness: Filtering methods were less sensitive to scoring-function power than VCP and CRSVP, enabling simpler scoring functions when matched to the data’s error distribution.This reduced dependence is particularly useful for real-world datasets where labeling is expensive.
  • Inference cost: RF used 36% of scoring-function calls on left-dense data, while VCP, CRSVP, and TWF used exactly ℓ calls per trajectory.LF and RF evaluate steps directionally until their threshold is crossed, so choosing an algorithm with suitable implicit bias can reduce inference cost.

6 Conclusion … A.2 Left (Right) Filtration

The paper concludes that filtration-based conformal prediction can precisely localize decisive errors in contiguous trajectory sets with lower inference compute, supporting debugging and agent improvement. The appendices provide formal coverage arguments for variable-class vanilla CP and establish nesting and score-membership properties for left filtration.

  • 6 Conclusion: Filtration-based conformal algorithms isolate decisive trajectory errors in contiguous sets while requiring less inference-time computation.The authors position these methods as components of agent-improvement pipelines for more efficient human debugging.
  • A Theorems and Proofs: The appendix supplies complete proofs for the theorems stated in the main text.It formally develops the assumptions and arguments underlying the proposed guarantees.
  • A.1 Vanilla Conformal Prediction: Variable-class vanilla conformal prediction retains valid coverage because the number of possible classes per datapoint does not affect exchangeability of conformal scores.The theorem is presented as an essentially standard conformal-prediction result for completeness.
  • A.1 Vanilla Conformal Prediction: The vanilla CP prediction set includes the true label whenever its score is at most the calibration quantile.The proof reduces coverage to the test score being among the k smallest exchangeable scores, yielding ⌈(n + 1)(1 −α)⌉/(n + 1).
  • A.2 Left (Right) Filtration: Left filtration returns nested valid suffixes as its threshold increases.The nesting follows because higher thresholds enlarge the valid-suffix set, while the candidate suffixes themselves are nested.
  • A.2 Left (Right) Filtration: Under monotonicity, the left-filtration conformal score equals the score of the suffix beginning at the decisive error.Specifically, SLF(x, y∗) = gLF(cj∗:ℓ).
  • A.2 Left (Right) Filtration: For a fixed threshold ˆq, the left-filtration score is at most ˆq exactly when the decisive error belongs to the filtered suffix.The proof shows the infimum is achieved because the finite suffix-score set produces a minimum valid threshold.

A.3 Two-Way Filtration · B Additional Details of Conformal Algorithms

The two-way filtration intersects left- and right-filtration prediction sets, yielding nested contiguous sets with conformal coverage guarantees. The additional CRSVP procedure constructs contiguous trajectory intervals by recursively traversing a binary tree and interpolating scores with uniform noise.

  • A.3 Two-Way Filtration: TWF defines its prediction set as the intersection FTWF(x; q) = FLF(x; q) ∩ FRF(x; q), with empty and universal boundary cases.Specifically, FTWF(x; 0) = ∅ and FTWF(x; ∞) = x.
  • A.3 Two-Way Filtration: For thresholds 0 ≤ q1 ≤ q2, TWF sets are nested: FTWF(x; q1) ⊆ FTWF(x; q2).This follows because both left- and right-filtration sets are nested and intersection preserves inclusion.
  • A.3 Two-Way Filtration: TWF assigns STWF(x, y*) by taking the maximum of the left and right scores, and under monotonicity simplifies them to gLF(cj*:ℓ) and gRF(c1:j*).The maximum-score expression follows from requiring y* to belong to both one-way filtration sets.
  • A.3 Two-Way Filtration: The TWF prediction set satisfies 1 − α ≤ P[y* ∈ CTWF(xn+1; q̂)] < 1 − α + 1/(n+1).The guarantee assumes exchangeable calibration and test data and uses CTWF(xn+1; q̂) = FTWF(xn+1; q̂).
  • A.3 Two-Way Filtration: Because intersection can produce empty prediction sets, TWF can fall back to predicting the single most likely step under gLF or gRF.Coverage remains valid marginally, but empty predictions may be undesirable in practice.
  • B Additional Details of Conformal Algorithms: CRSVP represents a trajectory as a binary tree whose leaves are individual steps and uses gCRSVP to identify the most likely leaf containing the decisive error.If the likely leaf is not the error, the tree is traversed upward to the first ancestor containing it.
  • B Additional Details of Conformal Algorithms: CRSVP interpolates scores with uniform noise u ∼ U(0, 1), recursively traverses upward until crossing q̂, and returns a contiguous subinterval of the trajectory.Parent nodes strictly contain their children, so leaf-to-root prediction sets are nested.

C Implementation Details and Additional Results · C.1 Additional Experiment Details

This block provides additional detail on the datasets, models, and experimental results used in the paper.

  • C.1 Additional Experiment Details: The section expands on the datasets used in the experiments.It identifies datasets as one focus of the additional detail.
  • C.1 Additional Experiment Details: The section provides further information about the models used.Models are explicitly listed among the topics covered.
  • C.1 Additional Experiment Details: The section gives more detail about the experimental results.Experimental results are another stated focus of the section.
  • C Implementation Details and Additional Results: The material concerns implementation details and additional results.The block is titled “C Implementation Details and Additional Results.”
  • C.1 Additional Experiment Details: The additional information covers both experimental inputs and outcomes.Datasets and models represent the experimental setup, while results represent outcomes.
  • C.1 Additional Experiment Details: The section is dedicated to elaborating on the paper’s experimental work.The passage characterizes the content as more detail about datasets, models, and experimental results.

C.1.1 Synthetic Dataset Generation · C.1.2 Context-Engineered LLM Implementation · C.1.3 Fine-tuned Scoring Function

The paper constructs synthetic failed trajectories with controlled decisive-error locations, uses role-specific LLM prompts for step-level scoring, and fine-tunes Qwen3-1.7B to estimate decisive-error likelihoods.

  • C.1.1 Synthetic Dataset Generation: Errors were generated uniformly by selecting one trajectory step and injecting instructions for the agent to fail there.The authors generated 1,200 failed GSM8k trajectories for each of DyLAN and MACNET, and similarly for MATH.
  • C.1.1 Synthetic Dataset Generation: The synthetic datasets explicitly controlled the location of the decisive error step along each execution trajectory.This construction supported evaluation of error attribution at known error locations.
  • C.1.2 Context-Engineered LLM Implementation: The context-engineered implementation used multiple role prompts to obtain step-level scores from different perspectives.The prompts were inspired by ECHO and were intended to improve evaluation robustness through diverse role-specific instructions.
  • C.1.2 Context-Engineered LLM Implementation: An LLM first summarized why the agent failed using the overall trace, then provided that information to four role-specific LLMs for scoring.The average of the role-generated scores was used as the resulting score.
  • C.1.3 Fine-tuned Scoring Function: 4,800 trajectories were generated across datasets and MAS architectures, with 4,000 used for fine-tuning and an 85%/15% training-validation split.The fine-tuning data were equally split across datasets, architectures, and error locations; 800 trajectories were held out.
  • C.1.3 Fine-tuned Scoring Function: The authors fine-tuned an LLM to estimate step-level likelihoods of being the decisive error step.Fine-tuning used MATH and GSM8k trajectories with synthetically injected errors because Who&When had limited data.
  • C.1.3 Fine-tuned Scoring Function: Qwen3-1.7B was fine-tuned for 20 epochs on an ℓ-way classification task producing a scalar score for each trajectory step.Each trajectory had one injected, labeled decisive-error step, and the score represented that step’s decisive-error likelihood.
  • C.1.3 Fine-tuned Scoring Function: Fine-tuning used an Nvidia GeForce RTX 5090, took roughly one day, and consumed less than 16 GB of GPU memory with batch size 32.The authors state that conformal calibration was computationally trivial after scoring calls were completed.

C.1.4 Rollback Experiment Implementation · C.2 Additional Experimental Results

The rollback procedure uses conformal prediction to identify a recovery interval, restores the MAS to its earliest step, and re-executes the task with context and instructions intended to avoid the prior mistakes.

  • C.1.4 Rollback Experiment Implementation: The system generates a conformal prediction set for a failed task or trajectory using the LF algorithm.This set identifies the steps considered relevant for rollback.
  • C.1.4 Rollback Experiment Implementation: Rollback restores the MAS state to the first or earliest step contained in the conformal set.Selecting the earliest identified step defines the recovery point.
  • C.1.4 Rollback Experiment Implementation: The restart includes instructions directing the agent to avoid repeating the same mistakes.These instructions are part of the corrected-version procedure shown in the rollback example.
  • C.1.4 Rollback Experiment Implementation: Figure 8 illustrates the procedure through the failed task, decisive error, final answer, and corrected versions after rollback.The example presents the task evolution before and after recovery.
  • C.1.4 Rollback Experiment Implementation: The agent then restarts the task from that rollback point rather than continuing the failed trajectory.The procedure re-executes the task after state restoration.
  • C.1.4 Rollback Experiment Implementation: During restart, the system supplies additional context from the failed trace to support correction of the final outcome.The failed trace provides information that is carried into re-execution.

C.2.1 Scoring Aggregation Methods · C.3 Prompts Used

The paper compares monotonic scoring aggregations for converting step-wise error likelihoods into set-wise scores, finding that RF performance is largely insensitive to scoring-function details. It also specifies naive, role-based, and rollback prompts that constrain error attribution outputs and guide recovery from detected mistakes.

  • C.2.1 Scoring Aggregation Methods: Step-wise error likelihoods are aggregated into set-wise scores, with normalized summation used in the main-text experiments.The aggregation is intended to obey monotonicity.
  • C.2.1 Scoring Aggregation Methods: Max aggregation uses the highest step-level score and can include a length penalty for sets longer than necessary.Unlike summation, max scores are not length-normalized because single-step maxima tend to be similar across trajectory lengths.
  • C.2.1 Scoring Aggregation Methods: LogSumExp provides a monotonic family interpolating between sum and max through an inverse-temperature parameter β.Higher β approaches max-like behavior, while β near zero approaches a monotonic mean-like function; length normalization is also added.
  • C.2.1 Scoring Aggregation Methods: RF is largely insensitive to the scoring function across Max, LogSumExp, and Sum aggregation variants, evaluators, and DyLAN and MACNET architectures.The comparison uses the Left-Dense GSM8K subset, three evaluator settings, and optional length penalties.
  • C.3 Prompts Used: The naive LLM prompt asks the model to locate the most critical mistake in a multi-agent conversation given the problem, correct answer, and wrong answer.It requires only a single probability between 0 and 1 for the likelihood that the critical error occurs in the segment.
  • C.3 Prompts Used: Context-engineered scoring uses Conservative, Liberal, Skeptical, and Pattern roles with distinct evidence, confidence, alternative-explanation, and reasoning-pattern instructions.The roles respectively emphasize explicit contradictions, indirect causes, missing evidence and alternatives, or repeated coordination issues.
  • C.3 Prompts Used: Role-based prompts require valid JSON containing an investigation summary, primary conclusion, agent, mistake step, confidence, evidence, reason, and alternative explanations.They also require exact evidence quotes and map confidence from 0.0–0.2 as Very unlikely through 0.8–1.0 as Very likely.
  • C.3 Prompts Used: Rollback prompts warn that history through a detected cut index contains wrong information, then ask the MAS to solve the task step by step using mostly correct prior context.The instructions provide the wrong conversation content, previous context, and task query as separate inputs.
Loading 2605.06788v1…