PaperlayerView exact revision on arXiv

Source-linked AI summary

Estimation of Propagated Risk in LLM-based Multi-Agent Systems

Shreyan Ganguly, Souradeep Chattopadhyay, Qisai Liu, Soumik Sarkar

arXiv:2610.33282v1cs.AIcs.MA

TL;DR

Downstream error risk depends on how receivers use upstream information, while existing propagation-aware approaches do not model interaction effects conditioned on sender correctness. IE-Prop learns separate effects for correct and incorrect senders and recursively propagates risk; across evaluated architectures and benchmarks, it generally improves downstream error-risk discrimination over receiver-local and propagation-based baselines.

  • Problem

    Existing propagation-aware approaches inherit upstream risk but do not model whether a realized sender–receiver interaction relates differently to receiver error depending on sender correctness.

    Source: §1 · pp. 1–2

  • Method

    IE-Prop learns separate interaction effects for correct and incorrect senders, combines them using estimated sender risk, and recursively propagates risk through the execution graph.

    Source: §1 · p. 2

  • Results

    Across three multi-agent architectures and three question-answering benchmarks, IE-Prop generally improves downstream error-risk discrimination over receiver-local and existing propagation-based baselines.

    Source: §9 · p. 8

  • Takeaways & Limitations

    Realized inter-agent interactions provide predictive information beyond receiver-local evidence, while the value of recursive risk propagation depends on information-flow structure.

    Source: §1 · p. 3, §8 · p. 8

    Evaluation covers three four-agent architectures, three question-answering benchmarks, and one model family, leaving generalization to larger and more diverse systems for future work.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Language Model (LLM) based multi-agent systems introduce a distinctive reliability challenge in which actions taken by upstream agents can affect the correctness of downstream agents. While prior work has investigated error propagation and developed methods for estimating uncertainty inherited from upstream agents, existing approaches do not explicitly model how realized sender-receiver interactions contribute to downstream error risk under different sender correctness states. In this work, we study agent-level error-risk estimation from multi-agent execution traces. We propose Interaction-Effect Propagation (IE-Prop), an interaction-conditioned model that learns separate effects for correct and incorrect upstream agents. For each sender-receiver interaction, the model combines these conditional effects using the sender's estimated error risk, and recursively propagates the resulting risk estimates through the execution graph. We evaluate our approach across three multi-agent architectures and three benchmarks. Across all evaluated architectures, our approach raises the final-node AUROC over the established baseline by approximately 0.05 points on average. Furthermore, tracking risk across multiple agent steps contributes a distinct +0.03 AUROC gain in sequential systems. These results show that realized inter-agent interactions provide predictive information beyond receiver-local evidence, while the value of multi-step risk tracking depends on the system's information-flow structure.

1 Introduction

Downstream error risk depends on realized sender–receiver interactions, not only upstream correctness. IE-Prop models these interactions and recursively propagates risk, addressing a gap in prior propagation-aware approaches.

2 Related Work

  • PropUQ-MAS treats inherited risk as a contamination contribution that cannot lower receiver risk as parent risk rises, motivating sender-correctness-conditioned interaction effects.IE-Prop instead learns separate effects for realized interactions under correct and incorrect sender states.Source: §1 · p. 2, §2 · p. 3

1 Introduction

  • IE-Prop learns separate signed interaction effects for correct and incorrect senders, combines them using estimated sender risk, and recursively propagates risk through the execution graph.The recursive procedure learns from node-level correctness labels without explicitly maintaining a joint distribution over upstream error states.Source: §1 · p. 2
  • Downstream risk depends on realized inter-agent interactions, while the size and source of the benefit vary with graph, data, and receiver evidence.The results also indicate that receiver responses matter even when upstream answers are correct.Source: §1 · p. 3

2 Related Work

  • Prior work estimates confidence for isolated generations or uncertainty across trajectories and whole-system executions; this paper focuses on realized sender–receiver interactions.Related work also studies preserving information during inter-agent communication in decentralized settings.Source: §1 · p. 2, §2 · p. 3
  • IE-Prop uses an additive log-odds structure related to logistic graphical models, enabling compositional risk propagation without explicitly maintaining a joint distribution over agents’ error states.Each incoming interaction contributes a correctness-conditioned adjustment to receiver error log-odds.Source: §2 · p. 4

3 Interaction-Effect Error-Risk Estimation

IE-Prop estimates agent error risk from completed execution traces by combining receiver-local evidence with realized sender–receiver interactions. It recursively propagates risk through the execution graph while conditioning interaction effects on sender correctness.

State-conditioned interaction model

  • Each realized sender–receiver interaction has separate signed effects on receiver error log-odds for correct and incorrect senders, allowing interactions to represent propagation or correction.The interaction representation uses the realized exchange, including the receiver’s output, so scoring is post-hoc.Source: §3 · p. 4

Inference with uncertain sender states

  • At inference, estimated sender error risks weight the two state-specific interaction effects, whose expected contributions are added to receiver-local evidence.The model averages interaction contributions in logit space before applying the sigmoid to obtain receiver risk.Source: §3 · pp. 4–5

Recursive risk propagation

  • Risk estimates are computed recursively in topological order, with each non-root node using its parents’ recursively estimated risks.In a sequential graph, a node’s risk can therefore reflect interactions further upstream through its parent’s risk.Source: §3 · p. 5

Approximation to marginal inference

  • The update approximates marginal inference: it applies the sigmoid to expected receiver log-odds rather than taking the expectation of the sigmoid.It costs O(k) rather than 2^k for k parents and is exact when both effects of each uncertain edge coincide.Source: §3 · p. 5

4 Training

The estimator is trained end to end on execution traces using node-level correctness labels. Joint optimization covers the receiver-local and interaction models, with evaluation on held-out questions.

Training

  • The model parameters are optimized jointly by minimizing binary cross-entropy between recursively predicted node risks and correctness labels across labeled nodes.Training uses execution traces generated from training questions.Source: §4 · p. 5
  • Gradients propagate through recursive risk computation to train both the receiver-local and interaction models; evaluation uses traces from held-out questions.The procedure also includes K-fold prediction.Source: §4 · p. 5

5 Evaluation Design

The evaluation compares receiver-local and interaction-based risk estimators across multi-agent architectures, benchmarks, and model sizes. It also tests whether sender-state conditioning and recursively accumulated parent risk add predictive value.

5 Evaluation Design

  • The main benchmark compares Receiver-Text, which uses receiver task, role, and output but ignores incoming messages, against estimators that analyze realized interactions.Receiver-Only uses verbalized uncertainty, Maximum Sequence Probability, and prediction entropy; Free learner estimates an unconstrained interaction adjustment, while IE-Prop mixes state-conditioned effects using parent risk.Source: §5 · pp. 5–6
  • Recursion Control replaces propagated parent risk with the parent’s local risk to test whether recursively accumulated upstream information adds value beyond local evidence and the observed interaction.PropUQ-MAS is also reported as an external reference, with final-node AUROC as the metric.Source: §5 · p. 5
  • Table 2 reports seed-averaged final-node AUROC for in-domain GSM8K and MedQA settings across Qwen3 0.6B, 1.7B, and 4B models.Its columns are grouped by dataset, uncertainty signal, and model size, with best and second-best values marked within each topology.Source: Table 2 · p. 6

6 Experimental Setup

  • The study evaluates sequential, fully connected, and specialist-hierarchy topologies using four-agent systems instantiated with Qwen3 models.In the sequential pipeline, each agent sees only its immediate predecessor; the fully connected system attends to preceding outputs, and the final specialist-hierarchy summarizer aggregates parallel responses.Source: §6 · p. 6
  • GSM8K, MedQA, and text-only WebQA are evaluated across dataset, topology, and model-size combinations, yielding 27 settings per method.Each agent receives a binary correctness label by comparing its extracted final answer with ground truth.Source: §6 · p. 6
  • In-domain evaluation trains on a dataset’s training split and tests on its held-out split, while transfer trains on two datasets and tests on the third.Transfer is conducted for each architecture and model size.Source: §6 · p. 6

7 Results

Across the evaluated settings, realized inter-agent interactions add predictive information beyond receiver-local evidence, and recursive propagation adds further information in sequential systems. The size of these benefits varies by architecture, model size, dataset, and receiver evidence.

7.1 Interaction information beyond receiver text

  • IE-Prop exceeds Receiver-Text in 46 of 54 combinations; mean AUROC gains appear at every evaluated model size across sequential, fully connected, and hierarchical systems.Across model sizes, gains were +0.015, +0.030, and +0.025 sequentially; +0.036, +0.060, and +0.123 fully connected; and +0.023, +0.087, and +0.068 in the hierarchy.Source: §7.1 · p. 7
  • On WebQA, IE-Prop improves over Receiver-Text in all 12 evaluated settings and achieves the highest AUROC in nine, but PropUQ-MAS leads at 4B sequentially.IE-Prop leads in most hierarchical settings and at the two smaller sequential model sizes; Figure 3 shows the sequential and hierarchical comparisons across Qwen3 sizes and uncertainty signals.Source: §7.1 · p. 7, Figure 3 · p. 7

7.2 State-conditioned effects

  • IE achieves higher seed-averaged AUROC than the Free Learner in all 54 displayed combinations, with average gains of approximately +0.026 sequentially, +0.021 fully connected, and +0.019 hierarchically.This supports sender-state conditioning as a useful predictive inductive bias in the evaluated settings, but does not establish distinct propagation and correction mechanisms.Source: §7.2 · p. 7

7.3 Downstream risk discrimination with correct upstream context

  • With correct upstream answers on sequential MedQA traces, IE assigns higher predicted risk to incorrect than correct receivers; AUROC is 0.68, 0.72, and 0.88 for the 0.6B, 1.7B, and 4B models.The result demonstrates predictive utility when the upstream answer is correct, but not independence from estimated sender risk, which the model still uses when combining interaction effects.Source: §7.3 · pp. 7–8

7.4 Contribution of recursive risk propagation

  • Recursive risk propagation improves final-node AUROC in all nine sequential dataset–model-size combinations, increasing the overall mean from 0.631 to 0.655 (+0.024).The average benefit is similar across model sizes; gains averaged over sizes are +0.026 on GSM8K, +0.026 on MedQA, and +0.019 on WebQA, showing information beyond local parent uncertainty and the immediate interaction.Source: §7.4 · p. 8, Figure 5 · p. 8

8 Discussion

The discussion frames downstream risk as dependent on both upstream correctness and receiver response, with interaction modeling and recursive propagation offering complementary benefits whose relative value depends on topology. The authors also bound the evidence to a limited evaluation scope and note deployment, labeling, and modeling caveats.

8 Discussion

  • Interaction modeling can improve discrimination without multi-hop accumulation in shallow hierarchies, while sequential systems benefit from recursively accumulated risk because earlier interactions pass through intermediate agents.The conclusion reports generally improved discrimination over receiver-local and existing propagation-based baselines across three architectures and three question-answering benchmarks.Source: §8 · p. 8, §9 · p. 8

Limitations

  • The evaluation covers three four-agent architectures, three question-answering benchmarks, and one model family, leaving generalization to larger and more diverse systems for future work.Differences in agent roles and prompting protocols also limit direct comparisons across architectures.Source: Limitations · p. 9
  • The additive interaction model approximates marginal inference, learned effects are predictive associations rather than causal relationships, and the estimator relies partly on automated correctness labels.The distinct contributions of sender-state conditioning and recursive propagation are not fully established in every topology.Source: Limitations · p. 9

Ethical Considerations

  • Risk estimates may create false confidence in incorrect outputs; benchmark evaluation guarantees neither correctness nor safety in real-world deployment, and the approach incurs computational and environmental costs.The authors also caution that estimates may not generalize reliably to unseen domains, model families, or architectures.Source: Ethical Considerations · p. 9

A Approximation Bound

The approximation replaces exact averaging of the receiver’s sigmoid risk over parent states with a scalar update, whose local discrepancy is bounded by the variance of the receiver logit. The bound applies even when parent states are correlated, but does not account for errors in propagated parent marginals.

A Approximation Bound

  • The discrepancy between Eπ[σ(Z_v)] and σ(µ) is bounded by one-half the maximum absolute second derivative of σ times Var(Z_v).The Taylor remainder gives the bound, with the linear term vanishing after taking expectations.Source: A Approximation Bound · p. 12
  • The bound holds for any parent-state distribution, including correlated states, and is exact when Var(Z_v) = 0.The error vanishes when the two effects coincide and grows with strong, uncertain, positively co-varying parent effects.Source: A Approximation Bound · p. 12
  • In a single-parent sequential system, variance is largest for a maximally uncertain parent; the fully connected system’s final agent also has a covariance term among its three parents.The three parents share the first agent as an ancestor.Source: A Approximation Bound · p. 12

B.1 Reverse-KL barycenter

The scalar composition rule has a reverse-KL barycenter interpretation, while its recursive risk update is an approximation to exact marginalization. Cross-dataset transfer shows that interaction information can remain useful beyond training datasets, but does not uniformly outperform a propagation-based reference.

B.1 Reverse-KL barycenter

  • The scalar update is the reverse-KL barycenter of state-conditioned Bernoulli receiver distributions, but it is not exact marginalization or a claim of empirical superiority.It depends on parent marginals even when parents are dependent; replacing exact marginals with propagated ones also accumulates upstream errors.Source: §B.1 · p. 12

C Induced Probabilistic Model

  • Conditioned on the completed trace, the nodewise conditionals form a Bayesian network over error indicators, but exact inference can require joint parent-state tables.Variable elimination is exponential in induced width; exact marginalization is cheap in the controlled sequential and fully connected systems described here.Source: C Induced Probabilistic Model · pp. 12–13, Figure 6 · p. 13

D Controlled QA: Details and Full Results

  • Training uses every labeled node as a target, and each fold model is selected by validation final-node AUROC before fold predictions are averaged.The shared setup uses five question-level folds and up to 30 epochs with patience 5.Source: D Controlled QA: Details and Full… · p. 13

E Cross-dataset transfer

  • IE improves over Receiver-Only most clearly on held-out MedQA across all three graphs, while its advantage over PropUQ-MAS is less consistent.Paired intervals overlap zero in seven of nine graph–held-out-dataset comparisons; sequential GSM8K favors IE, while fully connected GSM8K favors PropUQ-MAS.Source: E Cross-dataset transfer · p. 13, Figure 7 · p. 14

F Oracle-Supervised Sender-State Conditioning

Oracle-conditioned supervision substantially increases oracle-state swap gaps, supporting improved alignment between sender labels and learned heads. Deployed predictive performance, however, remains largely unchanged, and the swap-gap evidence does not establish causal error-transmission or correction effects.

F Oracle-Supervised Sender-State Conditioning

  • Oracle supervision leaves predictive performance largely unchanged, with mean deployed AUROC changes of +0.004, approximately 0, and −0.001 in sequential, fully connected, and hierarchical systems.The comparison is relative to the original IE model.Source: F Oracle-Supervised Sender-State… · p. 14
  • At λ = 0.1, oracle-state swap gaps reach +0.570, +0.740, and +0.679 for sequential, fully connected, and hierarchical systems, respectively.The diagnostics use cross-fitted training traces that also served in fold-level validation; independent test traces corroborate the trend in the sequential system.Source: F Oracle-Supervised Sender-State… · p. 14
  • Large swap gaps support improved sender-state alignment but do not identify causal transmission or correction effects, since sender–receiver correctness correlations can also produce them.At test time inference still uses the original soft recursive mixture without sender labels.Source: F Oracle-Supervised Sender-State… · p. 14

G Local Uncertainty Signals

The local risk estimator uses verbalized uncertainty, Maximum Sequence Probability, or prediction entropy as separate inputs. These signals capture self-reported confidence, generated-sequence probability, and token-level predictive dispersion, respectively.

G Local Uncertainty Signals

  • Each signal is evaluated independently as the local input to the risk estimator, whose local risk prediction also serves as the Receiver-Only baseline.All three signals use the same experimental protocol.Source: G Local Uncertainty Signals · pp. 14–15
  • Verbalized uncertainty converts the agent’s self-reported confidence into an uncertainty score, so larger values indicate greater self-reported uncertainty.The confidence value ranges from 0 to 1, with larger values indicating greater confidence before conversion.Source: G Local Uncertainty Signals · pp. 14–15
  • Maximum Sequence Probability uses length-normalized probability of the generated token sequence, with lower sequence probability indicating greater uncertainty.Length normalization reduces dependence on response length.Source: G Local Uncertainty Signals · p. 15
  • Prediction entropy measures uncertainty in next-token distributions, and the sequence-level signal aggregates token entropies over the generated output.Higher entropy indicates a more dispersed predictive distribution; unlike semantic entropy, it does not group responses by semantic equivalence.Source: G Local Uncertainty Signals · p. 15

H Evaluation Metric

The section evaluates error-risk estimates with AUROC on held-out final-node responses. It defines the metric as discrimination between incorrect and correct responses across thresholds and outlines the propagation and training procedure.

H Evaluation Metric

  • Each final-node response is labeled incorrect (E_i = 1) or correct (E_i = 0), and its predicted error risk is denoted r̂_i.AUROC summarizes the ROC curve, which plots true positive rate against false positive rate as threshold τ varies.Source: H Evaluation Metric · p. 15
  • AUROC is computed on held-out final-node responses, with seed-averaged results reported where applicable.The metric evaluates risk ranking across decision thresholds rather than probability calibration.Source: H Evaluation Metric · p. 15
  • An AUROC of 0.5 corresponds to random discrimination, while 1.0 indicates perfect separation between incorrect and correct responses.Higher AUROC indicates better discrimination of downstream errors.Source: H Evaluation Metric · p. 15
  • Propagation encodes each sender-receiver interaction from task, roles, and visible outputs, then processes agents in topological order using local signals and parent risks.The interaction encoder is frozen; root risks are returned as σ(b_v).Source: H Evaluation Metric · p. 16
  • Training uses question-level K-fold splits, binary cross-entropy over node risks, and AdamW updates, retaining the epoch with the best held-out final-node AUROC.The best epoch's node logits are recorded for each fold.Source: H Evaluation Metric · p. 16

Related papers

Loading 2610.33282v1…