PaperlayerView exact revision on arXiv

Source-linked AI summary

UMAS: System-Level Uncertainty Quantification for Multi-Agent LLM Systems

Hanwen Li, Jinhao Duan, Xiaoshuang Shi, Yue Zhang, Tianlong Chen, Kaidi Xu, Chenxi Yuan

arXiv:2610.33013v1cs.MAcs.AIcs.CLcs.LG

TL;DR

Single-agent UQ methods do not fully exploit MAS interaction structure, while additional sampling can add inference cost. UMAS estimates credibility-weighted uncertainty over an interaction trajectory, and improves AUROC over strong baselines across evaluated settings. The authors also report improved hallucination detection and selective-prediction outcomes.

  • Problem

    Existing UQ methods largely target single-agent settings, leaving how to use MAS interaction structure while keeping inference cost-neutral underexplored.

    Source: §1 · pp. 1–2

  • Method

    UMAS uses generation confidence to estimate peer credibility, update node-level uncertainty across interactions, and aggregate it into a system-level estimate.

    Source: §1 · pp. 2–3

  • Results

    Across backbones and datasets, UMAS improves AUROC over the best baseline; on Llama 4 Scout, Ave is 74.93 for UMAS versus 63.00 for the best baseline.

    Source: §5.2 · p. 7

  • Takeaways & Limitations

    UMAS’s uncertainty improvements translate to downstream gains in selective prediction and hallucination detection in collaborative MAS.

    Source: §1 · p. 3, §5.4 · p. 8

    UMAS requires token-level logits, limiting its applicability to fully black-box APIs that expose only completion text.

    Source: A Limitations · p. 13

Abstract

from arXiv · show

Multi-Agent Systems (MAS) are a promising paradigm for improving the reasoning capabilities of large language models, yet they remain prone to hallucinations and erroneous outputs. Uncertainty Quantification (UQ) is crucial for assessing system reliability, but existing methods are largely designed for single-agent settings; how to exploit MAS structure for UQ while keeping inference cost-neutral remains underexplored. We propose UMAS, a parameter-free, cost-neutral, system-level UQ framework that estimates the credibility of inter-agent influence from each generation's intrinsic confidence. UMAS uses these credibility signals to modulate how uncertainty is updated at each node, and then aggregates node-level uncertainty into a system-level estimate. Extensive evaluations on Debate and DyLAN across diverse benchmarks show that UMAS consistently outperforms state-of-the-art baselines by an average of 10.54 AUROC points. Beyond uncertainty estimation, UMAS enables hallucination detection and uncertainty-aware answer selection, improving MAS accuracy by up to 12.6 points on specific tasks while enhancing reliability. Code is available at https://github.com/Dby52-04/UMAS.

1 Introduction

The introduction argues that standard single-agent uncertainty quantification can be unreliable for multi-agent systems because it overlooks interaction trajectories and peer influence. UMAS estimates uncertainty across that trajectory, using peer credibility to update node confidence, and improves uncertainty reliability across evaluated settings.

1 Introduction

  • UMAS estimates node-level uncertainty at each interaction round, calibrates updates using peer credibility inferred from intrinsic confidence, and aggregates estimates into system-level uncertainty.The framework is presented as parameter-free and cost-neutral.Source: §1 · pp. 2–3
  • 10.54 AUROC points is UMAS’s average gain over the best-performing baseline across two backbones.The evaluation covers Debate and DyLAN across five tasks.Source: §1 · p. 2
  • Across Debate and DyLAN, two LLM backbones, and five benchmarks, UMAS improves uncertainty estimates and supports selective prediction and hallucination detection.The authors report downstream gains in reliability and system performance.Source: §1 · pp. 2–3
  • Standard single-agent UQ can become less reliable on MAS outputs even as MAS improves task accuracy, because terminal-answer estimates ignore interaction structure.The introduction attributes the mismatch to uncertainty propagation and inter-agent influence across multi-round trajectories.Source: §1 · pp. 1–2

2 Related Works

Prior UQ work spans token-probability, multi-sample consistency, hybrid, and sequential approaches, while MAS research develops collaborative and adaptive interaction architectures. UMAS focuses on uncertainty in collaborative updates.

2 Related Works

  • UQ methods include efficient token-probability approaches, multi-sample semantic consistency methods, and hybrids that combine confidence and consistency.The related work notes that token-probability methods can overlook semantic variability, while consistency methods require multiple samples.Source: §2 · p. 3
  • Prior multi-step UQ includes SAUP, which weights uncertainty by step but requires test-domain labels, and UProp, which samples single-agent trajectories.UMAS instead scores one realized multi-agent trajectory.Source: §2 · p. 3
  • MAS research includes role-playing, debate, dynamic-team, and query-specific architectures, while this work targets collaborative updates as a basis for MAS UQ.DiverseAgentEntropy is cited as a related approach using uncertainty metrics to prioritize reliable agents.Source: §2 · p. 3

3 Preliminary

The preliminaries describe sequence probability as a confidence basis and represent a multi-agent system as a directed graph of agent states connected by information dependencies.

3 Preliminary

  • Maximum Sequence Probability (MSP) is the probability of an output sequence given its input, factorized over tokens; its negative logarithm is a standard uncertainty score.MSP is described as a computationally efficient confidence proxy.Source: §3 · p. 3
  • The MAS is represented as a directed graph whose vertices record agent states across communication rounds.The graph has N agents and M rounds, with rounds equivalent to layers.Source: §3 · p. 3
  • An edge represents an information dependency when a node’s generation depends on predecessor states from the previous round.The vertex set spans agents and rounds.Source: §3 · p. 3

4 Method

UMAS derives confidence from each generated response, tracks credibility-weighted confidence updates across agent interactions, and averages dynamic confidence over the interaction graph to produce system-level uncertainty. UMASFIX separates the contribution of trajectory aggregation from credibility-aware updating.

4 Method

  • Each node’s raw confidence is its response’s log-probability given its input, and UMAS tracks a dynamic confidence adjusted according to peer evidence.A larger, less negative log-probability indicates higher confidence.Source: §4 · p. 4

4.1 Peer Credibility Estimation

  • UMAS estimates each update edge’s credibility from the peer’s dynamic confidence relative to the updater’s prior dynamic confidence.This relative score is a heuristic for peer influence, not a truth detector.Source: §4.1 · p. 4

4.2 Credibility-aware Confidence Updating

  • A bounded trust weight regulates the moving-average confidence update: low peer credibility retains historical confidence, while high credibility lets confidence track the current response.The normalized incoming edge scores are averaged and complemented to form the node-level trust weight.Source: §4.2 · pp. 4–5

4.3 System-level Uncertainty Aggregation

  • UMAS produces system-level uncertainty as the negative average dynamic confidence over the interaction graph, yielding a length-normalized score that prevents one node from dominating.It uses single-response confidence signals already produced during generation, avoiding additional sampling.Source: §4.3 · p. 5

4.4 UMASFIX : Disentangling Aggregation from Credibility Updating

  • UMASFIX uses trajectory aggregation with raw node confidence but skips credibility updates, allowing its comparison with UMAS and single-output baselines to separate their contributions.The authors report that both design choices contribute substantially and that UMAS requires no extra generations, model calls, or auxiliary models.Source: §4.4 · p. 5

5 Experiments

Across the evaluated MAS architectures, backbones, and benchmarks, UMAS improves uncertainty reliability and supports better response selection and hallucination detection. The experiments also examine performance across system and model scales, confidence metrics, and computational efficiency.

5.1 Experimental Setup

  • The evaluation covers Debate and DyLAN, two LLM backbones, five benchmarks, eight UQ baselines, and AUROC as the main metric.Multi-sample baselines use five samples; correctness is exact match for four tasks and pass@1 for MBPP.Source: §5.1 · pp. 6–7

5.2 Main Result

  • On Llama 4 Scout, the best baseline reached Ave 63.00 versus UMASFIX 70.32 and UMAS 74.93; on GPT-4o mini, the respective results were 64.86, 71.49, and 74.02 AUROC.Trajectory-level aggregation contributes +7.32/+6.63 AUROC over the best baseline across the two backbones; credibility-aware updating adds +4.61/+2.53, with larger contributions when peer confidences vary widely.Source: §5.2 · p. 7

5.3 Selective Prediction

  • 5.5–12.6 points of absolute accuracy gains over the original MAS were achieved by uncertainty-based selection, with UMAS or UMASFIX best in every evaluated cell.Selection chooses the least-uncertain of five independently generated trajectories.Source: §5.3 · p. 7, Table 2 · p. 7

5.4 Ablation Study

  • UMASFIX and UMAS outperform all baselines in specificity across varying F1 constraints, and UMAS is nearer the upper-right trade-off region in most settings.Hallucination detection flags responses whose uncertainty exceeds a threshold selected using validation data.Source: §5.4 · p. 8, Figure 2 · p. 8
  • Across agent counts and communication rounds from 2 to 4, UMAS achieves the best AUROC in every evaluated system-scale setting.UMASFIX and UMAS also consistently outperform all baselines across these scales.Source: §5.4 · p. 8, Figure 3 · p. 8
  • Across seven Llama variants from 1B to 400B parameters, UMAS consistently outperforms most baselines on four datasets.For models below 8B, the evaluation uses MMLU because MMLU-Pro exceeds their reasoning capability.Source: §5.4 · p. 8, Figure 4 · p. 9
  • With PE or PPL as the node-level confidence metric, UMAS beats the raw metric in 11 of 12 configurations.The exception is PPL on MBPP, where results are near parity: 59.87 versus 60.03.Source: §5.4 · pp. 8–9, Table 3 · p. 9
  • UMAS uses single-trajectory logits and lightweight CPU-bound probability calculations, avoiding multi-sample consistency methods' latency and hardware dependencies.The efficiency comparison reports latency and AUROC for Llama 4 Scout under Debate on MMLU-Pro.Source: §5.4 · p. 9, Table 4 · p. 9

6 Conclusion

UMAS addresses unreliable single-agent UQ on multi-agent outputs by aggregating uncertainty over the interaction graph and accounting for the credibility of peer influence.

6 Conclusion

  • Standard single-agent UQ can become unreliable on MAS outputs because it ignores trajectory-level interactions and influence credibility.UMAS aggregates uncertainty across the full interaction graph rather than relying only on the terminal response; this is the larger contributor to its gains.Source: §6 · p. 9
  • UMAS estimates peer-influence credibility from each generation’s intrinsic confidence, with the added benefit most pronounced when peer reliability varies.Credibility-aware weighting supplements the gains from full-graph aggregation.Source: §6 · p. 9
  • Across MAS paradigms, backbones, and benchmarks, UMAS consistently improves AUROC and benefits selective prediction and hallucination detection without extra sampling.Source: §6 · p. 9

A Limitations

The appendix describes UMAS’s limitations and responsible-use considerations, then specifies the collaborative MAS protocols used in the study.

A Limitations

  • UMAS requires token-level logits, limits its evaluation to collaborative protocols, and assumes cooperative peers; shared high-confidence misconceptions can evade its credibility signals.The cooperative-peer rule is not directly transferable to role-asymmetric proposer–critic settings without considering the critique’s content and role.Source: A Limitations · p. 13

B Broader Impact

  • UMAS estimates self-reported uncertainty, not factual correctness, so low uncertainty is not a guarantee of accuracy and ground-truth verification remains necessary.The paper also notes that the estimator could support both quality assurance and adversarial probing.Source: B Broader Impact · p. 13

C.1 Architecture

  • Debate uses independent candidate answers followed by iterative peer critique and revision, with the group reaching a consensus through prompting and context sharing.The framework is inspired by the “Society of Mind” theory.Source: §C.1 · p. 13
  • DyLAN dynamically organizes task-specific agents in a feed-forward communication structure, using an intermediate ranker to assess contributions and deactivate less effective agents.The study uses 4 agents and 3 rounds for DyLAN, versus 3 agents and 3 rounds for Debate.Source: §C.1 · p. 13

C.2 Task Accuracy

This section examines task accuracy in Debate and DyLAN relative to the single-agent Base across backbone–dataset pairs, where both systems improve accuracy in nearly every pair. It also illustrates UMAS’s answer selection and describes its three-phase uncertainty-estimation process.

C.2 Task Accuracy

  • Base uses the first agent’s first-round Debate response, generated independently before peer communication with decoding identical to the MAS run.This baseline is used for a fair comparison and represents single-agent inference.Source: §C.2 · p. 13, Table 5 · p. 14
  • Both Debate and DyLAN improve over Base across nearly every backbone–dataset pair, with task accuracy reported for five benchmarks.Table 5 compares Base, Debate (3 agents × 3 rounds), and DyLAN (4 agents × 3 rounds); the supplied passages do not provide cell values.Source: §C.2 · p. 13, Table 5 · p. 14

C.4 Example

  • In the Norcross example, the solutions select (E) moral agent because moral agency requires moral reasoning, unlike moral-patient status.The example explains that a moral patient can receive moral consideration without being capable of moral reasoning.Source: §C.4 · pp. 22, 25

D UMAS Algorithm

  • UMAS computes raw log-MSP confidence at each node, derives trust weights from sigmoid-transformed relative-confidence scores averaged over peers, updates confidence with a credibility-weighted moving average, and aggregates trajectory confidences by their negative average.The algorithm’s three phases are raw confidence computation, credibility-aware updating, and system-level aggregation.Source: D UMAS Algorithm · p. 26

E Dataset

This appendix describes the benchmark datasets and semantic-consistency optimization, documents the experimental setup, and illustrates confidence updating. It also reports an evaluation on Qwen 3.5-9B.

E Dataset

  • The five benchmark datasets cover general knowledge, algebra, medical multiple-choice questions, Python programming, and numerically challenging multi-step math problems.GSM-Hard uses larger, less common operands than GSM8K.Source: E Dataset · pp. 27–28

E.1 Examples

  • Table 7 presents representative examples from the QA datasets used in the study.The passage introducing the table identifies it as a set of dataset examples.Source: Table 7 · p. 28, §E.1 · p. 28

F Computational Optimization for Semantic Consistency

  • 55.50% vs. 48.90%: using extracted final answers raises SE’s average AUROC on Llama 4 Scout from 48.90% to 55.50%, while SentSAR rises from 55.28% to 57.45%.The condensed representation also reduces GPU memory requirements; verbose MAS generations can hinder entailment models.Source: F Computational Optimization for… · p. 29

G Hardware

  • About 20 minutes: the baseline suite takes this wall-clock time per setting on two A100 GPUs, while UMAS post-processing takes negligible CPU time and zero GPU-hours.LLM inference used API-hosted models; locally run entailment and similarity models used the compute node.Source: G Hardware · p. 29

Confidence Updating for Agent #3 at Round 2

  • −21.16 vs. −41.02: UMAS updates Agent #3’s Round-2 confidence using peer-trust weight w = 0.29, reducing its uncertainty relative to raw MSP.The passage attributes the update to preventing an overly verbose generation from masking agreement with the correct consensus.Source: Confidence Updating for Agent #3 at… · p. 31

I.1 Additional Backbone: Qwen 3.5-9B

  • 75.75 vs. 68.89: UMAS’s mean AUROC on Qwen 3.5-9B exceeds the strongest baseline by 6.86 points; UMAS or UMASFIX leads in nine of ten columns.The exception is Debate-MMLUP, where G-NLL scores 85.75 versus UMASFIX at 85.04.Source: Table 8 · p. 37, §I.1 · p. 37

I.2 Baselines Aggregated over Intermediate Generations

This section compares trajectory-aggregated consistency baselines with UMAS and tests whether gains persist across aggregation rules and methods.

I.2 Baselines Aggregated over Intermediate Generations

  • 45 generations per query: extending consistency-based baselines across three agents, three rounds, and five samples per node is costly; single-sample trajectory aggregation is the UMASFIX comparison.UMAS further replaces uniform aggregation with credibility-aware updates.Source: §I.2 · p. 37
  • 5.15 points: UMAS’s average AUROC exceeds the strongest consistency-based contender when those baselines are aggregated across every generation in the trajectory.The comparison covers Llama 4 Scout under Debate; Table 9 reports AUROC by dataset for trajectory-mean baselines and UMAS.Source: §I.2 · p. 37, Table 9 · p. 38
  • Trajectory averaging improves every recorded method; UMASFIX has the strongest aggregated cost-free log-probability score, and UMAS improves on it further.Table 10 compares final-response scores with trajectory means across 20 settings.Source: §I.2 · pp. 37–38, Table 10 · p. 38

I.3 Expected Calibration Error

On Llama 4 Scout under Debate, UMAS also performs best on average ECE, extending its reported evaluation beyond AUROC ranking quality.

I.3 Expected Calibration Error

  • UMAS achieves the best average Expected Calibration Error on Llama 4 Scout under Debate, where lower ECE is better.The reported measure assesses calibration, unlike AUROC, which measures ranking quality.Source: Table 11 · p. 38, §I.3 · p. 38

I.5 Alternative Aggregation Strategies

UMAS’s gains persist across tested aggregation strategies, while final-round-only scoring is weaker than uniform trajectory averaging. The reported peer-credibility weights vary across inputs and are approximately normally distributed.

I.5 Alternative Aggregation Strategies

  • All tested UMAS aggregation variants substantially outperform every baseline in the main results, regardless of aggregation strategy.The alternatives include depth-weighted, exponential-weighted, minimum, and maximum aggregation; Table 13 reports AUROC on GPT-4o mini under Debate.Source: §I.5 · p. 40, Table 13 · p. 40
  • 72.74 vs. 74.47 AUROC: final-round-only scoring is weaker than uniform averaging across the 20 settings, and early nodes retain useful signal when answers are later corrected.Final-round-only scoring removes the effect of early-round errors that are corrected later.Source: §I.5 · pp. 40–41

I.6 Distribution of Weight

  • Peer-credibility weights vary across inputs and are approximately normally distributed, with extremely high- or low-credibility updates occurring relatively rarely.Figure 6 presents the distribution of w(t)ᵢ.Source: §I.6 · p. 41

I.7 Robustness to the Correctness Definition

UMAS’s ranking is robust across alternative correctness definitions, while the analysis covers the 16 short-answer settings rather than the program-execution task.

I.7 Robustness to the Correctness Definition

  • The robustness check recomputes AUROC using five answer-extraction strategies and strict and lenient aggregates, with E1 and E5 agreeing on 99.5% of correctness labels.E1 and E5 agree on 97.8% of extracted answers.Source: §I.7 · p. 41
  • The analysis covers the four short-answer datasets, or 16 of Table 1’s 20 settings, because MBPP is scored by pass@1 execution rather than answer extraction.Source: §I.7 · p. 41
  • UMAS ranks first and UMASFIX second under all seven correctness definitions, with UMAS’s margin over the best baseline never below +8.19 AUROC (E4).Table 14 reports mean AUROC over the 16 short-answer settings.Source: §I.7 · p. 41, Table 14 · p. 41

I.8 Controlling for Response Length

Length controls test whether UMAS’s uncertainty estimates simply track response verbosity. UMAS retains an advantage within length quartiles and adds signal beyond length and MSP, while length normalization lowers its AUROC.

I.8 Controlling for Response Length

  • The controls test whether UMAS merely tracks verbosity across the 20 settings, using token count of the scored text as response length.Source: §I.8 · p. 41
  • 69.26 AUROC: after length stratification, UMAS retains an 8.20-point margin over the strongest baseline, down from 11.95 points.Length-stratified AUROC is computed within question-length quartiles and pooled by bin size; all methods except PPL lose AUROC, but rankings remain unchanged.Source: §I.8 · p. 41, Table 15 · p. 42
  • +5.56 AUROC: UMAS adds the most over a logistic regression using length alone, significant in 14 of 20 settings.Adding UMAS to a model using standardized MSP plus terminal and trajectory length still gains +4.05 AUROC, significant in 14 of 20 settings.Source: §I.8 · p. 41
  • Length normalization lowers UMAS AUROC from 74.47 to 69.96 and UMASFIX AUROC from 70.91 to 67.74, so most of the gain survives.The paper keeps unnormalized log-probabilities, consistent with Aichberger et al. [2].Source: §I.8 · pp. 41–42
Loading 2610.33013v1…