PaperlayerView exact revision on arXiv

Source-linked AI summary

Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantification

Kimia Hamidieh, Veronika Thost, Walter Gerych, Mikhail Yurochkin, Marzyeh Ghassemi

arXiv:2604.17112v1cs.AI

TL;DR

Self-consistency can miss confident errors when a model repeatedly gives the same wrong response. The paper estimates epistemic uncertainty from cross-model semantic disagreement and combines it with aleatoric uncertainty; across tasks, total uncertainty improves on AU.

  • Problem

    Self-consistency-based uncertainty can fail when a model repeatedly produces the same incorrect answer with high confidence.

    Source: §1 · p. 1

  • Method

    The paper estimates epistemic uncertainty from generated-text disagreement across a small, scale-matched ensemble and adds it to aleatoric uncertainty as total uncertainty.

    Source: §1 · pp. 1–2

  • Results

    Across ten long-form tasks and five reference models, TU consistently outperforms AU in AUROC and selective abstention; the largest reported AUROC gain is +0.15 on HotpotQA.

    Source: §1 · p. 2, §5.3 · p. 8

  • Takeaways & Limitations

    Combining AU and EU offers a practical path toward more comprehensive uncertainty estimation than single-model uncertainty scores, when multiple comparable models are available.

    Source: §6 · p. 10

    Response-level semantic similarity may underperform on tasks with many semantically distinct but correct answers, where disagreement need not reflect uncertainty.

    Source: §6 · p. 10

Abstract

from arXiv · show

Large language models (LLMs) often produce confident yet incorrect responses, and uncertainty quantification is one potential solution to more robust usage. Recent works routinely rely on self-consistency to estimate aleatoric uncertainty (AU), yet this proxy collapses when models are overconfident and produce the same incorrect answer across samples. We analyze this regime and show that cross-model semantic disagreement is higher on incorrect answers precisely when AU is low. Motivated by this, we introduce an epistemic uncertainty (EU) term that operates in the black-box access setting: EU uses only generated text from a small, scale-matched ensemble and is computed as the gap between inter-model and intra-model sequence-semantic similarity. We then define total uncertainty (TU) as the sum of AU and EU. In a comprehensive study across five 7-9B instruction-tuned models and ten long-form tasks, TU improves ranking calibration and selective abstention relative to AU, and EU reliably flags confident failures where AU is low. We further characterize when EU is most useful via agreement and complementarity diagnostics.

1 INTRODUCTION

Self-consistency can miss confident errors because a model may repeatedly produce the same wrong answer. The paper adds cross-model semantic disagreement as epistemic uncertainty (EU), combining it with aleatoric uncertainty (AU) in total uncertainty (TU).

1 INTRODUCTION

  • Self-consistency can fail when a model repeatedly produces the same incorrect answer with high confidence.The paper frames EU as uncertainty about whether the model's confidence is trustworthy for a given input.Source: §1 · p. 1
  • EU estimates cross-model semantic disagreement using generated responses from a small, scale-matched ensemble, without additional training.The paper presents this as a black-box approach to estimating uncertainty in model choice.Source: §1 · p. 1
  • Across ten long-form tasks and five reference models, TU consistently outperforms AU in AUROC and selective abstention, while EU flags confident incorrect responses.AU captures a model's internal variability, whereas EU measures divergence from other plausible models.Source: §1 · p. 2

2 RELATED WORKS

Prior uncertainty work has largely focused on response inconsistency, while EU approaches use Bayesian-inspired estimation, prompting, or verifiers. This paper situates its contribution among LLM ensemble applications.

2 RELATED WORKS

  • AU methods commonly sample multiple responses and assess semantic consistency; this paper adopts degree-based semantic dispersion as its AU baseline.The passage identifies Semantic Entropy as a related approach.Source: §2 · p. 2
  • Bayesian-inspired EU methods perturb embeddings, vary decoding temperatures, or use hidden-state entropy; explicit LoRA ensembles can improve calibration but add computational cost.The paper contrasts these approaches with its estimator, which uses generated text rather than logits or hidden states.Source: §2 · p. 2
  • Prompt-based methods query the same model iteratively, while verifier-based work uses inter-model disagreement; the paper notes their reported benefits or evaluations are limited in scope.The paper also says prior work has not fully explored model-space effects or AU–EU interactions.Source: §2 · p. 3
  • The paper connects classical uncertainty ensembles and recent LLM collaboration, verifier, and multi-model sampling work to uncertainty estimation.It studies LLM ensembles specifically from the viewpoint of uncertainty estimation.Source: §2 · p. 3

3 QUANTIFYING PREDICTIVE UNCERTAINTY USING RESPONSE SIMILARITY

The paper separates predictive uncertainty into within-model response variability and uncertainty about model choice, then estimates both from response similarity. It approximates EU with a surrogate model ensemble and sampled responses.

3.1 ALEATORIC UNCERTAINTY VIA INTRA-MODEL RESPONSE SIMILARITY

  • Aleatoric uncertainty (AU) measures how semantically dissimilar responses from the same model are across samples.The paper uses expected intra-model response similarity as the basis for this measure.Source: §3.1 · p. 3

3.2 EPISTEMIC UNCERTAINTY AS INTER-MODEL RESPONSE SIMILARITY

  • Epistemic uncertainty (EU) is modeled as divergence between a reference model and an ideal model, and estimated by comparing inter-model with intra-model response similarity.A mismatch between those similarities corresponds to higher EU.Source: §3.2 · p. 4
  • The surrogate ensemble must approximate plausible models; the paper uses similarly capable, scale-matched models from different vendors to provide diversity and uniform weighting.The ensemble's fidelity depends on how well it approximates the inaccessible ideal model distribution.Source: §3.2 · p. 4

3.3 EMPIRICAL ESTIMATES OF UNCERTAINTY METRICS

  • The empirical procedure samples responses from auxiliary models and the reference model, then estimates AU, EU, and TU from those responses.The authors report keeping the overall response-sampling budget at the magnitude used by self-consistency methods.Source: §3.3 · p. 5

4 EXPERIMENTAL SETUP

The experiments evaluate uncertainty estimates across instruction-tuned models and diverse long-form generation tasks. They assess correctness ranking and selective prediction against an AU baseline.

4 EXPERIMENTAL SETUP

  • The main experiments use five instruction-tuned models with approximately 7–9B parameters, sampling two responses per model for TU and ten responses for AU.This keeps the sampling budget the same across the two metrics.Source: §4 · p. 5
  • The evaluation spans QA, math reasoning, German-to-English translation, and summarization, with additional SimpleQA runs using GPT-4o and Claude 3.7 Sonnet.The QA datasets include both open-domain and multi-hop tasks.Source: §4 · pp. 5–6
  • Figure 2's comparison across AU levels shows EU separates incorrect from correct responses most strongly when AU is low.The primary AU baseline is Lin et al. (2023)'s implementation of aleatoric uncertainty.Source: Figure 2 · p. 6, §4 · p. 6
  • Uncertainty quality is measured by AUROC for separating incorrect from correct responses and by risk–coverage metrics for selective prediction.The study also reports accuracy at 90% and 80% coverage and AURC, where lower is better.Source: §4 · p. 6

5 RESULTS

EU flags confidently wrong answers when AU is low, but its usefulness depends on how models agree on correct answers. Across the evaluated tasks, combining EU with AU improves correctness calibration and selective abstention.

5.1 EPISTEMIC UNCERTAINTY FLAGS CONFIDENT FAILURES OF ALEATORIC UNCERTAINTY

  • In the lowest 5% of AU scores, EU remains significantly higher for incorrect generations, flagging confident errors that AU alone misses.The authors contrast this with approaches that treat low-AU predictions as reliable and only use cross-model comparisons above a threshold.Source: §5.1 · pp. 6–7

5.2 EPISTEMIC UNCERTAINTY, AGREEMENT, AND DIVERSITY

  • EU AUROC correlates positively with redundancy J (r = +0.72, p = 0.03) and negatively with complementarity G (r = −0.72, p = 0.03).This is opposite to the intuition that more disagreement necessarily makes EU more useful.Source: §5.2 · p. 7
  • EU is most effective when correct answers are nearly unique and similarly phrased across models, while harder unanswered inputs elicit diverse alternatives.High-J, low-G WMT16-de-en and CoQA show strong discrimination; diverse valid summaries in XSum inflate EU without improving ranking.Source: §5.2 · pp. 7–8

5.3 TOTAL UNCERTAINTY IMPROVES CORRECTNESS CALIBRATION

  • TU improves AUROC over AU on all benchmarks, with gains of +0.15 on HotpotQA, +0.14 on CoQA, and +0.13 on WMT16-de-en.These gains occur across five 7–9B instruction-tuned models.Source: Figure 4 · p. 8, §5.3 · p. 8
  • TU improves AUROC over AU on SimpleQA for GPT-4o (0.70 vs. 0.59) and Claude 3.7 Sonnet (0.58 vs. 0.53), with gains maintained on individual and pooled datasets.Per-model, per-dataset results are reported in Table 2.Source: §5.3 · pp. 8–9
  • TU attains mean AUROC 0.72 versus 0.64 for the strongest baseline, closeness centrality, across almost all datasets using Mistral-7B-Instruct-v0.3.The comparison covers baselines including token and sequence probabilities, semantic entropy, and self-consistency.Source: Figure 5 · p. 8, §5.3 · p. 9
  • TU still exceeds AU in AUROC with a single auxiliary model 0.43× the reference model’s size, and larger, more capable auxiliaries bring greater gains on TriviaQA.The authors also report higher AUROC than AU when the reference model is larger than auxiliary models, and improved AUROC with more sampled responses.Source: Figure 6 · p. 9, §5.3 · p. 9

5.4 TOTAL UNCERTAINTY IMPROVES SELECTIVE ABSTENTION

  • Across coverage levels and datasets, TU has the lowest risk with one exception; it also improves selective accuracy in nearly all cases and lowers AURC in all cases.On HotpotQA and XSum, TU improves C@90 by over 1.5 points and reduces AURC by over 20%.Source: Figure 7 · p. 9, Table 1 · p. 10, §5.4 · p. 10

6 CONCLUSIONS

The conclusion frames aleatoric and epistemic uncertainty as complementary signals and presents their combination as a more comprehensive uncertainty estimate. Its scope is bounded by reliance on multiple comparable models and response-level semantic similarity.

6 CONCLUSIONS

  • Self-consistency captures data ambiguity, while cross-model semantic disagreement captures uncertainty arising from model limitations.The conclusion describes these as complementary failure modes of language models.Source: §6 · p. 10
  • TU combines intra-model entropy and inter-model semantic divergence using black-box access to model outputs, and outperforms self-consistency-based methods across the evaluated models and datasets.The approach requires access to multiple comparable models.Source: §6 · p. 10
  • Response-level semantic similarity may underperform when tasks allow many semantically distinct correct answers, and disagreement then need not indicate uncertainty.The authors also note that homogeneous surrogate-model biases can cause cross-model disagreement to underestimate epistemic uncertainty.Source: §6 · p. 10

A.1 THEORETICAL INTERPRETATIONS OF EPISTEMIC UNCERTAINTY

This appendix interprets the proposed divergence as a one-sided kernel discrepancy between a model’s predictive distribution and an ensemble reference. Under stated kernel conditions, it bounds distributional mismatch and can identify low-AU disagreement with the ensemble.

Kernel and variational interpretation of D(ω || ω∗)

  • With a symmetric positive-definite similarity kernel, D can be rewritten using kernel mean embeddings in the reproducing kernel Hilbert space.The passage identifies Eq. 5 as the first two terms of squared maximum mean discrepancy and characterizes D as a one-sided kernel discrepancy.Source: §A.1 · p. 16

Variational-gap interpretation

  • The variational interpretation replaces negative log terms in classical KL with kernel similarity, yielding a semantic gap that projects the model distribution toward the ideal distribution while penalizing variability.The interpretation describes D as a semantic variational gap under the geometry induced by the kernel.Source: §A.1 · p. 16

Kernel and variational interpretation of D(ω || ω∗)

  • D is bounded above in absolute value by total variation distance; with equal embedding norms, D = 0 implies equal embeddings, and a characteristic kernel then implies equal predictive distributions.The stated results connect the kernel discrepancy to distributional mismatch under the specified conditions.Source: §A.1 · p. 16
  • Under the desired conditions and a bounded characteristic kernel, D flags high self-similarity paired with low ensemble agreement and requires only O(n^2) kernel evaluations for n responses per model.The paper presents D as a tractable proxy for epistemic uncertainty that is cheaper than KL-divergence evaluations or other EU approximations.Source: §A.1 · p. 17

A.2 AGREEMENT AND COVERAGE METRICS.

The appendix summarizes cross-model behavior using correctness overlap and the extra coverage available when an oracle can choose among models.

Agreement and coverage metrics

  • For each dataset, the authors form a binary matrix marking whether each model’s sampled response to each input is correct.The matrix is used to derive two coarse descriptors of cross-model behavior.Source: §A.2 · p. 17
  • Jaccard redundancy J is the mean pairwise Jaccard index of models’ correctly answered example sets; high J means models succeed on the same inputs.Source: §A.2 · p. 17
  • Oracle-gain diversity G measures additional coverage from an oracle choosing the correct model per example; high G means models get different examples right.Source: §A.2 · p. 17

A.3 ADDITIONAL EXPERIMENTAL SETUP

The additional setup uses inference-only generation and evaluation, with response sampling configured to compare uncertainty estimates under matched sample budgets.

Additional experimental setup

  • Experiments ran inference-only on two NVIDIA A100 80GB GPUs, using lm-evaluation-harness for generation and a local vLLM-hosted Llama-3-70B-Instruct judge for correctness.Correctness followed prior work by using only the first sampled response from each model.Source: §A.3 · p. 17
  • Ten responses per model were sampled for the first 100 prompts and used for AU, while Sections 5.3–5.4 used two responses per model for both TU and EU to match sample budgets.Generation used temperature 0.7 and top-p 0.9; SimpleQA outputs came from OpenAI and Anthropic APIs.Source: §A.3 · p. 17
  • Response semantic similarity was measured with cosine distance over sentence-T5-xl embeddings.For datasets absent from lm-evaluation-harness, the authors followed its prompt-formatting conventions and supplied code in supplementary material.Source: §A.3 · p. 17

A.4 ADDITIONAL RESULTS ON TOTAL UNCERTAINTY

Across the reported model–dataset evaluations, total uncertainty generally improves correctness discrimination over aleatoric uncertainty, with gains depending on model and task. The appendix also reports task-specific limitations and correlations with automatic quality metrics.

AUROC results per model and dataset

  • TU improves AUROC in nearly all model–dataset combinations, with the largest gains on HotpotQA; gains vary by model, and TU occasionally underperforms for stronger models on XSum.On XSum, the text suggests disagreement with less reliable auxiliary models as a possible reason; on GSM8K, gains concentrate among lower-performing models. Figures 8 and 9 and Table 2 present these comparisons.Source: §A.4 · p. 18, Figure 8 · p. 18, Figure 9 · p. 18
  • WMT16-de-en and CoQA show limited gains where reference-model accuracy exceeds 90%, while TruthfulQA’s diverse valid answers weaken the reliability of semantic disagreement as an EU signal.The passage also notes that TU corrects anomalously low AU for Mistral on CoQA.Source: §A.4 · p. 18, Figure 10 · p. 19

ROC curves

  • 0.746 vs. 0.707 AUROC: pooled ROC results favor TU over AU, with larger gains reported on HotpotQA, WMT16-de-en, and CoQA.Figure 12 reports better or comparable TU performance across individual tasks; the cited text identifies the three tasks with the largest gains.Source: Figure 11 · p. 19, §A.4 · p. 19, Figure 12 · p. 21

Comparison with baselines

  • The Mistral-7B baseline comparison reports TU as better calibrated with correctness than AU and the listed uncertainty baselines.The listed comparisons include token- and sequence-level confidence measures, semantic entropy, self-consistency, and other approaches.Source: §A.4 · p. 20, Table 3 · p. 21

Correlation with BLEU and ROUGE

  • On WMT16-de-en, AU and TU both correlate moderately negatively with BLEU; on XSum, EU’s ROUGE correlation is −0.213, while AU’s is positive.The authors interpret the XSum pattern as suggesting EU is a better proxy for quality degradation than AU in that setting.Source: §A.4 · p. 21, Table 4 · p. 21

A.5 THE EFFECT OF THE AUXILIARY MODEL SET ON TOTAL UNCERTAINTY

Appendix analyses show that TU’s advantage over AU persists across model sizes, auxiliary-set choices, sampling budgets, and free-form versions of multiple-choice tasks. They also identify low-AU errors as a setting where EU separates incorrect from correct responses, and examine computational costs and correctness-judge reliability.

A.5 THE EFFECT OF THE AUXILIARY MODEL SET ON TOTAL UNCERTAINTY

  • Across TriviaQA reference-model sizes, TU retains higher AUROC than AU—even for Qwen2.5-32B versus 7–9B peers—while noise-perturbed auxiliaries yield smaller TU–AU gains than diverse peers.Both AUROC scores tend to decrease as reference-model size increases.Source: Figure 13 · p. 22, §A.5 · p. 22

A.6 ABLATIONS

  • TU’s AUROC improves as auxiliary models increase for Mistral-7B-Instruct-v0.3, with decreasing run variance; across datasets, TU beats aleatoric baselines even with one auxiliary model, and sample returns diminish beyond 30 in most tasks.TU improvements are apparent with as few as 10 samples (n = 5 on the x-axis).Source: Figure 15 · p. 23, §A.6 · p. 23
  • TU’s AUROC advantage over AU persists with both sentence-T5-xl embeddings and deberta-large-mnli entailment probability as similarity measures.Source: Table 5 · p. 23, §A.6 · p. 23
  • The unweighted TU outperforms AU across benchmarks without task-specific calibration, while alternative weights can yield marginal gains on individual datasets.The comparison averages AUROC over five reference models.Source: Table 6 · p. 24, §A.6 · p. 24

A.7 ADDITIONAL RESULTS ON MULTIPLE-CHOICE QA TASKS

  • TU improves over AU in most adapted Big-Bench Hard tasks, with the largest gains when base-model performance is low; the tasks are evaluated as free-form responses.Source: §A.7 · p. 24, Table 7 · p. 24

A.8 EPISTEMIC UNCERTAINTY ANALYSIS

  • Across models and benchmarks, incorrect responses in low-AU buckets have higher EU than correct ones; this separation weakens at mid and high AU.The same pattern appears when disaggregated by dataset.Source: §A.8 · p. 25, Figure 17 · p. 25, Figure 18 · p. 25

A.9 COMPUTATIONAL COST

  • With five-way parallelism, TU returns to about 78 seconds while retaining calibration and abstention gains; on TriviaQA, TU with five samples outperforms AU with 20.TU’s five-sample comparison uses one sample from each of five models.Source: §A.9 · pp. 25–26, Table 8 · p. 26, Table 9 · p. 26

A.10 COMPUTING CORRECTNESS USING AN LLM JUDGE

  • The judges gave identical correctness probabilities in 93/100 cases, and human–judge disagreement was below 6% in an audit of 100 Mistral samples.The manual audit covered TriviaQA and TruthfulQA.Source: §A.10 · p. 27
Loading 2604.17112v1…