Source-linked AI summary
A Survey of Uncertainty Estimation Methods on Large Language Models
Zhiqiu Xia, Jinxuan Xu, Yuqian Zhang, Hang Liu
TL;DR
LLMs can produce fluent but misleading responses, while comprehensive surveys dedicated to their uncertainty estimation remain limited. This paper develops an LLM-centered inference-stage taxonomy, evaluates representative methods across datasets and domains, and identifies future directions. Its examples show that method performance varies across datasets and uncertainty settings, and the survey concludes with four method classes and extensive evaluations.
Problem
Fluent LLM outputs can be biased, hallucinated, or non-factual, while existing surveys do not comprehensively synthesize uncertainty estimation methods tailored to LLMs.
Method
The survey proposes an LLM-centered taxonomy of inference-stage uncertainty methods, emphasizes approaches requiring no additional data or model modifications, and evaluates representative methods across datasets and domains.
Results
Performance varies across datasets and uncertainty settings: 2S achieves the best performance on TruthfulQA, while SimpleQA favors 2S and Ptrue in ROC analysis and LN-E in ARC analysis.
Takeaways & Limitations
The survey organizes LLM uncertainty estimation into verbalizing, latent information, consistency-based, and semantic clustering methods while highlighting future research directions.
Takeaways & Limitations
The survey omits detailed methodological explanations, does not evaluate all introduced methods, and excludes literature outside inference-stage LLM uncertainty estimation.
Abstract
from arXiv · showhide
Large language models (LLMs) have demonstrated remarkable capabilities across various tasks. However, these models could offer biased, hallucinated, or non-factual responses camouflaged by their fluency and realistic appearance. Uncertainty estimation is the key method to address this challenge. While research efforts in uncertainty estimation are ramping up, there is a lack of comprehensive and dedicated surveys on LLM uncertainty estimation. This survey presents four major avenues of LLM uncertainty estimation. Furthermore, we perform extensive experimental evaluations across multiple methods and datasets. At last, we provide critical and promising future directions for LLM uncertainty estimation.
1 Introduction
LLMs can produce fluent but biased, hallucinated, or non-factual responses, motivating uncertainty estimation and a dedicated survey of methods tailored to LLMs. The survey introduces an inference-stage taxonomy, evaluates representative approaches across datasets and domains, and identifies future directions.
- Fluent LLM responses can still be biased, hallucinated, non-factual, or misaligned, making misleading outputs difficult to detect.
- Uncertainty estimation assigns confidence scores that can support decisions about whether an initial response should be accepted.
- Existing DNN uncertainty techniques are not easily transferable to LLMs because LLMs contain very large numbers of parameters.
- Prior LLM uncertainty surveys emphasize traditional DNN uncertainty, calibration, benchmarks, or applications rather than comprehensively examining core methods.
- This survey introduces an LLM-centered taxonomy, focuses on inference-stage methods without extra data or model modifications, and evaluates representative approaches across datasets and domains.
2 Uncertainty Sources in LLM
LLM uncertainty has two primary sources: aleatoric uncertainty inherent in data and epistemic uncertainty arising from the model’s lack of knowledge or understanding.
- Aleatoric uncertainty is inherent in data and can arise from ambiguous or incomplete prompts and multiple valid interpretations.
- Epistemic uncertainty reflects the model’s lack of knowledge or understanding when concepts or data are underrepresented in training.
3 Uncertainty Estimation in LLMs
LLM uncertainty estimation is organized around modeling response variation, confidence, and the information available during inference. The survey groups methods into verbalizing, latent-information, consistency-based, and semantic-clustering approaches, each using different signals and trade-offs.
- Problem Definition and Overview: LLMs generate responses autoregressively by sampling each next-token distribution conditioned on the prompt and previously generated tokens.The process ends at a preset token limit or when the end-of-sequence token is generated.
- Problem Definition and Overview: Uncertainty can be estimated from variation across responses, while confidence is treated as complementary to uncertainty and may guide answer acceptance.The survey describes response variance as an uncertainty measure and discusses majority voting or an additional response for final selection.
- Overview: The survey categorizes LLM uncertainty estimation into four classes: verbalizing, latent information, consistency-based, and semantic clustering methods.These categories structure the survey’s review of methods applicable during inference.
- Verbalizing Methods: Verbalizing methods prompt an LLM to produce an answer with a self-assessed confidence level, but reinforcement learning from human feedback can make answers over-confident.Variants include multiple candidates, multiple prompt-and-answer rounds, and verbal rather than numerical confidence expressions.
- Latent Information Methods: Latent-information methods use white-box outputs such as token probabilities, entropy, negative log-likelihoods, or hidden-state representations to estimate uncertainty.The survey also describes covariance- and attention-based measures derived from internal model representations.
- Consistency-based and Semantic Clustering Methods: Consistency-based methods compare outputs generated for an original prompt and paraphrased prompts, whereas semantic-clustering methods group semantically equivalent responses and compute uncertainty from cluster probabilities.Semantic equivalence can be assessed through mutual entailment, using an NLI classifier or a general-purpose LLM.
4 Evaluation
The evaluation compares representative uncertainty-estimation methods across five datasets using AUROC and AUARC, with performance varying by dataset and uncertainty type. Entailment-aware methods perform strongly on free-form questions, while 2S performs best on TruthfulQA and SimpleQA.
- Metrics: AUROC and AUARC, both ranging from 0 to 1, measure discrimination between correct and incorrect answers, with higher scores indicating better uncertainty estimation.The evaluation emphasizes whether estimated uncertainty distinguishes correct from incorrect responses.
- TruthfulQA: 2S achieves the best performance on TruthfulQA, where uncertainty is primarily epistemic and high-confidence responses are usually correct.Its ROC curve rises steeply, and its accuracy improves substantially at higher rejection rates.
- TriviaQA: NLI achieves the highest ROC performance on TriviaQA, while DSE achieves the highest AUARC as 2S and Perp accuracy deteriorate beyond a 0.5 rejection rate.TriviaQA permits multiple responses with the same meaning, introducing aleatoric uncertainty.
- GSM8K: On GSM8K, NLI and SE perfectly classify the highest-confidence responses, whereas INSIDE performs below random guessing on AUROC.NLI, DSE, and SE maintain stronger TPR gains as FPR increases, and the best methods reach perfect accuracy among the top 20% high-confidence responses.
- Cross-dataset comparison: NLI, SE, and DSE perform best on free-form questions because entailment relationships reduce aleatoric uncertainty and better estimate epistemic uncertainty.This pattern is reported when comparing TriviaQA and GSM8K.
- SimpleQA: On SimpleQA, 2S and Ptrue maintain low FPR while TPR approaches 1, and LN-E has the highest ARC performance after the rejection rate exceeds 0.8.NLI, SE, and DSE do not show superior performance on this challenging dataset.
5 Future Directions
The paper identifies benchmark design, method enhancement, and long-response uncertainty as future directions. It calls for evaluations that vary question type, difficulty, and ambiguity, while noting that current methods remain limited for long, multi-step responses.
- Uncertainty estimation benchmark: A dedicated uncertainty-estimation benchmark should include diverse question types, varying difficulty levels, and controllable ambiguity.These properties would directly evaluate uncertainty across general Q&A, mathematics, translation, and different ambiguity levels.
- Uncertainty estimation method enhancement: Uncertainty-estimation methods need enhancement because long responses involve inter-sentence relationships and large vocabularies that challenge current approaches.Segmenting long responses can ignore relationships across sentences, while consistency-based and semantic-clustering methods may become less effective.
- Uncertainty estimation method enhancement: Current uncertainty-estimation methods are predominantly validated on short answers and may not adequately address long, multi-step reasoning processes.Long-response uncertainty remains under-explored.
6 Conclusion
The survey organizes inference-stage uncertainty estimation for LLMs into four classes and supplements this taxonomy with extensive evaluations and future directions.
- The survey classifies LLM uncertainty estimation methods into verbalizing, latent information, consistency-based, and semantic clustering methods.
- The survey also provides extensive evaluations and identifies promising future directions for LLM uncertainty estimation.
7 Limitations
The survey identifies three limitations, mainly attributed to space constraints: abbreviated methodological explanations, incomplete evaluations, and exclusion of non-inference-stage literature.
- Space constraints limited detailed methodological explanations for various methods in the main text.
- The survey did not evaluate and report results for all introduced methods.
- The survey excludes literature that does not concern the inference stage of LLMs.
A Mathematical Formulation of the Methods
The appendix presents mathematical notation and representative formulations for latent-information, consistency-based, and semantic-clustering uncertainty methods, alongside dataset examples.
- A.1 Common notations: Common mathematical notations are introduced in Table 1 before the method-specific formulations.
- A.2 Latent Information Methods: Latent-information methods include average and maximum negative log-likelihood, true-token probability, predictive entropy, length-normalized entropy, and token-distribution aggregation measures.
- A.2 Latent Information Methods: The latent-information formulations also describe vector operations, covariance-based quantities, embeddings, centering, and eigenvalues.
- A.3 Consistency-based Methods: Consistency-based methods include sampling diversity, first-second-distance-based scoring, variation ratios, and similarity calculations.
- A.4 Semantic Clustering Methods: Semantic-clustering methods include semantic entropy and discrete semantic entropy.
- A.4 Semantic Clustering Methods: The appendix lists sample questions and answers from datasets including SciQ, TriviaQA, and SimpleQA, plus a daily-earnings word problem.
- A.4 Semantic Clustering Methods: Table 2 provides samples from each dataset used in the semantic-clustering-methods section.
B Detailed Explanation of AUROC and AUARC
The section explains AUROC as threshold-based discrimination between correct and incorrect responses, and AUARC as accuracy retention when low-confidence answers are rejected.
- AUROC plots true positive rate against false positive rate while varying the accepted confidence threshold.
- Responses are labeled positive or negative according to whether they are correct or incorrect relative to the ground truth.
- AUARC plots accepted-sample accuracy as increasingly low-confidence answers are rejected.
- The AUARC area measures how well uncertainty estimation maintains high accuracy after rejecting low-confidence answers.
- The section introduces a prompt used for question-and-answer tasks.
D Detailed Explanation of Dataset
The evaluation uses datasets spanning truthfulness, science, reading comprehension, summarization, and machine translation, with varied sizes and task designs.
- TruthfulQA: TruthfulQA contains 817 questions across 38 categories, including health, law, finance, and politics.It is designed to evaluate whether language models generate truthful answers to questions that may elicit human misconceptions.
- Supplementary datasets: Table 3 reports supplementary results on the AESLC summarization dataset and the WMT14 De-En machine-translation dataset.These datasets extend evaluation beyond question-answering tasks.
- SciQ: SciQ contains 13.7K multiple-choice science questions, and the experiments use its 1K-sample validation set.Its topics include biology, chemistry, earth science, and physics.
- TriviaQA: TriviaQA is a reading-comprehension benchmark with over 650K question-answer-evidence triples and complex, compositional questions.The benchmark includes diverse evidence sources.
E Supplementary Results
Supplementary experiments evaluate summarization and machine translation, revealing task-dependent uncertainty patterns and method behavior.
- AESLC: AESLC results are consistent with SimpleQA because both datasets exhibit high epistemic uncertainty.The passage explicitly characterizes this as a cross-dataset consistency in results.
- WMT14 De-En: WMT14 De-En has aleatoric uncertainty between that of multiple-choice and free-form question-answering datasets.This places the machine-translation task between those two Q&A settings on the reported uncertainty dimension.
- WMT14 De-En: Consistency-based and semantic-clustering methods show comparable performance on the machine-translation task.The paper links this comparable performance to the intermediate aleatoric-uncertainty setting reported for the task.