Source-linked AI summary
LM-Polygraph: Uncertainty Estimation for Language Models
Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun, Artem Vazhentsev, Sergey Petrakov, Kirill Fedyanin, Daniil Vasilev, Elizaveta Goncharova, Alexander Panchenko, Maxim Panov, Timothy Baldwin, Artem Shelmanov
TL;DR
LLMs can hallucinate, but existing UE research for text generation has been more theoretical than engineering-focused. LM-Polygraph unifies UE methods, benchmarking, and confidence-aware interaction, while experiments show current selective generation remains far from solved. The framework therefore offers practical tooling without eliminating the need for further UE research.
Problem
LLMs can produce misleading or erroneous outputs, while text-generation UE remains difficult because it must aggregate token uncertainty, handle huge output spaces, and account for semantic variation.
Method
LM-Polygraph provides unified Python implementations of UE techniques, an extendable evaluation benchmark, and tools for integrating uncertainty scores into LLM applications.
Results
Experiments find that white-box information-theoretic methods usually perform better, while overall selective generation remains far from solved.
Takeaways & Limitations
Uncertainty scores can inform users and developers about risks in model outputs and encourage skepticism toward potentially unreliable responses.
Takeaways & Limitations
Comprehensive UE evaluation remains open, and the benchmark should be expanded to include more tasks and datasets.
Abstract
from arXiv · showhide
Recent advancements in the capabilities of large language models (LLMs) have paved the way for a myriad of groundbreaking applications in various fields. However, a significant challenge arises as these models often "hallucinate", i.e., fabricate facts without providing users an apparent means to discern the veracity of their statements. Uncertainty estimation (UE) methods are one path to safer, more responsible, and more effective use of LLMs. However, to date, research on UE methods for LLMs has been focused primarily on theoretical rather than engineering contributions. In this work, we tackle this issue by introducing LM-Polygraph, a framework with implementations of a battery of state-of-the-art UE methods for LLMs in text generation tasks, with unified program interfaces in Python. Additionally, it introduces an extendable benchmark for consistent evaluation of UE techniques by researchers, and a demo web application that enriches the standard chat dialog with confidence scores, empowering end-users to discern unreliable responses. LM-Polygraph is compatible with the most recent LLMs, including BLOOMz, LLaMA-2, ChatGPT, and GPT-4, and is designed to support future releases of similarly-styled LMs.
1 Introduction
LLMs can hallucinate misleading or erroneous information, while uncertainty estimation for text generation remains difficult and fragmented. LM-Polygraph addresses this gap with unified UE tools, an extendable benchmark, and confidence-enriched LLM interaction.
- LLMs may generate plausible but false information when their knowledge is limited, creating challenges for practical deployment.
- Text-generation UE must aggregate token uncertainties across sequences, handle sampling and pruning, and account for semantic similarity beyond surface form.
- Existing text-generation UE research is fragmented, with non-comparable studies that hinder holistic conclusions.
- LM-Polygraph consolidates UE techniques through unified Python interfaces, an extendable benchmark, and tools for integrating uncertainty into LLM pipelines.
- The framework enriches model outputs with uncertainty scores that can help users judge whether responses are reliable.
2 Python Library
LM-Polygraph provides Python interfaces for state-of-the-art UE techniques across modern language-model architectures and publicly available models.
- LM-Polygraph implements state-of-the-art UE techniques with unified Python interfaces for Hugging Face models.
- The library supports both conditional seq2seq models and unconditional decoder-only language models.
- It has been tested with BLOOMz, Dolly v2, Alpaca, LLaMA-2, and Flan-T5.
3 Uncertainty Estimation Methods
LM-Polygraph organizes UE methods into white-box and black-box approaches with different access requirements and computational trade-offs. The methods span probability-based, ensemble, density-based, hybrid, self-evaluation, and response-similarity techniques.
- White-box methods require logits, internal representations, or model access, whereas black-box methods use only generated texts.
- White-box Methods: Information-based methods use token or sequence probabilities and are cheap and simple, but generally provide relatively low-quality baseline estimates.
- White-box Methods: Ensemble-based methods measure discrepancies among predictions from multiple model variants, including through mutual information.
- White-box Methods: Density-based methods detect out-of-distribution instances efficiently but require training data and primarily capture epistemic uncertainty.
- White-box Methods: Hybrid uncertainty quantification combines information-based detection of ambiguous instances with density-based detection of out-of-distribution instances.
- Black-box Methods: Black-box similarity methods generate K responses, construct a K × K similarity matrix, and aggregate it into an uncertainty score.
4 Demo
The demo application lets users interact with public or web-served LLMs while viewing uncertainty scores for their answers. It supports end-user trust assessment and researchers’ qualitative analysis of UE methods and responses.
- The application displays confidence scores alongside LLM answers during interactive use.Users can select a UE method and a publicly available model, or connect to services such as ChatGPT and GPT-4 using an API key.
- Users can assess whether model answers are reasonable to trust, while researchers can qualitatively analyze UE methods and LLM responses.
5 Evaluation Benchmark
LM-Polygraph provides an extensible Python benchmark for evaluating uncertainty-estimation methods in text generation. The evaluation covers machine translation, text summarization, and question answering using multiple datasets and models, with selective generation assessed by PRR.
- The benchmark runs one or multiple UE experiments through Python modules and allows researchers to add and evaluate novel methods uniformly.
- Experiments cover machine translation, text summarization, and question answering, using two datasets for each task.The datasets include WMT14 German-to-English and French-to-English, XSum and AESLC, and CoQA and bAbI QA.
- Experiments use Vicuna-v1.5-7B and Llama-v2-7B models.
- Selective generation rejects low-quality sequences based on uncertainty scores, and methods are compared using Prediction Rejection Ratio (PRR).Higher PRR values indicate better selective-generation quality.
6 Experimental Results
Information-theoretic white-box methods usually perform best for both evaluated models, while black-box results depend on the model. Overall, selective generation remains far from solved, especially on more difficult tasks.
- White-box information-theoretic methods usually achieve better performance for both Vicuna-v1.5-7B and LLaMA-v2-7B.They are generally easy to implement and computationally lightweight, although several Monte Carlo and semantic-entropy methods require repeated sampling.
- For black-box methods, Eccentricity has the best average performance for Vicuna, while LLaMA shows no clear advantage among the evaluated methods.
- Selective generation remains far from solved: performance is far from perfect across methods, models, and datasets, especially on XSum and WMT14.
7 Conclusion
LM-Polygraph aims to make LLM use safer by adding uncertainty information to model predictions. The framework is intended to support future UE methods and help detect and mitigate hallucinations in real-world applications.
- Uncertainty scores can inform users and developers about risks in model outputs and encourage skepticism toward potentially unreliable answers.
- The authors plan to expand LM-Polygraph with new UE methods to support detecting and mitigating LLM hallucinations.
Limitations
LM-Polygraph’s coverage is intentionally selective, its evaluation scope remains incomplete, and practical access depends on model, language, and deployment constraints.
- The framework omits techniques lacking strong prior performance, theoretical motivation, or sufficient distinctiveness.
- Comprehensive evaluation remains an open question, and the benchmark should expand its number of tasks and datasets.
- The demo cannot directly provide access to the largest public LLMs because running them is prohibitively expensive.Users can still access models such as ChatGPT by supplying an API key.
- Although LM-Polygraph supports common APIs for modern LLMs, future releases may require modifications.
- Valid uncertainty estimates currently cover only English outputs because implemented quality metrics and models are mostly English-specific.The authors plan to support custom quality metrics and scoring models.
Ethics Statement
The paper frames uncertainty estimation as complementary to methods for reducing harmful outputs and surveys diverse sequence-, token-, ensemble-, and representation-based techniques. These methods vary in computational requirements, model-access assumptions, and the level at which uncertainty is estimated.
- The experiments use publicly available datasets previously leveraged in LLM uncertainty-estimation research.
- Uncertainty scores can support safer use of LLMs, while certainty estimation does not replace model debiasing or other safeguards against harmful content.
- Sequence-level methods estimate uncertainty from probabilities, semantic clustering, mutual-information variants, entropy, or sampled generations.Semantic Entropy groups generated sequences into semantically homogeneous clusters before aggregating their probabilities.
- Token-level and ensemble methods derive uncertainty from token entropy, ensemble disagreement, pairwise KL divergence, or reverse mutual information.
- Representation-based methods include Mahalanobis Distance and Robust Density Estimation, using hidden representations and covariance-based distances.Robust Density Estimation reduces representation dimensionality with PCA and uses Minimum Covariance Determinant covariance estimates.
A.2 Black-box Methods
The paper presents black-box uncertainty methods that infer response uncertainty from similarities among multiple generated answers. These methods include lexical, entailment-based, graph-based, and embedding-based measures, with continuous alternatives addressing limitations of discrete semantic-set counts.
- Semantic similarity methods: Number of Semantic Sets measures uncertainty by counting distinct semantic interpretations, with higher counts indicating greater uncertainty.The measure compares responses using entailment and reports the resulting number of semantic sets.
- Graph-based methods: The integer-valued semantic-set measure can violate transitivity assumptions, motivating the continuous Sum of Eigenvalues of the Graph Laplacian.The graph-based alternative is proposed as a continuous counterpart.
- Graph-based methods: Degree Matrix uncertainty uses the corrected trace of pairwise response similarities, with larger average distances corresponding to larger uncertainty.The resulting measure is UDeg = 1 −trace(D)/K2.
- Embedding-based methods: Graph-Laplacian eigenvectors provide response coordinates for Eccentricity, which uses distance from the center as uncertainty or negative confidence.This approach addresses the limited embedding-space information available from pairwise similarities alone.
- Similarity functions: Lexical Similarity averages pairwise overlap-based scores such as ROUGE and BLEU, while NLI similarity uses entailment or contradiction probabilities from DeBERTa-large.The methods compare generated responses through lexical or semantic similarity.
D Dataset Statistics
The experiments use test sets from several datasets and provide configurations, scripts, and dataset statistics for reproducible evaluation. Confidence scores are calibrated from uncertainty estimates, but their quality depends heavily on calibration-data size and diversity.
- Dataset statistics: Experiments use all examples from dataset test sets, while density-based methods are trained on random subsets of 1000 training examples.Dataset statistics include training, validation, and test counts, text lengths, target lengths, and source and target languages.
- Experiment setup: The benchmark provides code and configuration examples for running Vicuna-v1.5-7b experiments on the CoQA question-answering dataset.The configuration structure is stated to remain the same for other datasets and models.
- Confidence calibration: Normalization maps uncertainty estimates to confidence scores in [0, 1], calibrated to represent expected generation quality such as RougeL.Calibration estimates quality by averaging metrics within uncertainty bins on validation data.
- Confidence calibration: The quality of normalized confidence scores depends heavily on the calibration dataset’s size and diversity, and more efficient universal calibration remains future work.The calibration dataset mixes datasets from different tasks and operating conditions.