Source-linked AI summary
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Umang Bhatt, Javier Antorán, Yunfeng Zhang, Q. Vera Liao, Prasanna Sattigeri, Riccardo Fogliato, Gabrielle Gauthier Melançon, Ranganath Krishnan, Jason Stanley, Omesh Tickoo, Lama Nachman, Rumi Chunara, Madhulika Srikumar, Adrian Weller, Alice Xiang
TL;DR
The paper addresses the gap between explainability and knowing when predictions may be wrong or exceed a model’s knowledge. It reviews how to assess, use, and communicate uncertainty as a complementary form of transparency, concluding that well-calibrated estimates can support fairness, decision-making, and trust, while evidence gaps remain.
Problem
Explainability may reveal why a model behaves as it does without showing whether its predictions are wrong or exceed its knowledge, motivating uncertainty as complementary transparency.
Method
The paper surveys machine learning, visualization/HCI, design, decision-making, and fairness literature on assessing, using, and communicating uncertainty.
Results
The review covers uncertainty quantification and its use for reducing model unfairness, supporting expert decisions, building trust, and communicating uncertainty to stakeholders.
Takeaways & Limitations
Well-calibrated and well-communicated uncertainty can improve transparency, support trust, and help stakeholders use model predictions more appropriately.
Takeaways & Limitations
Empirical understanding remains limited for how decision-makers use aleatoric versus epistemic uncertainty and perceive uncertainty in continuous outcomes.
Abstract
from arXiv · showhide
Algorithmic transparency entails exposing system properties to various stakeholders for purposes that include understanding, improving, and contesting predictions. Until now, most research into algorithmic transparency has predominantly focused on explainability. Explainability attempts to provide reasons for a machine learning model's behavior to stakeholders. However, understanding a model's specific behavior alone might not be enough for stakeholders to gauge whether the model is wrong or lacks sufficient knowledge to solve the task at hand. In this paper, we argue for considering a complementary form of transparency by estimating and communicating the uncertainty associated with model predictions. First, we discuss methods for assessing uncertainty. Then, we characterize how uncertainty can be used to mitigate model unfairness, augment decision-making, and build trustworthy systems. Finally, we outline methods for displaying uncertainty to stakeholders and recommend how to collect information required for incorporating uncertainty into existing ML pipelines. This work constitutes an interdisciplinary review drawn from literature spanning machine learning, visualization/HCI, design, decision-making, and fairness. We aim to encourage researchers and practitioners to measure, communicate, and use uncertainty as a form of transparency.
1 INTRODUCTION
Algorithmic transparency exposes information about model behavior, but explainability alone may not show when predictions are unreliable or beyond the model’s knowledge. The paper therefore presents uncertainty estimation and communication as a complementary form of transparency.
- Algorithmic transparency exposes information about a model’s behavior to stakeholders, while explainability provides reasons for that behavior.
- Uncertainty can indicate when stakeholders should trust predictions and help developers address fairness issues when estimates are well calibrated and communicated.
- Uncertainty is task-dependent: regression commonly uses error bars or confidence intervals, while classification uses probabilities to express confidence.
- A smaller regression interval indicates greater model certainty, whereas classification probabilities quantify confidence in a particular outcome.
Why do we care?
The paper uses cancer diagnosis to illustrate how uncertainty can affect fairness assessment, decision-making, trust, and communication. It emphasizes that uncertainty displays can convey rich predictive information but may also confuse users if poorly communicated.
- Why do we care?: The cancer-diagnosis scenario involves doctors, patients, and review boards using ML tools for diagnosis, understanding diagnoses, and reviewing decisions.
- Why do we care?: Dataset bias can create epistemic uncertainty and larger error rates for under-represented groups, complicating fairness assessment.
- Why do we care?: Considering uncertainty before using a model’s output may help doctors avoid over-relying on spurious predictions or under-relying on accurate ones.
- Why do we care?: Poorly communicated uncertainty may be incomprehensible and erode trust, such as when broad confidence intervals lead doctors to override model outputs.
- Why do we care?: Predictive distributions encode modes and tails, but systems may summarize them with means and error bars or defer uncertain cases to medical professionals.
2 MEASURING UNCERTAINTY
The paper distinguishes aleatoric uncertainty arising from data noise from epistemic uncertainty arising from limited knowledge, and reviews methods for quantifying both. It also emphasizes that calibrated uncertainty is necessary before uncertainty can function as transparency.
- Representing Uncertainty: Predictive distributions represent uncertainty over possible outcomes; classification commonly uses class probabilities, while regression often reports a predictive mean with error bars.Distribution summaries can include means, medians, confidence intervals, standard deviations, and quartiles.
- Sources of Uncertainty: Aleatoric uncertainty reflects noise or class overlap in data, whereas epistemic uncertainty reflects uncertainty about the function, parameters, or model class explaining observed data.Aleatoric uncertainty is irreducible through additional observations, while epistemic uncertainty can decrease with data from sparse input regions.
- Methods to Quantify Uncertainty: Bayesian methods marginalize predictions over plausible model weights, while Frequentist ensembles use disagreement among multiple fitted models to estimate epistemic uncertainty.The predictive posterior combines epistemic and aleatoric uncertainty, whereas ensemble disagreement captures model uncertainty.
- Methods to Quantify Uncertainty: Neural-network posteriors and predictive distributions are analytically and computationally intractable to compute exactly, creating a trade-off between approximation fidelity and scalability.More faithful approximations tend to be more computationally intensive and scale worse to larger models.
- Evaluating Uncertainty: Calibration assesses whether predictive uncertainty reflects the true target distribution, and it is distinct from predictive accuracy.Calibration supports decisions such as deferring uncertain cases to human experts and is a precursor to using uncertainty as transparency.
3 USING UNCERTAINTY
The paper examines how uncertainty can support fairness, decision-making, and trust in automated systems. It also identifies limits in current evidence, including unresolved questions about how people process different uncertainty types and persistent data constraints.
- 3.1 Uncertainty and Fairness: Uncertainty can reveal fairness problems caused by noisy measurements, labels, or under-represented populations.Measurement and label noise are aleatoric sources, while representation bias is epistemic and may be reduced with more diverse data.
- 3.1 Uncertainty and Fairness: Large epistemic uncertainty on deployment-representative validation data can indicate representation bias in the training data.Practitioners can then identify subgroups that are unequally represented between training and validation data.
- 3.2 Uncertainty and Decision-making: Decision-makers can use uncertainty estimates to avoid over-relying on spurious predictions or under-relying on accurate ones.In medical diagnosis, a reject option can route situations in which the system is likely wrong to a human professional.
- 3.2 Uncertainty and Decision-making: Uncertainty increases transparency by helping explain automated decisions and supporting human experts when systems defer cases.Experts may receive a predictive distribution or summary statistic when the reject option is triggered.
- 3.3 Uncertainty and Trust Formation: The empirical understanding of how decision-makers use aleatoric versus epistemic uncertainty remains limited, especially for continuous outcomes.The paper also identifies uncertainty processing and affective responses during trust formation as open questions for future research.
- 3.3 Uncertainty and Trust Formation: Well-measured and well-communicated uncertainty should support appropriate, calibrated trust rather than blindly increasing trust.It may help stakeholders distinguish different types of model capabilities through more precise trust judgments.
4 COMMUNICATING UNCERTAINTY
Communicating uncertainty is difficult because people vary in statistical understanding and systematically distort probabilities. The paper reviews categorical, numerical, and graphical methods, emphasizing stakeholder testing and user research when selecting representations.
- Communication challenges: Stakeholders may misinterpret uncertainty because numeracy differences and cognitive biases distort probability judgments.Examples include ratio bias, probability distortion, and framing effects.
- Communication methods: High-precision uncertainty displays can help experts but may be difficult for uninitiated audiences to understand.Communication methods range from categorical descriptions to full probability distributions.
- Categorical communication: Predefined uncertainty categories can reduce cognitive effort, but people may misunderstand their thresholds and intended probabilities.For example, people interpreted “very likely” as around 60%, although the IPCC guideline specifies greater than 90%.
- Numerical and graphical communication: Numerical and graphical formats offer alternative ways to communicate uncertainty, but each involves interpretive trade-offs.Frequency formats, icon arrays, quantile dot plots, and distribution plots translate probabilities or distributions into more concrete representations.
- Uncertainty requirements: Uncertainty representations should be tested with stakeholders, and teams should conduct user research to address different goals, needs, constraints, and stakeholder types.The reviewed findings are hypotheses for testing with real users rather than conclusive predictions of usability.
5 CONCLUSION
The paper frames uncertainty as a complementary form of algorithmic transparency and reviews how to quantify, apply, and communicate it. It concludes that transparency depends on calibrated estimates used coherently and communicated clearly for the relevant stakeholder and use case.
- Conclusion: The review connects uncertainty to fairness, decision-making, trust in automated systems, and communication requirements.It synthesizes literature from machine learning, visualization/HCI, design, decision-making, and fairness.
- Future work: Future work should study interactions between fairness, transparency, and uncertainty, including uncertainty communication, fairness perceptions, and regression calibration.The authors also encourage attention to uncertainty’s effects on models in deployment.
A UNCERTAINTY QUANTIFICATION METRICS
The appendix distinguishes uncertainty metrics by task and by whether they capture aleatoric, epistemic, or combined uncertainty. For classification, predictive entropy summarizes overall uncertainty, while expected entropy and mutual information isolate its components.
- Classification setting: Predictive distributions in classification can be approximated from T stochastic forward passes and summarized using uncertainty metrics.The predictive distribution averages outputs from stochastic models sampled through Monte Carlo passes.
- Classification setting: Predictive entropy measures overall predictive uncertainty, combining aleatoric and epistemic components.For deterministic point-estimate models, it captures only aleatoric uncertainty and ranges from 0 to log K.
- Classification setting: Expected entropy captures aleatoric uncertainty, whereas mutual information between model parameters and targets captures epistemic uncertainty.Predictive entropy can be recovered as the sum of expected entropy and mutual information.
- Classification setting: Variation ratio captures disagreement among predictions from multiple stochastic forward passes.It is based on how often the output class is predicted across T passes.
A.2 Regression setting
For regression, the paper models continuous targets with heteroscedastic Gaussian noise and approximates predictive uncertainty through Monte Carlo-induced Gaussian mixtures. Variance provides a practical metric that decomposes into aleatoric and epistemic components, while distribution summaries support visualization.
- A.2 Regression setting: The regression setting uses heteroscedastic Gaussian noise models whose predicted mean and variance may vary across input space.Marginalizing over model parameters with T Monte Carlo samples induces a Gaussian mixture over outputs.
- A.2 Regression setting: Because Gaussian-mixture entropy lacks a closed-form expression, the paper uses mixture variance as an uncertainty metric.The variance decomposes into aleatoric and epistemic components.
- A.2 Regression setting: Aleatoric variance represents irreducible data noise, while epistemic variance reflects lack of knowledge about model parameters.The two components can be expressed as an additive decomposition in regression.
- A.2 Regression setting: Percentiles and quantiles summarize probability mass, with quartiles dividing a predictive distribution into four equal-probability sections.A 20th percentile of 5 means values ≤5 contain 20% of the predictive distribution’s probability mass.
- A.2 Regression setting: Error bars, box plots, and violin plots display predictive-distribution summaries through spread, quartiles, whiskers, and sampled extrema.Figure 4 contrasts a full distribution with mean-and-standard-deviation error bars, box plots, and violin plots.
B CALIBRATION METRICS
The paper reviews calibration metrics for classification and regression predictive distributions, emphasizing that different metrics assess confidence, uncertainty, accuracy, or tail behavior. It also highlights important limitations and validation requirements for interpreting these measures.
- Validation and Scope: Calibration metrics should be computed on an independently sampled validation set, while RCE and TCE apply only to one-dimensional continuous targets.The paper illustrates independent validation with data collected from hospitals different from those used for training.
- Classification Metrics: Brier score measures the mean squared distance between predicted class probabilities and one-hot labels, with lower values indicating better performance.Unlike log-likelihood, it is bounded above and penalizes highly confident errors less strongly.
- Classification Metrics: Classification calibration metrics compare predictive confidence or uncertainty with empirical accuracy or error.ECE focuses on confidence, while UCE focuses on predictive uncertainty.
- Limitations: ECE can produce misleading conclusions because it is not a proper scoring rule and may report perfect calibration for predictions equal to the marginal label distribution.Naive multiclass ECE can also bias results toward low-probability bins.
- Classification Metrics: PAvPU combines accurate-and-certain and inaccurate-and-uncertain predictions to evaluate whether certainty tracks classification accuracy.Its component conditional probabilities are p(accurate | certain) and p(uncertain | inaccurate).
- Regression Metrics: Regression calibration transforms predictive distributions with their cumulative density functions and evaluates whether the transformed values resemble a uniform distribution on [0, 1].RCE assesses global calibration, while TCE focuses on extreme-value behavior.
C UNCERTAINTY QUANTIFICATION METHODS
The paper surveys uncertainty-quantification approaches, focusing mainly on methods for obtaining uncertainty estimates from deep learning models while noting that many techniques generalize more broadly.
- Scope: The appendix presents uncertainty-quantification approaches omitted from the main text for brevity, with emphasis on deep learning models.The authors note that these techniques often apply more generally.
C.1 Bayesian Methods
Bayesian methods represent uncertainty through distributions over plausible model parameters or predictions. The paper reviews approximate inference methods and contrasts their computational cost and fidelity.
- Bayesian Neural Networks: Bayesian neural networks place probability distributions over weights instead of assigning each weight a single fitted value.These distributions are obtained by leveraging or approximating Bayesian updating.
- Variational Inference: Variational inference approximates the posterior p(w|D) with a simpler parameterized distribution q_θ(w) by minimizing KL divergence and maximizing the ELBO.The resulting objective can be optimized with stochastic gradient descent.
- Approximate Inference: Approximate Bayesian methods include variational inference, stochastic gradient MCMC, Monte Carlo dropout, and SWAG.Monte Carlo dropout uses a multiplicative Bernoulli distribution, while SWAG fits a Gaussian approximation from optimization checkpoints.
- Predictive Distributions: Ensembles approximate predictive posteriors by combining predictions from multiple plausible models, which tend to disagree more in data-sparse regions.Agreement is typically greater in data-dense regions.
- Trade-offs: More faithful posterior approximations and better-calibrated uncertainty estimates generally require greater computational intensity and scale worse to larger models.The paper therefore presents method choice as dependent on the use case.
- Bayesian Non-Parametrics: Gaussian processes provide exact probabilistic reasoning and robust uncertainty estimates, but their computational cost grows cubically with the number of data points.They are described as especially suitable for small datasets of ≤5000 points; larger settings generally require approximate algorithms.
C.2 Frequentist Methods
Frequentist approaches can estimate uncertainty after training, including curvature-based sampling and single-pass latent-space methods. The section also describes temperature scaling as a cheap post-hoc calibration technique.
- Post-hoc Methods: Post-hoc frequentist methods obtain uncertainty estimates from non-probabilistic models independently of how those models were trained.Curvature around an optimum can be used to draw plausible weight samples.
- Deterministic Methods: Deterministic uncertainty quantification computes uncertainty as distance to a latent-space centroid using a single forward pass.The same principle is used to obtain epistemic uncertainty estimates in SNGP.
- Calibration: Temperature scaling learns a multiplicative validation-set factor for output logits as a cheap post-hoc way to improve neural-network calibration.The method applies to models trained without probabilistic outputs.
D OTHER ALGORITHMIC USE CASES FOR UNCERTAINTY
Uncertainty supports active learning, selective prediction, downstream optimization, and fairness assessment. These uses depend on meaningful uncertainty estimates and explicit definitions of fairness criteria.
- Active learning: Epistemic uncertainty identifies inputs whose labels would provide the greatest learning value in active learning.This is useful when labels are scarce or expensive, such as labels requiring medical professionals.
- Selective prediction: Models can abstain or defer predictions when high uncertainty indicates likely underperformance on underrepresented inputs.Rejection-option classification uses uncertainty to decide when predictions should be withheld or referred to experts.
- Downstream decision-making: Uncertainty estimates inform downstream ML and operational-research models about the validity of predicted inputs.Stochastic optimization can use distributional inputs to reduce solution costs in the presence of uncertainty.
- Fairness assessment: Fairness assessment commonly evaluates demographic parity, equal opportunity, and equalized odds using group-based prediction or error-rate comparisons.The paper defines groups through a binary sensitive attribute and distinguishes these fairness criteria by the rates they equalize.
- Fairness assessment: Equalized odds requires predicted outcomes and the sensitive attribute to be independent conditional on the actual outcome.This corresponds to equalizing true-positive and false-positive rates across groups.
E.2 Uncertainty and Fairness
Uncertainty intersects fairness through model specification, bias mitigation, and communication. The paper also presents public uncertainty visualization and pandemic forecasting as transparency applications.
- Uncertainty in model specification: Model uncertainty arises when the chosen hypothesis class cannot represent the true data-generating process, potentially biasing predictions.A simple explainable cancer-diagnostic model may omit nonlinear structure.
- Uncertainty in model specification: Expanding the hypothesis class, such as using deep neural networks with sufficiently large datasets, can potentially mitigate specification bias.The paper cautions that model uncertainty is difficult to disentangle from data uncertainty and is rarely detected or analyzed.
- Uncertainty and bias mitigation: Fairness interventions operate at pre-processing, in-processing, or post-processing stages, with uncertainty estimates informing each stage.These stages respectively modify training data, learning objectives, or post-training predictions.
- Uncertainty and bias mitigation: Meaningful and unbiased uncertainty estimates are required for reliable bias mitigation and communication to decision-makers.Deferring uncertain predictions to experts can overcome the incompatibility between calibration and equalized odds when group base rates differ.
- Uncertainty communication: COVID-19 forecasts are communicated to governments, organizations, and individuals for policy, resource allocation, risk judgments, and behavior.The paper identifies public-facing forecast hubs and uncertainty visualization as transparency tools in pandemic communication.