PaperlayerView exact revision on arXiv

Source-linked AI summary

Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling

Bairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas, Shiyu Chang, Yang Zhang

arXiv:2311.08718v2cs.CL

TL;DR

LLM uncertainty decomposition remains understudied, despite its relevance to trustworthy predictions and the practical limits of applying existing decomposition methods to black-box models. Input clarification ensembling varies and ensembles input clarifications to separate ambiguity-related aleatoric uncertainty from epistemic uncertainty. Experiments report reliable total uncertainty estimates and improved ambiguity detection relative to baselines.

  • Problem

    Decomposing LLM uncertainty remains understudied, while existing approaches are difficult to apply to black-box LLMs and may require costly model training.

    Source: §1 · p. 2

  • Method

    Input clarification ensembling generates input clarifications, ensembles predictions across them, and uses a mutual-information decomposition to estimate ambiguity-related aleatoric and epistemic uncertainty.

    Source: §1 · p. 2, §3.4 · p. 5

  • Results

    The method provides reliable total-uncertainty quantification comparable to baselines and achieves the best ambiguity-detection performance, significantly outperforming baselines.

    Source: §4.2 · p. 7, §4.3 · p. 8

  • Takeaways & Limitations

    When uncertainty is high, decomposition distinguishes whether users may need to supply model knowledge or make an ambiguous query more concrete.

    Source: §1 · p. 2

    For factoid QA and mathematical reasoning, the method relies on well-calibrated LLM predictions, and calibration may differ on particular downstream tasks.

    Source: Impact Statement · pp. 9–10

Abstract

from arXiv · show

Uncertainty decomposition refers to the task of decomposing the total uncertainty of a predictive model into aleatoric (data) uncertainty, resulting from inherent randomness in the data-generating process, and epistemic (model) uncertainty, resulting from missing information in the model's training data. In large language models (LLMs) specifically, identifying sources of uncertainty is an important step toward improving reliability, trustworthiness, and interpretability, but remains an important open research question. In this paper, we introduce an uncertainty decomposition framework for LLMs, called input clarification ensembling, which can be applied to any pre-trained LLM. Our approach generates a set of clarifications for the input, feeds them into an LLM, and ensembles the corresponding predictions. We show that, when aleatoric uncertainty arises from ambiguity or under-specification in LLM inputs, this approach makes it possible to factor an (unclarified) LLM's predictions into separate aleatoric and epistemic terms, using a decomposition similar to the one employed by Bayesian neural networks. Empirical evaluations demonstrate that input clarification ensembling provides accurate and reliable uncertainty quantification on several language processing tasks. Code and data are available at https://github.com/UCSB-NLP-Chang/llm_uncertainty.

1. Introduction

Reliable LLM predictions require not only measuring uncertainty but separating ambiguity-driven data uncertainty from model-knowledge uncertainty. Existing decomposition approaches are costly for black-box LLMs, motivating input clarification ensembling as an alternative.

1. Introduction

  • Uncertainty quantification measures model confidence, and higher uncertainty can signal that generated output should be clarified, evaluated, or rejected.Prior LLM work has quantified total uncertainty through methods including ensembles and prompted confidence reports.Source: §1 · p. 1
  • Uncertainty decomposition distinguishes epistemic uncertainty from missing model knowledge and aleatoric uncertainty from ambiguity or randomness in the data-generating process.Input ambiguity is a key source of aleatoric uncertainty in language tasks.Source: §1 · pp. 1–2
  • Separating uncertainty sources offers distinct responses: users can supply knowledge for high epistemic uncertainty or make queries more concrete for high aleatoric uncertainty.The paper treats ambiguity-driven aleatoric uncertainty as irreducible regardless of how well the model learns.Source: §1 · p. 2
  • Bayesian neural networks require training-procedure changes, while deep ensembles require training multiple model variants, making both approaches impractical or prohibitively expensive for LLMs.The paper frames effective LLM uncertainty quantification and decomposition as an open question under these constraints.Source: §1 · p. 2
  • Input clarification ensembling varies input clarifications, ensembles the resulting LLM predictions, and avoids modifying model parameters or inference procedures.The framework uses prediction disagreement across clarifications to gauge aleatoric uncertainty and assigns each clarified prediction’s remaining uncertainty to epistemic uncertainty.Source: §1 · p. 2

2. Related Work

Prior work addresses uncertainty quantification and calibration for LLMs, while ambiguity research examines ambiguous inputs and model handling. The paper positions its approach as quantifying total uncertainty while also decomposing it.

Uncertainty quantification

  • Intrinsic methods include Bayesian and ensemble approaches, while extrinsic methods use auxiliary models post hoc; the paper places its method in the intrinsic family.The paper says its approach is directly motivated by Bayesian neural networks.Source: §2 · pp. 2–3

Uncertainty Quantification and Model Calibration for LLMs

  • Existing LLM uncertainty methods estimate total uncertainty, whereas this work proposes quantifying total uncertainty and decomposing it into aleatoric and epistemic components.Prior approaches include semantic clustering, sampled-answer consistency, and directly prompted confidence estimates.Source: §2 · p. 3
  • Prior work reports relatively well-calibrated LLM predictions on factual QA and complex reasoning tasks when properly prompted, and this method relies on well-calibrated predictions.Calibration aligns predicted probabilities or confidence scores with actual event likelihoods.Source: §2 · p. 3

Modeling Ambiguity with language models

  • Ambiguity benchmarks and studies report that language models struggle to recognize and manage ambiguous inputs; this work uses aleatoric uncertainty to identify potential ambiguity.The paper reports that decomposing aleatoric uncertainty can enhance ambiguity detection performance.Source: §2 · p. 3

3. Methodology

Input clarification ensembling decomposes LLM predictive uncertainty by varying clarifications of an input, while keeping model parameters fixed. The method treats disagreement across clarifications as an estimate of input-ambiguity uncertainty and average uncertainty on clarified inputs as largely epistemic.

3.1. Notations and Problem Formulation

  • Total predictive uncertainty is entropy, while epistemic uncertainty reflects model knowledge and aleatoric uncertainty reflects irreducible uncertainty in the ground-truth distribution.The paper aims to estimate both components for LLMs.Source: §3.1 · pp. 3–4

3.2. Background: Bayesian Neural Networks and DEEP ENSEMBLES

  • Bayesian neural networks and DEEP ENSEMBLES decompose predictive uncertainty into model disagreement and average individual-model uncertainty, which can approximate epistemic and aleatoric uncertainty under assumptions.Bayesian neural networks model parameter distributions; DEEP ENSEMBLES use multiple trained models.Source: §3.2 · p. 4

3.3. Do BNN and DEEP ENSEMBLES work for LLMs?

  • For LLMs, parameter access and compute make fine-tuning multiple models difficult, while in-context ensembles do not optimize the training objective and yield inaccurate decomposition.On AmbigQA, aleatoric uncertainty differed only slightly between ambiguous and unambiguous questions under the tested in-context ensemble.Source: §3.3 · p. 4

3.4. Input Clarification Ensembling

  • Input clarification ensembling averages predictions across texts that reduce input ambiguity, using their disagreement to estimate aleatoric uncertainty caused by ambiguity.The framework keeps the LLM parameters fixed; average uncertainty on clarified inputs is mostly attributed to epistemic uncertainty.Source: §3.4 · p. 5
  • Clarifications are texts appended to an input to reduce its ambiguity, and predictions are ensembled over their distribution while the model parameters remain constant.The method’s mutual-information term estimates aleatoric uncertainty caused by input ambiguity; the paper mainly focuses on that component.Source: §3.4 · p. 5

3.5. Input Clarification

  • The method focuses on instruction and question ambiguity, and uses a clarification LLM prompted with instructions and examples or fine-tuned to generate clarifications.The framework can separately assess ambiguity in different input components and is described as generalizable beyond those studied.Source: §3.5 · pp. 5–6

3.6. Improving Performance via Soliciting Clarifications

  • The framework proposes using uncertainty estimates to identify unclear input components and offer users clarification options for improving their queries.For example, high uncertainty attributed to an ambiguous instruction can prompt feedback and suggested clarifications.Source: §3.6 · p. 6

4. Experiments

The experiments assess total-uncertainty reliability and uncertainty decomposition through mistake detection, ambiguity detection, a monotonicity check, and answer recall. Across these evaluations, the method provides reliable uncertainty estimates, identifies ambiguity, lowers aleatoric uncertainty after clarification, and improves answer recall with more clarifications.

4. Experiments

  • The experiments evaluate total-uncertainty quality through mistake detection and decomposition through ambiguity detection, monotonicity, and answer recall.Source: §4 · p. 6

4.1. Experiment Configurations

  • Experiments use gpt-3.5-turbo-0613 as the default LLM, and prompt it to cluster semantically equivalent generated answers for output-distribution estimation.For the main mistake-detection evaluation, the authors use 200 validation examples each from NQ and GSM8K.Source: §4.1 · p. 6, §4.2 · p. 7

4.2. Quantifying Total Uncertainty

  • The proposed method provides reliable total-uncertainty estimates, with performance comparable to the baselines on mistake detection.The authors report that it remains effective on GSM8K, where ASK4CONF estimates uncertainty poorly.Source: §4.2 · p. 7

4.3. Uncertainty Decomposition

  • The method achieves the best ambiguity-detection performance, while ground-truth disambiguations further improve performance and dataset ambiguity affects results.Performance is lower on AmbigQA, where ambiguities are subtle, than on AmbigInst, where they are explicit; fine-tuned LLaMA clarification generation takes under 10 minutes on 4×80G H100 GPUs.Source: §4.3 · p. 8
  • Qualitative examples attribute uncertainty to input ambiguity in an underspecified FM-dial question and to epistemic uncertainty in a clear question about the Royal Proclamation.Source: §4.3 · p. 8

4.4. Monotonicity Check

  • Aleatoric uncertainty drops significantly after clarification on AmbigQA and AmbigInst, supporting the clarification network’s effectiveness.Source: §4.4 · p. 9

4.5. Recall of Correct Answers

  • Recall of target answers consistently increases with more clarifications, significantly outperforming direct answers to ambiguous inputs.The improvement is more significant on AmbigInst than on AmbigQA.Source: §4.5 · p. 9

5. Conclusion

The paper proposes input clarification ensembling for decomposing LLM uncertainty. Experiments affirm that it provides reliable uncertainty quantification and separates total uncertainty into aleatoric and epistemic components.

  • The framework leverages input clarifications in a structure symmetric to BNN methods, making it more suitable for black-box LLMs.The authors identify a more effective clarification module as future work.Source: §5 · p. 9

Impact Statement

The paper frames its uncertainty-quantification work as supporting users in identifying sources of LLM uncertainty and improving trustworthiness and reliability. Its stated limitation is reliance on well-calibrated LLMs for factoid QA and mathematical reasoning.

Impact Statement

  • The framework aims to help users pinpoint sources of LLM uncertainty, with trustworthiness and reliability as its principal application scenario.The authors also state that they assess evaluations and datasets for harmful content or adverse impacts.Source: Impact Statement · p. 9
  • The method relies on well-calibrated LLMs for factoid QA and mathematical reasoning.The authors note that calibration may differ on particular downstream tasks.Source: Impact Statement · pp. 9–10

A.1. Additional Results

Additional ambiguity-detection results show that three model-confidence methods have AUROC scores below 60. The appendix also describes their confidence signals and supervised fine-tuning of Llama-3-8B-Instruction on AmbigQA.

A.1. Additional Results

  • The additional confidence measures are SELFCONSISTENCY, SAMPLE REPETITION, and SAMPLE DIVERSITY, based on sampled-answer frequency or answer diversity.SELFCONSISTENCY samples 10 answers at temperature 0.7; SAMPLE DIVERSITY counts unique answers, and SAMPLE REPETITION resamples answers generated with greedy decoding.Source: §A.1 · p. 13
  • All three model-confidence methods score below 60 AUROC for identifying ambiguous inputs.Table 3 reports average aleatoric uncertainty for unambiguous and ambiguous questions.Source: §A.1 · p. 13, Table 3 · p. 13

A.2. Supervised Fine-tuning for Clarification Generation

  • Llama-3-8B-Instruction was fine-tuned on the full AmbigQA training set for five epochs using batch size 16 and learning rate 2e-5; the lowest-validation-loss checkpoint (epoch 2) was tested.Source: §A.2 · p. 13

A.3. Implementation details for baselines

The appendix specifies procedures for mistake and ambiguity detection, and points to dataset-specific prompts for clarification generation. Its fine-tuning template uses an original question and a ground-truth clarification.

A.3. Implementation details for baselines

  • Mistake detection estimates the answer distribution, selects its most frequent answer, and prompts ChatGPT to judge correctness before AUROC and F1-threshold evaluation.Correct answers are treated as positive examples for the F1 threshold search.Source: §A.3 · p. 13
  • For mistake detection, ASK4CONF estimates confidence by prompting the LLM about its most frequent sampled answer rather than directly generating an answer.The implementation uses the “Verb. 2S top-1” method and its corresponding prompts.Source: §A.3 · p. 13
  • For ambiguity detection, the listed signals are total uncertainty, BNN* aleatoric uncertainty, and ASK4CONF’s ambiguity confidence score.Source: §A.3 · p. 14

A.4. Prompts for Our Clarification Model

  • The clarification-generation prompts are specified separately for Natural Question and GSM8K, AmbigQA, and AmbigInst.The fine-tuning template has slots for the original question and its ground-truth clarification.Source: §A.4 · p. 14, Figure 5 · p. 14

A.5. Details of the Qualitative Results

The appendix describes sampling question paraphrases as clarifications and post-processing the resulting answers. It uses an LLM prompt for answer extraction and handles refusals and invalid clarifications separately.

A.5. Details of the Qualitative Results

  • For the GSM8K illustration, the full question asks for sightseeing time after giving activity durations and their shares of total time.Source: §A.5 · p. 14
  • Five LLM-generated paraphrases of each question serve as clarifications for uncertainty quantification.The prompt asks the LLM to paraphrase a question so it is easier to understand and solve.Source: §A.5 · p. 14

A.6. Prompt the LLM for Answer Extraction

  • The authors use an LLM prompt to extract answers and cluster semantically equivalent outputs, reporting better performance than NLI models for this task.The prompt is shown in Figure 11.Source: §A.6 · pp. 14–15
  • Refusal-like answers map to “Unknown”; all-Unknown clarifications are dropped, while remaining Unknown answers increase epistemic uncertainty through a frequency adjustment.The adjustment distributes each Unknown occurrence evenly across the other unique answers.Source: §A.6 · p. 15

B.1. Dataset Creation

The dataset combines ambiguous tasks generated and clarified with ChatGPT with manually disambiguated tasks drawn from an existing instruction dataset. Its examples pair inputs with clarification-dependent answers and filter cases where different clarifications yield the same answer.

B.1. Dataset Creation

  • 15 ambiguous tasks were generated from ChatGPT-produced task descriptions and ambiguity causes, with open-ended tasks filtered out and ground-truth clarifications generated from those causes.The descriptions follow the SELF-INSTRUCTION pipeline; the final ambiguous-instruction set contains 214 input questions.Source: §B.1 · pp. 15–16
  • Each ambiguous task received 15 inputs paired with answers for the ground-truth clarifications, and inputs with identical answers across clarifications were removed.The resulting ambiguous-instruction set contains 214 input questions.Source: §B.1 · p. 16
  • Ten tasks from Instruction Induction were manually clarified and paired with 15 input-output examples each, yielding 25 tasks and 364 inputs overall.The ten selected tasks include letter and word operations, comparisons, arithmetic, and antonyms.Source: §B.1 · p. 16

B.2. Dataset Examples

The examples show instruction-input pairs across list operations, numerical tasks, sentence processing, and prompts that depend on information in tables. Several illustrate how an instruction can leave the relevant operation or criterion unspecified.

B.2. Dataset Examples

  • Prompts ask for sorting names or objects, alphabetizing data, and rearranging objects in ascending order.Examples include a list of names, an animal list, and a table of objects with size, weight, color, date, and price fields.Source: §B.2 · pp. 16–18
  • Numerical tasks include averaging a list with rounding, counting listed objects, finding a middle value, and determining a square root.The average example supplies four decimal-valued numbers, while the counting example includes repeated objects.Source: §B.2 · pp. 16–19
  • Table-based instructions ask for football-player rankings, the largest city, or file organization by date.The examples provide player statistics, city population and land area, and file creation and last-modified dates.Source: §B.2 · pp. 17–18
  • Other prompts ask for a country's capital or movie classification using a film's MPAA, IMDb, and Rotten Tomatoes ratings.The movie example gives ratings for Toy Story 4.Source: §B.2 · p. 19
  • Language-focused examples ask for a sentence's length or index in a longest-sentence selection, and for identifying a sentence's subject.The inputs include a sentence and three numbered lines from Hamlet.Source: §B.2 · pp. 16, 20
Loading 2311.08718v2…