Source-linked AI summary
Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models
Kaitlyn Zhou, Dan Jurafsky, Tatsunori Hashimoto
TL;DR
As language models are increasingly used for factual tasks, it matters whether they interpret expressions of confidence, sources, and information limits. This paper builds a typology of epistemic markers and tests them in question prompts, finding that certainty markers can reduce accuracy. It also examines how these expressions appear in pretraining data and discusses limits to the study’s scope.
Problem
Because language models are increasingly used for factual tasks, it is important to understand how they interpret linguistic cues about confidence, sources, and information limits.
Source: §1 · p. 1Method
The authors develop a typology of epistemic markers and inject verbal and numerical markers into trivia-question prompts to measure effects on generation.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
As language models expand into general language use, their interpretation of naturalistic expressions of uncertainty and certainty warrants attention to possible opportunities and harms.
Source: §9 · p. 9The study calls for testing more models and datasets and exploring multi-shot prompting; long-form and dialogue generation are beyond its scope.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
The increased deployment of LMs for real-world tasks involving knowledge and facts makes it important to understand model epistemology: what LMs think they know, and how their attitudes toward that knowledge are affected by language use in their inputs. Here, we study an aspect of model epistemology: how epistemic markers of certainty, uncertainty, or evidentiality like "I'm sure it's", "I think it's", or "Wikipedia says it's" affect models, and whether they contribute to model failures. We develop a typology of epistemic markers and inject 50 markers into prompts for question answering. We find that LMs are highly sensitive to epistemic markers in prompts, with accuracies varying more than 80%. Surprisingly, we find that expressions of high certainty result in a 7% decrease in accuracy as compared to low certainty expressions; similarly, factive verbs hurt performance, while evidentials benefit performance. Our analysis of a popular pretraining dataset shows that these markers of uncertainty are associated with answers on question-answering websites, while markers of certainty are associated with questions. These associations may suggest that the behavior of LMs is based on mimicking observed language use, rather than truly reflecting epistemic uncertainty.
1 Introduction
As language models are used for factual tasks, the paper asks how naturalistic epistemic language affects their generation. It introduces a typology of markers and finds that certainty expressions can lower accuracy.
1 Introduction
- Prior linguistic-calibration work focused on mapping model probabilities to ordinal outputs, while epistemic expressions also convey sources, speaker commitment, and assertion or presupposition.Source: §1 · p. 1
- The authors use zero-shot prompts that inject verbal and numerical markers into trivia questions and measure their effects on model accuracy.Source: §1 · p. 1
- High-certainty expressions lower accuracy, contrary to the expectation that certainty cues would improve responses.The paper reports this result for expressions such as “I’m certain it’s” and “I’m 100% sure.”Source: §1 · p. 2
- The study develops a typology of epistemic markers, including hedges, factive verbs, and evidentials, to examine their effects on language generation.Source: §1 · pp. 1–2
- Qualitative and quantitative analyses investigate possible origins of the unexpected behavior, and the findings point to shortcomings in models’ interpretation of uncertainty expressions.Source: §1 · p. 2
2 Expressions of Certainty and Uncertainty: Linguistic Background
The paper distinguishes linguistic markers that weaken or strengthen commitment, as well as expressions that indicate information sources. It uses these distinctions to build a taxonomy of uncertainty templates for question answering.
2 Expressions of Certainty and Uncertainty: Linguistic Background
- Hedges weaken category membership or speaker commitment, whereas strengtheners mark certainty and factive verbs presuppose their complement is true.Source: §2 · p. 2
- Evidentials indicate where information came from and can express either certainty or uncertainty; the study distinguishes explicit source mentions from indirect or unspecified sources.Source: Figure 2 · p. 2, §2 · p. 3
2.1 Expressions of Uncertainty Typology
- The authors combined literature, brainstorming, and crowdsourcing to develop and classify templates across linguistic dimensions, including factives, evidentials, sources, and personal pronouns.Source: §2.1 · p. 3
2.2 Amazon Mechanical Turk Details
- Mechanical Turk workers supplied additional expressions; the authors reviewed nine five-example samples and excluded approximators because trivia answers are discrete.Source: §2.2 · p. 3
3 Methods
The methods inject epistemic markers into trivia questions and evaluate model responses across several question-answering datasets. The study also describes its prompting procedures and filters answers to support gold-answer scoring.
3 Methods
- Fifty minimal-pair templates were applied per question across TriviaQA, Natural Questions, CountryQA, and Jeopardy; CountryQA included 53 in-vocabulary questions.The other three datasets used random subsets of 200 questions each.Source: §3 · p. 3
- Findings are aggregated across templates with shared characteristics, with 95% confidence intervals calculated using bootstrap resampling or t-tests.Source: §3 · p. 3
3.1 Additional Prompting Details
- Prompts used Q:/A: delimiters and avoided trailing spaces; the paper reports model-specific experiment dates from February through August 2023.Source: §3.1 · p. 4
3.2 Challenges in Open-Ended Generation
- Filtering out multiword and out-of-vocabulary answers supports gold-answer probability scoring, while the retained high-frequency answers help reduce question-difficulty variation.Source: §3.2 · p. 4
4 The Impact of Uncertainty on Language Generation
The section tests how verbal uncertainty cues affect GPT-3 question answering and finds that accuracy varies substantially with the wording of the prompt. Weakeners outperform strengtheners, and evidential markers often help, while high-certainty templates can reduce accuracy.
4.1 Variation in GPT3 Responses Across Verbal Uncertainties
- Accuracy varied by up to 80% across prompts on the same questions, including 14% for “We realize it’s...” versus perfect accuracy for many CountryQA templates.In TriviaQA, “I’m certain it’s...” reached 42% accuracy, versus 56% for “I would need to double check but maybe it’s...”.Source: §4.1 · p. 4
- Across four datasets, weakeners averaged 47% accuracy versus 40% for strengtheners; factive verbs consistently hurt, while evidentials improved performance in three datasets.Factive verbs drive the strengthener disadvantage, and evidential markers significantly improve results in three of four datasets.Source: §4.1 · pp. 4–5, Figure 3 · p. 5
- Across six tested models, hedges outperformed boosters, non-factive verbs outperformed factives, and evidentials outperformed non-evidentials.The additional model tests reproduce the main results on TriviaQA.Source: Table 1 · p. 5, §4.1 · p. 5
- Plausibility shields, sources, and personal pronouns showed no consistent improvement across the four datasets.Source: §4.1 · p. 5
4.2 Expressions of Uncertainty Compared to the Standard Prompting Method
- Across six models and four datasets, six templates significantly outperformed standard prompting (t-test, p < 0.05); on TriviaQA, one template achieved 66% accuracy versus 63% for standard prompting.In Natural Questions, seven templates outperformed standard prompting; six were expressions of uncertainty.Source: §4.2 · p. 5
5 Why Does Certainty Hurt?
The authors test whether prompt perplexity explains the accuracy differences and examine how weakeners affect answer probabilities. They find no perplexity-accuracy correlation, while pretraining-text patterns offer a possible account of why certainty cues can mislead models.
5.1 Certainty Affects Performance Independent of Perplexity
- Template perplexity was not correlated with prompt accuracy (Pearson’s ρ = -0.03), weakening the explanation that prompt frequency or length caused the observed differences.Source: §5.1 · p. 6
5.2 A Redistribution of Probability Mass When Prompted with Weakeners
- Among correctly predicted answers, probability-on-gold was lower for weakeners than strengtheners in NaturalQA (42% vs 45%), JeopardyQA (47% vs 51%), and TriviaQA (53% vs 55%).Probability-on-gold is the summed probability assigned to answer aliases.Source: §5.2 · p. 6
- Alternative-token entropy was higher with weakeners than strengtheners across all four datasets, indicating probability was spread more evenly across alternative answers.The authors interpret this as evidence that weakener accuracy gains do not come from greater confidence in the answer.Source: Table 2 · p. 6, §5.2 · p. 6
5.3 Certainty Used in Questions Instead of Answers
- In Stack Exchange text, certainty phrases appeared less than half as often in answers as questions (104 vs 280 instances per million words), while uncertainty phrases appeared about twice as often in answers (436 vs 222).Source: §5.3 · p. 6
- Qualitative coding found “I’m sure” often used in questions to establish group membership or isolate an issue, while “I think” often softened instructions or corrections in answers.The authors suggest models may mimic these observed uses when responding to epistemic markers.Source: §5.3 · pp. 6–7
6 The Impact of the Degree of Uncertainty on Performance
The section tests whether numerical certainty markers track the accuracy of generated answers. They are poorly calibrated, and accuracy tends to fall at the certainty extremes, especially at 100%.
6 The Impact of the Degree of Uncertainty on Performance
- The numerical-marker evaluation used seven templates across 0%, 10%, 30%, 50%, 70%, 90%, and 100%, with 50 questions sampled per dataset.Source: §6 · p. 7
- Numerical uncertainty markers were poorly calibrated: expected calibration error ranged from 0.50 to 0.30, with 0 being best; “I’m 90% sure” yielded 57% accuracy on TriviaQA.Source: §6 · p. 7
- Accuracy usually peaked between 70% and 90% certainty, then dropped at 100%; 21 of 28 templates using 100% had lower probability-on-gold than their 90% counterparts.Accuracy also dropped sharply for templates using 0%.Source: §6 · p. 7
7 Why Does 100% Certainty Hurt?
The section explores why 100% certainty prompts hurt performance, proposing that numerical expressions may be used non-literally in training text. Its Pile analysis finds few examples of literal certainty and more expressions of uncertainty in questions than answers.
7 Why Does 100% Certainty Hurt?
- In a 100-example sample, 44 uses of “100%” involved negation and half expressed uncertainty, while only three indicated certainty; uncertainty uses were nearly twice as common in questions as answers (14 vs 8).The authors hypothesize that negation and the relative rarity of certainty uses contribute to reduced performance.Source: §7 · p. 7
- The authors propose that “100%” may appear in hyperbolic or non-literal uses in training text, and report spikes in percentage frequencies at 50%, 95%, and 100% in the Pile.The frequency visualization uses the first million samples queried from the Pile dataset.Source: §7 · p. 7, Figure 5 · p. 8
- Figure 4 shows accuracy dropping between 90% and 100% injected uncertainty and increasing between 0% and 10%.Source: Figure 4 · p. 8
8 Related Work
Prior model-uncertainty work has emphasized confidence extraction and calibration, while this work connects language generation with research on speaker commitment through natural expressions of certainty and uncertainty.
9 Related Work
- Prior work studies confidence extraction, model calibration, and linguistic calibration, including efforts to reduce overconfidence or teach verbal calibration.The cited work includes Kuhn et al. (2023), Sun et al. (2022), Gleave and Irving (2022), Mielke et al. (2022), Kadavath et al. (2022), and Lin et al. (2022).Source: §8 · p. 8
- Epistemic expressions span speaker commitment and discourse features such as hedges, factivity, evidentiality, and whether knowledge is asserted or presupposed.Prior linguistic work examines factivity and projection, while computational work covers factuality, veridicality, commitment, bias, hedges, and modality.Source: §8 · pp. 8–9
- This work bridges speaker-commitment research and natural language generation by examining how models interpret commitment through expressions of certainty and uncertainty.Source: §8 · p. 9
9 Discussion and Conclusion
The paper reports that certainty expressions and factive verbs reduce accuracy, and discusses how models handle generated uncertainty, idiomatic language, and attributions. It also identifies risks from unverified attributions and limits in interpreting idiomatic expressions.
Can Models Generate Expressions of Uncertainty?
- Preliminary experiments find that numerical-calibration methods transfer poorly to generating verbal epistemic markers, while uncertainty expressions are slightly better calibrated than certainty expressions.Source: §9 · p. 9
Navigating Idiomatic Language
- Models appear to mimic humans' use of certainty when they are not actually certain, yet fail to recognize idiomatic versus literal meanings in the question-answering setup.The authors identify accurate interpretation of idiomatic versus literal expressions as future work.Source: §9 · p. 9
Verified Attribution
- Some high-performing templates use falsely injected attributions such as “Wikipedia says,” which can give downstream users a believable attribution without verifying its accuracy.Source: §9 · p. 9
9 Discussion and Conclusion
- Accuracy drops when naturalistic certainty expressions, including strengtheners and factive verbs, are present; the paper traces the effect to their use in pretraining data.Source: §9 · p. 9
Limitations
The study's scope is bounded by its model, dataset, prompting, language, and generation settings; the authors identify several extensions for future work.
Limitations
- The work identifies scaling to other models, increasing dataset variety, and testing multi-shot prompting as limitations.The authors describe the results as robustly tested but say scalability would benefit the community if more compute and time were available.Source: Limitations · p. 9
- Multi-shot prompting, long-form generation, and dialogue generation remain outside the study's scope.The authors describe multi-shot prompting as a realistic scenario that would expand exploration of epistemic markers.Source: Limitations · p. 9
- The study examines English hedges and boosters, although expressions of uncertainty vary across cultures and contexts.Syntactic, idiomatic, and pragmatic differences in hedges are identified as follow-up topics.Source: Limitations · p. 9
Ethics Statement
The ethics discussion highlights risks from unverified attributions and from generating certainty expressions that may encourage over-reliance on AI.
Ethics Statement
- Unverified evidentials can sound convincing to downstream users while providing little guarantee that statements are accurate.Source: Ethics Statement · p. 9
- Teaching models to emit uncertainty expressions when unsure, rather than certainty expressions, may be a safer design choice for human-computer interactions.The authors warn that generating certainty expressions could further exacerbate over-reliance on AI, including when systems are wrong.Source: Ethics Statement · p. 10
A Appendix
The appendix tests how models emit uncertainty and how prompt and training-example design affect performance. It reports weak calibration alongside effects from marker placement, numerical uncertainty, and other linguistic features.
A.1 When LMs Emit Their Own Uncertainty
- The in-context dataset contains 48 examples distributed across probability-on-gold buckets from 0 to 100 in increments of 10.The method follows a 50-sample few-shot setup, with the stated 48 examples covering confidence levels uniformly.Source: §A.1 · p. 13
A.1.1 Experiment Details
- Models were trained to emit strengtheners above a confidence threshold and weakeners below it, emitting nothing otherwise.For example, “I’m sure” was appended to an example when the probability-on-gold exceeded 0.5.Source: §A.1.1 · p. 13, §A.1.2 · p. 13
A.1.2 Prompting Perturbations
- Accuracy remained approximately 83% across thresholds 0.3, 0.5, 0.7, and 0.9; later experiments used a 0.5 threshold and random sample order.The perturbations also tested ascending and descending sample order and expressions placed after answers.Source: §A.1.2 · p. 13
A.1.3 Results
- 63% vs 80% accuracy for uncertainty prefixes versus suffixes; probability-on-gold was 40% vs 67%, respectively.The authors suggest that asking for the answer immediately may improve accuracy. Weakener emission coincided with 74% accuracy versus 83% when no weakener was emitted; strengtheners yielded 79% accuracy either way. Weakener templates reached macro-F1 0.56 versus 0.53 for certainty templates, against a 0.45 random-guessing baseline, while entropy varied with marker emission.Source: §A.1.3 · pp. 13–14, Table 3 · p. 14
A.2 Perplexity vs Accuracy
- ρ = -0.03 for the correlation between GPT-3 davinci perplexity and uncertainty-expression accuracy on TriviaQA questions.Source: Figure 6 · p. 14
A.3 Performance of Additional Linguistic Features
- Plausibility shields, sources, and personal pronouns show mixed effects, without consistent significant accuracy gains or drops.Source: Figure 8 · p. 15
A.4 Additional Tables and Results
- Supplementary results show accuracy variation across numerical uncertainties, alongside tables of Stack Exchange marker counts and six-feature marker coding.Other appendix tables list top templates for individual datasets and across GPT models.Source: Figure 9 · p. 16, Table 5 · p. 16, Table 6 · p. 17