PaperlayerView exact revision on arXiv

Source-linked AI summary

When Linguistic and Internal Confidence Diverge in Large Language Models

Hefan Zhang, Bingquan Zhang, Ming Cheng, Saeed Hassanpour, Weicheng Ma, Soroush Vosoughi

arXiv:2608.28382v1cs.CLcs.AI

TL;DR

LLM confidence reports are used to help decide when answers should be trusted, yet it remains unclear whether they align with models’ internal probability signals. The paper evaluates this alignment across tasks, models, and prompting regimes, finding weak instance-level alignment and divergence across confidence measures.

  • Problem

    Although LLM confidence reports help decide whether answers should be trusted, filtered, abstained from, or combined, their alignment with internal probability signals remains unclear.

    Source: §1 · p. 1

  • Method

    The paper systematically evaluates linguistic and internal confidence across model families, tasks, and prompting regimes, comparing association, magnitude agreement, calibration, and generation-task semantic-entropy uncertainty.

    Source: §1 · p. 2, §5.4 · p. 6

  • Results

    Across 10 tasks and 30 models, instance-level alignment is weak; linguistic dispersion can preserve rank-order signal but does not guarantee magnitude agreement or calibration.

    Source: §9 · p. 8, §10 · p. 9

  • Takeaways & Limitations

    Linguistic confidence should not be treated as calibrated probability and should be used downstream only with distribution-aware validation or held-out calibration.

    Source: §10 · p. 9

    The evaluation compares available confidence channels rather than identifying true uncertainty, and semantic entropy depends on sampling and NLI-based clustering choices.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Users often ask large language models (LLMs) to report how confident they are, but it is unclear whether such linguistic confidence tracks the model's internal confidence. We study this question across 8 classification tasks, 2 generation tasks and 30 models from three families. For classification, we compare linguistic confidence with logits-based confidence along three axes: association, magnitude agreement and calibration. For generation, we test whether linguistic confidence tracks semantic-entropy-based uncertainty. The axes frequently diverge. Instance-level association is weak on average, although it improves on easier items and for stronger base models. Instruction-tuned models often report higher confidence and sometimes show higher association, but they also have larger confidence gaps and worse calibration. Prompt design mostly changes the distribution of reported confidence. Attitude cues inflate confidence without improving alignment, while score exemplars can preserve rank-order signal when they avoid collapsed confidence values. Regression analyses show that distributional properties of confidence scores explain much of the observed alignment pattern, with model metadata playing a smaller role after controls. These results support a lossy-channel view of linguistic confidence. A more dispersed verbal confidence distribution can carry useful rank information, but it does not make the scores calibrated. Linguistic confidence should therefore be evaluated with multi-axis diagnostics before being used in downstream reliability pipelines.

1 Introduction · 2 Related Work · 3 Confidence Measurements and Evaluation

The paper finds that linguistic confidence only inconsistently tracks internal confidence, and evaluates these signals separately for association, magnitude agreement, and calibration. It describes how the signals are measured across classification and generation tasks.

1 Introduction

  • LLMs are used to provide confidence signals for downstream decisions, but internal probabilities may be unavailable, leaving practitioners with linguistic self-reports.The paper asks whether verbal confidence aligns with an internal probability signal, rather than treating logits-based confidence as ground-truth uncertainty.Source: §1 · p. 1
  • Across 10 tasks and 30 models, instance-level association is weak on average, while easier tasks and stronger base models tend to show higher association.Instruction-tuned models often report higher confidence, but apparent association gains do not reliably improve calibration. More dispersed verbal scores can preserve rank-order information, yet do not ensure calibration and may increase magnitude mismatch.Source: §1 · pp. 1–2

2 Related Work

  • Prior work often evaluates verbal confidence against correctness; this study separately analyzes cross-channel association and calibration across models, tasks, and prompting regimes.It also examines task difficulty, model properties, prompting, and confidence-score distributions as factors associated with the outcomes.Source: §2 · p. 2

3 Confidence Measurements and Evaluation

  • For classification, the study compares linguistic confidence with logits-based confidence; for generation, it compares linguistic confidence with semantic entropy over sampled responses.Logits-based probabilities serve as an internal confidence proxy, while semantic entropy is used where answer-token probabilities are not well defined.Source: §1 · p. 1, §3 · p. 3

3.1 Linguistic Confidence

  • Models are prompted to answer and report confidence on a 0–10 scale, which is divided by 10 for comparison with probabilities.The prompt defines 0 as no belief in the answer and 10 as full confidence.Source: §3.1 · p. 3

3.2 Internal Confidence Proxies

  • Classification confidence is the softmax probability of the selected answer among candidate options; generation uncertainty uses semantic entropy over clusters of equivalent sampled responses.Softmax probabilities are an internal proxy, not ground-truth uncertainty; higher semantic entropy means lower confidence, so the study negates it for association analyses.Source: §3.2 · p. 3

3.3 Evaluation Axes

  • Association uses Pearson and Spearman correlations, magnitude agreement uses Euclidean distance, and calibration uses expected calibration error against correctness.For generation, association with negated semantic entropy is the primary cross-channel metric; distance and ECE are used for classification. Correlation, distance, and ECE capture distinct forms of agreement, so the paper reports them jointly.Source: §3.3 · pp. 3–4

4 Experimental Setting

The study evaluates linguistic confidence across classification and generation tasks using 30 models from three model families, with standardized prompts and robustness tests.

4 Experimental Setting

  • The evaluation covers 8 classification tasks and 2 generation tasks, including CoQA (Reddy et al., 2019) and TriviaQA (Joshi et al., 2017).Five evaluation sets with more than 200 examples are randomly downsampled to 200 instances; the other tasks use complete evaluation sets.Source: §4 · p. 4
  • The study uses 30 publicly available LLaMA, Mistral, and Qwen models spanning different sizes, versions, and tuning types.Table 6 lists all models.Source: §4 · p. 4
  • Standardized prompts use two-shot examples for base models and system-prompt format requirements for instruction-tuned models, with additional prompt robustness tests.Tests include confidence-exemplar search, confidence-elicitation ablation, and attitude-based perturbations.Source: §4 · p. 4

5 Results

Across classification and generation, linguistic confidence carries limited rank-order signal that varies with task, model, and prompt. Aggregate trends can obscure weak instance-level alignment, and association does not establish calibration.

5.1 Overall Alignment: Confidence Measures Align Weakly

  • Across settings, instance-level mean Pearson association was r = 0.135, versus pooled-setting Pearson r = 0.4830; base models showed weak association while instruction-tuned models showed negligible aggregate association.Instruction-tuned models often concentrate logits-based and linguistic confidence near high values, compressing variation; the base-model pattern does not imply reliability.Source: Figure 1 · p. 4, §5.1 · pp. 4–5

5.2 Task-Level Variation: Association Tends to Be Higher on Easier Tasks

  • Across eight tasks, the normalized-accuracy association slope was 0.360 (95% CI [−0.842, 1.562], p = 0.491), versus 1.067 (95% CI [0.075, 2.058], p = 0.0396) excluding Ruin Names.Thirty of 45 model–prompt slopes were positive (mean = 0.034, 95% CI [−0.155, 0.223]); Ruin Names had low accuracy but high association, potentially reflecting surface familiarity rather than calibration.Source: §5.2 · p. 5

5.3 Model-Level Variation: Model Properties Affect Alignment

  • Base-model association tends to rise with size, while instruction-tuned association is flat or falls; instruction tuning’s small Pearson gain is nonsignificant despite significantly larger distance and linguistic ECE.Confidence-score dispersion helps explain these patterns; across families, association generally rises from LLaMA to Mistral and Qwen as distance and ECE fall, but version updates are less consistent.Source: Figure 3 · p. 5, §5.3 · pp. 5–6

5.4 Generation Setting: Association Remains Task- and Model-Dependent

  • On generation tasks, instruction-tuned models usually show stronger association than base models, and TriviaQA shows higher association than CoQA.The comparison uses association with negated semantic entropy; it indicates rank-order information, not calibrated uncertainty.Source: §5.4 · p. 6

5.5 Prompt-Level Variation: Prompt Design Mainly Changes Score Distributions

  • Criticism, approval, and irrelevant cues raised reported confidence while reducing association; a calibration instruction did not significantly improve alignment.Adjacent but non-identical confidence exemplars worked better than identical or very large scores, which collapsed or reduced useful variation; removing confidence elicitation did not change accuracy and had limited effects on logits statistics.Source: §5.5 · pp. 6–7

6 Distributional Properties Explain Much of the Pattern

Confidence-distribution properties were more recurring predictors of alignment than model metadata, and linguistic dispersion could improve association without ensuring calibration. The paper interprets linguistic confidence as a lossy channel from internal confidence to user-facing scores.

6.1 Regression Modeling

  • 0.570 for distance, 0.644 for ECE-style cross-channel mismatch, and 0.498 for correlation were the OLS models’ respective R2 values.Logits statistics also predicted magnitude agreement, while model metadata generally had smaller and less consistent effects than confidence-distribution statistics.Source: §6.1 · p. 7
  • Linguistic dispersion predicted higher correlation but also greater distance and higher confidence-to-confidence ECE, while higher mean confidence predicted greater distance and ECE.Dispersion can preserve rank-order signal while worsening magnitude agreement and cross-channel ECE; high reported scores were also associated with greater mismatch.Source: §6.1 · p. 7

6.2 A Lossy-Channel Interpretation

  • When many internal states map to the same verbal score, linguistic confidence loses instance-level distinctions; a broader score range can preserve more rank-order information.Confidence collapse near a few high values limits the score’s ability to distinguish instances.Source: §6.2 · p. 7
  • On eight classification tasks with gpt-4.1-nano, finer-grained confidence provided a more dispersed weighting signal and improved aggregation accuracy.The paper cautions that broader score ranges may still be shifted upward, nonlinear, or poorly matched to correctness, so dispersion alone does not guarantee calibration.Source: §6.2 · pp. 7–8, Table 3 · p. 8

7 Linguistic Confidence as a Weighting Signal · 8 Operational Guidance · 9 Discussion

Linguistic confidence can provide a weak ranking signal, but it is not a transparent or necessarily calibrated readout of internal uncertainty. Its use depends on score distribution and task-specific validation, so it should be treated as a distinct channel with measurable failure modes.

7 Linguistic Confidence as a Weighting Signal

  • In an eight-task aggregation study, numeric confidence weighting outperformed level-confidence and answer-only voting, illustrating useful rank-order signal but not calibration.The study sampled 16 outputs per prompt and used gpt-4.1-nano; numeric scores ranged from 0 to 10, while level labels mapped to weights 3, 2, and 1.Source: §7 · p. 8

8 Operational Guidance

  • Treat linguistic confidence as a signal to validate, not a probability: check score spread, held-out association with correctness or an internal proxy, and high-confidence cases that other evidence marks uncertain.Before using confidence for abstention, filtering, or sample weighting, verify these properties on held-out data.Source: §8 · p. 8
  • For white-box systems, mismatches between high verbal and low internal confidence can flag weak probability support; for black-box systems, validate on the same task and use only dispersed scores for weak ranking.Collapsed or prompt-sensitive scores should not control abstention thresholds.Source: §8 · p. 8

9 Discussion

  • Linguistic confidence is a separate channel that sometimes carries rank-order information but becomes uninformative at the instance level when scores cluster near high values.This helps explain why aggregate plots may look encouraging even when instance-level correlations remain weak.Source: §9 · p. 8
  • Association does not guarantee calibration: linguistic confidence can track logits-based confidence yet remain systematically overconfident, while small distance between concentrated signals can conceal little ranking information.Reliability assessments should report association, magnitude agreement, and calibration together.Source: §9 · p. 8
  • Prompting can change confidence distributions and careful exemplars can avoid score collapse, but an encouraging tone can inflate confidence without improving alignment.Asking a model to reflect its internal probabilities does not by itself make verbal confidence track them.Source: §9 · p. 8
  • The analysis is diagnostic, not causal: it neither identifies training changes that create confidence distributions nor shows that changing variance alone fixes confidence; mean, variance, and saturation help explain alignment patterns.The authors therefore frame linguistic confidence as a channel with measurable failure modes, rather than as a direct probability estimate.Source: §9 · pp. 8–9

10 Conclusion · Limitations · Ethical Considerations

Across 10 tasks and 30 models, linguistic confidence aligns weakly with internal confidence signals and should be treated as a lossy channel rather than a calibrated probability. The study also emphasizes limits in its proxies and scope, and warns that unsupported confidence claims can create unwarranted trust.

10 Conclusion

  • Across 10 tasks and 30 models, instance-level alignment is weak; easier tasks and stronger base models tend to show higher association, while instruction-tuned models often remain overconfident.Attitude cues inflate reported confidence without improving alignment, and distributional properties explain much of the observed pattern.Source: §10 · p. 9
  • Linguistic confidence is a lossy channel that may support downstream weighting when validated, but dispersion does not guarantee magnitude agreement or calibration.The authors recommend multi-axis diagnostics, checking for collapsed or inflated distributions, and distribution-aware validation or held-out calibration.Source: §10 · p. 9

Limitations

  • The study covers a limited set of opensource models and two proxies; logits-based confidence is not ground-truth uncertainty, and semantic entropy depends on choices such as sampling and NLI-based clustering.The goal is to compare available confidence channels, not identify true uncertainty.Source: Limitations · p. 9
  • Most experiments are correlational, and regression controls do not establish causality or explain why model versions change confidence distributions.The study lacks access to the training details needed to explain those changes.Source: Limitations · p. 9
  • Repeated subsampling reduces concern that trends depend on one 200-example subset, but estimates remain variable; the small, single-model black-box study is illustrative only.It cannot evaluate the paper’s cross-channel diagnostics.Source: Limitations · p. 9
  • Future work should test additional internal uncertainty proxies, especially for generation, and examine post-hoc confidence elicitation after models complete their answers.The proposed elicitation protocol requires a separate generation setting and is outside the present experiments.Source: Limitations · p. 9

Ethical Considerations

  • High linguistic confidence can create unwarranted trust and should not serve as a calibrated probability without task-specific validation or as the sole basis for high-stakes decisions.Multi-axis reporting, mismatch detection, and held-out calibration are intended to reduce overreliance, but diagnostics are not guarantees of reliability; the study used existing benchmarks and public models without human-subject data.Source: Ethical Considerations · pp. 9–10

A Prompt Design and Robustness Analyses … A.3 Confidence-Elicitation Ablation

The appendix documents confidence-elicitation prompts for classification and generation, then examines prompt sensitivity, confidence-elicitation ablation, and attitude-based perturbations.

A Prompt Design and Robustness Analyses

  • The appendix covers prompting setups and robustness analyses, including exemplar search, confidence-elicitation ablation, and attitude-based prompt perturbations.It describes prompts for both classification and generation tasks.Source: A Prompt Design and Robustness Analyses · p. 13

A.1 Classification Prompt Design

  • For base models, classification Prompt 1 elicits confidence alongside the answer, while Prompt 2 also explains confidence scores before elicitation.Prompt 1 is adapted from Xiong et al. (2023).Source: §A.1 · p. 13
  • For instruction-tuned classification models, the system prompt specifies confidence reporting and output-format constraints.Source: §A.1 · p. 13
  • Classification exemplar settings are (5, 6) for base Prompt 1, (4, 5) for base Prompt 2, and score 2 for instruction-tuned models.The rationale and quantitative comparison are reported in Appendix G.1.Source: §A.1 · p. 13

A.2 Generation Prompt Design

  • Generation prompts follow Kuhn et al. (2023), add confidence instructions following Xiong et al. (2023), and evaluate CoQA and TriviaQA.Source: §A.2 · p. 13
  • CoQA uses zero-shot context-and-question prompts, while TriviaQA uses ten-shot prompts; confidence is averaged across five generations for semantic-entropy estimation.Source: §A.2 · p. 13

A.3 Confidence-Elicitation Ablation

  • The ablation removes linguistic-confidence instructions while keeping other prompt settings unchanged, retaining two-shot examples for base models and format constraints for instruction-tuned models.Appendix G.2 reports comparisons of accuracy, logits statistics, and calibration.Source: §A.3 · pp. 13–14

A.4 Attitude-Based Prompt Perturbation … D Distance and ECE trends by Model Size

The appendix finds that 200-example evaluations recover larger-pool trends without detectable systematic distortion, while model-size trends show that confidence association, distance, and calibration can behave differently. Attitude perturbations test whether reported confidence changes with conversational framing.

A.4 Attitude-Based Prompt Perturbation

  • The study perturbed original prompts with criticism, approval, and irrelevant cues, using three variants per attitude category to test framing sensitivity in reported confidence.The cues are shown in Figure 13; full quantitative results are reported in Appendix G.3.Source: §A.4 · p. 14, Figure 13 · p. 15

B Dataset Stability Analysis

  • For five tasks, the stability analysis drew 500 subsamples of 200 examples across nine model–prompt settings and recomputed accuracy, correlations, ECE, and cross-channel distance.CoLA and Temporal Sequences used full evaluation sets; QNLI and MMLU used enlarged pools, while QQP used a random 5,000-example proxy full set.Source: B Dataset Stability Analysis · p. 14
  • Across 585 metric rows, larger-pool estimates fell within subsampling 95% intervals for 581 rows and all 225 rows involving the five conclusion-linked metrics.Median absolute differences were 0.024–0.027, and 93.9%–95.3% of estimates fell within ±0.10; the authors report no detectable systematic distortion from the 200-example evaluations.Source: B Dataset Stability Analysis · pp. 14–15

C Model Selection

  • The study selected 30 models from the LLaMA, Mistral, and Qwen families and used gpt-4.1-nano for the small-scale aggregation study.The aggregation model was accessed through the OpenAI API on May 4, 2026, at temperature 0.7.Source: C Model Selection · p. 15

D Distance and ECE trends by Model Size

  • The model-size correlation plots show correlations tending to rise with size for base models but plateauing for instruction-tuned models.Figure 14's error bars indicate the standard deviation of correlation values within each model group.Source: D Distance and ECE trends by Model Size · p. 15, Figure 14 · p. 15
  • Model-size trends show that association, magnitude agreement, and calibration measure distinct aspects of confidence reliability.Base-model correlations often rise with scale, while distance and ECE may move differently; instruction-tuned models can show lower distance or ECE as accuracy improves even when correlation does not.Source: D Distance and ECE trends by Model Size · p. 15

E Distribution of Logits-based and Linguistic Confidence · F Distribution of Correlation Scores and Paired t-Test Analyses

Alignment varied across model families and versions alongside shifts in linguistic-confidence distributions. In larger instruction-tuned models, logits-based confidence often saturated near 1 as correlation declined.

E Distribution of Logits-based and Linguistic Confidence

  • For base models, correlation generally increases with model size; CoLA distributions show more diverse, less concentrated linguistic confidence, with Qwen2.5 7B an exception to the distributional pattern.Qwen2.5 7B produces more monotonous linguistic confidence than its 3B model.Source: E Distribution of Logits-based and… · p. 15
  • In instruction-tuned models, logits-based confidence grows larger and often approaches 1 with model size, and this saturation reduces correlation between confidence measures.Source: E Distribution of Logits-based and… · p. 15

F Distribution of Correlation Scores and Paired t-Test Analyses

G Effects of Prompt Design … G.3 Prompt Perturbation

Prompt design can shift linguistic confidence without improving its alignment with internal confidence: attitude cues inflate reports while weakening their association with logits. Exemplar-score spacing and confidence elicitation also affect reported variability or base-model logits, while task accuracy remains unchanged when confidence is not requested.

G.1 Hyperparameter Search

  • Exemplar-score searches measured correlation and accuracy across three tasks and three models, using paired scores for base prompts and a single score from 1 to 10 for instruction-tuned prompts.The instruction-tuned search also included a random baseline.Source: §G.1 · pp. 17–18
  • Performance and correlation tend to rise when exemplar scores are close but not identical; identical scores reduce confidence variance and instance-level association.Very large exemplar scores push outputs toward 8–10 and compress their variability.Source: §G.1 · p. 18
  • The selected exemplar scores are (5, 6) for Base Prompt 1, (4, 5) for Base Prompt 2, and 2 for the Instruct prompt.These settings balance correlation and accuracy.Source: §G.1 · p. 18

G.2 Ablation Studies

  • Removing confidence elicitation leaves normalized accuracy unchanged, but modestly shifts base-model logits and logits variance; instruction-tuned models show no significant changes.The setup repeats the main experiment across all eight tasks with other settings held fixed.Source: §G.2 · p. 18

G.3 Prompt Perturbation

  • Attitude cues raise reported confidence while decreasing its association with logits, so the inflation is not grounded in internal probabilities.Approval has the largest effect, criticism is also positive, and even irrelevant cues increase average confidence.Source: §G.3 · p. 19
  • The appended calibration instruction does not significantly change average confidence or correlation, unlike attitude cues, which weaken their association with logits.Attitude testing covered eight tasks on three models, with three cue variants each for criticism, approval, and irrelevance.Source: §G.3 · pp. 18–19

H Generation Task Details … H.3 Semantic Entropy and Logits-Based Confidence

The appendix details how linguistic confidence and internal uncertainty are compared across generation and classification settings. In classification, linguistic confidence aligns more closely with logits-based confidence than with negated option-level predictive entropy.

H.1 Prompt Design for Generation Tasks

  • Generation-task prompts add random 0–10 confidence examples after each question-answer pair in CoQA or each TriviaQA demonstration, then average confidence across five sampled generations.The setup follows Kuhn et al. (2023), with a confidence-query instruction following Xiong et al. (2023).Source: §H.1 · p. 19

H.2 Full Results on Generation Tasks

  • For generation tasks, Pearson correlation compares average linguistic confidence across five generated sequences with negated semantic entropy for each prompt.Entropy is negated because higher entropy indicates lower model confidence; results are presented in Table 11.Source: §H.2 · p. 19

H.3 Semantic Entropy and Logits-Based Confidence

  • Semantic entropy captures generative uncertainty through semantic diversity among responses, unlike logits-based confidence, which reflects token-level probabilities.It is intended for settings where exact answer spans or salient token positions are difficult to identify.Source: §H.3 · p. 20
  • For classification, semantic entropy is computed from softmax probabilities over the N options obtained from final-layer logits.The corresponding linguistic confidence averages scores elicited by inserting each candidate option into the prompt and asking for confidence individually.Source: §H.3 · p. 20
  • The classification comparison uses Qwen2.5-14B to correlate linguistic confidence with logits-based confidence and negated semantic entropy under two prompt templates.Results are presented in Table 12.Source: §H.3 · p. 20
  • Across 16 task-prompt settings, logits-based confidence is positively correlated with linguistic confidence in 15, while association with negated option-level predictive entropy is generally weaker and often negative.The authors interpret this as evidence that token-probability confidence and entropy capture different internal views of reliability.Source: §H.3 · p. 21

I Regression Model

The section specifies separate regression models for three outcomes—DIST_LINGUISTIC-LOGITS, ECE_LINGUISTIC-LOGITS, and CORRELATION—using model, task, confidence-distribution, and performance predictors.

I Regression Model

  • The analysis independently models DIST_LINGUISTIC-LOGITS, ECE_LINGUISTIC-LOGITS, and CORRELATION as continuous outcomes.Full OLS coefficients for these outcomes are shown in Tables 13–15.Source: I Regression Model · p. 21, Table 13 · p. 23, Table 14 · p. 24
  • Predictors cover model metadata, task characteristics, linguistic and logits confidence distributions, and performance-related measures.The listed predictors include model family, version, type, size; task and number of choices; confidence means and standard deviations; accuracy measures; and linguistic and logits ECE.Source: I Regression Model · p. 21
  • Categorical predictors are one-hot encoded with a dropped reference; zero-variance columns are removed and an intercept is included.Source: I Regression Model · p. 21
  • Ridge standardized effects are also reported for all three outcomes to rank effect sizes under collinearity.They appear in Tables 16–18.Source: I Regression Model · p. 21, Table 16 · p. 26, Table 17 · p. 26

Related papers

Loading 2608.28382v1…