PaperlayerView exact revision on arXiv

Source-linked AI summary

When Linguistic and Internal Confidence Diverge in Large Language Models

Hefan Zhang, Bingquan Zhang, Ming Cheng, Saeed Hassanpour, Weicheng Ma, Soroush Vosoughi

arXiv:2608.28382v1cs.CLcs.AI

TL;DR

The paper asks whether linguistic confidence reflects LLMs’ internal confidence, a question left open by prior work on verbalized confidence. Across tasks, models, and confidence measures, it finds weak and structured alignment, supporting a lossy-channel view and recommending multi-axis validation before downstream use.

  • Problem

    Prior work shows verbalized confidence is prompt-sensitive and often overconfident after instruction tuning, but whether it aligns with models’ internal probability signals remains unclear.

  • Method

    The paper systematically compares linguistic confidence with logits-based confidence across classification tasks and with semantic-entropy-based uncertainty for generation, using multi-axis diagnostics and regression analyses.

  • Results

    Across 10 tasks and 30 models, instance-level alignment is weak; distributional properties explain much of the pattern, while linguistic dispersion can preserve rank-order signal without guaranteeing calibration.

  • Takeaways & Limitations

    Linguistic confidence is a lossy channel that may support downstream weighting when validated but should not be treated as a calibrated probability.

  • Takeaways & Limitations

    The comparison does not identify true uncertainty, and the correlational regression analyses do not establish causal mechanisms.

Abstract

from arXiv · show

Users often ask large language models (LLMs) to report how confident they are, but it is unclear whether such linguistic confidence tracks the model's internal confidence. We study this question across 8 classification tasks, 2 generation tasks and 30 models from three families. For classification, we compare linguistic confidence with logits-based confidence along three axes: association, magnitude agreement and calibration. For generation, we test whether linguistic confidence tracks semantic-entropy-based uncertainty. The axes frequently diverge. Instance-level association is weak on average, although it improves on easier items and for stronger base models. Instruction-tuned models often report higher confidence and sometimes show higher association, but they also have larger confidence gaps and worse calibration. Prompt design mostly changes the distribution of reported confidence. Attitude cues inflate confidence without improving alignment, while score exemplars can preserve rank-order signal when they avoid collapsed confidence values. Regression analyses show that distributional properties of confidence scores explain much of the observed alignment pattern, with model metadata playing a smaller role after controls. These results support a lossy-channel view of linguistic confidence. A more dispersed verbal confidence distribution can carry useful rank information, but it does not make the scores calibrated. Linguistic confidence should therefore be evaluated with multi-axis diagnostics before being used in downstream reliability pipelines.

1 Introduction

The paper asks whether linguistic confidence reflects models’ internal confidence signals rather than merely correctness judgments. Across multiple evaluation axes, it finds that these channels often diverge and that confidence-score distributions strongly shape their alignment.

  • Motivation and approach: The study distinguishes linguistic confidence from internal proxies, using logits-based confidence for classification and semantic entropy for generation.The authors explicitly treat these proxies as accessible comparison channels, not ground-truth uncertainty.
  • Evaluation axes: Association, magnitude agreement, and calibration measure different properties and can yield conflicting assessments of confidence quality.A score may track another score’s ordering while remaining miscalibrated or mismatched in magnitude.
  • Main findings: Across 10 tasks and 30 models, instance-level association is weak on average but tends to improve on easier tasks and for stronger base models.Instruction-tuned models often report higher confidence, without reliably improving calibration.
  • Prompt effects: Prompt changes mainly shift the mean and variance of linguistic confidence, while attitude cues raise verbal confidence without improving grounding in internal probabilities.The paper therefore treats prompting as affecting score distributions more than cross-channel alignment.
  • Distributional interpretation: More dispersed linguistic scores can preserve rank-order information, whereas collapsed scores lose instance-level distinctions; dispersion alone does not ensure calibration.The paper describes linguistic confidence as a lossy channel whose useful rank signal still requires validation.
  • Contribution: The paper evaluates confidence across model families, tasks, and prompting regimes using separate measures for cross-channel association and correctness-based calibration.Its contribution is a systematic multi-axis comparison rather than treating correlation as calibration.

4 Experimental Setting

The experimental setting spans eight classification tasks, two generation tasks, and 30 publicly available models from three model families. Standardized prompts support task evaluation, while additional prompt tests probe confidence robustness.

  • Tasks: The benchmark suite includes eight classification tasks spanning syntax, inference, paraphrase, causal, lexical, temporal, and broad-knowledge reasoning, plus CoQA and TriviaQA generation tasks.The classification tasks include CoLA, QNLI, QQP, Cause And Effect, Conceptual Combinations, Ruin Names, Temporal Sequences, and MMLU.
  • Models: The study uses 30 publicly available models from the LLaMA, Mistral, and Qwen families, covering different sizes, versions, and tuning types.The model inventory is listed in Table 6.
  • Prompting: Standardized prompt templates constrain answer and confidence formats, with two-shot examples for base models and system-prompt requirements for instruction-tuned models.The study also tests confidence exemplars, confidence-elicitation ablations, and attitude-based perturbations.

5 Results

The results show that linguistic and internal confidence align weakly at the instance level, with structured variation by task difficulty, model type, generation setting, and score distribution. Aggregate associations can conceal individual-example divergence, while prompt and model changes often alter confidence distributions without ensuring calibration.

  • Task-level variation: Association tends to weaken as task difficulty increases, although the pooled task-level slope is not statistically reliable.Across eight tasks, the primary estimate is slope = 0.360, 95% CI [−0.842, 1.562], p = 0.491.
  • Overall alignment: Instance-level association is weak, averaging r = 0.135 despite stronger aggregate relationships across settings.This gap matters for abstention, filtering, and selection, which rely on individual-example confidence.
  • Task-level variation: Thirty of 45 model–prompt slopes are positive, but the mean slope is small and uncertain, indicating heterogeneous task and model effects.The mean slope is 0.034 with 95% CI [−0.155, 0.223].
  • Model-level variation: For base models, association rises with model size as linguistic confidence becomes more varied, whereas instruction-tuned models plateau or decline amid confidence saturation.Instruction-tuned models can show slightly higher correlation than base models, but also significantly larger distance and linguistic ECE.
  • Generation setting: Generation results generally show stronger association for instruction-tuned models and higher association on TriviaQA than CoQA, consistent with task- and model-dependent alignment.The generation comparison uses linguistic confidence against negated semantic entropy as the internal uncertainty proxy.
  • Prompt-level variation: Confidence exemplars preserve more association when scores are adjacent but non-identical, while attitude cues raise verbal confidence and reduce alignment with logits.Identical or very large exemplars collapse confidence variation, and calibration instructions do not significantly improve the relationship.

6 Distributional Properties Explain Much of the Pattern

Distributional properties explain much of the observed confidence alignment: dispersion can preserve rank-order information, while mean shifts and incompatible ranges worsen magnitude and calibration measures. Model metadata generally contributes less consistently after distributional controls.

  • Regression evidence: Distributional statistics jointly explain substantial variation in correlation, distance, and cross-channel ECE.OLS models attain R2 values of 0.498 for correlation, 0.570 for distance, and 0.644 for ECE-style mismatch.
  • Distributional drivers: Greater linguistic dispersion is associated with higher correlation, but also greater distance and higher confidence-to-confidence ECE.Dispersion preserves rank-order signal while potentially worsening magnitude agreement and cross-channel calibration.
  • Distributional drivers: Higher linguistic mean confidence is associated with greater distance and higher confidence-to-confidence ECE.Mean shifts can make reported and internal confidence occupy less compatible ranges.
  • Magnitude agreement: Higher logits mean is associated with smaller distance and lower cross-channel ECE, indicating more compatible score ranges.This comparison uses logits-based confidence as an internal proxy, not as ground-truth uncertainty.
  • Model metadata: Model family, version, type, and size generally have smaller and less consistent effects than confidence-distribution statistics.Model size is a partial exception for correlation, but its effect does not recur consistently for distance or cross-channel ECE.
  • Lossy-channel interpretation: A broader linguistic score range can preserve rank-order information, but shifted, nonlinear, or poorly matched scores can still have poor calibration.Dispersion therefore supports association without guaranteeing calibrated confidence.

7 Linguistic Confidence as a Weighting Signal

The paper examines linguistic confidence as a possible weighting signal for aggregating multiple model outputs. In the reported setting, finer-grained confidence improves aggregation accuracy, but confidence remains a separate channel that requires validation rather than calibration assumptions.

  • Aggregation study: Confidence-weighted aggregation improves over majority voting in the gpt-4.1-nano study, with numeric confidence performing best.The study samples 16 outputs per prompt across eight classification tasks.
  • Interpretation: The aggregation result illustrates the use of rank-order signal rather than demonstrating that linguistic confidence is calibrated.The paper recommends validating the signal on the target setting before deployment.
  • Practical policy: Linguistic confidence should be validated before controlling abstention, filtering, or sample weighting.Checks should include score spread, association with correctness or an internal proxy, and high-confidence cases with contradictory evidence.
  • Practical policy: High linguistic confidence paired with low internal confidence identifies cases where the model sounds certain despite weak probability support.This mismatch is most directly useful in white-box systems; black-box systems require held-out validation instead.
  • Diagnostics: Association, magnitude agreement, and calibration should be reported together because they can diverge.Instruction-tuned models can be overconfident despite moving in the same direction as logits-based confidence.
  • Prompt design: Prompting can alter confidence distributions and careful exemplars can avoid score collapse, but prompt tone can inflate confidence without improving alignment.Asking models to reflect internal probabilities does not by itself ensure verbal tracking.
  • Scope: The analysis is diagnostic rather than causal and does not identify training changes that create a given confidence distribution.It also does not claim that manipulating variance alone will fix confidence.

10 Conclusion

Across tasks and models, linguistic confidence aligns weakly with internal confidence, and dispersion can preserve rank information without ensuring magnitude agreement or calibration. The paper therefore recommends multi-axis validation and cautions against treating linguistic confidence as a calibrated probability.

  • Instance-level alignment is weak across 10 tasks and 30 models, while easier tasks and stronger base models tend to show higher association.
  • Instruction-tuned models often remain overconfident despite sometimes showing higher association with internal confidence.
  • Linguistic dispersion can preserve rank-order signal, but it does not guarantee magnitude agreement or calibration.
  • The study is limited to a set of open-source models and two internal confidence proxies, neither of which represents ground-truth uncertainty.
  • Most experiments are correlational, and regression controls do not establish causal mechanisms or explain training-related distributional changes.
  • The black-box aggregation study is small, uses one model outside the main suite, and serves only as an illustration.

A Prompt Design and Robustness Analyses

The appendix describes prompt formats for classification and generation, confidence-score exemplars, confidence-elicitation ablations, and attitude-based perturbations. These designs aim to standardize elicitation while testing prompt sensitivity and whether confidence reporting changes model behavior.

  • A Prompt Design and Robustness Analyses: The appendix documents prompting setups and robustness analyses for classification and generation confidence elicitation.
  • A.1 Classification Prompt Design: Base models use two classification prompt templates, while instruction-tuned models receive confidence and format constraints through the system prompt.
  • A.1 Classification Prompt Design: Confidence exemplars use scores (5, 6) and (4, 5) for base-model prompts, while instruction-tuned models use score 2 as the format example.
  • A.2 Generation Prompt Design: Generation experiments cover CoQA with zero-shot prompting and TriviaQA with ten-shot prompting, averaging confidence across five sampled generations.
  • A.3 Confidence-Elicitation Ablation: Confidence-elicitation ablations remove confidence instructions while preserving other prompt settings and formatting constraints.

A.4 Attitude-Based Prompt Perturbation

The robustness analyses test attitude-based prompt perturbations, dataset-size stability, and model-size trends across confidence metrics. Larger-pool checks broadly recover the 200-example trends, while association, distance, and calibration can move differently with scale.

  • A.4 Attitude-Based Prompt Perturbation: Attitude perturbations prepend criticism, approval, or irrelevant cues, each with three variants, to test conversational framing effects.
  • B Dataset Stability Analysis: Figures 13 and 10–12 provide examples of attitude, TriviaQA, and no-confidence-elicitation prompts used in the robustness analyses.
  • B Dataset Stability Analysis: Larger evaluation pools assess whether the 200-example limit systematically changes results across five affected tasks and nine model–prompt settings.
  • B Dataset Stability Analysis: The median absolute difference between subsample and larger-pool estimates is 0.024–0.027, with 93.9%–95.3% within ±0.10.
  • B Dataset Stability Analysis: Larger-pool estimates fall within the 95% subsampling interval for 581 of 585 metric rows, while individual estimates retain sampling variability.
  • D Distance and ECE trends by Model Size: The study uses 30 models from the LLaMA, Mistral, and Qwen families, with model-size plots separating correlation, distance, and ECE trends.

E Distribution of Logits-based and Linguistic Confidence

Model-family and model-version analyses link confidence alignment patterns to the distribution of linguistic scores, while showing that correlation, distance, and ECE need not change together. Changes in dispersion coincide with divergent alignment trends, but the paper does not claim a causal training explanation.

  • Correlation generally increases with model size for base models but decreases for instruction-tuned models as logits-based confidence saturates.
  • Paired t-tests compare correlation, confidence-logit distance, and ECE across model families and versions.
  • Across families, correlation generally rises from LLaMA to Mistral and Qwen alongside lower distance and ECE, but within-family changes are less uniform.
  • LLaMA base-model confidence standard deviation increases from 1.16 to 1.32, whereas Qwen3 14B base models decrease from 1.23 to 0.71.
  • Higher dispersion can preserve rank information, while compressed confidence distributions reduce expressivity and weaken association.
  • Training and post-training changes may influence confidence dispersion, but the available analyses do not establish a direct causal link.

G Effects of Prompt Design

Prompt design changes how linguistic confidence is distributed and can alter its association with internal confidence without changing task accuracy. Score exemplars preserve association when they avoid confidence collapse, whereas attitude cues inflate reported confidence while reducing alignment.

  • G.1 Hyperparameter Search: The exemplar study compares weighted correlation and accuracy across three tasks, three models, and two base prompt templates.Figures 22 and 23 summarize averages weighted by task sample size.
  • G.1 Hyperparameter Search: Close but nonidentical exemplar scores increase correlation and performance by preserving variation in linguistic confidence.Identical scores reduce variance and lower instance-level association; very large scores compress outputs toward 8–10.
  • G.1 Hyperparameter Search: The selected exemplar settings are (5, 6) for Base Prompt 1 and (4, 5) for Base Prompt 2.
  • G.2 Ablation Studies: Removing confidence elicitation leaves normalized accuracy unchanged but modestly shifts average logits and logits variance for base models.One comparison also shows a change in logits ECE; instruction-tuned models show no significant metric changes.
  • G.3 Prompt Perturbation: Attitude cues consistently raise reported linguistic confidence, with approval producing the largest increase and irrelevant cues also increasing averages.The tested cues include criticism, approval, and irrelevance variants across eight tasks and three models.
  • G.3 Prompt Perturbation: Higher reported confidence from attitude cues coincides with lower association with logits-based confidence.The calibration instruction does not significantly change average confidence or correlation.

H.1 Prompt Design for Generation Tasks

The generation-task analyses elicit confidence alongside sampled answers and compare it with semantic entropy, while additional classification analyses contrast entropy with logits-based confidence. These measures are not interchangeable: linguistic confidence is closer to token-probability confidence in the reported classification settings.

  • H.1 Prompt Design for Generation Tasks: Confidence is elicited by appending a query and assigning random 0–10 scores after demonstrations or question-answer pairs.Generated confidence is averaged across five sampled generations for each prompt.
  • H.2 Full Results on Generation Tasks: Semantic entropy and average linguistic confidence are computed across five generated sequences, then compared using Pearson correlation.Semantic entropy is negated for correlation because higher entropy indicates lower confidence.
  • H.2 Full Results on Generation Tasks: Semantic entropy measures semantic diversity among generated responses rather than concentration in token-level probabilities.It is used when exact answer spans or salient token positions are difficult to identify.
  • H.3 Semantic Entropy and Logits-Based Confidence: For N-option classification, semantic entropy is computed from the softmax probabilities p1, p2, . . . , pN obtained from final-layer logits.
  • H.3 Semantic Entropy and Logits-Based Confidence: Option-level linguistic confidence averages scores obtained by inserting each candidate option separately and requesting confidence.Qwen2.5-14B is evaluated under two prompt templates against logits-based confidence and negated semantic entropy.
  • H.3 Semantic Entropy and Logits-Based Confidence: Table 9 reports paired comparisons of logits-based statistics under prompting conditions, including means, t-statistics, and p-values.
  • H.3 Semantic Entropy and Logits-Based Confidence: Across 16 task-prompt settings, logits-based confidence correlates positively with linguistic confidence in 15 settings.Association with negated option-level predictive entropy is generally weaker and often negative.

I Regression Model

The regression analysis models distance, calibration error, and correlation using model, task, confidence-distribution, and performance predictors. It uses one-hot encoding and ridge-standardized effects alongside full OLS coefficients to rank effects under collinearity.

  • The analysis independently models DIST_LINGUISTIC-LOGITS, ECE_LINGUISTIC-LOGITS, and CORRELATION as continuous outcomes.
  • Predictors combine model metadata, task characteristics, confidence-distribution statistics, and accuracy or calibration measures.
  • Categorical model and task fields are one-hot encoded with a dropped reference, while zero-variance columns are removed and an intercept is included.
  • The accompanying tables organize correlations, calibration-error outcomes, and internal-metric comparisons for the broader analysis.
  • Full OLS coefficients and ridge cross-validated standardized effects are reported for all three outcomes.Ridge effects rank predictor magnitudes under collinearity.
  • The reported coefficient table includes distributional, performance, task, model-family, version, model-type, and task predictors.
Loading 2608.28382v1…