PaperlayerView exact revision on arXiv

Source-linked AI summary

Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patterns

Hanqi Xiao, Vaidehi Patil, Hyunji Lee, Elias Stengel-Eskin, Mohit Bansal

arXiv:2509.24988v2cs.CLcs.AI

TL;DR

LLM confidence estimation is important, but prior approaches often rely on a model judging its own correctness, an advantage the experiments find is generally absent. The paper trains correctness models on historical data from multiple LLMs and tests other ways to inject history. Its generalized model outperforms single-model baselines and transfers across models, with additional gains from history-based methods and downstream selective prediction.

  • Problem

    LLM confidence estimation often relies on a model predicting its own correctness, but whether a model has privileged self-knowledge remains an open question.

    Source: §1 · pp. 1–2

  • Method

    The paper trains correctness models on historical correctness data from multiple LLMs and also tests in-context examples and post-hoc calibration as ways to encode history.

    Source: §1 · pp. 2–3

  • Results

    Training Qwen3-8B on eight models’ correctness data beats eight separately trained CMs by 2.22% accuracy and .041 AUROC on average, with improvements on all target models.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The findings support treating reliable correctness prediction as a transferable, model-agnostic skill learned from correctness history rather than a model-specific self-introspection skill.

    Source: §5 · p. 9

    Transfer to new datasets is harder for a GCM than transfer across model families.

    Source: §3.4 · p. 7

Abstract

from arXiv · show

Generating accurate and calibrated confidence estimates is critical for deploying LLMs in high-stakes or user-facing applications, and remains an open challenge. Prior research has often framed confidence as a problem of eliciting a model's "self-knowledge", i.e., the ability of an LLM to judge whether its own answers are correct; this approach implicitly assumes that there is some privileged information about the answer's correctness that is accessible to the model itself. However, our experiments reveal that an LLM attempting to predict the correctness of its own outputs generally performs no better than an unrelated LLM. Moreover, we hypothesize that a key factor in building a "Correctness Model" (CM) is exposure to a target model's historical predictions. We propose multiple methods to inject this historical correctness information, creating a Generalized Correctness Model (GCM). We first show that GCMs can be trained on the correctness data from many LLMs and learn patterns for correctness prediction applicable across datasets and models. We then use CMs as a lens for studying the source of correctness prediction ability and its generalization, systematically controlling their training data and finding that answer phrasing is a strong predictor for correctness. We further explore alternative methods of injecting history without training an LLM, finding that including history as in-context examples can help improve correctness prediction, and post-hoc calibration can provide complementary reductions in calibration error. We evaluate GCMs based on Qwen3-8B across 5 model families and the MMLU and TriviaQA datasets, as well as on a downstream selective prediction task, finding that reliable LLM confidence estimation is a generalizable and model-agnostic skill learned by systematically encoding correctness history rather than a model-specific skill reliant on self-introspection.

1. Introduction

The introduction questions whether LLMs have privileged self-knowledge for predicting correctness and argues that historical correctness data can support model-agnostic prediction. It introduces correctness models trained on multiple models’ histories and reports gains across models, datasets, and selective prediction.

1. Introduction

  • Self-predictors show little to no advantage: on MMLU, Llama3-70B trained on its own confidence prediction achieves 77.6% versus Qwen2.5-72B’s 78.2%.Similar patterns appear in training-free settings and when question and answer are both provided.Source: §1 · p. 2
  • Correctness models (CMs) estimate calibrated P(is correct) scores, and the paper trains them on historical correctness data from multiple LLMs.Varying the training distribution, test settings, postprocessing, and input features provides a way to study correctness estimation.Source: §1 · p. 2
  • Multi-model CMs generalize across model families and sizes, but less well across datasets; question, response, and predicted answer each contribute, with answer phrasing playing a substantial role.Some cross-model gains also generalize through similarities between tasks models find difficult.Source: §1 · p. 2
  • In-context examples improve correctness prediction for larger models, while post-hoc calibration can adapt a CM to dataset-wise OOD settings.The paper treats these as alternative ways to encode history without finetuning.Source: §1 · p. 3
  • Training Qwen3-8B on eight models’ correctness data beats eight separately trained CMs by 2.22% accuracy and .041 AUROC on average, improving on every target model.The GCM also exceeds Llama3-70B’s self-emitted confidences on MMLU by 2.4% absolute accuracy and .265 AUROC, and reduces Qwen3-32B’s logit-confidence ECE from .073 to .029.Source: §1 · p. 3

2. Methods and Experimental Setup

The methods define correctness prediction as estimating answer correctness from a query and response, then compare trained and training-free ways to encode historical correctness. Experiments use multiple model families, datasets, and confidence metrics to assess predictive performance and calibration.

2. Methods and Experimental Setup

  • A Correctness Model is any system estimating whether an answer is correct from a query and response; for MMLU, the response and selected answer choice are distinguished.The work treats prompting, probing, auxiliary models, finetuning, and post-hoc calibrators as possible CMs.Source: §2 · p. 3
  • The main analysis uses MMLU, with transfer experiments on TriviaQA and an additional Spider evaluation; responses are free-form and judge-graded for correctness.The study constructs 18 correctness datasets from 10 models across MMLU and TriviaQA, spanning five model families and sizes from 3B to 72B.Source: §2 · p. 3
  • Unless otherwise stated, confidence is logit-based P(True), elicited by asking whether a model response correctly answers its prompt.Verbalized confidences are also explored, while post-hoc methods map historical probabilities using held-out data and calibration procedures such as spline calibration.Source: §2 · pp. 3–4
  • Specific CMs are finetuned on one correctness dataset, while answerless CMs predict correctness from the query alone; both use Qwen3-8B initialization by default.The setup uses LoRA, rank 32, batch size 16, one epoch, and 70% of a dataset, except where ablations vary these choices.Source: §2 · pp. 3–4
  • The GCM finetunes Qwen3-8B on concatenated correctness data from eight LLMs, conditioning on each generating model’s name and performance history.Training datapoints and steps are matched against the baseline of eight separate CMs.Source: §2 · p. 4
  • RQ2B compares answerless CMs with answer-only models that use the query and selected answer letter; the latter provides an intermediate input ablation.Table 1 presents the settings compared in RQ2B.Source: §2 · p. 4, Table 1 · p. 4
  • For training-free ICL, the method retrieves the five most semantically similar historical examples and adds them to the CM prompt before eliciting verbalized confidence.This setting uses prompts five times the standard length; post-hoc calibration instead maps historical scores or confidences to empirical correctness on held-out data.Source: §2 · p. 4
  • Evaluation reports accuracy, expected calibration error, adaptively binned RMS calibration error, and AUROC on a held-out 25% of each correctness dataset.The evaluation uses close to 3,500 examples and the same questions across datasets to prevent train-test contamination for GCMs.Source: §2 · p. 4

3. Results

The results find little advantage for self-prediction and show that correctness models trained across LLMs transfer across models, while transfer across datasets is harder. Ablations identify response phrasing as useful, and historical examples or post-hoc calibration offer alternative ways to improve prediction.

3.1. RQ1: Models have Little Self-Knowledge for Correctness Estimation

  • Qwen2.5-72B and Llama3-70B show no self-prediction advantage in the answerless setting, where either model predicts the other's or its own correctness about equally well.In the answerful setting, Qwen2.5-72B predicts both models better, which the authors attribute to its higher MMLU accuracy (83.5% versus 78.3%).Source: Figure 2 · p. 4, §3.1 · p. 5

3.2. RQ2a: Generalization of CMs Trained to Predict Multiple LLMs

  • With equal data and training time, the GCM beats SCMs by at least 3% accuracy on Llama3.1-8B and Gemma3-27B, with ECE ≤.03; cross-dataset transfer is less reliable.On TriviaQA, the GCM has lower accuracy and ECE .105 versus the SCM's .023, despite similar AUROC; the authors also report GCM advantages on Spider and held-out models, with some held-out comparisons matching rather than beating SCMs.Source: §3.2 · pp. 5–6, Table 6 · p. 6

3.3. RQ2b: Conditioning Factors Relevant to Prediction and Generalization

  • Ablations show that query, answer, and response phrasing each contribute to correctness prediction, while GCMs gain more than SCMs from answer-only conditioning.The GCM improves 7% from answerless to answer-only versus 4% for the SCM; removing model identity negatively affects calibration, accuracy, and AUROC, though the GCM still beats the answer-only SCM.Source: §3.3 · pp. 6–7, Table 7 · p. 7

3.4. RQ2c: Alternative Methods for Encoding History

  • In-context examples improve Qwen3-32B's accuracy by 2.6% and reduce ECE by 4.7% when predicting Gemma3-27B, but Qwen3-8B shows no gains.Post-hoc calibration can bring cross-dataset GCM performance to match the target SCM using 5% of target data, and can further reduce ECE.Source: §3.4 · p. 7, Table 8 · p. 7

3.5. Recommendations for Correctness Prediction

  • The recommended GCM with post-hoc calibration improves selective prediction, achieving lower risk than self-emitted Llama3-70B confidences or an SCM at the same coverage.For TriviaQA, it beats an OOD-trained SCM on AUROC and matches it on accuracy and ECE; the authors recommend ICL when training is not possible, despite its higher inference cost.Source: §3.5 · pp. 7–8, Figure 4 · p. 8

4. Related Work

Prior work studies LLM confidence through self-reports, hidden-state probes, and fine-tuning, alongside research on reward and quality estimation. This paper situates GCMs as a history-aware approach to correctness estimation with cross-model and cross-dataset generalization.

Self-Knowledge and Confidence Calibration

Correctness Models and Cross-Model Transfer

  • Correctness models range from self-reported confidence to hidden-state probes and direct fine-tuning, while surrogate models have been reported to outperform GPT self-reports in one-to-one transfer (Shrivastava et al., 2023).Uncertainty estimation instead focuses more closely on intrinsic uncertainty, including semantic uncertainty (Kuhn et al., 2023).Source: §4 · pp. 8–9

Reward Models, LLM Judges, and Quality Estimation

  • Reward models and quality estimation assess output reward or quality, but are not trained with knowledge of which model produced the output (Lambert et al., 2025; Tan et al., 2025; Zheng et al., 2025).The paper contrasts this with correctness modeling that uses historical information about generating models.Source: §4 · p. 9

Assessor Models

  • Assessors predict a model's chance of answering correctly from the query alone, supporting routing or preemptive rejection (Hernández-Orallo et al., 2022; Pacchiardi et al., 2025; Zhou et al., 2022; Pacchiardi et al., 2024).The paper treats this as an answerless ablation of its correctness models.Source: §4 · p. 9

Downstream Applications

  • Prior work applies correctness estimation to hallucination detection, interpretability, reasoning, semantic parsing, and system routing; GCMs add a cross-model, history-aware framework that generalizes across models and datasets.The cited application examples include hallucination detection (Zhou et al., 2025; Li et al., 2024; 2025a) and routing (Hu et al., 2024; Wang et al., 2024a; Ong et al., 2024).Source: §4 · p. 9

5. Conclusion

The paper argues that correctness prediction benefits from historical data across models rather than relying on model-specific self-introspection. Its GCMs generalize across models and improve downstream selective prediction.

5. Conclusion

  • GCMs improve selective prediction, with the reported benefit stemming from their generalization ability.The conclusion also reports that GCMs outperform models trained to predict themselves and self-emitted confidences of much larger models.Source: §5 · p. 9
  • A GCM trained to predict correctness across many LLMs learns transferable strategies and incurs no penalty when predicting models other than itself.The paper describes this as a benefit of the finding that LLMs have little self-knowledge for correctness prediction.Source: §5 · p. 9
  • The authors conclude that reliable LLM confidence estimation is a generalizable, model-agnostic skill learned by systematically encoding correctness history.Source: §5 · p. 9

A. Extensions of RQ1 Results: Do models have an advantage when predicting their own correctness?

Across the extended self-prediction tests, models show no consistent advantage at predicting their own correctness. On TriviaQA, GCM with post-hoc calibration outperforms SCMs and other methods.

A. Extensions of RQ1 Results: Do models have an advantage when predicting their own correctness?

  • Across the extended tests, self-prediction shows no consistent advantage, including for Qwen2.5-72B and Llama3-70B, which share hidden-layer count and hidden size.The paper notes that these shared dimensions make their LoRA interactions expected to be similar.Source: A. Extensions of RQ1 Results: Do… · p. 14
  • Training-free predictors show no systematic self-prediction advantage: diagonal AUROC entries are not consistently higher than off-diagonal entries.The tests cover five target models and five correctness models.Source: A. Extensions of RQ1 Results: Do… · pp. 14–15

B.1. TriviaQA Results

  • On TriviaQA, GCM with post-hoc calibration outperforms SCMs and other methods.Source: §B.1 · p. 16

B.2. Joint Training on MMLU and TriviaQA

Jointly training on MMLU and TriviaQA correctness data improves accuracy and AUROC on both datasets, but calibration changes differ by dataset.

  • Joint training improves accuracy and AUROC on both datasets; calibration improves on MMLU but ECE and RMSCE are slightly worse on TriviaQA.The comparison is against GCMs trained only on the corresponding target dataset.Source: §B.2 · p. 16
  • Dataset-specific post-hoc calibration may still benefit calibration when training jointly across datasets.Source: §B.2 · p. 16

B.3. Additional Evaluations for GCMs vs SCMs

A repeat comparison using a different training seed found that GCMs continued to outperform SCMs on MMLU and TriviaQA.

  • GCMs outperform SCMs on MMLU and TriviaQA in a repeat run with a different seed, with accuracy impacts within .2% and patterns unchanged.The repeat used the same dataset examples and a batch of 16 SCMs.Source: §B.3 · p. 16

B.4. MMLU

On MMLU, GCM outperforms SCMs on accuracy and AUROC across all eight model cases, while calibration improvements are smaller and not uniform.

  • Accuracy improved by +.022 on average (.829 vs. .807) and AUROC by +.041 on average (.866 vs. .825), with all 8 cases improving on both metrics.Source: §B.4 · pp. 16–17
  • ECE changed by –.002 on average (.024 vs. .026), while RMSCE changed by –.004 on average (.079 vs. .083), with lower values better.ECE improved in 5 cases and degraded in 3; RMSCE improved in 7 cases and degraded in 1.Source: §B.4 · pp. 16–17

B.5. TriviaQA

On TriviaQA, General Correctness Models (GCMs) outperform Specific Correctness Models (SCMs) on average accuracy and AUROC, while calibration changes vary across models. The appendix also examines unlimited-training performance, experimental comparisons, and what historical information must contain.

B.5. TriviaQA

  • On TriviaQA, GCMs averaged +.021 accuracy (.865 vs. .844) and +.031 AUROC (.901 vs. .869) over SCMs; AUROC improved in all eight cases.The largest accuracy gain was +.042 for Qwen3-8B supplement; the largest AUROC gain was +.047 for Qwen2.5-32B-Instruct.Source: §B.5 · pp. 17–18
  • ECE fell –.0005 on average (.0233 vs. .0238), while RMSCE fell –.0018 (.0775 vs. .0793), with mixed per-model calibration changes.ECE improved in four cases and degraded in four; RMSCE improved in five and degraded in three.Source: §B.5 · p. 17

C.1. Unlimited Training Time Ablation

  • In an ablation with an unlimited amount of training time until overfitting occurs, the General Model (GCM) still outperforms specific models (SCMs).Source: §C.1 · p. 18

C.2. Figure 1 Experimental Settings

  • Figure 1 compares the SCM and GCM on Llama3.1-8B, conditioning ablations, and Qwen3-32B with versus without ICL; its RQ1 panel repeats Figure 2’s Answerless setting.The GCM comparison also includes Llama3.1-8B logit confidences.Source: §C.2 · p. 18

C.3. Discussion on ChatGPT’s Memory System and Similar Techniques for Injecting History

  • Useful history must encode historical correctness, not merely general memory; sparse memories may omit prior performance and generations.The discussion notes that ChatGPT has a history function but distinguishes it from direct historical correctness information.Source: §C.3 · p. 18

C.4. Discussion on the Effects of Batch Size on Calibration Error

Batch size materially affects calibration in the tested CM training setup, while post-hoc calibration methods produce small and method-dependent changes on TriviaQA.

C.4. Discussion on the Effects of Batch Size on Calibration Error

  • For SCMs and GCMs, batch size affects calibration: batch size 1 is especially detrimental, sizes above 32 can also harm ECE, and the authors use 16.The batch-size choice is based on the reported calibration observation.Source: §C.4 · pp. 18–19

C.5. Further Discussion on Post-hoc Calibration

  • Spline calibration is used in the main results with two or four knots; unlike Platt scaling, it need not compress probabilities and may retain high-confidence predictions.The authors chose spline calibration after observing positive calibration-error results and its ability to preserve a smooth probability distribution in some cases.Source: §C.5 · p. 19
  • On TriviaQA, most post-hoc methods change ECE by less than one percentage point on average; temperature and spline scaling improve it slightly, while four other methods reduce calibration quality.The methods associated with reduced calibration quality are Platt scaling, isotonic regression, ENIR, and BBQ.Source: §C.5 · p. 19

D. Expanded Related Work

Prior work studies self-reported confidence, correctness prediction, reward or quality estimation, and assessors, but the paper emphasizes history-aware correctness prediction across models and datasets.

Self-Knowledge and Confidence calibration

  • Calibration research includes prompted confidence and correctness-label fine-tuning, while studies also examine unanswerable questions, hidden activations, and black-box elicitation.Kadavath et al. (2022) reported reasonably calibrated prompted probabilities for larger LLMs; Kapoor et al. (2024) argued prompting alone is insufficient.Source: D. Expanded Related Work · p. 20

Correctness Models and Cross-Model Transfer

  • Earlier correctness models include self-reported confidence, hidden-state probes, and fine-tuned LLMs; prior transfer findings remained in one-model-to-one-model settings.Shrivastava et al. (2023) found untrained LLaMA models could sometimes predict GPT confidences more accurately than GPT’s self-reported probabilities.Source: D. Expanded Related Work · p. 20

Reward Models, LLM Judges, and Quality Estimation

  • Reward models and quality estimation generally omit which model produced an answer and historical performance, whereas this work conditions on model identity and history.The paper distinguishes calibrated confidence estimates from reward models’ unbounded scalars and quality-estimation labels that generally lack probabilistic meaning.Source: D. Expanded Related Work · p. 21

Assessor Models

  • Assessors predict potential response correctness from the query alone; the paper treats this as an Answerless Correctness Model that omits the response.Assessors are also used for routing and preemptive rejection.Source: D. Expanded Related Work · p. 21

Downstream Applications

  • Prior applications of correctness estimation include hallucination detection, truthfulness, interpretability, reasoning, semantic parsing, and routing.The paper situates its cross-model framework within this range of downstream applications.Source: D. Expanded Related Work · pp. 21–22

Correctness Models and Cross-Model Transfer

  • The paper presents its GCM as a cross-model, history-aware framework that generalizes across models and datasets.It aggregates correctness patterns across models, contrasting with prior one-model-to-one-model transfer work.Source: D. Expanded Related Work · pp. 20, 22

E. Additional Details on Experimental Setup

The appendix documents model-training settings, dataset construction, additional Spider evaluation, retrieval, and prompt formats used for correctness prediction.

E. Additional Details on Experimental Setup

  • GCMs and SCMs use LoRA rank 32, batch size 16, alpha 16, dropout 0.0, learning rate 1e-5, and query/value matrices, with cross-entropy training on correctness labels.Loss is masked except for the yes/no token used to obtain P(True); training uses bf16 mixed precision and DeepSpeed ZeRO stage 2.Source: E. Additional Details on Experimental… · p. 22
  • GCM training concatenates correctness datasets from block 1 or 3, depending on dataset; block 2 supplies OOD evaluations, and SCMs train on each listed dataset.The average MMLU dataset is 198 tokens and the average TriviaQA response is 70.4 tokens.Source: E. Additional Details on Experimental… · p. 22, Table 34 · p. 23
  • Spider tests text-to-SQL correctness with exact-match labels; its GCM predicts six target models using 32B-scale models and a 75/25 training/test split.The setup omits 70B-scale models for time and follows the Table 2 experiment apart from those changes.Source: E. Additional Details on Experimental… · p. 22
  • Semantic ICL retrieves examples using Chroma’s default embedding function, all-MiniLM-L6-v2 at the time of writing.This retrieval supports the semantic ICL examples setting.Source: E. Additional Details on Experimental… · p. 22

SCM / GCM / Answer-only / P(True)

Answerless Setting

  • The Answerless prompt predicts whether a named model will answer a prompt correctly, using the prompt without a response.Training examples include a ground-truth yes/no label, which is removed at inference.Source: Answerless Setting · p. 23

Verbalized Confidence

  • Verbalized Confidence prompts request a calibrated percent probability that the current model response correctly answers the current prompt, in a specified output format.The prompt asks for concise analysis before the probability.Source: Verbalized Confidence · pp. 23–24

ICL Verbalized Confidence

  • ICL verbalized prompts append retrieved past question-response examples with correctness labels; lower retrieval distance means greater similarity, and labels are absent except in examples.Template placeholders are dynamically populated; retrieved documents contain formatted training examples, while current prompts and completions come from the target-model dataset.Source: ICL Verbalized Confidence · p. 24, Notes for Reproducibility · pp. 24–25
Loading 2509.24988v2…