PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating Persistent Calibration under Evolving Model Knowledge

Victor Wang, Thomas Hofweber, Mohit Bansal, Elias Stengel-Eskin

arXiv:2609.38797v1cs.LGcs.AIcs.CL

TL;DR

The paper asks whether confidence estimates can remain calibrated as a model’s knowledge changes. It evaluates estimators across open-model checkpoints using knowledge contrast sets, finding that existing methods lag oracle methods on those sets even when full-set calibration is strong. Multi-checkpoint training improves contrast-set calibration.

  • Problem

    The paper asks whether a confidence estimator can track a model’s changing knowledge, a question relevant to trustworthy continually learning systems.

    Source: §1 · p. 1

  • Method

    The authors evaluate confidence estimators across open-model checkpoints and use knowledge contrast sets of questions answered correctly by one checkpoint and incorrectly by another.

    Source: §1 · p. 2, §2 · p. 3, §3.1.1 · p. 5

  • Results

    On Marin 8B Jeopardy, self-consistency’s full-set AUC is 0.886 versus oracle training’s 0.890, while contrast-set AUC is 0.760 versus 0.825.

    Source: §3.2 · p. 6

  • Takeaways & Limitations

    Multi-checkpoint training improves contrast-set calibration in both oracle and non-oracle settings, suggesting a way to identify confidence features robust across checkpoints.

    Source: §1 · p. 3

    The authors hypothesize that many confidence functions can fit one checkpoint, while only some use meta-knowledge features that generalize to others.

    Source: §1 · p. 2

Abstract

from arXiv · show

As AI systems move from static repositories to agents that are capable of continual adaptation and learning, maintaining their trustworthiness means equipping the models backing them with the ability to produce confidence estimates that dynamically reflect their changing skills and knowledge. We introduce the problem of persistent calibration, which requires a confidence estimator to faithfully reflect the knowledge contained in a model as that knowledge changes, without recurring supervision. We operationalize this by examining persistent calibration across checkpoints of open models, asking whether confidence estimators trained on earlier checkpoints can generalize to later ones. Specifically, we aim to shed light on whether confidence is dependent on knowledge, a question with implications for the reliability of confidence estimates. To measure this relationship, we define and evaluate calibration on knowledge contrast sets: subsets containing questions that one checkpoint answers correctly and another checkpoint answers incorrectly, reflecting a change in knowledge. We show that both inference-time and fine-tuning methods fall short on contrast-set calibration compared to oracle methods trained on future checkpoints, even for methods that are well-calibrated on the full dataset. We provide evidence for the hypothesis that persistent calibration is challenging because there is a vast space of possible confidence functions that are well-calibrated on a given checkpoint, out of which only some rely on meta-knowledge features that would generalize to other checkpoints. Towards improving contrast-set calibration, we show that multi-checkpoint training helps, suggesting an avenue for identifying confidence features that remain robust across changing knowledge.

1 INTRODUCTION

Continually learning models need confidence estimates that remain calibrated as their knowledge changes. The paper introduces persistent calibration and contrast-set evaluation, finding that existing methods struggle to track knowledge changes while multi-checkpoint training improves contrast-set calibration.

1 INTRODUCTION

  • Persistent calibration requires confidence estimates to remain calibrated to a model’s evolving knowledge without repeated recalibration.The paper studies whether estimators trained on earlier checkpoints generalize to later checkpoints.Source: §1 · pp. 1–2
  • Knowledge contrast sets contain questions answered correctly by one checkpoint and incorrectly by another, testing whether confidence reflects checkpoint-specific knowledge.The contrast-set evaluation is intended to distinguish meta-knowledge from static heuristics such as question difficulty.Source: §1 · p. 2
  • The authors hypothesize that persistent calibration is difficult because many confidence functions fit one checkpoint, but only some use meta-knowledge features that generalize.The paper connects this question to adaptive effort, where confidence can guide retrieval, tool use, reasoning, or abstention.Source: §1 · pp. 2–3
  • Existing inference-time and fine-tuning methods show limited contrast-set calibration despite strong full-set calibration; on Marin 8B Jeopardy, self-consistency matches oracle full-set AUC but trails on contrast-set AUC.Self-consistency and oracle AUC are 0.886 and 0.890 on the full set, versus 0.760 and 0.825 on the contrast set.Source: §1 · p. 3
  • Multi-checkpoint training improves contrast-set calibration, while confidence estimators with equal performance on one checkpoint can behave very differently on others.The authors suggest this heterogeneity makes robust meta-knowledge features difficult to identify.Source: §1 · p. 3

2 PERSISTENT CALIBRATION

This section defines persistent calibration and describes how checkpoint-based evaluation tests whether confidence tracks changing knowledge. It motivates contrast-set evaluation and outlines calibration metrics and a normalization procedure.

2 PERSISTENT CALIBRATION

  • Persistent calibration asks confidence estimators to faithfully reflect a model’s knowledge as it changes; training-based methods use earlier checkpoints for later evaluation, while inference-time methods apply directly to evaluation checkpoints.By default, the checkpoint generating an answer also produces its confidence estimate.Source: §2 · p. 3
  • Knowledge contrast sets contain questions for which two checkpoints differ in answer correctness, addressing the confound that overlapping knowledge can yield strong calibration without checkpoint-specific faithfulness.Faithful estimates should be higher for the checkpoint that answers correctly; checkpoint-agnostic question features cannot capture that difference.Source: §2 · pp. 3–4
  • For contrast-set questions, the method normalizes checkpoint confidences using p1(1 − p2)/Z and p2(1 − p1)/Z, restoring contrast-set calibration if the original confidences are calibrated and correctness is conditionally independent given them.Z = p1(1 − p2) + p2(1 − p1).Source: §2 · p. 4
  • The evaluation reports AUC, Brier score, and Expected Calibration Error for full and contrast sets, plus class-balanced variants for improvement and regression cases.The class-balanced variants control for the prior that the later checkpoint tends to be correct.Source: §2 · p. 4

0. These

The passage specifies the scale and interpretation of the contrast-set delta metrics.

0. These

  • Contrast-set delta metrics range from −1 to 1; random guessing gives Δ0 = 0, while always identifying the correct answer gives Δ0 = 1.Evaluation covers multiple future checkpoints, which are aggregated when reporting metrics.Source: §0 · p. 4

3 EXPERIMENTS AND RESULTS

Across open-model checkpoints, the experiments assess whether confidence methods track changing knowledge without repeated recalibration. On knowledge contrast sets, inference-time and earlier-checkpoint methods generally lag oracle training, while multi-checkpoint training improves calibration.

3.1 EXPERIMENTAL SETUP

  • Experiments use Olmo 3 7B and 32B checkpoints and Marin 8B, evaluating on TriviaQA, Jeopardy, and BioASQ, with BioASQ testing domain transfer.For Olmo 3, earlier checkpoints train non-oracle methods and later checkpoints are used for evaluation; Marin’s first three checkpoints train and last three evaluate.Source: §3.1 · p. 4

3.1.1 CONFIDENCE ESTIMATION METHODS

  • Methods include self-consistency, fine-tuned verbalized confidence, checkpoint-index confidence, and generalized correctness models; oracle adapters train directly on evaluation checkpoints.Non-oracle adapters instead train on earlier checkpoints, testing whether learned confidence features transfer to later ones.Source: §3.1.1 · p. 5

3.2 RESULTS

  • On Marin 8B Jeopardy, SC’s full-set AUC of 0.886 nearly matches oracle’s 0.890, but contrast-set AUC is 0.760 versus 0.825, showing full-set calibration can miss persistent calibration.The paper therefore treats contrast-set calibration as a distinct criterion for persistent calibration.Source: Table 1 · p. 6, §3.2 · p. 6
  • The oracle-training advantage observed in-domain largely disappears on BioASQ, where oracle and non-oracle methods both perform similarly poorly.The experiments therefore limit the observed oracle advantage to the training domain.Source: §3.2 · p. 6
  • SC generally beats surrogate and copy ablations, while non-oracle training has mixed results; copy ablations exceed 0.8 full-set AUC but achieve 0.5 contrast-set AUC.These ablations test whether apparent transfer can be explained by knowledge already present in an earlier checkpoint or by inter-checkpoint correctness correlation.Source: §3.2 · pp. 6–7, Table 2 · p. 7
  • Multi-checkpoint training matches or exceeds single-checkpoint and respective-checkpoint baselines; ablations indicate that both more data and multiple checkpoint views contribute.Among the ablations, single-checkpoint data performs better, suggesting access to multiple checkpoint weights contributes most to the gains.Source: Table 3 · p. 7, §3.2 · p. 7, Table 4 · p. 8
  • Figure 3 shows stronger forward than backward transfer; random labels for the second checkpoint generally collapse its AUC without penalizing the first checkpoint.The transfer analysis uses Olmo 3 7B on TriviaQA, with Jeopardy results in Appendix B.6.Source: Figure 3 · p. 8
  • Although GCM matches oracle performance on Olmo 3 7B, its oracle gap grows for Marin 8B and Olmo 3 32B as its backbone makes more contrast-question errors.The authors argue that an external verifier’s calibration does not transfer to more capable models.Source: §3.2 · p. 8

4 ANALYSIS

The analysis investigates why confidence estimators can transfer across checkpoints on full datasets yet fail to track changing knowledge. It finds transfer is asymmetric and confidence functions that work on one checkpoint can diverge sharply on another.

GCM

  • GCM’s contrast-set AUC headroom over oracle training grows for larger models when its Qwen3-8B backbone knows fewer answers.The authors interpret this as evidence that a static external verifier does not adapt to the evaluated model’s capability.Source: Figure 2 · p. 8

Transfer hypothesis

  • The authors hypothesize that transfer is difficult because only some of the many confidence functions calibrated on one checkpoint use meta-knowledge features that generalize.They examine full-set transfer to understand factors that may enable transfer and challenges that may prevent persistent calibration.Source: §4 · pp. 8–9

Transfer Asymmetry

  • AUC was 0.870 for forward transfer from 10% to 100% training, versus 0.671 for backward transfer from 100% to 10%.Adapters trained at 10% and 100% had nearly equal AUCs on 100% (0.870 vs. 0.885), but different AUCs on 10% (0.860 vs. 0.671).Source: §4 · p. 9

Random Labels

  • Training on C2 with random labels for C3 generally plummeted C3 performance without penalizing C2 performance.The result illustrates that an estimator can perform on one checkpoint but not another, supporting the proposed heterogeneity of viable confidence features.Source: §4 · p. 9

5 RELATED WORK

Prior confidence-estimation work spans several method families and has documented domain-transfer difficulties for trained methods. This paper highlights checkpoint-specific features as another source of brittleness and centers evaluation on knowledge contrast sets.

Confidence Estimation

  • Confidence-estimation methods include generation consistency, verbalized confidence, fine-tuning, and external verifiers.Trained methods achieve the best in-domain calibration but often struggle with domain transfer due to domain-specific signals (Srey et al., 2026; Ying et al., 2026).Source: §5 · p. 9
  • This work highlights checkpoint-specific features as another source of brittleness, which multi-checkpoint training partially alleviates.The passage describes this as an additional dimension beyond domain-specific transfer difficulties.Source: §5 · p. 9

Continual Learning

  • Continual-learning approaches for LLMs include model editing, retrieval, and continued pre-training; this work focuses on pre-training as the phase in which models acquire most knowledge.Prior work found model editing can lead to underconfidence in token-probability confidence, while a frozen probe can show viable forward transfer during pre-training (Liu et al., 2026).Source: §5 · p. 9
  • The paper distinguishes its approach by centering evaluation on knowledge contrast sets to isolate dependence between knowledge and confidence.The passage also notes prior work on continual learning and metacognition.Source: §5 · p. 9

6 CONCLUSION

The conclusion frames persistent calibration as confidence tracking a model’s evolving knowledge without repeated recalibration, while noting substantial headroom and potential difficulty finding robust meta-knowledge features. The accompanying appendix explains contrast-set conditioning and the metrics used to evaluate confidence.

6 CONCLUSION

  • Persistent calibration requires confidence estimates to reflect dynamically evolving knowledge without repeated recalibration, but substantial headroom remains despite multi-checkpoint training.The authors suggest that checkpoint-specific confidence functions may hinder identifying meta-knowledge features that generalize.Source: §6 · p. 10

A.1 CONDITIONING ON CONTRAST SET MEMBERSHIP

  • Contrast-set membership can make globally calibrated confidence appear miscalibrated: when each checkpoint is 80% accurate independently, each is expected to be 50% accurate on questions where their correctness differs.The appendix normalizes confidence pairs for contrast-set evaluation; its example maps (0.8, 0.8) to (0.5, 0.5), assuming relative probabilities of the two contrast outcomes match the independent case despite checkpoint carryover.Source: §A.1 · pp. 15–16

A.2 EVALUATION METRICS

  • Discriminability measures separation of correct from incorrect answers, whereas calibration asks whether accuracy among answers assigned confidence p equals p.AUC measures discriminability; Brier score measures calibration and discriminability; ECE measures calibration, estimated here with SmoothECE.Source: §A.2 · p. 16

A.3 AGGREGATION OF METRICS OVER CHECKPOINTS

The appendix specifies how evaluation metrics are aggregated across checkpoints and how datasets are processed. It avoids pooling where pooling could conceal checkpoint-level calibration differences and describes dataset-specific filtering and question handling.

A.3 AGGREGATION OF METRICS OVER CHECKPOINTS

  • Full-set AUC and Brier score are computed on pooled checkpoint data, while ECE is averaged across checkpoints to avoid cancellation that can hide miscalibration.For contrast sets, metrics are averaged across checkpoint pairs rather than pooled; AUC and Brier score use normalized confidence pairs, while ECE uses one checkpoint’s confidences to avoid cancellation.Source: §A.3 · p. 16
  • Contrast sets are included only when they contain at least 500 questions for TriviaQA and Jeopardy or 300 for BioASQ.The paper also states that different checkpoint pairs’ contrast sets are not pooled because the normalization assumes exactly one correct answer in each pair.Source: §A.3 · pp. 16–17

A.4 DATA PROCESSING

  • TriviaQA and Jeopardy questions are shuffled and deduplicated; Jeopardy categories are screened for factual-recall suitability, and low-quality questions are blacklisted.The resulting train and test splits contain 4,999 and 4,960 TriviaQA questions, and 4,997 and 4,998 Jeopardy questions, respectively.Source: §A.4 · pp. 17–18
  • BioASQ evaluation retains factoid and list questions, accepts any correct answer for list questions, and excludes lists with more than 10 answer groups.After processing, BioASQ has 1,690 factoid and 1,029 list questions; TriviaQA, Jeopardy, and BioASQ are the three factual-recall datasets used.Source: §A.4 · pp. 17–18

A.5 AUDIT OF DATA QUALITY

The appendix audits judgment quality, specifies evaluation and implementation procedures, and tests whether contrast-set results depend on the normalization method. Confidence estimators still underperform oracle training on contrast-set calibration with fitted normalization.

A.5 AUDIT OF DATA QUALITY

  • The TriviaQA audit found agreement with LLM judgments except for incomplete answers, and judged the contrast set similarly high quality to the full set.The auditor checked 50 contrast-set and 50 non-contrast-set questions; incomplete references occurred in 3/50 and 2/50, respectively.Source: §A.5 · p. 18

A.6 SELF-CONSISTENCY IMPLEMENTATION

  • Self-consistency uses probability-weighted beam search, groups equivalent answers with an NLI model, and assigns the top candidate’s probability as confidence.The beam size is 10; equivalent answers require mutual entailment as the NLI model’s argmax.Source: §A.6 · p. 18

A.7 LORA TRAINING CONFIGURATION

  • LoRA confidence adapters use one training epoch and binary cross-entropy, while retaining each checkpoint’s existing token embedding and unembedding matrices.The passage reports that using the existing LM head was crucial for forward transfer in early experiments.Source: §A.7 · p. 19

A.8.1 BACKGROUND ON THE REPORTED METRIC

  • Reported metric values average results across random seeds, with deterministic estimators ignoring the seed.For contrast-set metrics, the evaluated data concatenate checkpoint pairs restricted to questions on which their correctness differs.Source: §A.8.1 · p. 19

A.8.2 PAIRED BOOTSTRAP TEST

  • The paired bootstrap accounts for shared-question dependence and stratifies replicates by correctness classes for full-set metrics or improvement and regression for contrast-set metrics.A result is underlined when its one-sided 95% confidence interval contains zero.Source: §A.8.2 · pp. 19–20, §A.8 · p. 19

A.9 MODEL CHECKPOINTS; A.10 CONTRAST SET STATISTICS; B RESULTS AND ANALYSIS

  • The experiments report checkpoint identities and performance, contrast-set sizes, and significance-adjusted comparisons across appendix results.The passage identifies Tables 7–10 as reporting checkpoint data and contrast-set size statistics, and describes the result-marking convention for Appendix B.Source: §A.9 · p. 20, §A.10 · p. 20, B RESULTS AND ANALYSIS · p. 20

B.1 EMPIRICAL SUPPORT FOR CONDITIONING ON CONTRAST SET MEMBERSHIP

  • The fitted normalization maps pairs of checkpoint confidences to the probability that the first checkpoint is correct, and its results retain the oracle-training gap.The fitted Bradley-Terry model uses evaluation confidences and correctness labels, making the downstream results generous.Source: §B.1 · pp. 20–21
  • Fitted normalization preserves the finding that confidence estimators underperform oracle training on contrast-set calibration.The passage reports the same trends as Table 11, indicating robustness to the normalization implementation.Source: §B.1 · p. 21

B.2 ORACLE TRAINING VERSUS EXISTING METHODS

Existing confidence methods leave headroom relative to oracle training, including on contrast sets where full-set calibration can be saturated.

B.2 ORACLE TRAINING VERSUS EXISTING METHODS

  • SC and non-oracle training consistently fall short of oracle training on contrast-set calibration, even when full-set calibration is saturated.Post-hoc isotonic calibration makes SC’s full-set ECE negligible but does not improve its contrast-set AUC.Source: §B.2 · p. 21, Table 8 · p. 22
  • SC outperforms its surrogate and copy ablations in 5/6 contrast-set AUC cases, while non-oracle training does so in 3/6.A surrogate checkpoint can still distinguish correct from incorrect answers when differing checkpoints make different predictions.Source: §B.2 · p. 22

B.3 ABLATION OF MULTI-CHECKPOINT AND MULTI-ANSWER TRAINING

The ablations associate multi-checkpoint training gains chiefly with access to multiple checkpoints’ weights, while GCM and domain-transfer results delimit where calibration performance holds.

B.3 ABLATION OF MULTI-CHECKPOINT AND MULTI-ANSWER TRAINING

  • Multi-checkpoint and multi-answer training often lead to substantial calibration improvements in the reported ablations.The passage identifies Tables 14 and 15 as reporting these ablation results.Source: §B.3 · p. 22
  • Multi-checkpoint training outperforms its ablations, with most of its gain attributed to viewing multiple checkpoints’ weights rather than only adding data.The single-checkpoint data ablation matches unablated multi-checkpoint training on the reported class-balanced metrics.Source: §B.3 · p. 22

B.4 GCM RESULTS

  • GCM matches oracle training on contrast-set AUC only for Olmo 3 7B and falls short on full-set AUC, especially for larger models.The authors attribute its nontrivial contrast-set AUC to GCM knowing many correct answers in the contrast sets.Source: §B.4 · p. 23

B.5 DOMAIN TRANSFER

  • When transferring from TriviaQA or Jeopardy to BioASQ, oracle training and other methods drop to similarly low performance.The authors limit the oracle’s advantage in these experiments to the training domain.Source: §B.5 · p. 23

B.6 TRANSFER ASYMMETRY ANALYSIS

Transfer is asymmetric across checkpoints, and later checkpoints show greater cross-seed variation in confidence-adapter features.

B.6 TRANSFER ASYMMETRY ANALYSIS

  • Forward transfer is more robust than backward transfer, and later-checkpoint performance can become catastrophic without penalizing an earlier checkpoint.Figure 4 reports this pattern for checkpoint transfer and for multi-checkpoint training with random labels on the second checkpoint.Source: §B.6 · p. 23, Figure 4 · p. 24

B.7 CROSS-SEED VARIANCE ON DIFFERENT CHECKPOINTS

  • SC and non-oracle training remain below oracle training on contrast-set calibration even when full-set calibration is saturated.The reported post-hoc calibration of SC makes full-set ECE negligible but does not improve contrast-set AUC; fitted normalization preserves the same trend.Source: Table 11 · p. 25, Table 12 · p. 26
  • Mean and median correlations are positive across layers, suggesting later checkpoints tend to have a larger space of viable confidence features.Variance is computed within each question, so dataset question heterogeneity is not represented in this measure.Source: §B.7 · pp. 24–26

C RELATED WORK

Prior work uses contrast sets to test whether models rely on spurious features, while metacognition research examines models’ awareness of their behavior. This paper connects these lines by testing whether confidence estimators track knowledge changes across checkpoints.

C RELATED WORK

  • Continual-training-induced knowledge contrast sets let the paper test whether confidence estimators track changes in model knowledge.Earlier work used contrast sets to expose spurious-feature reliance and evaluated model performance on human-written or automatically generated sets.Source: C RELATED WORK · pp. 26–27
  • Confidence estimators trained on one checkpoint can nontrivially reflect a future checkpoint’s knowledge, but this is not consistent across models.The paper frames persistent calibration as a testbed for metacognition.Source: C RELATED WORK · p. 27
  • SC beats its ablations in 5/6 contrast-set AUC cases, whereas non-oracle training beats its ablations in 3/6.Source: Table 13 · p. 28
  • Multi-checkpoint training often yields substantial gains, though on Jeopardy the single-checkpoint data ablation matches it on class-balanced contrast-set metrics Δb.The Jeopardy table reports that multi-checkpoint training still beats its two ablations overall.Source: Table 14 · p. 29, Table 16 · p. 30, Table 15 · p. 30
  • GCMs underperform oracle training, especially for larger models, and this pattern persists after post-hoc calibration.The comparison motivates meta-knowledge because a static external verifier does not adapt to model capability.Source: Table 17 · p. 31
  • Oracle training’s advantage does not hold when transferring from TriviaQA or Jeopardy to BioASQ.The table caption links this result to prior work on domain-specific correctness signals.Source: Table 18 · p. 32
  • For Olmo 3 7B on TriviaQA, checkpoint index has positive mean and median Spearman correlation with adapter variance across layers.The caption suggests later checkpoints offer a larger space of viable confidence features for adapters to capture.Source: Table 19 · p. 33

Related papers

Loading 2609.38797v1…