Source-linked AI summary
Auditing Conformal Prediction for Language Models Under Inference-Time Model Shift
Achref Doula, Max Mühlhäuser
TL;DR
Conformal prediction for language models is commonly evaluated with matched calibration and deployment configurations, although deployment choices may change. This paper formalizes inference-time model shift and audits it across models, benchmarks, and configurations. Matched evaluation can substantially overstate deployment coverage, motivating reporting inference configuration as an experimental axis.
Problem
Current language-model CP evaluations typically use the same inference configuration for calibration and deployment, leaving reliability under configuration changes unexamined.
Source: §1 · pp. 1–2Method
Results
Takeaways & Limitations
The authors argue that CP evaluations should treat inference configuration as a first-class axis and report mismatch sensitivity alongside deployment-assumption-aligned baselines.
Source: §6 · p. 9The audit uses a finite deployment grid and does not address richer variation such as continuous temperatures or temporal drift in serving policies.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Conformal prediction (CP) for language models is typically evaluated under a matched protocol, in which calibration and deployment use the same post-training inference configuration, such as quantization mode and inference temperature. In deployment, these choices often differ from those used at calibration to meet memory or throughput constraints or serving policies. We argue that this exposes a limitation in current CP evaluation protocols for language models. We formalize the underlying phenomenon as inference-time model shift: a change in the conformal score distribution induced by inference configuration rather than by the input--label distribution, and theoretically characterize how it breaks split-conformal exchangeability even when the task distribution and trained model are unchanged. We then evaluate its consequences across 8 language models, 4 finite-label benchmarks, and 12 inference configurations. We find that matched evaluation can substantially overstate deployment-time coverage, and that the effect is heterogeneous across models and datasets. We argue that inference configuration should be reported as a first-class experimental axis in CP evaluations for language-model systems.
1 Introduction
Matched conformal prediction (CP) evaluates reliability when calibration and deployment share an inference configuration, but deployment choices can change after calibration. The paper formalizes inference-time model shift and finds that matched evaluation can overstate deployment coverage, motivating evaluation across inference configurations.
1 Introduction
- Inference-time model shift changes the conformal score distribution through inference configuration while the input–label distribution remains fixed, so calibration and deployment scores need not be exchangeable.Inference configuration includes choices such as quantization mode and temperature that shape the predictive distribution without changing the trained model.Source: §1 · pp. 1–2
- On MMLU-PRO, matched calibration reached mean coverage 0.902, versus 0.639 under mismatch and 0.508 in the worst case, with deployment fixed to nf4 at T=0.5.The comparison covers eight small language models and a nominal target of 0.90; the task distribution and trained model remain unchanged.Source: Figure 1 · p. 2, §1 · p. 2
- The audit covers eight language models, four finite-label benchmarks, and twelve configurations; discrete-label tasks permit direct comparison of labels, predictive distributions, and conformal scores across configurations.The scope isolates inference-configuration mismatch from confounds associated with open-ended generation.Source: §1 · p. 2
- Mismatch effects are heterogeneous across models and datasets, and are neither monotone across configurations nor consistent in direction.Low-temperature regimes concentrate failures, while aggressive quantization amplifies but does not solely cause them.Source: §1 · p. 2
- The authors propose reporting matched validity alongside mismatch sensitivity and baselines aligned with deployment assumptions.Their empirical audit spans eight language models, four finite-label benchmarks, and twelve inference configurations.Source: §1 · p. 2
2 Related Work
Prior work studies conformal prediction beyond matched data conditions and applies it to language-model reliability, while calibration and quantization research examines related deployment choices. This paper distinguishes its focus: whether conformal reliability persists when inference configuration changes between calibration and deployment.
2 Related Work
- Prior conformal work studies failures under covariate shift, label shift, and distribution drift, typically where the data-generating distribution or dependence structure changes [9] [37] [12] [29] [30].This paper instead keeps the task distribution fixed and studies inference-configuration changes.Source: §2 · p. 3
- Language-model CP research covers multiple-choice QA, generated candidate sets, factuality, API-only settings, and open-ended generation, usually under a fixed inference setup or other uncertainty targets [23, 31, 27, 5, 35, 42, 39, 43, 16].This paper asks whether calibration under one post-training configuration remains reliable under another.Source: §2 · p. 3
- Temperature scaling and quantization are established calibration and serving topics, but prior work does not formulate their calibration-to-deployment mismatch as a distinct conformal-validity problem [17] [26].The paper connects these choices to exchangeability of conformal scores and CP evaluation design.Source: §2 · p. 3
3 A Theoretical Characterization of Inference-Time Model Shift
The paper defines inference-time model shift as a change in conformal score distributions caused by inference configuration while the task distribution remains fixed. When calibration and deployment score distributions differ, split-conformal exchangeability—and therefore its usual coverage guarantee—need not hold.
3 A Theoretical Characterization of Inference-Time Model Shift
- Inference-time model shift occurs when two inference configurations induce different distributions of conformal scores despite a fixed task distribution.For configuration c, the model produces predictive distribution π(c)(x), and the nonconformity score V(c) induces distribution F_c.Source: §3 · p. 3
- At calibration, the configuration is observed with the input, label, and predictive distribution; at test time, the conformal layer receives the predictive vector while configuration may be latent.This setup distinguishes known calibration configurations from potentially unobserved deployment configurations.Source: §3 · p. 4
- When calibration uses c_cal and deployment uses c_dep with F_c_cal ≠ F_c_dep, their scores are generally not exchangeable, so the usual split-conformal proof no longer applies.The usual coverage guarantee therefore need not hold across configurations.Source: §3 · p. 4
- The failure comes from a changed score-generating mechanism, not a changed task distribution; matched evaluation remains appropriate for a fixed, known deployment configuration.Matched evaluation need not answer the broader deployment question when configuration can vary after calibration.Source: §3 · p. 4
4 Reliability Targets Under Deployment Uncertainty
The section defines reliability targets for deployment uncertainty and introduces two conservative procedures: a global maximum-threshold hedge and an adaptive latent-configuration hedge.
4 Reliability Targets Under Deployment Uncertainty
- When deployment configuration is unobserved, reliability should remain meaningful for individual latent deployment states; fixed-known and stable-mixture deployment call for different targets.Pooled calibration is included as a baseline for stable-mixture validity.Source: §4 · p. 4
4.1 Max-threshold calibration
- MAXTHR uses the largest configuration-specific threshold to hedge globally, yielding marginal validity for any deployment distribution on configurations.Its validity assumes configuration-specific calibration data and a test point are exchangeable with respect to the score function.Source: §4.1 · pp. 4–5
4.2 Latent-robust calibration
- LATENT-ROBUST uses observed predictive vectors to identify plausible deployment configurations and adapt the threshold to that set.Its two-stage error-budget split can make it more conservative than MAXTHR, producing larger prediction sets.Source: §4.2 · p. 5
5 Evaluation
The evaluation audits calibration–deployment mismatch across language models, finite-label benchmarks, and inference configurations. Mismatch reduces coverage, with effects varying across datasets and models and persisting when quantization or temperature is averaged over.
5.1 Experimental Setup
- The audit covers eight small language models, four finite-label benchmarks, and a 12-configuration grid of three quantization modes and four temperatures.It compares six procedures, including matched, mismatch, pooled, and robust calibration; the main metric is empirical coverage.Source: §5.1 · p. 6
5.2 Experiment 1: Reliability Under Calibration–Deployment Mismatch
- Across four datasets, MATCHED tracks the nominal 0.90 target, while MISMATCH WORST falls below it for almost every model; on MMLU-PRO, MISMATCH WORST averages 0.508.MAXTHR and LATENT-ROBUST restore target coverage in nearly all cells, with increased prediction-set size.Source: §5.2 · pp. 6–7, Figure 2 · p. 7
- Mismatch effects are heterogeneous: DBPEDIA-14 MISMATCH MEAN spans 0.795–0.946 across models, while MOBILEACTIONS MISMATCH WORST falls below target for 6 of 8 models.The MISMATCH MEAN–MISMATCH WORST gap is consistently large, so averaging off-diagonal calibration choices understates worst-case reliability costs.Source: §5.2 · p. 7
5.3 Experiment 2: Quantization Effects at Fixed Deployment Temperature
- With deployment temperature fixed at T = 0.5 and quantization averaged, MISMATCH WORST remains below target for all 8 models on every dataset.MMLU-PRO averages are 0.906 for MATCHED and 0.520 for MISMATCH WORST; low-temperature deployment remains mismatch-sensitive regardless of deployment quantization.Source: §5.3 · pp. 7–8, Figure 3 · p. 8
5.4 Experiment 3: Temperature Effects at Fixed Deployment Quantization
- With deployment quantization fixed to nf4 and temperatures averaged, MISMATCH WORST falls below target for all models on every dataset, while MISMATCH MEAN does so less consistently.MMLU-PRO averages are 0.901 for MATCHED and 0.719 for MISMATCH WORST; neither temperature nor quantization alone accounts for the degradation.Source: §5.4 · p. 8, Figure 4 · p. 9
6 Discussion
Inference configuration can change conformal scores and undermine split-conformal validity even when the task distribution and trained model remain fixed. Because mismatch effects vary across settings, the paper argues that evaluation should report configuration sensitivity and acknowledges limits in predicting those effects.
6 Discussion
- Matched-only evaluation can miss invalid split-conformal coverage when inference configuration changes the score distribution between calibration and deployment.The task distribution and trained model can remain fixed while this shift occurs.Source: §6 · p. 9
- Low-temperature deployment is the most mismatch-sensitive regime in these experiments, while aggressive quantization can amplify—but does not solely cause—the effect.Effects are not monotonically ordered across configurations; their magnitude and direction vary by dataset and model family.Source: §6 · p. 9
- The paper argues that CP evaluations should treat inference configuration as a first-class axis and report deployment assumptions, mismatch sensitivity, and a deployment-aligned baseline when configuration is unknown.The proposed reporting target may be a fixed configuration, stable mixture, or configuration unobserved at test time.Source: §6 · p. 9
- The analysis characterizes when mismatch matters but does not predict its coverage effects, and its finite deployment grid omits continuous temperatures and temporal serving-policy drift.The audit therefore does not address these richer forms of deployment variation.Source: §6 · p. 9
7 Conclusion
The conclusion identifies inference-time model shift as a distinct CP failure mode and argues that matched-only evaluation can overstate deployment reliability. The appendix supplies technical development for its guarantees and reliability targets under different deployment assumptions.
7 Conclusion
- Inference-time model shift can alter conformal scores and break matched-calibration assumptions despite a fixed task distribution, motivating evaluation that treats inference configuration as part of experimental design.The audit spans eight small language models, four datasets, and quantization and temperature settings.Source: §7 · p. 10
A Additional Theory and Full Statements
- The appendix formalizes reliability targets under deployment assumptions and provides full proofs for the main-text statements, while clarifying that it does not characterize every mismatch effect.It also recalls the split-conformal threshold and notation used in the paper.Source: A Additional Theory and Full Statements · p. 14
A.1 Setup and split-conformal preliminaries
- The split-conformal preliminaries define calibration scores and their order statistics, then use a threshold to form a prediction set and state standard validity and threshold monotonicity facts.These preliminaries establish notation and tools used in the subsequent technical development.Source: §A.1 · p. 15
A.2 Proof of Proposition 3.2
The appendix shows why changing inference configuration can break split-conformal exchangeability and distinguishes the resulting reliability targets. It also establishes when pooled calibration supports mixture-average reliability and where that guarantee stops.
A.2 Proof of Proposition 3.2
- When calibration and test scores have different configuration-dependent distributions, they are generally not exchangeable, so the usual split-conformal coverage guarantee need not hold.The proof gives a score-level counterexample with calibration scores 0 and a test score 1, yielding coverage 0.Source: §A.2 · p. 15
A.3 Reliability targets under deployment uncertainty
- Fixed-configuration, mixture-marginal, and configuration-conditional validity target, respectively, one known state, average reliability over a deployment mixture, and validity separately for every configuration.MATCHED, POOLED, and MAXTHR align with these targets, respectively; LATENT-ROBUST's stated theorem guarantees unconditional coverage, not configuration-conditional coverage.Source: §A.3 · p. 16
A.4 Pooled calibration and mixture-marginal validity
- Pooled calibration is mixture-marginally valid when calibration and test tuples are exchangeable under the same stable deployment mixture.This does not guarantee validity within each configuration, so pooled calibration may not suit individual latent deployment states.Source: §A.4 · p. 17
A.5 Proof of Proposition 4.1
The appendix establishes validity results under configuration-specific exchangeability, then clarifies the guarantees’ limits and documents the latent-setting implementation and experimental scope.
A.5 Proof of Proposition 4.1
- The MAXTHR set contains each configuration-specific set, so it achieves at least 1 − α coverage conditional on every deployment configuration and marginally under any deployment distribution.The proof uses set containment to transfer each configuration-specific guarantee, then obtains marginal validity by averaging across configurations.Source: §A.5 · p. 17
A.6 Proof of Theorem 4.2
- LATENT-ROBUST guarantees marginal coverage at 1 − α_stage1 − β_stage2 under its two exchangeability assumptions, but splitting the error budget can make its sets larger than MAXTHR’s.The Stage-1 thresholds use α_stage1 < α, which can inflate prediction sets despite the adaptive threshold.Source: §A.6 · pp. 18–19
A.7 Remarks and non-claims
- The theory assumes no universal ordering of configurations: mismatch may cause undercoverage, overcoverage, or mild effects, and the framework does not predict which outcome occurs.MATCHED, POOLED, MAXTHR, and LATENT-ROBUST address different deployment assumptions and reliability targets.Source: §A.7 · p. 19
B More on related work
- The related-work comparison focuses on whether prior evaluations treat inference configuration as an axis for conformal validity; matched calibration targets known, fixed deployment configuration.Source: B More on related work · p. 19, Table 1 · p. 20
C.1 Datasets and split sizes; C.2 Models, score functions, and deployment grid
- The supplementary benchmark uses four datasets, eight small language models, three nonconformity scores, and a 12-configuration grid formed from three quantization modes and four temperatures.The split sizes are fixed across the reported experiments.Source: §C.1 · p. 20, §C.2 · pp. 20–21
C.3 Latent split implementation and setting
- The latent implementation uses a Stage-2 logistic-regression router on set-size signatures, calibrated on held-out Stage-2 data to form plausible configurations and an adaptive threshold.Stage-1 thresholds are fixed first; the router is held fixed when configuration nonconformity scores are computed on Stage-2 calibration data.Source: §C.3 · p. 21
C.4 Quantization and temperature implementation.
The implementation varies quantization and temperature while holding the checkpoint and benchmark pipeline fixed; these choices alter the predictive distributions used to compute conformal scores.
C.4 Quantization and temperature implementation.
- Quantization changes numerical representation without changing the checkpoint, training, prompt, labels, or data splits, and no model weights are updated across configurations.The compared modes are fp16, int8, and nf4.Source: §C.4 · p. 21
- Temperature scales token logits before candidate-label scoring, after which length-normalized log-likelihoods are normalized over the finite candidate set to obtain predictive probabilities.Source: §C.4 · pp. 21–22
C.4 Quantization and temperature implementation.; C.5 Compute resources
- Changing quantization or temperature changes the score-generating procedure while leaving the task distribution, trained checkpoint, prompt, labels, and data splits unchanged.The experiments ran sequentially on one RTX A6000 GPU, with CPU resources handling preprocessing, calibration, routing, and aggregation.Source: §C.4 · p. 22, §C.5 · p. 22
D Additional Main-Benchmark Results
Across deployment settings, calibration–deployment mismatch remains consequential but varies across tasks and models; robustness methods trade larger prediction sets for more reliable coverage.
D.1 Deployment fixed to nf4@T = 0.5
- Mismatch often substantially degrades coverage under deployment fixed to nf4@T = 0.5, with the strongest failures on MMLU-Pro and greater stability for many models on MobileActions.MAXTHR and LATENT-ROBUST generally recover coverage by increasing average prediction-set size, especially on harder tasks.Source: §D.1 · p. 23, Table 5 · p. 23, Table 6 · p. 23
D.2 Deployment temperature fixed to T = 0.5
- With deployment temperature fixed at T = 0.5 and quantization averaged, several CommonsenseQA and MMLU-Pro model–dataset pairs remain strongly mismatch-sensitive, while effects still differ across model families.Source: §D.2 · p. 23, Table 7 · p. 24, Table 8 · p. 24
D.3 Deployment quantization fixed to nf4
- With deployment quantization fixed to nf4 and temperature averaged, substantial mismatch effects persist, supporting temperature as a primary driver while neither configuration dimension alone explains the phenomenon.The effects also vary across models, including different degrees of undercoverage and conservativeness.Source: §D.3 · pp. 23–24
E Calibration–Deployment Mismatch Structure
- Pairwise heatmaps show matched diagonal cells generally near target but configuration-dependent off-diagonal undercoverage and overcoverage that differ materially across models.Coverage panels show absolute coverage, while coverage-gap panels isolate deviations from the nominal target.Source: E Calibration–Deployment Mismatch… · p. 24
F.1 Robustness to the score function
Robustness checks find that mismatch sensitivity persists across score functions and target miscoverage levels, while reliability gains from robust methods carry set-size costs.
F.1 Robustness to the score function
- The qualitative mismatch pattern persists across APS, LAC, and RAPS; APS and RAPS are often more conservative than LAC, but changing scores does not remove the risk that matched-only reporting overstates deployment reliability.Source: §F.1 · pp. 24–25
- Score-function differences are typically larger than run-to-run variability, consistent with systematic differences in score construction rather than benchmark instability.Source: §F.1 · p. 25
F.2 Robustness to the target miscoverage level
- Across α values of 0.05, 0.1, 0.2, and 0.3, MATCHED tends to remain near target, mismatch baselines can under-cover, and robust methods recover coverage more reliably, with set-size costs changing alongside the target level.The qualitative method ranking remains stable across target levels.Source: §F.2 · p. 25
G LATENT-ROBUST Diagnostics
LATENT-ROBUST diagnostics connect its conservativeness to how well the latent deployment configuration can be inferred, with broader hedging carrying an efficiency cost. The section also reports diagnostic scope, claim boundaries, and run-to-run variability.
G LATENT-ROBUST Diagnostics
- Larger inferred configuration sets hedge across more plausible deployment states and correspond to a larger efficiency cost.The diagnostics describe conservativeness as mediated by the difficulty of inferring the latent deployment configuration.Source: G LATENT-ROBUST Diagnostics · p. 25
- Adaptive hedging is driven primarily by latent-configuration predictions rather than repeated reversion to the global worst-case threshold, and the diagnostics are reasonably stable across runs.The tables report standard deviations for the diagnostic quantities.Source: G LATENT-ROBUST Diagnostics · pp. 25–26
- Tables 23–26 report LATENT-ROBUST diagnostics on CommonsenseQA, DBPedia-14, MMLU-Pro, and MobileActions under the main-paper LAC setting.The diagnostics include empirical coverage of the latent configuration-set predictor and the average size of the predicted configuration set.Source: G LATENT-ROBUST Diagnostics · p. 25, Table 23 · p. 41, Table 24 · p. 42
1. Claims
- The paper limits its claims to finite-label benchmarks and post-training inference-configuration changes, excluding continuous temperatures and temporal serving-policy drift.The limitations also characterize the work primarily as a characterization rather than a mechanistic theory.Source: §1 · p. 43
7. Experiment statistical significance
- Experiments use three independent random splits, with appendix tables reporting mean ± standard deviation for coverage and average prediction-set size.The paper also describes aggregating over deployment axes before summarizing across runs.Source: §7 · p. 44