Source-linked AI summary
Perplexity Cannot Always Tell Right from Wrong
Petar Veličković, Federico Barbero, Christos Perivolaropoulos, Simon Osindero, Razvan Pascanu
TL;DR
Perplexity is widely used to evaluate sequential models, but its relationship to prediction accuracy can make it an unreliable model-selection signal. The paper combines Transformer-continuity reasoning with confidence–accuracy analysis and parity-task checkpoint observations. It finds that confidence gains can place models on the wrong side of iso-perplexity, especially out of distribution.
Problem
Perplexity is a popular, easy-to-compute evaluation measure, but the paper examines whether it can be trusted to select models when accuracy and confidence diverge.
Source: §1 · p. 1Method
The paper uses Transformer-continuity reasoning and analyzes confidence–accuracy relationships, with parity prediction providing an out-of-distribution setting for checkpoint study.
Source: §4 · p. 5, §4.4 · p. 8Results
In the studied out-of-distribution regime, many models fall on the wrong side of iso-perplexity as confidence increases, and the best OOD-accuracy checkpoint is among the worst in OOD perplexity.
Source: §4.5 · p. 8Takeaways & Limitations
A confident prediction can coexist with a failure that has log-perplexity arbitrarily close to zero, so perplexity may not reveal that failure.
Source: §5 · p. 8The paper offers no alternative to perplexity when accuracy cannot be measured, and instead proposes diagnostic approaches for deciding when model-selection protocols may need reconsideration.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Perplexity -- a function measuring a model's overall level of "surprise" when encountering a particular output -- has gained significant traction in recent years, both as a loss function and as a simple-to-compute metric of model quality. Prior studies have pointed out several limitations of perplexity, often from an empirical manner. Here we leverage recent results on Transformer continuity to show in a rigorous manner how perplexity may be an unsuitable metric for model selection. Specifically, we prove that, if there is any sequence that a compact decoder-only Transformer model predicts accurately and confidently -- a necessary pre-requisite for strong generalisation -- it must imply existence of another sequence with very low perplexity, but not predicted correctly by that same model. Further, by analytically studying iso-perplexity plots, we find that perplexity will not always select for the more accurate model -- rather, any increase in model confidence must be accompanied by a commensurate rise in accuracy for the new model to be selected.
1. Introduction
The paper argues that perplexity can favor incorrect predictions and develops theoretical and empirical evidence that confidence can undermine its usefulness for model selection. It studies this through Transformer continuity, iso-perplexity analysis, and language-model checkpoints.
1. Introduction
- Perplexity combines confidence and correctness, so a less accurate but better-calibrated model may receive a lower score and be preferred.The paper frames its contributions as evidence that miscalibrated confidence can damage perplexity’s model-selection power.Source: §1 · pp. 1–2
- For a wide class of decoder-only Transformers, confident correct prediction on a sufficiently long sequence implies another sequence that is predicted incorrectly but has log-perplexity approaching zero.The paper also validates the continuity-based observation on bitstring-copy tasks with a custom Transformer and Gemma 3 4B.Source: §1 · p. 2
- Iso-perplexity curves reveal regions where confidence rises too much relative to accuracy improvement, leaving a model unselected against weaker alternatives.The analysis studies these curves under assumptions about homogeneity of confidence.Source: §1 · p. 2
- Checkpoint analysis of parity prediction finds that distribution shifts are a key driver of models landing on the wrong side of iso-perplexity curves.The study tracks checkpoints throughout language-model training.Source: §1 · p. 2
2. Related work
Prior work documents disconnects between likelihood or perplexity and sample quality, including in long-context language modeling. This paper connects those observations to confidence calibration and Transformer continuity, and argues that accuracy and confidence can interact asymmetrically.
2. Related work
- Likelihood can diverge from sample quality: models may achieve good likelihood with poor samples, or assign higher likelihood to out-of-distribution images.The cited examples concern generative models, including VAEs and flow-based models [Theis et al. (2015); Nalisnick et al. (2019)].Source: §2 · p. 2
- For language generation, Nucleus Sampling was reported to produce better generations with lower likelihoods than likelihood-maximizing approaches such as beam search [Holtzman et al. (2019)].Source: §2 · p. 2
- Long-context studies report low perplexity despite inaccessible or missed information, while this paper argues that accuracy and confidence have a detrimental asymmetric relationship [Gelberg et al. (2025); Peng et al. (2023); Liu et al. (2024); Hsieh et al. (2024)].Fang et al. (2025) attribute misleading long-context perplexity to aggregation; this paper says its account also extends to confidence–accuracy offsets [Fang et al. (2025)].Source: §2 · p. 2
- Calibration work discusses neural-network overconfidence and argues that perplexity minimization can create regions where a confident but wrong model scores better than a hesitant, more accurate one [Guo et al. (2017); Kadavath et al. (2022)].Source: §2 · p. 2
- The paper uses Pasten et al. (2025)’s continuity result, built on decoder-only Transformer limitations, to show that accurate confident prediction of one long sequence entails another sequence with low perplexity that the model fails to predict.The cited theoretical background includes representation collapse in decoder-only Transformers [Barbero et al. (2024)].Source: §2 · p. 2
3. Log-perplexity of wrong next-token predictors can arbitrarily approach zero
For compact-position-embedding decoder-only Transformers, confidently copying an infinite bitstring entails wrong-copy sequences with log-perplexity arbitrarily close to that of correct sequences. The section extends this analysis to stochastic sampling and learning dynamics, then reports copy-task experiments consistent with the predicted convergence.
3.1. Preliminaries
- The bitstring copy task supplies a simple setting where correctness and perplexity are easy to define: after a bit sequence and stop symbol, the model must reproduce the bits exactly.The theoretical analysis assumes compact position embeddings and deterministic greedy decoding; the authors note this setup is less common in contemporary use.Source: §3.1 · p. 3, §3.2 · p. 3, §3.3 · p. 3
3.2. Deterministic sampling
- For any sufficiently long prefix αN, a bitstring βN exists that the model copies incorrectly while its perplexity differs from αN’s by an arbitrarily small amount.This convergence result is the basis for the confident-copying proposition.Source: §3.2 · p. 3
- With greedy decoding, confident correct copying of every finite prefix implies wrong-copy sequences whose log-perplexity approaches the correct sequence’s value; at certainty, the wrong sequences’ perplexity tends to zero.The results assume a decoder-only Transformer with compact position embeddings and a bitstring copy task.Source: §3.2 · p. 3
3.3. Stochastic sampling
- For stochastic sampling, the authors fold temperature into an effective confidence level and distinguish regimes by Nγ and NεN when assessing copy reliability.When Nγ is small, stochastic bit flips are unlikely on the baseline sequence.Source: §3.3 · p. 4
- When Nγ ≪1 and NεN ≪1, stochastic sampling is likely to copy the baseline αN correctly but to fail on βN because continuity tethers output to αN.If the sequence is too short for the theory to apply, the authors cannot make concrete claims about βN; when Nγ is not small, the model may also fail to copy αN reliably.Source: §3.3 · p. 4
3.4. Implications on learning dynamics
- As the correct sequence’s loss approaches zero, the gradient on the incorrectly copied βN sequence vanishes under Proposition 3.2’s conditions.The authors state that the vanishing training signal means such an incorrect sample cannot be jointly learned.Source: §3.4 · p. 4
3.5. Empirical analysis
- In both a trained copy-task Transformer and Gemma 3 4B, the αN–βN distribution gap shrinks as N increases, βN copying probability collapses, and their log-perplexities converge.Gemma’s patterns are noisy because its larger token set can lead to premature newline or end-of-turn predictions, but the authors report that these issues do not change the overall metric trends.Source: §3.5 · pp. 4–5
- The analysis mainly relies on perplexity’s averaging process, while the authors separately motivate studying a pointwise setup that emphasizes model confidence.They note that a shorter-token metric such as LongPPL would not necessarily suffer the identified smoothing effects.Source: §3.5 · p. 5
4. An analytic view into confidence
The section analyzes when changes in model confidence and accuracy cause perplexity to misrank models, then examines whether such failures occur during parity-model training. The empirical results show that these failures are especially pronounced out of distribution.
4. An analytic view into confidence
- An increase in confidence must be matched by sufficient accuracy improvement for perplexity to recognize the new model as better.Under the stated assumptions, the analysis derives a critical accuracy for increased confidence.Source: §4 · p. 5, §4.2 · p. 6
4.1. Preliminaries
- The analysis treats confidence and accuracy as scalars in a binary classifier that makes every decision with identical confidence.This simplifying setup makes log-perplexity expressible from accuracy and confidence.Source: §4.1 · pp. 5–6
4.2. Iso-perplexity curves
- Iso-perplexity curves show that greater confidence often requires accuracy gains exceeding 5–10 percentage points, while perfect confidence requires perfect accuracy.For an already accurate but unconfident model, confidence can also improve perplexity despite a substantial accuracy drop.Source: §4.2 · pp. 6–7
4.5. Results and Discussion
- OOD, the highest-accuracy checkpoint is among the worst in perplexity, and perplexity-accuracy correlation is positive rather than the IID value of r = −0.94.The authors conclude that many OOD checkpoints fall on the wrong side of iso-perplexity, an effect tied to increased confidence.Source: Figure 4 · p. 7, §4.5 · p. 8
4.3. How often are we on the wrong iso-perplexity side?
- The iso-perplexity analysis assumes identical confidence, which may not fit any observed real-data accuracy and log-perplexity pair.The authors also note that curves become difficult to interpret near their singularity at γ = 0.5.Source: §4.3 · p. 7
- To assess misranking in practice, the authors compare log-perplexity and micro-F1 across checkpoints and use averaged Shannon entropy as a confidence proxy.Pairs with lower perplexity but lower accuracy indicate misranking; ideally, perplexity and accuracy have Pearson correlation r ≈ −1.Source: §4.3 · p. 7
4.4. Parity task setup
- Parity provides a simple target procedure that is difficult for Transformers to reproduce when generalizing to longer bitstrings.The experiment trains on bitstrings up to size 16 and evaluates out of distribution at size 128, tracking checkpoints during training.Source: §4.4 · p. 8
4.5. Results and Discussion
- OOD checkpoints with low entropy can retain predictive power while harming perplexity, unlike IID entropy reductions that accompany better performance and lower loss.The best-performing observed OOD model also has one of the dataset’s lowest entropies.Source: §4.5 · p. 8
5. Conclusions
The paper argues that perplexity’s averaging can hide confident mistakes, while the metric itself can favor less-confident predictors. It offers no replacement where accuracy cannot be measured, but proposes diagnostics to help identify when model-selection protocols may need reconsideration.
5. Conclusions
- Averaging token losses can leave a confident mistake with log-perplexity arbitrarily close to zero.The authors present this as theoretical evidence that averaging can produce misleading outcomes.Source: §5 · p. 8
- Perplexity itself is skewed toward less-confident predictors, especially in long-context settings, and iso-perplexity analysis reveals unfavorable regions.The paper says high confidence combined with a perplexity objective can construct the averaging-related cases described above.Source: §5 · p. 8
- The paper offers no alternative when accuracy cannot be measured, but proposes diagnostics to identify when model-selection protocols may need reconsideration.Source: §5 · p. 8
A. Proof of Lemma 3.1.
The proof uses continuity of compact-position-embedding decoder-only Transformers to construct an incorrect bitstring whose perplexity approaches that of a correctly copied prefix. The construction flips one bit while preserving the final symbol, then uses continuity to bound the prediction change.
A. Proof of Lemma 3.1.
- Under the lemma’s assumption, the Transformer confidently copies every finite prefix of an infinite bitstring.Source: A. Proof of Lemma 3.1 · p. 10
- For sufficiently long prefixes, the resulting bitstring is not correctly copied yet has perplexity arbitrarily close to the original prefix’s.The proof says continuity misclassifies the altered sequence while smoothing the perplexity spike from that misclassification.Source: A. Proof of Lemma 3.1 · p. 10
- Continuity for equal-length inputs with the same last symbol lets the proof flip one bit while keeping the model’s output close.The proof chooses a bit flip at a position that keeps the relativised Hamming distance below the continuity bound.Source: A. Proof of Lemma 3.1 · p. 10
B. Proof of Corollary 3.7
The corollary shows that, under Proposition 3.2’s conditions, the gradient of standard autoregressive cross-entropy vanishes on a sequence the Transformer does not correctly copy. The proof relies on compactness to bound the logits Jacobian and on convergence in loss to convergence in predicted targets.
B. Proof of Corollary 3.7
- The corollary concerns standard autoregressive cross-entropy for a compact decoder-only Transformer with compact position embeddings.Source: B. Proof of Corollary 3.7 · p. 10
- If the model’s loss on αN tends to zero, the conditions of Proposition 3.2 imply vanishing loss gradient on the incorrectly copied βN.Source: B. Proof of Corollary 3.7 · p. 10
- The proof bounds the logits Jacobian using compactness and uses convergence in loss to establish convergence of predicted targets.Source: B. Proof of Corollary 3.7 · p. 11