PaperlayerView exact revision on arXiv

Source-linked AI summary

No Universal Signal Predicts Sample-Level LLM Regression under Version Updates

Jia Sheng, Yiwei Lu

arXiv:2608.13607v1cs.AIcs.CLcs.LG

TL;DR

LLM updates can improve aggregate accuracy while still making previously correct individual samples incorrect. This paper evaluates inference-time signals for predicting these regressions and finds that the most useful signal depends on the task and model update, with some cross-version signals supporting selective fallback.

  • Problem

    Aggregate accuracy does not reveal sample-level regressions, motivating research into whether inference-time signals can predict which previously correct samples will become incorrect after updates.

    Source: §1 · pp. 1–2

  • Method

    The paper compares single-model and cross-version signals for predicting per-sample negative flips across MCQ, math reasoning, and code-generation tasks.

    Source: §3.1 · pp. 3–4

  • Results

    Signal effectiveness is task- and update-dependent: confidence leads on MCQ and simpler math, while likelihood/KL signals lead more often on harder math and code, with no universal winner.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    Practitioners should match regression signals to the task and update rather than defaulting to confidence, while some cross-version signals can support selective fallback.

    Source: §1 · p. 2, §5 · p. 10

    The proposed task-output-structure explanation is consistent with the results but remains a hypothesis rather than a mechanistic explanation.

    Source: §5 · p. 10

Abstract

from arXiv · show

Frontier LLMs are updated frequently and typically outperform their predecessors in aggregate. But aggregate gains say little about individual samples: an update can still cause sample-level regression, where a response correct under the old model becomes incorrect under the new one. This paper studies how to predict such regressions from signals available at inference time. We compare single-model signals (confidence, logit margin, attention entropy) against cross-version signals (output KL divergence, likelihood drift, token-level KL, representation drift) under a unified added-value test that isolates each signal's gain over a confidence baseline. Across six benchmarks in three task families (multiple-choice question answering, or MCQ; math reasoning; code generation) and six model update pairs, we find that (1) signal effectiveness is task-dependent: confidence is strongest on MCQ and simpler math, while likelihood/KL signals give the most frequent gains on harder math and code; (2) no signal is universally best across model updates either; and (3) some cross-version signals stay informative even when confidence fails, including without labels, which supports a proof-of-concept selective fallback that routes high-risk samples back to the old model. Practitioners can use these task-level patterns to choose which regression signal to trust for a given update. Code is available at https://github.com/jiashengsally/llm-regression-signals.

1 Introduction

This paper studies inference-time signals for predicting sample-level negative flips when updated LLMs regress on examples previously answered correctly. Across tasks and model updates, signal effectiveness depends on task and update, while some cross-version signals enable selective fallback when confidence is unreliable.

Motivation

  • Aggregate accuracy gains can conceal negative flips, where an updated model fails on samples the previous version answered correctly.The paper distinguishes aggregate improvement from sample-level reliability.Source: §1 · p. 1

Approach

  • The study compares cheap inference-time single-model signals, including confidence, logit margin, and attention entropy, with cross-version signals based on behavioral shifts.Cross-version signals compare old and new models on the same sample rather than inspecting parameter changes.Source: §1 · p. 1

Deployment

  • Some cross-version signals remain informative without label-based filtering and support a proof-of-concept selective fallback when confidence is unreliable.High-risk samples can be routed back to the old model as a guardrail.Source: §1 · pp. 1–2

Findings

  • Signal effectiveness is task-dependent, and no single signal wins universally across model updates.The study organizes this comparison around whether signals add value beyond old-model confidence and whether the best signal depends on task family.Source: §1 · p. 2
  • Combining multiple cross-version signals rarely improves over the best individual signal, indicating competition rather than complementarity.The result addresses whether combining signals produces a stronger predictor.Source: §1 · p. 2
  • Negative flips are not fully explained by old-model hardness or low confidence, and this relationship weakens as tasks become harder.This motivates looking beyond single-model confidence when anticipating regressions.Source: §1 · p. 2

2 Related Work

Prior work measures or forecasts model-update regressions mainly post hoc, through controlled fine-tuning, or at aggregate level. This paper instead targets sample-level regression during real version updates using inference-time signals evaluated under a unified protocol.

Regression Measurement and Mitigation

  • Negative flips and the Reliable Change Index measure sample-level behavioral changes, but RCI is post hoc while this work predicts regression from a single greedy decode at inference time.Cacioli’s RCI uses repeated sampling, whereas the paper’s prediction setting uses one greedy decode per model.Source: §2 · p. 2

Aggregate-Level Cross-Version Signals

  • Aggregate cross-version KL results are debated, whereas this work studies sample-level prediction under real updates and finds likelihood/KL effectiveness primarily task-dependent.Shenfeld et al. report forward KL as strong and consistent, while Chen et al. find its value limited and inconsistent.Source: §2 · p. 3

Sample-Level Forecasting

  • Prior sample-level forgetting forecasts assume controlled fine-tuning on enumerable tasks and repeated fine-tuning statistics, so they do not directly apply to real version updates at inference time.Jin and Ren (2024, 2026) use associations with upstream examples and a low-rank task-by-example matrix, respectively.Source: §2 · p. 3

Signal Toolbox

  • The evaluated signals come from established methods—including maximum-softmax confidence, margin sampling, attention entropy, and hidden-state geometry—and are compared under a unified protocol rather than newly proposed.These signals require white-box access to hidden states, attention weights, and output logits, motivating the study’s focus on open-weight models.Source: §2 · pp. 2–3

3 Method

The method defines sample-level negative flips and inference-time signals, then evaluates each signal’s added value over a confidence baseline. It covers MCQ, math reasoning, and code generation with deterministic answer generation and task-specific correctness checks.

Tasks and evaluation

  • The study covers MCQ, math reasoning, and code generation, using option matching, normalized final-answer extraction, and unit-test execution to determine correctness.Code failures include syntax, runtime, test, and timeout errors.Source: §3.1 · pp. 3–4

Generation and evaluation

  • Answers are generated with greedy decoding at temperature 0, producing reproducible outputs and deterministic negative-flip labels under fixed generation budgets.Generations reaching their cap are excluded because truncation corrupts labels and trajectory-level signals.Source: §3.1 · p. 3, §3.2 · p. 4

Negative flips

  • A negative flip occurs when the old model is correct but the new model is incorrect, with prediction restricted to the old-model-correct subset.Correctness is represented by c_old(x) and c_new(x) in {0, 1}.Source: §3.1 · p. 4

Inference-time signals

  • Signals are grouped into single-model uncertainty measures and cross-version measures that quantify how the update changes predictions, likelihoods, token distributions, or representations.Single-model signals include confidence, logit margin, and attention entropy; cross-version signals include output KL/JSD, likelihood drift, top-k token KL, and representation drift.Source: §3.2 · pp. 4–5

Added-value test

  • Each candidate is evaluated by its added value over LR(conf_new), a logistic-regression confidence baseline, using baseline-versus-augmented 5-fold cross-validation and reporting ΔCV-AUROC.Confidence is selected because it is simple and applies to every update pair, whereas representation drift requires matched hidden sizes and only three pairs.Source: Figure 2 · p. 4, §3.3 · p. 6

4 Experiments

The experiments evaluate regression signals across three task families and multiple same-family model updates under a unified added-value protocol. Results show task- and update-dependent effectiveness: confidence is strongest for MCQ and simpler math, while likelihood/KL signals most often help on harder math and code.

Experimental setup

  • Experiments cover five same-family update pairs, three task families, and instruction-tuned open-weight models with 7B–9B parameters.MCQ, math reasoning, and code generation are evaluated without explicit reasoning-phase decoding.Source: §4.1 · p. 6
  • The evaluation uses MMLU-Pro, GPQA Main, GSM8K, MATH-full, HumanEval, and MBPP with task-appropriate grading and a single-decode protocol.MCQ uses answer accuracy, while code generation is graded by unit tests.Source: §4.1 · p. 6

MCQ tasks

  • New-model confidence is the strongest predictor on MCQ, while old-model margin and attention entropy rarely add value over it.Output-distribution KL and JSD are reported separately as direct measures of answer-distribution change.Source: §4.2 · p. 8

Math reasoning

  • On MATH-full, new-trajectory likelihood drift and token KL give recurring gains for three of five update pairs, while representation drift is strongest for two matched pairs.On GSM8K, likelihood/KL gains are small and inconsistent, so new-model confidence is usually sufficient.Source: §4.2 · p. 8

Code generation

  • Likelihood drift and token KL give the most frequent gains on HumanEval and MBPP, though their magnitude and direction remain update-dependent.Trajectory-level signals can capture failures that confidence misses because code correctness depends on the entire generated program and unit-test outcomes.Source: §4.2 · p. 8

Robustness and fallback

  • Some cross-version signals remain informative without label-based filtering: likelihood drift changes from 0.80 to 0.76 on HumanEval for Qwen2→2.5.Other reported shifts are 0.74 to 0.72 on MBPP for Llama-3→3.1 and 0.62 to 0.59 on MATH-full for Qwen2.5→3.Source: §4.4 · pp. 9–10

5 Conclusion

Across updates and benchmarks, no single inference-time signal universally predicts sample-level LLM regressions: confidence is strongest for MCQ and simpler math, while likelihood/KL signals help on harder math and code. The task-output-structure explanation remains a hypothesis, and the framework could extend to other deployment-time changes.

5 Conclusion

  • No single inference-time signal predicts sample-level regressions across five same-family updates, one Llama-2→3 boundary case, and six benchmarks.New-model confidence wins on MCQ and simpler math, whereas cross-version likelihood/KL signals win on harder math and code.Source: §5 · p. 10
  • Old-model confidence alone cannot explain which samples flip, while some cross-version signals remain informative.These findings support using signals beyond confidence when predicting regressions after model updates.Source: §5 · p. 10

Limitations

  • The task-output-structure account is consistent with the results but remains a hypothesis rather than a mechanistic explanation.Future work could extend the framework to agentic scaffolding or system-prompt changes that may similarly cause sample-level regression.Source: §5 · p. 10

A Technical Appendix

The appendix documents the study configuration, expanded per-cell results, and additional analyses, including signal definitions, benchmark details, selective fallback, and label-free evaluations. It also reports implementation settings and representative per-cell AUROC results across update pairs and task benchmarks.

Study configuration

Expanded results

  • Tables A4–A6 provide baseline CV-AUROC values, added-signal CV-AUROC changes, accuracy gains, and old/new trajectory variants for MCQ, math, and code.The baseline uses confnew alone; cross-version signals in math and code have old-trajectory and new-trajectory variants.Source: A Technical Appendix · p. 14, Table A4 · p. 16, Table A5 · p. 17

Computing environment

  • Experiments used one NVIDIA L40S or H100 GPU per job, 16 CPU cores, and 64 GB of memory, with Ubuntu 24.04.4 and the listed Python, PyTorch, Transformers, scikit-learn, and SciPy versions.These settings specify the appendix’s computing and software environment.Source: A Technical Appendix · p. 14

Additional analyses

  • The appendix reports label-free standalone AUROC with bootstrap 95% confidence intervals for evaluated signals on the Mold-correct subset and full evaluation set.Representation drift is unavailable for update pairs with unequal hidden sizes and is reported only for three eligible pairs.Source: Table A10 · p. 21

Label-free results

  • 0.65 [0.61, 0.69] is the Mold-correct-subset AUROC for representation drift on Qwen2→2.5 MMLU-Pro, versus 0.53 [0.50, 0.57] on the full evaluation set.For Gemma 1.1→2 MMLU-Pro, representation drift is unavailable.Source: A Technical Appendix · p. 22
  • 0.81 [0.67, 0.93] is the Mold-correct-subset AUROC for confnew on Qwen2.5→3 HumanEval, versus 0.79 [0.65, 0.91] on the full evaluation set.The appendix also reports HumanEval likelihood-drift variants, including 0.69 [0.55, 0.81] for Gemma 1.1→2 with the old trajectory.Source: A Technical Appendix · pp. 24–25

A.1 A Negative-Flip Case Study

The Qwen2.5→3 case study shows a negative flip caused by misreading the question, not by faulty arithmetic. The old model answers correctly, while the new model produces an incorrect total after undercounting weekday classes.

A.1 A Negative-Flip Case Study

  • The case study reproduces the full reasoning for a GSM8K negative flip from Qwen2.5 to Qwen3.The outputs were lightly reformatted, while their numbers and reasoning steps remained unchanged.Source: §A.1 · p. 26
  • The old model correctly counts 33 classes, 495 students, and earns $7425.It computes 25 weekday classes plus 8 Saturday classes, then multiplies by 15 students and $15 per student.Source: §A.1 · p. 26
  • The new model incorrectly counts 13 classes, 195 students, and earns $2925.It interprets the weekdays as five days with one class per day before adding Saturday’s 8 classes.Source: §A.1 · p. 26
  • The regression is a reading-comprehension error: Qwen3 treats five weekday classes per day as one class per weekday.The correct interpretation is 25 weekday classes, not 5; the resulting calculation itself is not erroneous.Source: §A.1 · p. 26

Related papers

Loading 2608.13607v1…