PaperlayerView exact revision on arXiv

Source-linked AI summary

No Universal Signal Predicts Sample-Level LLM Regression under Version Updates

Jia Sheng, Yiwei Lu

arXiv:2608.13607v1cs.AIcs.CLcs.LG

TL;DR

LLM updates can improve aggregate accuracy while still making previously correct individual samples incorrect. This paper evaluates inference-time signals for predicting these regressions and finds that the most useful signal depends on the task and model update, with some cross-version signals supporting selective fallback.

  • Problem

    Aggregate accuracy does not reveal sample-level regressions, motivating research into whether inference-time signals can predict which previously correct samples will become incorrect after updates.

  • Method

    The paper compares single-model and cross-version signals for predicting per-sample negative flips across MCQ, math reasoning, and code-generation tasks.

  • Results

    Signal effectiveness is task- and update-dependent: confidence leads on MCQ and simpler math, while likelihood/KL signals lead more often on harder math and code, with no universal winner.

  • Takeaways & Limitations

    Practitioners should match regression signals to the task and update rather than defaulting to confidence, while some cross-version signals can support selective fallback.

  • Takeaways & Limitations

    The proposed task-output-structure explanation is consistent with the results but remains a hypothesis rather than a mechanistic explanation.

Abstract

from arXiv · show

Frontier LLMs are updated frequently and typically outperform their predecessors in aggregate. But aggregate gains say little about individual samples: an update can still cause sample-level regression, where a response correct under the old model becomes incorrect under the new one. This paper studies how to predict such regressions from signals available at inference time. We compare single-model signals (confidence, logit margin, attention entropy) against cross-version signals (output KL divergence, likelihood drift, token-level KL, representation drift) under a unified added-value test that isolates each signal's gain over a confidence baseline. Across six benchmarks in three task families (multiple-choice question answering, or MCQ; math reasoning; code generation) and six model update pairs, we find that (1) signal effectiveness is task-dependent: confidence is strongest on MCQ and simpler math, while likelihood/KL signals give the most frequent gains on harder math and code; (2) no signal is universally best across model updates either; and (3) some cross-version signals stay informative even when confidence fails, including without labels, which supports a proof-of-concept selective fallback that routes high-risk samples back to the old model. Practitioners can use these task-level patterns to choose which regression signal to trust for a given update. Code is available at https://github.com/jiashengsally/llm-regression-signals.

1 Introduction

This paper studies inference-time signals for predicting sample-level negative flips when updated LLMs regress on examples previously answered correctly. Across tasks and model updates, signal effectiveness depends on task and update, while some cross-version signals enable selective fallback when confidence is unreliable.

  • Motivation: Aggregate accuracy gains can conceal negative flips, where an updated model fails on samples the previous version answered correctly.The paper distinguishes aggregate improvement from sample-level reliability.
  • Approach: The study compares cheap inference-time single-model signals, including confidence, logit margin, and attention entropy, with cross-version signals based on behavioral shifts.Cross-version signals compare old and new models on the same sample rather than inspecting parameter changes.
  • Deployment: Some cross-version signals remain informative without label-based filtering and support a proof-of-concept selective fallback when confidence is unreliable.High-risk samples can be routed back to the old model as a guardrail.
  • Findings: Signal effectiveness is task-dependent, and no single signal wins universally across model updates.The study organizes this comparison around whether signals add value beyond old-model confidence and whether the best signal depends on task family.
  • Findings: Combining multiple cross-version signals rarely improves over the best individual signal, indicating competition rather than complementarity.The result addresses whether combining signals produces a stronger predictor.
  • Findings: Negative flips are not fully explained by old-model hardness or low confidence, and this relationship weakens as tasks become harder.This motivates looking beyond single-model confidence when anticipating regressions.

2 Related Work

Prior work measures or forecasts model-update regressions mainly post hoc, through controlled fine-tuning, or at aggregate level. This paper instead targets sample-level regression during real version updates using inference-time signals evaluated under a unified protocol.

  • Regression Measurement and Mitigation: Negative flips and the Reliable Change Index measure sample-level behavioral changes, but RCI is post hoc while this work predicts regression from a single greedy decode at inference time.Cacioli’s RCI uses repeated sampling, whereas the paper’s prediction setting uses one greedy decode per model.
  • Aggregate-Level Cross-Version Signals: Aggregate cross-version KL results are debated, whereas this work studies sample-level prediction under real updates and finds likelihood/KL effectiveness primarily task-dependent.Shenfeld et al. report forward KL as strong and consistent, while Chen et al. find its value limited and inconsistent.
  • Sample-Level Forecasting: Prior sample-level forgetting forecasts assume controlled fine-tuning on enumerable tasks and repeated fine-tuning statistics, so they do not directly apply to real version updates at inference time.Jin and Ren (2024, 2026) use associations with upstream examples and a low-rank task-by-example matrix, respectively.
  • Signal Toolbox: The evaluated signals come from established methods—including maximum-softmax confidence, margin sampling, attention entropy, and hidden-state geometry—and are compared under a unified protocol rather than newly proposed.These signals require white-box access to hidden states, attention weights, and output logits, motivating the study’s focus on open-weight models.

3 Method

The method defines sample-level negative flips and inference-time signals, then evaluates each signal’s added value over a confidence baseline. It covers MCQ, math reasoning, and code generation with deterministic answer generation and task-specific correctness checks.

  • Tasks and evaluation: The study covers MCQ, math reasoning, and code generation, using option matching, normalized final-answer extraction, and unit-test execution to determine correctness.Code failures include syntax, runtime, test, and timeout errors.
  • Generation and evaluation: Answers are generated with greedy decoding at temperature 0, producing reproducible outputs and deterministic negative-flip labels under fixed generation budgets.Generations reaching their cap are excluded because truncation corrupts labels and trajectory-level signals.
  • Negative flips: A negative flip occurs when the old model is correct but the new model is incorrect, with prediction restricted to the old-model-correct subset.Correctness is represented by c_old(x) and c_new(x) in {0, 1}.
  • Inference-time signals: Signals are grouped into single-model uncertainty measures and cross-version measures that quantify how the update changes predictions, likelihoods, token distributions, or representations.Single-model signals include confidence, logit margin, and attention entropy; cross-version signals include output KL/JSD, likelihood drift, top-k token KL, and representation drift.
  • Added-value test: Each candidate is evaluated by its added value over LR(conf_new), a logistic-regression confidence baseline, using baseline-versus-augmented 5-fold cross-validation and reporting ΔCV-AUROC.Confidence is selected because it is simple and applies to every update pair, whereas representation drift requires matched hidden sizes and only three pairs.

4 Experiments

The experiments evaluate regression signals across three task families and multiple same-family model updates under a unified added-value protocol. Results show task- and update-dependent effectiveness: confidence is strongest for MCQ and simpler math, while likelihood/KL signals most often help on harder math and code.

  • Experimental setup: Experiments cover five same-family update pairs, three task families, and instruction-tuned open-weight models with 7B–9B parameters.MCQ, math reasoning, and code generation are evaluated without explicit reasoning-phase decoding.
  • Experimental setup: The evaluation uses MMLU-Pro, GPQA Main, GSM8K, MATH-full, HumanEval, and MBPP with task-appropriate grading and a single-decode protocol.MCQ uses answer accuracy, while code generation is graded by unit tests.
  • MCQ tasks: New-model confidence is the strongest predictor on MCQ, while old-model margin and attention entropy rarely add value over it.Output-distribution KL and JSD are reported separately as direct measures of answer-distribution change.
  • Math reasoning: On MATH-full, new-trajectory likelihood drift and token KL give recurring gains for three of five update pairs, while representation drift is strongest for two matched pairs.On GSM8K, likelihood/KL gains are small and inconsistent, so new-model confidence is usually sufficient.
  • Code generation: Likelihood drift and token KL give the most frequent gains on HumanEval and MBPP, though their magnitude and direction remain update-dependent.Trajectory-level signals can capture failures that confidence misses because code correctness depends on the entire generated program and unit-test outcomes.
  • Robustness and fallback: Some cross-version signals remain informative without label-based filtering: likelihood drift changes from 0.80 to 0.76 on HumanEval for Qwen2→2.5.Other reported shifts are 0.74 to 0.72 on MBPP for Llama-3→3.1 and 0.62 to 0.59 on MATH-full for Qwen2.5→3.

5 Conclusion

Across updates and benchmarks, no single inference-time signal universally predicts sample-level LLM regressions: confidence is strongest for MCQ and simpler math, while likelihood/KL signals help on harder math and code. The task-output-structure explanation remains a hypothesis, and the framework could extend to other deployment-time changes.

  • 5 Conclusion: No single inference-time signal predicts sample-level regressions across five same-family updates, one Llama-2→3 boundary case, and six benchmarks.New-model confidence wins on MCQ and simpler math, whereas cross-version likelihood/KL signals win on harder math and code.
  • 5 Conclusion: Old-model confidence alone cannot explain which samples flip, while some cross-version signals remain informative.These findings support using signals beyond confidence when predicting regressions after model updates.
  • Limitations: The task-output-structure account is consistent with the results but remains a hypothesis rather than a mechanistic explanation.Future work could extend the framework to agentic scaffolding or system-prompt changes that may similarly cause sample-level regression.

A Technical Appendix

The appendix documents the study configuration, expanded per-cell results, and additional analyses, including signal definitions, benchmark details, selective fallback, and label-free evaluations. It also reports implementation settings and representative per-cell AUROC results across update pairs and task benchmarks.

  • Study configuration: Tables A1–A3 define model-update pairs, evaluated signals, and benchmark formats, sizes, grading rules, and decoding limits.Generations reaching the token cap are excluded, so final analysis sets may be smaller.
  • Expanded results: Tables A4–A6 provide baseline CV-AUROC values, added-signal CV-AUROC changes, accuracy gains, and old/new trajectory variants for MCQ, math, and code.The baseline uses confnew alone; cross-version signals in math and code have old-trajectory and new-trajectory variants.
  • Computing environment: Experiments used one NVIDIA L40S or H100 GPU per job, 16 CPU cores, and 64 GB of memory, with Ubuntu 24.04.4 and the listed Python, PyTorch, Transformers, scikit-learn, and SciPy versions.These settings specify the appendix’s computing and software environment.
  • Additional analyses: The appendix reports label-free standalone AUROC with bootstrap 95% confidence intervals for evaluated signals on the Mold-correct subset and full evaluation set.Representation drift is unavailable for update pairs with unequal hidden sizes and is reported only for three eligible pairs.
  • Label-free results: 0.65 [0.61, 0.69] is the Mold-correct-subset AUROC for representation drift on Qwen2→2.5 MMLU-Pro, versus 0.53 [0.50, 0.57] on the full evaluation set.For Gemma 1.1→2 MMLU-Pro, representation drift is unavailable.
  • Label-free results: 0.81 [0.67, 0.93] is the Mold-correct-subset AUROC for confnew on Qwen2.5→3 HumanEval, versus 0.79 [0.65, 0.91] on the full evaluation set.The appendix also reports HumanEval likelihood-drift variants, including 0.69 [0.55, 0.81] for Gemma 1.1→2 with the old trajectory.

A.1 A Negative-Flip Case Study

The Qwen2.5→3 case study shows a negative flip caused by misreading the question, not by faulty arithmetic. The old model answers correctly, while the new model produces an incorrect total after undercounting weekday classes.

  • A.1 A Negative-Flip Case Study: The case study reproduces the full reasoning for a GSM8K negative flip from Qwen2.5 to Qwen3.The outputs were lightly reformatted, while their numbers and reasoning steps remained unchanged.
  • A.1 A Negative-Flip Case Study: The old model correctly counts 33 classes, 495 students, and earns $7425.It computes 25 weekday classes plus 8 Saturday classes, then multiplies by 15 students and $15 per student.
  • A.1 A Negative-Flip Case Study: The new model incorrectly counts 13 classes, 195 students, and earns $2925.It interprets the weekdays as five days with one class per day before adding Saturday’s 8 classes.
  • A.1 A Negative-Flip Case Study: The regression is a reading-comprehension error: Qwen3 treats five weekday classes per day as one class per weekday.The correct interpretation is 25 weekday classes, not 5; the resulting calculation itself is not erroneous.
Loading 2608.13607v1…