PaperlayerView exact revision on arXiv

Source-linked AI summary

Generalization Is Stability, Not Accuracy: Multi-Axis Evaluation of LLMs

Nagham Omar, Mahmoud Jabarin, Maya Rozenshtein, Rom Himelstein, Avi Mendelson, Amit LeVi

arXiv:2610.01428v1cs.CLcs.AIcs.LG

TL;DR

Existing evaluations often rely on aggregate scores or a single behavioral signal, leaving per-instance stability across variation families and behavioral axes incompletely assessed. The paper introduces SAGO to measure behavioral change across these dimensions. Across eleven models and six datasets, instability is widespread and no model generalizes uniformly.

  • Problem

    Existing evaluations often use aggregate scores or a single behavioral signal, limiting assessment of per-instance stability across datasets, variation families, and behavior.

    Source: §2 · p. 3

  • Method

    SAGO evaluates per-instance behavioral stability across semantically equivalent variations, datasets, and axes including generation, activations, confidence, and mirroring.

    Source: §1 · p. 2

  • Results

    Across eleven LLMs and six datasets, no model generalizes uniformly; generation consistency is most sensitive, while content stability and response mirroring are independent dimensions.

    Source: §1 · p. 2

  • Takeaways & Limitations

    High benchmark performance is necessary but insufficient; reliable generalization requires per-instance stability across deployment-relevant domains, variations, and behavioral axes.

    Source: §5 · p. 8

Abstract

from arXiv · show

Generalization in large language models (LLMs) is the ability to produce consistent and semantically stable outputs when the same input is expressed in different ways. Existing work typically evaluates generalization through aggregate accuracy on a single prompt format, task, or set of variations, which conflates robustness with overall benchmark performance. In this work, we show generalization evaluation at the level of individual examples, across multiple input variants, and across different aspects of model behavior, focusing on variability rather than reducing performance to a score that can be improved through narrow training or other ways that obfuscate generalization evaluation. Following this view, we introduce the Stability-Aware Generalization Objective (SAGO), a framework that measures how much model behavior changes for the same input under different variations and benchmarks, capturing variability across several dimensions including generation consistency, internal activations, confidence, and response mirroring. We show that many commonly used models exhibit statistically significant and consistent generalization instability: no model generalizes uniformly, behavioral axes capture independent failure modes, and cross-dataset variation can reverse model rankings.

1 Introduction

The paper frames generalization as preserving behavior across semantically equivalent inputs and argues that aggregate performance can miss per-example instability. It introduces SAGO to evaluate that stability across datasets, variation families, and behavioral axes, and finds no model generalizes uniformly.

1 Introduction

  • Generalization means preserving performance across semantically equivalent input variations, although users express the same goal and context in different ways.Source: §1 · p. 1
  • Single-domain, single-variation, and aggregate-score evaluations can miss substantial behavioral differences across individual examples.Variant failures can be offset by successes, leaving similar overall performance despite changing behavior.Source: §1 · p. 2
  • SAGO evaluates generalization as per-instance stability across semantically equivalent variations, multiple datasets, and multiple variation families.It compares baseline behavior with behavior under controlled variations.Source: §1 · p. 2
  • The multi-axis metric measures changes in activation geometry, generation consistency, confidence and uncertainty, and response mirroring.Mirroring may reflect intended alignment or unintended instability.Source: §1 · p. 2
  • Across eleven LLMs and six datasets, no model generalizes uniformly; generation consistency is most sensitive, while content stability and response mirroring are independent dimensions.Source: §1 · p. 2

2 Related Work and Background

Prior work examines distribution shifts, prompt-level variation, and behavioral signals beyond accuracy, but these lines do not together assess per-instance, multi-axis stability across datasets and variations.

2 Related Work and Background

  • Distribution-shift evaluations summarize performance on held-out data but do not measure whether behavior flips for the same input expressed differently.Source: §2 · p. 3
  • Prompt-level studies cover semantic rewrites and surface changes, but often isolate one domain or variation family and use aggregate metrics that obscure per-example instability.Source: §2 · p. 3
  • Hidden-state geometry, generation similarity, confidence, and stylistic accommodation have been studied as separate behavioral signals.Combining them within one per-instance analysis had not been attempted.Source: §2 · p. 3
  • Even large benchmarks spanning datasets and variation families may use only one behavioral signal, leaving instability across axes and individual examples unresolved.Source: §2 · p. 3

3 Stability-Aware Generalization Objective (SAGO)

SAGO evaluates generalization as per-instance stability across meaning-preserving prompt variations, datasets, and behavioral axes.

3.1 Notation and Objective

  • SAGO tests whether each behavioral axis remains unchanged for every prompt under each meaning-preserving variation; signed deltas retain direction, while RMS captures magnitude.A violation indicates generalization failure on that axis, and its magnitude characterizes severity.Source: §3.1 · p. 4

3.2 Benchmark Construction

  • The benchmark combines controlled input variations across datasets, using social-register changes, surface noise, and structural rewrites that preserve meaning.The variation set also specifies subtype, strength, and position when applicable.Source: §3.2 · p. 4

3.3 Stability Generalization Score

  • Normalized axis deltas are aggregated by per-prompt RMS across variations, which registers instability even when every variation causes the same nonzero shift.Normalization places deltas on a common scale while retaining zero as the reference for no behavioral change.Source: §3.3 · p. 5
  • Dataset and overall SGS scores average per-prompt instability, with equal per-prompt weighting across datasets; lower scores indicate stronger generalization.SGS is zero only when the model satisfies the no-change condition for every prompt on that axis.Source: §3.3 · p. 5
  • The significance test compares mean per-prompt instability with a practical tolerance, using ε = 0.05 for primary results and also reporting ε = 0.01 and 0.10.Rejecting the null establishes instability above the chosen threshold.Source: §3.3 · p. 5

3.4 Metrics

  • Activation geometry tracks representational drift, while BLEU and BERTScore separate lexical changes from meaning-level drift that accuracy alone collapses.The activation measure uses last non-padding hidden states; BLEU and BERTScore compare variant and baseline responses.Source: §3.4 · p. 5
  • Confidence measures capture log-probability and entropy changes, while response mirroring tracks surface-property accommodation without presuming whether it is intended alignment or instability.Entropy is length-invariant; sequence log-probability is length-sensitive. Together, the four axes capture distinct sensitivities that aggregate metrics can hide.Source: §3.4 · pp. 5–6

4 Experiments

The experiments evaluate SAGO across instruction-tuned models, datasets, input variations, and behavioral axes. They find widespread instability, distinct axis-specific failure profiles, and dataset-dependent rankings.

4.1 Experimental Setting

  • The study evaluates eight open-source models from three families and three API-accessed closed-source models on six datasets, sampling N=16 prompts per dataset.The six datasets span factual QA and creative instruction-following; closed-source XAct and XProb are unavailable, and XEnt is available only for GPT-5.4.Source: §4.1 · p. 6
  • SAGO varies social register, surface noise, and structural form; it uses three positions for non-structural variants and retains variants above BERTScore > 0.85 when semantic preservation is not guaranteed by construction.Primary results use ε=0.05; mirroring is measured with a judge, heuristics, or response-length checks depending on variation type.Source: §4.1 · pp. 6–7

4.2 Experimental Results

  • Every evaluated model has significant generation-consistency and mirroring instability, and every open-source model exceeds ε=0.05 on at least one axis; no model generalizes uniformly.Figure 2a and Table 1 identify the pooled and per-dataset SGSX results, respectively.Source: §4.2 · p. 7
  • Generation consistency and confidence can diverge: Q3.5-9B has the open-source pool’s lowest ∆-BERT (0.080) and ∆-BLEU (0.520), but highest ∆-Prob (0.088).G4-E4B has the pool’s highest generation instability, with ∆-BERT 0.178 and ∆-BLEU 0.837, compared with G-2B at 0.097 and 0.084 and G-7B at 0.084 for ∆-BERT.Source: §4.2 · p. 8
  • For L-8B, ∆-Cos varies 3.7× and ∆-MR more than doubles across datasets, enough to reverse model rankings.Total SGSX pools all six datasets with equal per-prompt weight.Source: §4.2 · p. 8

5 Discussion

The discussion frames generalization as per-instance behavioral stability across relevant domains, variation families, and axes, rather than as aggregate accuracy or a single robustness score. It also specifies boundaries on the conclusions and directions for further study.

5 Discussion

  • Generalization is a structured stability profile: stability on one axis, dataset, or variation family does not establish stability elsewhere, and similar benchmark scores can conceal per-instance changes.The discussion recommends scoping claims to the evaluated axis, domain, and variation family and comparing profiles rather than rankings.Source: §5 · p. 8
  • SAGO measures where behavior changes under controlled semantically equivalent variation, but does not identify causal mechanisms; multilingual generalization remains untested because the datasets are English-only.The discussion also notes that ∆-BLEU may reflect harmless paraphrasing, N=16 may miss rare or worst-case failures, and GPT-4o-mini judgments and limited API access constrain some measurements.Source: §5 · p. 8
  • The discussion concludes that reliable generalization requires per-instance behavior to remain stable across the domains, variation families, and axes relevant to deployment; high benchmark performance alone is insufficient.It proposes multilingual and further task-family evaluation, causal-mechanism analysis, and testing whether training gains transfer across axes and datasets.Source: §5 · p. 8

A Variation Strength Examples

  • Tables 2–4 illustrate variation families at evaluated strength levels using a TruthfulQA prompt, with all examples applied globally.The example prompt asks who is the richest person in the world who did not finish high school.Source: A Variation Strength Examples · p. 13

B Tolerance Threshold Robustness

The appendix tests robustness across three tolerance thresholds and reports that significance patterns differ by behavioral axis. Its primary conclusions hold at ε=0.01 and ε=0.05, while ε=0.10 serves as a stress test.

B Tolerance Threshold Robustness

  • ∆-BLEU remains significant at p < .001 for every model with the axis available at all three thresholds, while ∆-BERT significance at ε = 0.10 remains for only 4/2/2 models at p < .05 / .01 / .001.The appendix reports per-model total SGS values and aggregate rejection counts by axis across ε ∈ {0.01, 0.05, 0.10}.Source: B Tolerance Threshold Robustness · p. 13
  • The activation axis is robust at ε ≤ 0.05, while mirroring remains significant across all three thresholds.For ∆-Cos, the reported rejection counts are 8/8/8 and 7/7/7 at ε = 0.01 and 0.05; for ∆-MR they are 9/9/9, 8/8/8, and 8/8/8 across the thresholds.Source: B Tolerance Threshold Robustness · p. 13
  • At ε = 0.10, ∆-Prob loses all significance and ∆-Ent is weak even at ε = 0.01; the main qualitative conclusions hold at ε = 0.01 and ε = 0.05.The appendix treats ε = 0.10 as a stress test, not a recommended operating point.Source: B Tolerance Threshold Robustness · p. 14

C Full Results

The appendix provides full per-dataset SGS results across axes for the open-source models and reports the closed-source entropy results available for GPT-5.4. These results are described as consistent with the patterns in Section 4.2.

C Full Results

  • Table 7 covers eight open-source models across six datasets and all six axes; the full per-dataset results are consistent with Section 4.2.The appendix identifies Table 6 as the corresponding closed-source confidence-axis result for GPT-5.4.Source: C Full Results · p. 14
  • Table 6 reports per-dataset SGS for GPT-5.4 on ∆-Ent at ε = 0.05, the only closed-source entropy result because GPT-5.4 alone returns the required per-token probabilities.Significance markers test H1: SGS > ε.Source: Table 6 · p. 14

D Prompt Sample Size Ablation

The prompt-sample ablation identifies the smallest sample size that yields stable per-prompt instability estimates. Five of six axes plateau by N=16, which the authors adopt as the operating point.

D Prompt Sample Size Ablation

  • N=16 is sufficient for stable SGSX estimation: five of six axes plateau by this sample size and remain flat through N=128.The authors select the smallest N whose bootstrap band stabilizes, using the most conservative value across models and axes.Source: D Prompt Sample Size Ablation · p. 15
  • Each subsample size is evaluated by drawing prompts without replacement from TruthfulQA and repeating each draw 500 times.For each draw, the authors track the bootstrap distribution of standard deviations of per-group means across sampled prompts.Source: D Prompt Sample Size Ablation · p. 15
  • Data collection stops at N=128 after stability is confirmed, while N=16 is retained as the operating point.Source: D Prompt Sample Size Ablation · p. 15

E Sensitivity Analysis

The sensitivity analysis tests how stylistic variation choices affect measured behavior. It treats placement positions and evaluated strength levels as distinct effects that should remain in the variation set.

E Sensitivity Analysis

  • All three positions—PREFIX, SUFFIX, and GLOBAL—and all evaluated strength levels induce meaningfully distinct behavioral effects on TruthfulQA.The authors conclude that both dimensions must be retained in the variation set for a complete picture of stylistic generalization.Source: E Sensitivity Analysis · p. 15
  • Structural rewriting variants include prompt compression or expansion by a multiplier and conversion between interrogative and imperative sentence forms.Source: Table 4 · p. 16
  • The length-variation examples range from a terse query to an expanded version asking the same question.Source: E Sensitivity Analysis · p. 16

E.1 Placement Ablation

Placement position changes measured instability profiles across behavioral axes and models. No single position dominates across them, so collapsing positions would lose behavioral signal.

E.1 Placement Ablation

  • Global variants produce the largest effects on generation consistency and mirroring across the reported axes and models.Source: §E.1 · p. 16
  • Prefix and suffix placements show distinct sensitivity profiles on confidence and activation axes.Source: §E.1 · p. 16
  • No placement dominates across all axes and models, so all three positions must be included in the variation set.Source: §E.1 · p. 16

E.2 Strength Ablations

Strength levels produce distinct, model- and axis-dependent behavioral effects rather than acting as a uniform scaling factor. The reported subtypes also differ in their response patterns.

E.2 Strength Ablations

  • Different strength levels produce meaningfully different values, with effects that are not monotone and do not saturate uniformly across models and axes.The authors conclude that all evaluated strength levels are needed to capture the full range of stylistic generalization failures.Source: §E.2 · pp. 16–17
  • Spacing effects grow with strength and have model-dependent saturation points, while generation consistency axes respond most at moderate strengths.Source: Figure 5 · p. 18
  • Case-variation detectability grows across most axes and plateaus above strength 50, with activation geometry showing the largest and most consistent response.Source: Figure 6 · p. 18
  • Punctuation strength effects are clearest on generation consistency and activation axes, while confidence axes remain largely flat.Source: Figure 7 · p. 20
  • Rude politeness variants produce larger, more consistent behavioral shifts than polite variants across most models and axes.Source: Figure 8 · p. 20
  • Compression produces larger, more model-consistent shifts than expansion on generation consistency and mirroring, while confidence responds more strongly to expansion.Source: Figure 9 · p. 20

Related papers

Loading 2610.01428v1…