Source-linked AI summary
Measuring Stability and Failure Behavior in Language Models Under Structured Perturbations
Samira Golsefid
TL;DR
Language-model accuracy alone does not show how reasoning degrades under perturbation or whether models respond appropriately when questions are unanswerable. The paper introduces a validity-gated, graded framework spanning seven stress families and evaluates four models on 4,473 tests. It finds family-specific collapse, with conflicting instructions and impossible-premise questions exposing consistent weaknesses across models.
Problem
Single accuracy scores provide limited evidence about degradation under perturbation and appropriate failure on unanswerable reasoning questions.
Method
The framework applies a measured-severity ladder across seven perturbation families, validity-gates tests, and reports Accuracy, magnitude-weighted Stability, and baseline-relative Collapse Point.
Results
Failures are family-specific across four models, with all models reaching Collapse Point under conflicting instructions and performing poorly on impossible-premise Knowledge Boundary questions.
Takeaways & Limitations
Graded, multi-family evaluation reveals deployment-relevant weaknesses that aggregate accuracy reporting leaves invisible.
Takeaways & Limitations
The study covers one grade-school-math task family and 100 seed problems, while LLM-based gating and judging may introduce evaluator error or systematic bias.
Abstract
from arXiv · showhide
Language models are usually judged by a single accuracy score, which does not reveal how their performance degrades as inputs are perturbed. We present a graded, multi-family, failure-aware framework for stress-testing reasoning models. It perturbs each problem along a multi-level severity ladder across seven families: six that preserve the answer, paraphrase, input noise, formatting, irrelevant context, context load, and conflicting instructions, and a Knowledge Boundary family that removes answerability so that refusal becomes the correct response. Every test is validity-gated and labeled by its measured severity, and each model is summarized by per-level Accuracy, a magnitude-weighted Stability, and a per-family Collapse Point defined relative to the model's own baseline. Instantiated on the same 100 seed problems used by GSM-Symbolic, expanded into 4,473 gated tests and run on four models spanning capability tiers, the framework exposes structure that an aggregate score hides: the level at which a model fails is family-specific rather than global, and two stressors expose consistent weaknesses across all models: conflicting instructions and questions built on an impossible premise. Recognition of unanswerability is otherwise uneven, reliable on missing information and fabricated evidence but weak on impossible premises. These failure points are invisible to standard accuracy reporting.
1 Introduction
The paper introduces a graded, failure-aware framework that evaluates reasoning models across diverse perturbation families rather than relying on aggregate accuracy. Across four models, failures are family-specific, with consistent weaknesses under conflicting instructions and impossible-premise questions.
- The framework measures reasoning-model degradation across seven perturbation families using Accuracy, magnitude-weighted Stability, and baseline-relative Collapse Point.Tests are validity-gated and labeled by measured rather than requested severity.
- It combines six answer-preserving stress families with Knowledge Boundary tests where unanswerability makes refusal correct.The benchmark admits meaning-preserving or genuinely unanswerable tests.
- The evaluation expands 100 GSM-Symbolic seed problems into 4,473 gated tests across four models spanning capability tiers and deployment settings.
- Every model degrades under perturbation, but collapse levels differ by family rather than forming a single global threshold.For example, gpt-4o-mini fails on context-related families that gemini-2.5-flash handles cleanly.
- All models reach Collapse Point under conflicting instructions and perform poorly on impossible-premise Knowledge Boundary questions.They compute answers instead of flagging the impossibility, even though stronger models reliably refuse missing-information and fabricated-evidence questions.
2 Related Work
Prior evaluation work studies accuracy, robustness, and failure behavior largely in separate settings. Existing perturbation benchmarks typically use isolated stressors, single intensities, or scalar accuracy drops, motivating a unified graded evaluation.
- Behavioral-testing and multi-metric frameworks extend evaluation beyond accuracy but do not measure degradation under intensifying stress for reasoning models.CheckList emphasizes invariance tests, while HELM reports seven metrics across sixteen scenarios.
- Meaning-preserving changes can substantially alter reported accuracy, including format changes that move accuracy by up to 76 points on a single task.This motivates reporting ranges rather than single accuracy points.
- Existing math benchmarks apply one or a few perturbation types, such as GSM-Symbolic’s clause changes, GSM-Plus’s eight perturbation types, and GSM-IC’s irrelevant sentences.GSM-Symbolic reports that a seemingly relevant clause can cut accuracy by up to 65%.
- Most prior perturbation studies use isolated or single-intensity tests and reduce outcomes to accuracy drops rather than degradation curves or failure characterizations.
- Research on abstention, false premises, calibration, truthful answers, sycophancy, and conflicting instructions generally examines failure behavior apart from robustness.The paper connects these literatures by measuring refusal on the same severity ladder as other stressors.
3 Framework
The framework evaluates reasoning models across seven graded stress families, using validity gates and achieved severity to distinguish meaning-preserving robustness from appropriate refusal. It summarizes behavior with Accuracy, magnitude-weighted Stability, and baseline-relative Collapse Point metrics.
- Perturbation families: Seven perturbation families apply increasing stress to surface form, surrounding context, task instructions, or answerability itself.The six meaning-preserving families alter inputs while preserving the answer; Knowledge Boundary instead makes questions genuinely unanswerable.
- Severity assignment: Severity is assigned from measured perturbation rather than requested generator level, so degradation curves reflect achieved stress.A candidate generated at L3 but measured as L2 is recorded at L2.
- Validity gating: Every candidate must pass family-specific validity gates before admission, including meaning-preservation checks for most answer-preserving families and epistemic validation for Knowledge Boundary.Structural/Format relies on structural checks because its edits leave words and numbers unchanged; Knowledge Boundary tests must be genuinely unanswerable.
- Per-test scoring: Output correctness is exact numeric agreement within 0.1% relative tolerance for answer-preserving families, but Knowledge Boundary uses a four-point behavioral refusal rubric.A perfect Knowledge Boundary score rejects or flags the unanswerable question, while a zero score is a confident answer treating the impossible premise as legitimate.
- Aggregate metrics: Accuracy averages correctness equally, whereas Stability weights correctness by perturbation magnitude and therefore emphasizes harder tests.Success on low-magnitude but failure on high-magnitude perturbations yields Accuracy > Stability.
- Aggregate metrics: Collapse Point is the lowest severity whose accuracy falls below τ = 0.80 × acc(L0), making failure thresholds relative to each model’s baseline.Collapse Point is undefined when accuracy never falls below the threshold.
4 Experiment
The experiment evaluates four language models on fixed grade-school math problems subjected to validity-gated, graded perturbations across seven families. The resulting suite contains 4,473 admitted tests from 5,673 candidates, with responses scored using family-specific procedures and baseline-anchored metrics.
- Dataset: 100 GSM-Symbolic seed problems provide verified numerical answers, baseline items, and sources for all perturbed variants.Fixing the seeds supports comparison with prior work while bounding compute cost.
- Test generation and gating: 5,673 generated candidates yielded 4,473 admitted tests after structural, meaning-preservation, and other validity gates.The admission rate was 78.9%, with 1,200 candidates rejected.
- Test generation and gating: Each admitted test is labeled by measured rather than requested severity, separating intended perturbation intensity from obtained intensity.This labeling is applied after the validity-gated generation pipeline.
- Systems and protocol: Four systems—o4-mini, gemini-2.5-flash, gpt-4o-mini, and phi4-mini—span reasoning-oriented, general-purpose, and open-weight capability tiers.The models are evaluated under identical test content and include both API and local deployment settings.
- Scoring and metrics: The evaluation combines 4,473 perturbed tests with 300 baseline evaluations and computes per-family Accuracy, Magnitude, Stability, and Collapse Point.Collapse Point is anchored to each model’s own baseline, while Knowledge Boundary responses use a four-point epistemic rubric.
5 Results
Across four models, accuracy declines with measured severity, but the location and type of failure vary by perturbation family. Conflict Instruction Stress and Knowledge Boundary are shared weak points, while unanswerability recognition is strongest for missing information and fabricated evidence and weakest for impossible premises.
- Accuracy across severity levels: Accuracy falls from L0 to L5 for every model: 0.98 to 0.65 for o4-mini, 0.94 to 0.46 for gemini-2.5-flash, 0.89 to 0.41 for gpt-4o-mini, and 0.72 to 0.41 for phi4-mini.Degradation is gentle through L1–L4 and concentrated at L5, which carries impossible-premise Knowledge Boundary tests.
- Stability by family: Meaning-preserving surface families remain robust for stronger models, with Stability ≥0.96 for o4-mini and ≥0.91 for gemini-2.5-flash.Stability is magnitude-weighted, rewarding correctness under stronger perturbations.
- Collapse points: Every model collapses under Conflict Instruction Stress and Knowledge Boundary, but Collapse Point levels differ by model and family.Collapse Point is the first level below 80% of that model’s baseline; “—” indicates no collapse through L5.
- Knowledge Boundary behavior: Unanswerability recognition ranges from 0.92–0.95 for missing information and 0.78–0.90 for fabricated evidence, but falls to 0.07–0.08 for three API models on impossible premises.Models often carry impossible-premise arithmetic through and return a number instead of flagging impossibility.
- Behavioral patterns: Figure 2 places meaning-preserving families in stable or robust quadrants, while Knowledge Boundary occupies the high-magnitude, below-threshold failure region.The plot uses mean perturbation magnitude on x and output stability on y, with a stability threshold of 0.80.
6 Discussion and Conclusion
The study argues that graded, multi-family evaluation reveals failure structure hidden by aggregate accuracy, while noting important scope and evaluator limitations. It concludes that the framework is reusable and should be extended across domains, datasets, models, and severity levels.
- Discussion and Conclusion: Aggregate accuracy hides family-specific collapse profiles and recurring weaknesses under conflicting instructions and impossible-premise questions.These behaviors matter for deployment but remain invisible to standard accuracy reporting.
- Limitations: The study is limited to grade-school math and 100 seed problems, so broader domains and larger seed sets are needed to test generality.The authors identify task-family and seed-set scope as limits on generalization.
- Limitations: LLM-based entailment, validity gating, and response judging may introduce evaluator error or systematic judge bias.Future work should validate samples against human annotations and assess inter-judge agreement.
- Discussion and Conclusion: The framework is designed to remain informative as models improve because its diagnostic structure outlasts any single model.Its degradation curves, magnitude-weighted stability, per-family Collapse Points, and behavioral quadrants provide the diagnostic structure.
- Future Work: Future extensions include more task domains, larger seed sets, uniform multi-model protocols, and finer severity ladders.The proposed extensions add surface-family bands and additional Knowledge Boundary types to resolve failure locations more precisely.
Appendix A. Prompt templates
The appendix specifies distinct answer formats for the evaluated models and a Knowledge Boundary instruction that changes behavior when a problem is unanswerable.
- Model Prompt Templates: o4-mini, gemini-2.5-flash, and gpt-4o-mini use a JSON-only response format containing the final numerical answer.The template requires the exact object format {"ground_truth": <number>} with no units, dollar signs, or commas.
- Model Prompt Templates: phi4-mini receives a template requiring step-by-step reasoning followed by a standalone line formatted as #### <number>.The final response must contain a plain number without units, currency symbols, commas, or words.
- Knowledge Boundary: Knowledge Boundary prompts instruct all models to refuse questions with false premises, impossible scenarios, fabricated entities, or insufficient information.Answerable problems instead require a final plain-number line formatted as #### <number>.