PaperlayerView exact revision on arXiv

Source-linked AI summary

To Reason but Not to Stop: Understanding and Recovering Pruned Large Reasoning Models

Minh-Anh Nguyen, Nguyen T. Ha Lan, Xiyan Fu, Dung D. Le, Wei Lu

arXiv:2610.19934v1cs.CL

TL;DR

Pruning can reduce large reasoning models’ accuracy, but whether this reflects damaged reasoning or another failure remains unclear. The paper diagnoses a major role for non-termination and proposes TGAS, a training-free steering method. Across models, pruning pipelines, and math and code benchmarks, TGAS improves termination and accuracy while reducing repetitive reasoning.

  • Problem

    Whether pruning-related accuracy degradation reflects lost reasoning ability or another failure remains unclear.

    Source: §1 · p. 1

  • Method

    TGAS adds a shared hidden-state direction associated with terminating rather than looping continuations to the residual stream at inference, without retraining or weight updates.

    Source: §1 · p. 2

  • Results

    TGAS consistently improves termination and accuracy across six LRMs, three pruning pipelines, and math and code benchmarks while reducing repetitive reasoning.

    Source: §1 · p. 2

  • Takeaways & Limitations

    At higher sparsity, some capability is genuinely lost, and TGAS recovers only part of the gap to the dense model.

    Source: §6 · p. 8

    The evaluation covers unstructured pruning, distilled or SFT-trained models up to 14B parameters, and math and code; transfer to RL-trained or larger models and open-ended tasks remains open.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Weight pruning can degrade the accuracy of large reasoning models (LRMs), yet it remains unclear whether it damages reasoning itself. We find that much of this degradation stems from a failure to stop rather than to reason. As sparsity increases, pruned LRMs increasingly fail to terminate within the token budget, often after the correct answer has appeared in their reasoning, instead falling into repetitive loops, even though they still recognize where the dense model would stop. We further find that the hidden-state difference between terminating and looping continuations is consistent across pruned checkpoints, suggesting a shared direction associated with committing to an answer. Building on this, we propose Termination-Guided Activation Steering (TGAS), a training-free method that adds this direction to the residual stream at inference time. Across six LRMs, three pruning pipelines, and math and code benchmarks, TGAS improves termination and overall accuracy while reducing repetitive reasoning and generation length, showing that this failure is actionable.

1 Introduction

Pruning-related accuracy loss may reflect difficulty stopping rather than loss of reasoning, but the distinction was unclear. The authors identify looping as a major failure mode and introduce TGAS to steer pruned models toward termination.

1 Introduction

  • Across pruning settings, AIME-25 accuracy decreases as non-termination becomes more frequent.Figure 1 uses SparseGPT with OBC calibration, a 16,384-token budget, and averages over four runs.Source: Figure 1 · p. 1, §1 · p. 1
  • Prior work addresses pruning-related accuracy loss through improved calibration and post-pruning retraining or fine-tuning, while whether pruning damages reasoning itself remains unclear.LRMs’ long chain-of-thought traces also make inference expensive, motivating pruning as a cost-reduction approach.Source: §1 · p. 1
  • Much pruning-related degradation stems from failure to stop rather than failure to reason, and non-termination increasingly dominates errors as sparsity grows.Many non-terminating traces already contain the correct answer; on problems solved by the dense model, pruned models remain largely accurate when they terminate.Source: §1 · p. 1
  • Pruned models retain the dense model’s stopping signal under replay but revisit earlier reasoning during free-running generation, producing repetitive loops.Many looping traces have already reached the correct answer.Source: §1 · p. 1
  • A consistent hidden-state contrast between terminating and looping continuations motivates TGAS, which adds that direction to the residual stream at inference without retraining or weight updates.Across six LRMs, three pruning pipelines, and math and code benchmarks, TGAS improves termination and accuracy while reducing repetitive reasoning.Source: §1 · p. 2

2 Background and Related Work

The background describes retraining-free unstructured pruning methods and calibration approaches for reasoning traces. The related-work discussion distinguishes TGAS’s target—pruning-induced non-termination—from methods addressing other reasoning behaviors.

Pruning for LRMs

  • SparseGPT compensates for pruning errors using second-order information, while Wanda scores weights by magnitude and input-activation norm.Both are retraining-free unstructured pruning methods; recent LRM methods also adapt calibration using on-policy chain-of-thought, reasoning-step importance, or self-generated traces.Source: §2 · p. 2

Related Works

  • Prior work describes repetitive loops in LRMs and reports that quantization can prolong them; this paper distinguishes pruning-amplified looping from failures to reason.The paper reports that pruning amplifies looping while largely preserving the stopping signal.Source: §2 · p. 2
  • TGAS targets pruning-induced non-termination using contrasting continuations of the same prefix, unlike SEAL’s reflection suppression or Plan-CoT’s reasoning-length modulation.Its direction is reported as weakly aligned with SEAL’s.Source: §2 · p. 2

3 Understanding Pruning-Induced Reasoning Degradation

The section finds that pruning-related accuracy loss is largely associated with non-termination, even when traces reach correct answers; under moderate pruning, stopping recognition remains largely intact while repetitive reasoning becomes more likely.

3.1 Do Pruned Models Lose the Ability to Reason?

  • From 50% to 60% sparsity, MATH-500 non-termination rises from 3.8% to 16.8% for DS-Qwen-7B and from 2.4% to 16.4% for DS-Qwen-14B, while terminated-but-wrong rates stay nearly unchanged.Accuracy among terminated generations remains high as sparsity grows.Source: Table 1 · p. 3, §3.1 · p. 3
  • Increasing the budget from 16,384 to 65,536 tokens cuts non-termination from 42.5% to 12.5% for 7B at 50% sparsity, but leaves accuracy nearly unchanged and barely reduces non-termination at 60–70%.The larger budget recovers only 0–3 additional correct traces across settings.Source: §3.1 · p. 3, Figure 3 · p. 3
  • At 60% sparsity, 41.4% and 39.2% of non-terminating DS-Qwen 7B and 14B traces contain the correct answer, appearing at median positions of 13.0% and 11.9% of trace length.These answers appear roughly 2.1k and 1.9k tokens into the traces, respectively, before models continue to the token limit.Source: §3.1 · p. 3, Figure 2 · p. 3

3.2 Why Do Pruned Models Fail to Terminate?

  • At 50–60% sparsity, teacher-forced </think> probability exceeds 0.92 for 7B and 0.85 for 14B, with median rank 0; at 70%, its probability weakens substantially.The signal remains more likely at the dense model’s STOP position than more than 50 tokens earlier, but at 70% its log-probability falls below ordinarily generated tokens.Source: §3.2 · p. 4, Table 2 · p. 4
  • At decision points, pruning shifts preference toward cyclic continuations: DS-Qwen-7B’s Δcyc rises from −15.3 dense to −5.0 at 70% sparsity, while OT3-7B’s rises from −23.1 to −5.6.Predictive entropy changes little for DS-Qwen-7B but increases for OT3-7B, whereas Δcyc shifts consistently.Source: §3.2 · pp. 4–5, Table 3 · p. 4
  • At 70% sparsity, DS-Qwen 7B and 14B reach loop-persistence probabilities of 0.307 and 0.324, and loop-entry probabilities of 0.085 and 0.081, respectively, versus at most 0.029 for dense models.The analysis labels paragraphs CYCLIC when their 8-word n-gram set has Jaccard similarity ≥0.5 with an earlier paragraph, and PROGRESS otherwise.Source: §3.2 · pp. 4–5, Table 4 · p. 5

4 Methodology

The method derives a hidden-state direction contrasting terminating and looping continuations, checks whether it is consistent across pruned checkpoints, and uses it to steer generation toward termination.

4 Methodology

  • TGAS uses a hidden-state direction separating terminating from looping continuations under the same prefix to steer pruned models toward commitment without weight updates.Source: §4 · p. 5

4.1 Constructing the Termination Direction

  • For each eligible branch point, the method contrasts terminating and non-terminating sampled continuations, averaging their hidden-state differences across branch points to isolate commitment versus looping.The classes share the same problem and prefix, so problem-specific content cancels; hidden-state averages use a common endpoint to exclude looping tails.Source: §4.1 · p. 5

4.2 Direction Consistency Across Checkpoints

  • Directions from different pruning levels are highly consistent: cosine similarity is at least 0.87 for DS-Qwen-7B at 50–70% pruning and at least 0.83 for DS-Qwen-1.5B at 40–60%.This supports reusing one direction per model across sparsity levels.Source: §4.2 · p. 6, Figure 5 · p. 6

4.3 Steering at Inference

  • TGAS constructs a direction at intermediate sparsity, selects the layer with the highest held-out ROC AUC, rescales the direction, and adds it to every token’s residual stream during generation.The direction is reused across sparsity levels of the same model; positive α steers toward commitment and negative α in the opposite direction.Source: §4.3 · p. 6

5 Experiments

Across training-free comparisons, TGAS improves accuracy and termination, reduces repetitive reconsideration and generation length, and transfers across pruning methods when capability remains. Training-based recovery helps more at higher sparsity, while ablations expose trade-offs in steering and baseline settings.

Targeting Termination

  • TGAS beats standard inference by 1.3–3.7 average accuracy points at every model-sparsity setting, raises termination in 16 of 18 pairs, and leads training-free average accuracy in all six settings.Source: §5 · p. 6, Table 5 · p. 7

Training-Free Baselines

  • Forced-Answer is competitive when its budget matches problem difficulty, No-Repeat often harms accuracy especially on code, and SEAL can raise termination at an accuracy cost.Source: §5 · p. 7

Training-Based Recovery

  • Masked-LoRA raises LCB accuracy from 14.7% to 33.4% for DS-Qwen-7B and from 37.6% to 51.8% for 14B at 60% sparsity, consistent with recovering capability lost to aggressive pruning.Unlike TGAS, it requires a separate adapter and optimization procedure for each model and sparsity level.Source: §5 · pp. 7–8, Figure 6 · p. 8

Generalization Across Pruning Methods

  • TGAS outperforms the pruned baseline in all eight RAC and Wanda settings and achieves the best accuracy in seven, but all training-free methods remain far below dense performance when capability is scarce.Transfer holds across pruning algorithms and calibration sets provided the pruned model retains sufficient capability.Source: §5 · p. 7, Table 6 · p. 8

Diagnosed Failure

  • At 60% sparsity, TGAS reduces decision points from 12.3 to 5.8 per 1k tokens for 7B and 11.5 to 6.8 for 14B, while cutting generation length 12.3%.Across all sparsity levels, generation length falls 7.6% on average.Source: §5 · p. 7, Table 7 · p. 8, Table 8 · p. 8

Ablations

  • Increasing α improves TGAS termination, especially on LCB, while shorter Forced-Answer budgets and stricter No-Repeat blocking hurt accuracy; 50 samples per class suffice for robustness tests.The ablations also find that directions from different checkpoints of the same model remain effective.Source: §5 · p. 8, Figure 6 · p. 8

6 Conclusion

The conclusion distinguishes recoverable termination failures under moderate pruning from genuine capability loss at higher sparsity, and presents TGAS as a post-pruning intervention.

6 Conclusion

  • Under moderate pruning, a substantial and recoverable share of accuracy loss stems from disrupted termination rather than incorrect reasoning.Source: §6 · p. 8
  • As sparsity increases, pruned models may reach the correct answer and recognize where to stop under teacher forcing, yet loop and fail to commit within the token budget.Source: §6 · p. 8
  • At higher sparsity, some capability is genuinely lost and can be recovered only by updating weights; TGAS instead steers toward termination without retraining.Source: §6 · p. 8

Limitations

The work is limited to post-pruning recovery, with its diagnosis and intervention evaluated within specified model, task, sparsity, and token-budget boundaries.

Limitations

  • The interventions recover degraded behavior after pruning; they do not change the pruning process or prevent degradation during pruning.Source: Limitations · p. 9
  • Evaluation covers unstructured sparsity, distilled or SFT-trained LRMs up to 14B parameters, and mathematics and code; generalization to RL-trained models, larger models, or open-ended tasks remains open.Source: Limitations · p. 9
  • Analysis uses a 16,384-token budget and requires one round of continuation sampling per pruned model; at high sparsity, TGAS recovers only part of the gap to dense performance.Source: Limitations · p. 9

Ethical Considerations

The work does not involve new human-subject data collection, but its interventions can change whether models terminate or continue reasoning.

Ethical Considerations

  • Because the methods alter generation behavior, including termination, evaluate task-specific reliability before deployment.The work reports no new data collection involving human subjects.Source: Ethical Considerations · p. 9

A.1 Pruning Methods for Large Language Models

Prior work compresses language and reasoning models with varied pruning strategies, while this paper’s experimental setup examines pruning-related answer presence in reasoning traces.

A.1 Pruning Methods for Large Language Models

  • LLM pruning includes unstructured methods such as SparseGPT and Wanda and structured methods such as LLM-Pruner, ShortGPT, and Sheared-LLaMA.These methods differ in what they remove and whether they use fine-tuning or continued pre-training for recovery.Source: §A.1 · pp. 11–12

A.2 Pruning Methods for Large Reasoning Models

  • Structured approaches remove coupled structures or layers, while calibration-data studies report that calibration choice affects pruning quality, especially at high sparsity.Related LRM work adapts calibration to generated reasoning trajectories and studies compression effects on reasoning.Source: §A.1 · p. 12, §A.2 · p. 12

B Preliminaries

  • One-shot pruning commonly reconstructs layer outputs on calibration inputs; SparseGPT uses second-order information, while Wanda scores weights by magnitude and activation norm.RAC adds generated reasoning activations to calibration, and OBC weights tokens by reasoning-step causal importance.Source: B Preliminaries · pp. 12–13

C. Detailed Experimental Setup

  • The protocol tests three LRM families with 10–70% unstructured pruning of decoder linear layers, using SparseGPT or Wanda with OBC, RAC, or C4 calibration.Embeddings and the language-model head remain dense.Source: §C.1 · p. 13

C.1 Generation Setup

  • Evaluation covers held-out MATH-500, AIME-25, and LiveCodeBench problems, spanning mathematical reasoning, competition problems, and code generation.LiveCodeBench problems post-date model training and test cross-domain transfer from math-derived steering trajectories.Source: §C.1 · p. 13

C.2 Measuring Answer Presence in Reasoning Divergence

  • At 60% sparsity, 41.4% of Qwen-7B and 39.2% of Qwen-14B non-terminating traces contain the correct answer under the Conclusive criterion.Conclusive is the primary answer-presence measure because it captures a reached correct result that was not transitioned into a final answer.Source: §C.2 · p. 14

C.3 Independent Validation with an LLM Judge

The appendix independently checks answer-presence measurements and describes recovery, steering, and diagnostic procedures for pruned reasoning models. It reports that non-termination and cyclic continuation increase with pruning, while stopping signals can remain detectable at the dense model’s stopping position.

C.3 Independent Validation with an LLM Judge

  • On the same random sample, Conclusive achieves 96.0% agreement with the independent judge (κ = 0.898), while Explicit achieves 84.8% agreement (κ = 0.782), supporting Conclusive as the primary measure.Source: §C.3 · pp. 14–15

C.4.1 Masked LoRA Recovery

  • Masked-LoRA updates only surviving weights, but its results mix recovery with distillation from verified dense-model traces and serve as an upper-bound reference only at 60%–70% sparsity.The method trains on fixed correct traces generated by dense DeepSeek-R1-Distill-Qwen 14B.Source: §C.4.1 · pp. 15–16

C.4.2 Forced-Termination Intervention.

  • Forced termination tests whether an existing reasoning trace suffices for a final answer; no-repeat decoding instead bans tokens that would complete an n-gram already in context.The forced-termination procedure appends </think> after a token budget; the no-repeat default is n = 32.Source: §C.4.2 · p. 16, §C.4.3 · p. 16

C.4.4 SEAL: Steerable Reasoning Calibration of LRMs

  • SEAL steers toward execution thoughts using a contrast with reflection and transition thoughts, applying its activation shift at thought boundaries.Its thought categories are assigned by keyword rules, and its direction is built from category-specific hidden-state averages.Source: §C.4.4 · p. 16

C.4.5 Termination-guided Activation Steering

  • Directions from different pruned checkpoints can remain highly similar; TGAS constructs one vector per model and evaluates it across pruning conditions and separate math and code benchmarks.The calibration problems are disjoint from evaluation sets, and the selected layer is assessed on held-out branch points.Source: Figure 9 · p. 17, §C.4.5 · p. 17

D.1 Answer Retention and Termination in Pruned Models on Dense-Model-Correct Samples

  • On AIME-25, Qwen-7B non-termination rose from 9% at 10% sparsity to 73% at 70%; a larger token budget recovered no traces at 70% sparsity.Non-termination exceeded terminated-wrong rates in 10 of 12 model-benchmark pairs at 60% sparsity.Source: §D.1 · p. 20, §D.2 · p. 20

D.4 Comprehensive Evaluation of Transition of Pruned LRMs

  • Teacher-forced preferences shift toward cyclic continuations as sparsity rises, and free-running loop entry reaches roughly 3–6× dense rates for DS-Qwen at 70% sparsity.Loop persistence also rises, though it peaks at 60% in three of four DS-Qwen blocks.Source: §D.4 · pp. 21–22

D.5 Comprehensive Commitment Directions Analysis

Termination directions remain aligned across pruning checkpoints and transfer across sparsity levels, while TGAS outperforms or matches training-free alternatives broadly; training-based recovery is stronger under severe pruning but requires optimization and separate adapters.

D.5 Comprehensive Commitment Directions Analysis

  • Termination-direction alignment exceeds label-shuffled similarity in all 15 checkpoint pairs, and branch-point identity controls align 0.143–0.52 less strongly in all 15 pairs.These controls indicate the alignment reflects the termination contrast rather than shared weights or branch-point identity alone.Source: §D.5 · pp. 23–24
  • Transferred directions on DS-Qwen reach 0.916–0.991 ROC AUC, exceeding the 95th percentile of shuffled-label controls in 29 of 30 pairs.Together with the cosine results, transfer supports reusing one termination direction across sparsity levels of a model.Source: §D.5 · pp. 24–25

E.1 Comprehensive Evaluation of Training-free Methods

  • TGAS achieves the best or tied-best accuracy on 53 of 72 checkpoint-benchmark pairs, versus 10 for Forced-Answer, and improves termination over Base on 65.Its mean accuracy gain rises from 1.6 points on dense models to 3.7 points at 60% sparsity; TGAS also beats SEAL on 62 of 72 pairs, while No-Repeat reduces accuracy on 41 of 72.Source: §E.1 · p. 27

E.2 Comparison of Training-Free and Training-Based Methods

  • At 60% sparsity, Masked-LoRA achieves stronger recovery than training-free methods, but requires optimization and a separate LoRA adapter for each sparsity level.At 50% sparsity, training-free methods match or outperform Masked-LoRA without additional training.Source: §E.2 · p. 27

E.3 Headroom and Recovery Analysis

TGAS captures much of the accuracy headroom available from committing to answers already stated in non-terminating traces, while primarily reducing repeated reconsideration. The alternatives reveal limits: forced answering depends on problem difficulty, and No-Repeat can disrupt useful patterns without reliably restoring accuracy.

E.3 Headroom and Recovery Analysis

  • At 60% sparsity, TGAS captures 90% of oracle headroom for DS-Qwen-1.5B and 70% for DS-Qwen-7B, and exceeds the oracle for DS-Qwen-14B.At 70% sparsity, it captures 82% and 37% for DS-Qwen-7B and 14B, respectively.Source: §E.3 · p. 28, Table 22 · p. 28
  • At 60% sparsity, TGAS turns 25.8%, 36.1%, and 44.2% of non-terminating generations into correct ones for DS-Qwen-1.5B, 7B, and 14B, respectively.It does not improve terminated-wrong generations over a rerun baseline, supporting the interpretation that it restores termination rather than correcting wrong answers.Source: §E.3 · p. 28

F.1 Behavioral Changes in Pruned Models After Steering

  • TGAS makes ∆cyc more negative in all 12 settings and reduces decision points by 24%–56%, with the largest reductions at 60% sparsity.For 7B and 14B, reconsideration frequency returns to approximately dense-model levels.Source: §F.1 · p. 32, Table 26 · p. 32

F.2 Ablation Study of Forced-Answer

  • Forced-Answer has no single effective budget across models and sparsity levels, and choosing one requires estimating each problem’s difficulty.A budget suited to difficult problems may unnecessarily truncate easier ones.Source: §F.2 · p. 32

F.3 Ablation Study of No-Repeat

  • With n = 32, No-Repeat can improve termination at moderate sparsity, but accuracy recovery remains limited and LiveCodeBench degradation persists even for dense models.The authors suggest blocking short repeated spans can disrupt valid reasoning and code patterns.Source: §F.3 · p. 32

F.4 Ablation Study of Steering Strength

The ablations show that TGAS’s effect depends on steering strength but is robust to sample count, pruning method, and direction-construction sparsity. Comparisons and significance tests find broad accuracy and termination gains, alongside lower generation cost.

F.4 Ablation Study of Steering Strength

  • For DeepSeek-R1-Distill-Qwen-7B at 60% sparsity, LiveCodeBench termination rises from 1.8% at α = −2 to 34.2% at α = 0 and 57.9% at α = +2.Stronger steering mainly improves termination rather than accuracy, while reversing the direction can substantially degrade both.Source: §F.4 · pp. 32–33, Table 29 · p. 34

F.5 Ablation on Number of Sampled Created Directions

  • Using 50 samples per class achieves comparable steering performance to 100 samples or the full set; on 60%-sparse Qwen-7B LiveCodeBench, termination stays within 52.9%–57.6%.Without steering, termination is 34.2%.Source: §F.5 · p. 33, Table 30 · p. 34

F.6 Generalization Across Pruning Methods

  • Under RAC and Wanda pruning, TGAS raises termination over Base on 16 of 18 checkpoint-benchmark pairs and improves accuracy on all 18, with best accuracy on 15.At 60% RAC sparsity, Qwen-7B LiveCodeBench termination rises from 2.4% to 21.8%, and accuracy from 2.4% to 13.4%.Source: §F.6 · p. 34, Table 32 · p. 37

F.7 Cross-Testing with Different Directions

  • Directions built at different sparsity levels vary by at most 0.6 accuracy points and 4.1 termination points on average across evaluated settings.Larger individual differences appear only on AIME-25, whose 30 problems yield standard deviations of 8–13 points.Source: §F.7 · p. 35, Figure 11 · p. 36

F.8 Comparison with SEAL

  • TGAS and SEAL directions show weak positive alignment, with cosine similarities of 0.146–0.306 under Layer-mean averaging.When SEAL’s direction is built on the 50% checkpoint, its average accuracy is 27.6%, below TGAS at 30.2%.Source: §F.8 · p. 35, Table 34 · p. 37

F.9 Comparison with Plan-CoT

  • TGAS is best or tied for accuracy in 14 of 16 checkpoint-benchmark pairs and has the highest termination rate in 14 of 16 pairs versus Plan-CoT.For Qwen-7B AIME-25, termination is 68.3% versus 54.2% at 50% sparsity and 54.2% versus 45.0% at 60% sparsity.Source: §F.9 · pp. 35–36, Table 35 · p. 38

F.10 Statistical Significance of TGAS Gains

  • Across six models, four checkpoints, and three benchmarks, TGAS matches or exceeds Base in 64 of 72 settings and raises mean accuracy from 46.5% to 49.1%.From dense to 60% sparsity, mean accuracy gain rises from +1.6 to +3.7 points while token reduction grows from 4.4% to 12.3%. Significant pooled accuracy gains occur in three of four tested model-benchmark pairs.Source: §F.10 · p. 36, Table 33 · p. 36

Related papers

Loading 2610.19934v1…