Source-linked AI summary
Reasoning-Aware Compression: Identifying and Protecting Vulnerable Reasoning Circuits for Energy-Efficient LLM Deployment
Leonard Twagirayezu, Prasenjit Mitra
TL;DR
The paper addresses the risk that uniform quantization damages reasoning circuits and can increase deployment energy. It profiles per-module INT4 vulnerability across five reasoning benchmarks using a held-out calibration split, then selectively restores sensitive modules to FP16. Vulnerability is task- and architecture-dependent, while selective protection can achieve accuracy-energy trade-offs unavailable to uniform compression, including a ProofWriter gain for R1-Qwen-7B.
Problem
Uniform quantization can damage critical reasoning circuits, while existing methods do not adequately capture reasoning-specific vulnerability and energy effects.
Method
The framework measures per-module INT4 vulnerability across 196–224 layer-projection pairs using a held-out calibration split, then selectively restores the most sensitive modules to FP16.
Results
Vulnerability varies by task and architecture, and selective compression can reach otherwise inaccessible accuracy-energy trade-offs, including R1-Qwen-7B Top-10% on ProofWriter at +12.00 pp over FP16 and −9.7% energy.
Takeaways & Limitations
Deployment choices should account for reasoning task and per-module vulnerability rather than assume one universal protection strategy.
Takeaways & Limitations
Experiments used a single Tesla V100-SXM2 GPU type, so energy measurements and chain-length effects may differ on other hardware.
Abstract
from arXiv · showhide
Large Reasoning Models (LRMs) impose substantial energy costs during deployment, yet current compression methods apply uniform quantization across all components, risking damage to critical reasoning circuits. We present a reasoning-aware compression framework that benchmarks quantization conditions across five reasoning benchmarks, GSM8K, FOLIO, MATH-500, ProofWriter, and MuSiQue, with hardware-level GPU energy measurement; profiles per-module INT4 vulnerability across all 196-224 (layer, projection) pairs via a perturbation sweep on a held-out calibration split, then selectively restores the most sensitive circuits to FP16. Three findings emerge. First, INT4 quantization can increase energy by extending reasoning chains; a 25% power reduction becomes a net energy increase on GSM8K. Second, vulnerability is task-dependent: attention projections are more critical for mathematical reasoning, and sensitivity patterns differ by architecture in logical inference. Third, selective compression achieves Pareto-optimal points inaccessible to uniform methods: R1-Qwen-7B Top-10% on ProofWriter gains +12 pp over FP16 at -9.7% energy, validated on held-out data across five reasoning benchmarks.
1 Introduction
Large Reasoning Models make deployment costly because extended reasoning traces require substantial memory and generate many tokens. The paper addresses this gap by identifying quantization-vulnerable reasoning circuits and selectively protecting them rather than compressing all components uniformly.
- Large Reasoning Models require substantial GPU memory and generate thousands of reasoning tokens per query, making deployment expensive for resource-constrained applications.
- The central question is which layer-projection pairs are most vulnerable and how protecting them trades off accuracy against energy savings.
- Existing quantization methods often use general-corpus calibration and do not capture reasoning-specific compression vulnerability or module-level energy effects.
- The framework targets extended chain-of-thought sequences, where quantization noise can propagate across many reasoning steps and energy can depend strongly on output length.
- Its contributions include per-module vulnerability profiling across 196–224 pairs, evidence that INT4 can lengthen outputs and increase energy, and vulnerability-guided selective compression.
2 Experimental Setup
The study evaluates two distilled DeepSeek-R1 models across five reasoning datasets and five compression conditions. It measures accuracy, output length, power, and energy using held-out evaluation data and hardware-level GPU sampling.
- Two DeepSeek-R1 distilled models are evaluated: Llama-8B and Qwen-7B, representing two architecture families.
- The five benchmarks cover arithmetic, first-order logic, competition mathematics, formal deduction, and multi-hop question answering.
- Each model is evaluated under five compression conditions.
- Energy per query is computed from mean sampled GPU power multiplied by wall-clock inference time.
- Metrics are averaged across eight experiments of 30 samples each, with calibration and held-out evaluation data separated 50/50.
3 Methodology
The methodology first compares grouped attention and MLP protection, then profiles individual layer-projection vulnerability through calibration-only perturbations. The most vulnerable modules are restored to FP16 within an otherwise INT4 model at several protection levels.
- Grouped Protection: Grouped protection restores either attention or MLP projections to FP16 to identify which projection type matters more for each reasoning task.
- Perturbation Sweep: The perturbation sweep quantizes each layer-projection pair individually on calibration data, measures accuracy, restores the module, and records its vulnerability.
- Perturbation Sweep: The sweep covers 196 Qwen-7B pairs and 224 Llama-8B pairs across five benchmarks, excluding MuSiQue from the combined ranking because near-zero accuracy makes scores noisy.
- Selective Compression: Selective compression restores only the most vulnerable ranked pairs to FP16 after loading all modules in INT4.
- Selective Compression: Protection levels are K ∈ {0.10, 0.20, 0.30, 0.40}, compared against random and least-vulnerable module selections.
4 Results
Quantization effects vary sharply by task, model, and protected modules: INT4 can improve accuracy yet increase total energy through longer reasoning traces. Selective restoration identifies Pareto improvements on some benchmarks, but vulnerability rankings and benefits do not generalize uniformly.
- Grouped quantization: INT4+Attention gives the best mathematical accuracy for both models, including MATH-500 gains of +3.75 pp for Qwen-7B and +2.50 pp for Llama-8B over FP16.On Qwen-7B MATH-500, it also lowers energy by 7.1% versus FP16.
- Grouped quantization: Full INT4 performs best on ProofWriter, reaching 77.08% for Qwen-7B and 41.67% for Llama-8B, while FOLIO gains are architecture-dependent.Qwen-7B ProofWriter improves +7.50 pp over FP16; Llama-8B FOLIO improves +3.75 pp.
- Grouped quantization: MuSiQue is the most compression-sensitive task: every INT4 condition matches or underperforms FP16, and no selective strategy reliably improves accuracy.The best Full INT4 results are 8.19% for Qwen-7B and 15.72% for Llama-8B.
- Grouped quantization: 25% lower power under Full INT4 can become 22.9% higher energy on Llama-8B GSM8K when output tokens rise 22.8%.The longer chain increases total energy despite lower per-token power draw.
- Per-module vulnerability: Vulnerability is localized and task-dependent: ProofWriter reaches drops of 50.00 pp for Qwen-7B and 36.67 pp for Llama-8B, while MuSiQue shows zero vulnerability across pairs.Most individually quantized pairs produce zero accuracy drop on most datasets, motivating selective protection.
- Selective compression: Top-10% protection on ProofWriter Qwen-7B reaches 84.00%, exceeding FP16 by 12.00 pp while reducing energy 9.7%.It protects 19 of 196 pairs and uses 1,743 J versus 1,931 J for FP16 on held-out data.
5 Conclusion
The framework identifies reasoning-circuit vulnerability through perturbation sweeps and selectively restores sensitive layer-projection pairs to FP16. It finds that energy, task-dependent vulnerability, and accuracy-energy trade-offs must be evaluated jointly for deployment.
- Perturbation sweeps across 196–224 layer-projection pairs identify vulnerable reasoning circuits on held-out calibration data.Selective restoration then protects only the most sensitive pairs rather than applying uniform protection.
- INT4 can increase total energy when longer reasoning chains outweigh power savings, including a 22.9% GSM8K energy increase from 22.8% more output tokens.The reported case converts a 25% power reduction into higher total energy.
- Vulnerability is task-dependent: attention projections are more critical for arithmetic reasoning, while logical-inference sensitivity varies across architectures and projection types.No universal protection strategy is optimal across all task types.
- R1-Qwen-7B Top-10% on ProofWriter reaches 84.00% accuracy, +12.00 pp over FP16, while reducing energy by 9.7% on held-out evaluation data.This is a Pareto-optimal operating point unavailable to uniform methods.
- Future work should reduce restored-layer memory overhead, develop task-adaptive rankings that generalize across datasets, and evaluate 30B–70B models.
Limitations
The study’s conclusions are bounded by hardware, sample-size, and training-regime limitations. These constraints affect portability, statistical precision on near-zero-accuracy tasks, and comparison with training-aware compression methods.
- Experiments use only Tesla V100-SXM2 GPUs on PSC Bridges-2, so energy measurements and chain-length effects may differ on other hardware.Replication on A100 or H100 hardware remains future work.
- Thirty-sample evaluations are insufficient for reliable statistical inference on near-zero-accuracy MuSiQue compression conditions.A single correct answer changes accuracy by 3.33 pp, so MuSiQue differences are directional rather than precise estimates.
- The study evaluates only post-training compression methods, leaving potential accuracy recovery from QAT or fine-tuned structured pruning untested.Those alternatives require training data and substantial additional compute.
Ethics Statement
The work evaluates compression of publicly available language models on publicly available benchmarks and reports no human-subject involvement. Its stated goal is to support environmentally sustainable AI deployment.
- The study uses publicly available language models and benchmarks, with no human subjects involved.
- Reducing energy consumption is presented as supporting environmentally sustainable AI deployment.
A Perturbation Sweep Vulnerability Heatmaps
The perturbation sweep visualizes per-module INT4 vulnerability across projection types and layers, using accuracy drops to identify circuits for FP16 restoration. The heatmaps cover R1-Qwen-7B and R1-Llama-8B, while Table 7 summarizes maximum drops and safe pairs.
- Heatmap interpretation: Each heatmap cell represents the accuracy drop when one layer–projection pair is individually quantized to INT4.White cells indicate zero accuracy drop, while darker cells identify more vulnerable circuits prioritized for FP16 restoration.
- R1-Qwen-7B: Figure 1 maps R1-Qwen-7B vulnerability across projection types and layer indices 0–27.
- R1-Llama-8B: Figure 2 maps R1-Llama-8B vulnerability across projection types and layer indices 0–31.Its sparser pattern is consistent with Full INT4 frequently matching FP16 accuracy on Llama-8B.
- Summary statistics: Table 7 reports each model’s most vulnerable pair through the maximum accuracy drop and counts pairs with zero or negative drop as safe.
B Top-K vs. Random-K vs. Bottom-K Ablation
The ablation compares vulnerability-guided, random, and least-vulnerable module protection across datasets and models. Its advantage is task- and model-dependent: Top-K protection is clearest on MATH-500, while other settings show weak or reversed discrimination.
- Ablation design: Table 8 compares Top-K%, Random-K%, and Bottom-K% protection across four datasets and two models, with Full INT4 as a reference.
- MATH-500: On MATH-500, Top-K% consistently outperforms Random-K% and Bottom-K% at K=10–20% for both models.For Qwen-7B at Top-10%, accuracy is 66.67% versus 65.69% for Random-10% and 62.92% for Bottom-10%.
- FOLIO and GSM8K: On FOLIO and GSM8K, all three strategies perform within one standard deviation at K=10–20% and degrade similarly at K=30–40%.
- ProofWriter: On ProofWriter with Llama-8B, Random-K% and Bottom-K% outperform Top-K% at low K.This indicates that the calibration-split ranking does not fully generalize to that evaluation setting.
C Statistical Significance Tests
The statistical tests find no significant accuracy difference between INT4 conditions and FP16, while energy differences are significant in 9 of 30 comparisons. The analysis uses held-out results from five experiments, limiting statistical power for accuracy tests.
- Accuracy tests: No accuracy difference between INT4 conditions and the FP16 baseline reaches significance at p<0.05.The tests use held-out evaluation results from n=5 experiments.
- Energy tests: Energy differences reach significance in 9 of 30 comparisons.This is reported despite the small sample size of five experiments with 30 samples each.
- Statistical power: The authors attribute the limited accuracy-test power to five experiments at 30 samples each.