Source-linked AI summary
QuantLRM: Quantization of Large Reasoning Models via Fine-Tuning Signals
Nan Zhang, Eugene Kwek, Yusen Zhang, Muyu Pan, Suhang Wang, Prasenjit Mitra, Rui Zhang
TL;DR
Low-bit PTQ for LRMs underuses fine-tuning information, despite the potential importance of weight updates. QuantLRM converts update patterns into channel-importance scores that protect both ends of the update-magnitude range. It consistently improves LRM quantization, including an average improvement of 6.55% on an RL-fine-tuned model, and can obtain signals through pseudo-fine-tuning when checkpoints are unavailable.
Problem
Existing PTQ methods use activation or second-order signals while largely ignoring fine-tuning traces that may indicate weight importance for LRM quantization.
Method
QuantLRM fits restricted quadratic functions to weight updates, combines their average scores with zero-update counts, and uses the resulting channel importance in quantization loss scaling.
Results
QuantLRM consistently improves quantization across SFT, DPO, and RL fine-tuned LRMs and reasoning benchmarks, with an average improvement of 6.55% on an RL-fine-tuned model.
Takeaways & Limitations
Fine-tuning traces provide a strong and lightweight importance signal for PTQ, with QuantLRM compatible with mainstream inference kernels and comparable speedup to state-of-the-art methods.
Takeaways & Limitations
QuantLRM may require gathering fine-tuning signals through pseudo-fine-tuning when a pre-fine-tuned checkpoint is unavailable.
Abstract
from arXiv · showhide
Weight-only quantization is important for compressing Large Language Models (LLMs). Inspired by the spirit of classical magnitude pruning, we study whether the magnitude of weight updates during reasoning-incentivized fine-tuning can provide valuable signals for quantizing Large Reasoning Models (LRMs). We hypothesize that the smallest and largest weight updates during fine-tuning are more important than those of intermediate magnitude, a phenomenon we term "protecting both ends". Upon hypothesis validation, we introduce QuantLRM, which stands for weight quantization of LRMs via fine-tuning signals. We fit simple restricted quadratic functions on weight updates to protect both ends. By multiplying the average quadratic values with the count of zero weight updates of channels, we compute channel importance that is more effective than using activation or second-order information. We run QuantLRM to quantize various fine-tuned models (including supervised, direct preference optimization, and reinforcement learning fine-tuning) over four reasoning benchmarks (AIME-120, FOLIO, temporal sequences, and GPQA-Diamond) and empirically find that QuantLRM delivers a consistent improvement for LRMs quantization, with an average improvement of 6.55% on a reinforcement learning fine-tuned model. Also supporting non-fine-tuned LRMs, QuantLRM gathers effective signals via pseudo-fine-tuning, which greatly enhances its applicability.
1. Introduction
QuantLRM addresses the limited competitiveness of existing low-bit PTQ for LRMs by using fine-tuning traces to identify important weights. It protects both the smallest and largest updates and consistently improves quantization across fine-tuning strategies and reasoning benchmarks.
- Method: QuantLRM computes channel importance by fitting restricted quadratic functions to updates and multiplying average scores by each channel’s count of zero updates.The resulting scores are injected into the quantization loss as scaling factors.
- Motivation: Existing PTQ methods are less competitive for low-bit quantization of LRMs because they were designed for general-purpose LLMs and discard fine-tuning traces.Current PTQ typically uses activation distributions or second-order information instead.
- Key hypothesis: Weight-update magnitude alone is insufficient, motivating protection of both the smallest and largest updates rather than intermediate magnitudes.The paper links extremely small updates to general capabilities and validates this hypothesis empirically.
- Evaluation: QuantLRM evaluates SFT-, DPO-, and RL-fine-tuned LRMs across AIME-120, FOLIO, temporal sequences, and GPQA-Diamond.Experiments include R1 distillation models, Olmo 3, and Qwen 3.
- Results: The method consistently outperforms strong PTQ baselines on W3A16 quantization and supports mainstream LRMs with minimal model-specific adaptation.The contribution is framed as state-of-the-art quantization performance across a broad experimental set.
2. Related Work
Related work covers PTQ methods that use calibration signals such as activations, second-order information, or output matching, alongside studies connecting fine-tuning traces to parameter importance. The paper positions its hypothesis validation against these existing protection signals.
- Post-Training Quantization: PTQ methods generally avoid retraining and use small calibration datasets with forward passes to make quantization decisions.Examples include AWQ, GPTQ, GPTAQ, and ANY4/ANY3.
- Fine-Tuning Traces in LLMs: Existing research studies fine-tuning traces for interpretation, safety, and parameter-importance analysis, but does not provide fine-grained importance scores.The cited work includes natural-language interpretation of updates and low-dimensional parameter changes in BERT-sized models.
- Hypothesis Validation: The hypothesis-validation setup compares protection signals by quantizing R1-Distill-Qwen-32B to 3-bit while retaining 5% of weights in 16-bit.The table identifies “SFT both ends” as the best protection signal.
3. Methodology
QuantLRM converts fine-tuning traces into channel-importance signals for weight-only quantization by protecting both the smallest and largest updates. It maps update magnitudes with restricted quadratics, accounts for zero updates, and injects the resulting scores into AWQ-style scaling.
- Key idea: QuantLRM uses fine-tuning signals to compute channel importance for quantizing LRMs.The method validates protection of both update-magnitude extremes and applies the resulting importance during quantization.
- Hypothesis validation: Directly using absolute weight updates underperforms activation-based importance, motivating a richer protection signal.Activation with the AWQ calibration set achieves the highest overall scores among the tested initial settings.
- Protecting both ends: Protecting both ends assigns high importance to the smallest and largest updates rather than intermediate magnitudes.The design uses a mapping function whose outputs are high at both extremes and low near the median.
- Zero-update handling: Zero updates are handled separately and counted within each channel to strengthen protection of the far-left end.The two restricted functions are fit on positive updates, while zero-update counts are multiplied with the mapping scores to obtain channel importance.
- Signal injection: QuantLRM injects channel importance into an AWQ-style quantization loss by searching scaling strength and then scaling weights before standard quantization.The method searches α over 20 candidates in [0, 1] and iteratively saves the best scaling factors for target modules.
4. Experiment Setup
The experiments evaluate QuantLRM across diverse reasoning tasks, model families, fine-tuning pipelines, and weight-only PTQ baselines. They also address calibration choices, model-size adaptations, inference measurement, and pseudo-fine-tuning for unavailable checkpoints.
- Benchmarks: QuantLRM is evaluated on AIME-120, FOLIO, temporal sequences, and GPQA-Diamond to cover mathematical, logical, temporal, and scientific reasoning.The benchmarks span 120 complex mathematical questions, logical reasoning, temporal reasoning, and multiple STEM domains.
- Calibration: The calibration setup uses each baseline’s default data, with QuantLRM sharing AWQ’s smaller calibration set and also testing verified reasoning chains.The reasoning-chain calibration set matches the size used by other baselines but is larger than AWQ’s default.
- Models and fine-tuning: The study covers SFT, DPO, and RL fine-tuned LRMs across R1 distillation models, Olmo-3-7B-Think, and Qwen3-1.7B.Qwen3-1.7B is additionally used in a pseudo-fine-tuning setting to collect quantization signals.
- Baselines: QuantLRM is compared with GPTQ, GPTAQ, AWQ, and ANY3 as weight-only PTQ baselines.For DPO and RL models, the reported comparisons use GPTQ and AWQ because of their competitive SFT-model scores.
- Implementation adaptations: Model-size adaptations include slicing MLP weight updates when large matrices create overflow-prone zero-update counts.This adaptation is used for R1-Llama-70B and R1-Qwen-32B; activation multiplication additionally helps R1-Llama-70B but does not generally replace the core signal.
- Evaluation protocol: The main analysis focuses on W3A16, while 4-bit quantization is also measured for performance and latency.Inference accuracy uses vLLM, and latency uses the AWQ kernel after 4-bit quantization because the kernel supports only 4-bit LRMs.
5. Results and Analysis
QuantLRM consistently improves ultra-low-bit reasoning-model quantization across fine-tuning types, while ablations, pseudo-fine-tuning, efficiency measurements, and 4-bit results examine its design and practical scope.
- Overall Results: QuantLRM reaches state-of-the-art 3-bit reasoning performance with one of the smallest calibration datasets.The overall results cover SFT, DPO, and RL fine-tuned LRMs.
- Overall Results: 3.03% is the largest average-score improvement over PTQ baselines across the three SFT models.On R1-Llama-70B, the improvement over the strongest PTQ baseline is 2.12%; on R1-Qwen3-8B it is 1.65%.
- Overall Results: 6.55% is the largest average-score improvement over competitive baselines on DPO- and RL-fine-tuned LRMs.The authors report consistent gains across fine-tuning types and attribute the smallest DPO gain to relatively weak fine-tuning signals.
- Ablation Study: The ablation’s best R1-Qwen-32B design counts zero updates and excludes them when fitting the quadratic functions, improving average score by 5.95%.On the 8B models, protecting both ends outperforms activation-based signals, while the preferred zero-update treatment varies by model.
- Pseudo-Fine-Tuning: Pseudo-fine-tuning produces progressively stronger QuantLRM performance as training steps accumulate, eventually matching or surpassing AWQ after 1956 training steps.QuantLRM surpasses GPTQ across all reported training steps; AWQ and GPTQ remain horizontal baselines because they require no training.
- Efficiency: QuantLRM adds 2 minutes and 27 seconds of offline preparation, while quantization takes roughly the same time as AWQ and inference latency is identical.The methods use the same AWQ inference kernel.
- 4-bit Quantization: In W4A16, QuantLRM remains competitive and stable, while the paper emphasizes W3A16 because 4-bit performance leaves little room for improvement.The reported 4-bit average scores of all three methods surpass the 16-bit model in the cited experiment.
6. Conclusion
QuantLRM quantizes large reasoning models using fine-tuning signals, protecting the smallest and largest weight updates. It consistently improves quantized reasoning accuracy while offering inference flexibility and comparable speedup to leading methods.
- QuantLRM gathers weight updates from SFT, DPO, and RL fine-tuning and assigns high importance to the smallest and largest updates.
- QuantLRM consistently improves large reasoning model quantization performance.
- QuantLRM is compatible with vLLM and AWQ kernels and offers inference flexibility with comparable speedup to state-of-the-art quantization methods.
- Fine-tuning traces provide a strong and lightweight importance signal for post-training quantization.
Impact Statement
The paper aims to advance machine learning and identifies no specific societal consequences that require highlighting.
- The work aims to advance the field of Machine Learning.
- The authors identify no potential societal consequence that they feel must be specifically highlighted.
A. A Smaller Calibration Set of QuantLRM and AWQ
QuantLRM uses one of the smallest calibration sets while still outperforming other quantization baselines. The setup uses 128 calibration samples with shorter sequences than several comparison methods.
- QuantLRM still outperforms other baselines despite using one of the smallest calibration sets.
- GPTQ, GPTAQ, and ANY3 use 128 C4 samples of length 2048 tokens, whereas AWQ and QuantLRM use 128 samples of length 512.
B. Prompts Used for Speed Analysis
The speed analysis measures QuantLRM and AWQ with seven short reasoning prompts. Each prompt generates 200 reasoning tokens, and median latency is computed.
- Inference speed is measured for QuantLRM and AWQ using seven short prompts.
- The prompts ask models to analyze, enumerate approaches, solve and revise, derive, justify, decompose, or identify risks and safeguards.
- Each prompt generates 200 reasoning tokens, and the analysis computes median latency.