What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study

• The study tests 3-bit and 2-bit weight-only quantization across supervised-fine-tuning and reinforcement-learning models and multiple reasoning benchmarks.Source: §1 · pp. 1–2
• Knowledge distillation is the preferred QAT objective, boosting reasoning models trained through either supervised fine-tuning or reinforcement learning.Source: §1 · p. 2
• PTQ initialization saves training cost and stabilizes QAT, especially during its early stages.Source: §1 · p. 2
• With knowledge distillation as a cold start, reinforcement learning on quantized models can deliver further performance gains.Source: §1 · p. 2
• Aligning the QAT training-data domain with the PTQ calibration set yields faster, more stable training.Source: §1 · p. 2
• 44.53%: Reasoning-QAT surpasses GPTQ on Qwen3-0.6B’s MATH-500 score under 3-bit quantization and outperforms PTQ and QAT baselines across multiple backbones and benchmarks.On DeepSeek-R1-Distill-Qwen-1.5B, it beats representative QAT baselines by up to 4.75% on average.Source: §1 · p. 2

2.1 Post-training Quantization for Reasoning Models

• Quantization converts bfloat16 model parameters into low-bit integer weights, which are dequantized for subsequent operations in weight-only quantization.Weight-activation quantization stores both weights and activations as low-bit integers and uses integer kernels.Source: §2.1 · p. 2
• 11.67%↓ on AIME-120 and 12.80%↓ on MATH-500 contrast with 1.03%↓ on Winogrande and 3.13%↓ on Hellaswag for R1-Qwen-1.5B under PTQ.The reported reasoning-task degradation is larger than the degradation on non-reasoning tasks; similar observations are reported for Qwen3-4B.Source: §2.1 · p. 2, Figure 1 · p. 3

2.2 Quantization-aware Training for Reasoning Models

• QAT simulates low-precision inference during training, using fake quantization in the forward pass and a straight-through estimator to pass gradients to original weights.The section states that QAT's benefits for reasoning models remain unclear.Source: §2.2 · pp. 2–3
• The section asks which objectives best suit low-bit QAT and how SFT, knowledge distillation, and reinforcement learning can support stable training.It notes that standard QAT often reuses cross-entropy, while reasoning models commonly use teacher-generated data or reinforcement learning.Source: §2.2 · p. 3
• QAT can require substantial extra training after severe low-bit degradation, making sample and time efficiency important when retraining budgets are limited.The section also asks how training-data domain, quality, and alignment with calibration data affect performance.Source: §2.2 · p. 3

3.1 Setups

• The study evaluates 2-bit and 3-bit group-wise weight-only quantization with group size 128 across SFT- and RL-trained models and reasoning benchmarks.The setup includes DeepSeek-R1-Distill-Qwen-1.5B, Qwen3-0.6B, and Qwen3-4B, with OpenR1-Math used for low-bit fine-tuning and RL.Source: §3 · p. 3, §3.1 · p. 4

3.2 QAT Training Objectives: SFT or KD?

• Under W3G128, KD limits average accuracy drops to 8.06% on R1-Qwen-1.5B and 9.26% on Qwen3-4B, versus SFT drops of 10.51% and 21.40%.KD performs better across SFT- and RL-trained models; the authors hypothesize its output-distribution alignment preserves uncertainty structure better than hard-label SFT.Source: §3.2 · p. 4

3.3 Training Efficiency of QAT

• GPTQ+KD starts with higher accuracy and lower loss than RTN+KD, outperforms it, and converges faster on MATH-500.KD also achieves higher accuracy and faster convergence than SFT under GPTQ initialization.Source: §3.3 · p. 5, Figure 2 · p. 6, Figure 3 · p. 6

3.4 QAT with Reinforcement Learning

• Zero-RL QAT collapses, while RL after a KD cold start improves accuracy by approximately 46%.KD restores sampling capability and reward density; after cold start, RL increases reward and accuracy while reducing response length and entropy.Source: §3.4 · p. 5, Table 2 · p. 5, Figure 4 · p. 6

3.5 The Choice of QAT Training Dataset

• Across QAT dataset configurations, reasoning-domain training data—especially numina+openr1—achieves the highest accuracy, while general-text wiki+wiki yields substantially lower accuracy, underscoring the importance of domain choice for final performance.Source: §3.5 · p. 6

4 Reasoning-QAT: The Optimized Workflow

• The workflow is presented as practical guidance validating the dependencies identified in the study, not as a new algorithm.The authors state that satisfying those dependencies is sufficient for stable low-bit training.Source: §4 · p. 6, §4.1 · p. 6

4.1 The Reasoning-QAT Workflow

• Reasoning-QAT first initializes with PTQ, then distills from the full-precision model to recover performance and establish an RL cold start, before applying GRPO.The cold-start design avoids the collapse observed when RL is applied directly to heavily quantized models.Source: §4.1 · p. 7, Figure 6 · p. 7

4.2 Empirical Evaluations

• Under W2G128, Qwen3-4B MATH-500 improves from 4.80% (GPTQ) to 78.27%; under W3G128, Qwen3-0.6B average score rises from 12.61% (GPTQ) to 31.67%.The reported comparisons cover representative PTQ baselines across models and reasoning benchmarks; Reasoning-QAT also achieves 58.51% average accuracy against reproduced QAT baselines.Source: §4.2 · p. 7, Table 3 · p. 8, Table 4 · p. 8

4.3 Analysis of Workflow Components

• GPTQ starts at 52.42% AVG versus RTN at 10.41%; KD raises accuracy from 55.27% to 56.45%, and GRPO raises it to 58.51%.The workflow reaches 58.51% AVG against a 63.56% BF16 upper bound.Source: §4.3 · p. 8

5 Conclusion

• The paper presents a systematic empirical study of QAT for reasoning models under extreme low-bit quantization.Source: §5 · p. 8
• For reasoning models under extreme low-bit quantization, Reasoning-QAT consolidates findings into a three-stage workflow that validates them and achieves strong recovery compared to PTQ baselines.Source: §5 · p. 8
• The study identifies KD as the preferred recovery objective, PTQ initialization as beneficial, RL as a source of additional gains with a viable cold start, and PTQ–QAT domain alignment as relevant to convergence and accuracy.Source: §5 · p. 8

Limitations

• Cross-domain generalization to coding and science remains limited, particularly for smaller models under extreme low-bit settings.Source: Limitations · p. 9
• QAT training uses primarily math-centric data, which effectively recovers performance on mathematical benchmarks.Source: Limitations · p. 9
• The empirical study validates training workflows rather than designing new quantization methods.Source: Limitations · p. 9

A.1 Training Implementations Details

• Detailed training hyperparameters are listed in Tables 6 and 7.Source: §A.1 · p. 11

A.2 Complete Factorial Combinations on Weight-Only Settings

• The component assessment covers PTQ initialization, KD, and GRPO for 3-bit groupwise weight-only quantization on R1-Qwen-1.5B.Source: §A.2 · p. 11
• GPTQ initialization yields a 42.01% average improvement over RTN initialization in the reported comparison.Source: §A.2 · p. 11
• With RTN initialization, KD achieves a 45.09% gain versus 42.64% for SFT; KD’s average accuracy advantage over SFT is 2.45% under RTN and 1.18% under GPTQ.Source: §A.2 · p. 11
• GRPO further boosts KD performance by 1.29% under RTN and 2.06% under GPTQ, while GPTQ provides a better starting point for RL training than RTN.Source: §A.2 · p. 11

• Reasoning-QAT loads FlatQuant transformation matrices and uses network-wise QAT adjustments to address quantization-error propagation across layers.This differs from FlatQuant’s layer-wise correction in isolation.Source: §A.3 · p. 11
• The KD stage uses 48,000 OpenR1-Math sequences of length 8,192, with higher learning rates for FlatQuant transformation matrices and clipping factors than for standard weights.The respective learning rates are 1 × 10−6, 5 × 10−5, and 5 × 10−4.Source: §A.3 · p. 11

Comparison with Representative PTQ Methods

• Qwen3-4B’s average score rises from 58.28% with FlatQuant to 60.78% with Reasoning-QAT in W4A4KV4.The authors note that QuaRot results for Qwen3-4B are omitted because its hidden size is incompatible with the standard kernel’s required Hadamard transformation.Source: §A.3 · pp. 11–12

A.4 Additional Evaluation on General Domains

• Under W3G128, Reasoning-QAT maintains performance comparable to the PTQ baseline across general-domain tasks and models, with a +2.45% PIQA gain on R1-Qwen-1.5B.The authors report similar stability in W4A4KV4.Source: §A.4 · p. 12

A.5 Analysis on Initialization for W4A4KV4 QAT

• With QuaRot initialization, KD raises MATH-500 from 1.20% to 11.20%, but the subsequent RL stage collapses because the model lacks a viable starting policy.The authors identify FlatQuant or equivalent outlier suppression as a prerequisite for the RL stage in this W4A4 scenario.Source: §A.5 · pp. 12–13
• QAT from RTN initialization collapses, supporting the authors’ conclusion that outlier suppression is needed for stable W4A4 training.The comparison is against FlatQuant initialization.Source: §A.5 · p. 13

A.6 Artifacts and Licenses

• The listed models are released under MIT or Apache 2.0 licenses, and the listed training and calibration datasets under Apache 2.0 or CC BY-SA.DeepSeek-R1-Distill-Qwen-1.5B uses MIT; the Qwen3 family and OpenR1-Math use Apache 2.0; Wikitext-2 uses CC BY-SA.Source: Models · p. 13, Datasets · p. 13, §A.6 · p. 13

Datasets

• The evaluation benchmarks are publicly available and used under their original licenses, and the authors report that data subsets contained no personally identifying information or offensive content.The listed benchmarks include GSM8K, MATH-500, AIME-120, GPQA-Diamond, and LiveCodeBench.Source: Datasets · p. 13

A.7 Potential Risks and Ethical Considerations

• Quantized models may inherit biases or hallucinations from their full-precision backbones; the authors report no additional safety risks from their methods.Source: §A.7 · p. 14

A.8 AI Writing Assistance

• AI assistants were used for grammatical correction and text polishing, while the authors verified the scientific claims, experimental designs, and results.Source: §A.8 · p. 14