Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization

• Layer-wise PTQ is practical because it quantizes parameters layer-by-layer without retraining or backpropagation.Source: §1 · p. 1
• Quantization errors accumulate across layers and degrade layer-wise PTQ performance, particularly in low-bit settings.Source: §1 · p. 2
• QEP modifies the layer-wise objective to propagate and compensate for accumulated errors with comparable computational complexity.Source: §1 · p. 2
• QEP is orthogonal to existing PTQ methods and integrates with any layer-wise PTQ pipeline.Source: §1 · p. 2
• QEP significantly improves GPTQ [Frantar et al., 2022], AWQ, and QuIP [Chee et al., 2023], especially at 2-bit quantization.Source: §1 · p. 2, Figure 1 · p. 2

Compensation-based layer-wise PTQ

• Compensation-based methods quantize weights sequentially using calibration-derived Hessians while compensating for subsequent unquantized weights.Source: §2 · p. 2

Rotation-based layer-wise PTQ

• Rotation-based methods preprocess weights with structured rotations to redistribute weight magnitudes more uniformly.Source: §2 · p. 3

Salience-based layer-wise PTQ

• Salience-based methods preserve important weights through mixed precision, while AWQ uses global scaling to simplify deployment and maintain accuracy.Source: §2 · p. 3

• Recent advances emphasize nonlinear quantization and block-wise or global fine-tuning, while the foundational layer-wise optimization remains largely unchanged since GPTQ [Frantar et al., 2022].Source: §2 · p. 3
• QEP complements and remains orthogonal to nonlinear quantization and structured extensions.Source: §2 · p. 3

Post-training quantization

• PTQ converts pretrained parameters into discrete representations drawn from a finite set of 2^b quantization levels.Source: §3 · p. 3
• Calibration-based PTQ seeks quantized parameters that minimize deviation from the original model’s performance.Source: §3 · p. 3

Layer-wise PTQ

• Layer-wise PTQ processes layers sequentially from input to output while solving independent optimization problems for each layer.Source: §3 · p. 3
• The layer-wise reconstruction objective uses X_l as input activations and a Hessian H_l := X_lX_l^T that can be precomputed and cached.Source: §3 · p. 3
• Existing methods use either quantized or full-precision activations, with no consensus on which produces better quantization outcomes.Source: §3 · p. 3
• GPTQ [Frantar et al., 2022] uses quantized activations and sequential residual correction, whereas other methods use distinct strategies within the same framework.Source: §3 · p. 4

• The experiment compares original and quantized Transformer-block compositions using the squared output discrepancy defined for each block.Source: §4 · p. 4
• Only the first 10 Transformer blocks are quantized while later blocks remain full precision, isolating error propagation beyond the quantized region.Source: §4 · p. 4
• Quantization errors show approximately exponential accumulation within quantized layers and continued growth through subsequent unquantized layers.Source: §4 · p. 4
• Independent layer-wise optimization ignores propagated errors from earlier layers and fails to correct accumulated errors, exacerbating later growth.Source: §4 · p. 4

5 QEP: Quantization Error Propagation

• QEP propagates quantization errors across layers and compensates for their accumulation, addressing a core limitation of independent layer-wise PTQ.The framework is designed as a general, lightweight, and scalable enhancement to existing layer-wise methods.Source: §5 · pp. 4–5

5.1 Problem Reformulation

• The reformulated objective compares full-precision and quantized outputs using their respective upstream inputs, enabling each layer’s weights to correct preceding errors.Unlike independent reconstruction, the optimal quantized weights generally differ from direct approximations of the original weights when upstream errors are nonzero.Source: §5.1 · p. 5, §5.2 · p. 5

5.2 Weight Correction

• QEP’s corrected-weight reformulation restores the quadratic structure needed for Hessian-based optimization and integrates with existing layer-wise PTQ methods.The correction term uses accumulated error and an empirical Hessian from quantized activations, while the discrete optimization retains the structure of the baseline objective.Source: §5.2 · p. 5
• Theorem 5.2 bounds QEP’s output quantization error by that of existing layer-wise PTQ under the stated Lipschitz-continuity assumptions.The parameter sets compared are those optimized by the QEP and baseline objectives, respectively.Source: §5.2 · p. 6

5.3 Controlling Propagation Strength

• The tunable propagation strength α_l ∈[0, 1] balances error correction against overfitting, with α_l = 1 fully correcting and α_l = 0 recovering the baseline setting.The parameter acts analogously to regularization and is especially important for preventing overfitting in large MLP blocks.Source: §5.3 · p. 6
• Setting α_l = 0 selectively can eliminate correction costs for expensive layers, reducing correction time by approximately one-third to one-half while acting as implicit regularization.The paper highlights high-dimensional MLP activations as a source of expensive correction terms and identifies adaptive α_l tuning as future work.Source: §5.3 · p. 6

Experimental setup

• Experiments cover RTN, GPTQ, AWQ, and QuIP on Llama-2, Llama-3, and Mistral models using INT4, INT3, and INT2 weight-only quantization.Evaluations use perplexity on WikiText2, PTB, and C4 plus zero-shot accuracy on ARC Easy, PiQA, and StoryCloze.Source: §6 · p. 7

Perplexity

• QEP substantially reduces perplexity across nearly all tested layer-wise PTQ methods and quantization levels, with QEP-enhanced QuIP achieving the best tested perplexity.At INT2, QEP makes practical perplexity achievable where linear-quantization baselines often degrade severely.Source: §6.1 · p. 7, Table 1 · p. 8

Zero-shot tasks

• QEP improves zero-shot accuracy consistently, with the largest gains at INT2; QEP-enhanced QuIP at INT2 matches RTN and GPTQ at INT3 on Llama-2-70B.Source: §6.1 · p. 8, Table 2 · p. 9

Runtime

• QEP correction requires significantly less computation time than other layer-wise PTQ quantization processes, with QEP+RTN time dominated by correction-term preprocessing.Using the same calibration dataset for correction and quantization further reduces preprocessing overhead.Source: §6.1 · p. 8, Table 3 · p. 8

Robustness

• QEP-enhanced RTN improves performance across C4, PTB, and WikiText2 calibration sets, unlike GPTQ, whose performance varies substantially with the calibration distribution.This supports the propagation-control mechanism as a way to reduce calibration overfitting.Source: Table 4 · p. 8, §6.1 · p. 9

Conclusion

• QEP explicitly propagates and compensates for accumulated quantization errors, substantially improving layer-wise PTQ performance, especially at low bit-widths.The conclusion frames this as progress from revisiting layer-wise PTQ fundamentals rather than relying only on non-linear or block-wise quantization.Source: §7 · p. 9

Limitations

• QEP relies on a small calibration set, so its performance remains sensitive to calibration-data quality despite less overfitting than GPTQ and AWQ.Source: §7 · p. 9
• QEP introduces a tunable per-layer propagation-strength parameter α_l, but automatic learning of α_l is left for future work.A fixed α_l = 1/2 works well in most cases.Source: §7 · p. 9

Theoretical scope

• The theoretical analysis uses calibration activations, quantized weights, quantized activations, and empirical Hessians, with invertibility assumptions stabilized by adding ρI when needed.Source: B Additional Theoretical Results · pp. 14–15

Objective reformulation

• The proof rewrites W_lX_l as W_l X̂_l + W_lδ_l, expressing the objective in terms of quantized activations and the layer quantization error.Source: §B.1 · p. 15

Optimization

• Because the objective is strictly convex in the quantized weights when the quantized Hessian is invertible, its stationary point is the unique minimizer.Source: §B.1 · p. 15

Closed-form solution

• The stationary condition is rearranged and multiplied by the inverse quantized Hessian to obtain the closed-form solution.Source: §B.1 · p. 15

Conclusion

• The resulting closed-form expression completes the proof as the unique minimizer of the Frobenius-norm objective.Source: §B.1 · p. 15

• Activation differences at the output layer grow exponentially with depth under standard layer-wise PTQ, to first order in quantization noise.This follows when layers are quantized independently without accounting for downstream effects.Source: §B.2 · p. 15
• The activation error is recursively decomposed into contributions from each layer’s weight quantization error and propagated through subsequent layers.The expansion starts from δ1 = 0 and uses the layerwise activation recurrence.Source: §B.2 · pp. 15–16
• Bounding relative layer errors produces the recurrence a_{l+1} ≤ r + (1 + r)a_l, which also grows geometrically with depth.Here r bounds the relative weight quantization error.Source: Then the final activation mismatch… · p. 17
• Under Lipschitz activations, the normalized error satisfies a recurrence that yields exponential growth when γl∥Wl∥2 ≤ 1 + ε.The resulting bound scales with (1 + ε)^(L−1).Source: Then the final activation mismatch… · pp. 17–18
• A one-dimensional linear network gives the exact lower bound |δL| ≥ (L −1)cE(1 + ε)^(L−2)C for uniformly perturbed weights.This construction demonstrates that the depth dependence is not merely an artifact of loose analysis.Source: Then the final activation mismatch… · p. 18

• QEP achieves a global Lipschitz upper bound on output mismatch no larger than the baseline bound.The proof establishes a stronger per-layer inequality before iterating it through the remaining layers.Source: Define the per-layer pre-activation… · pp. 19–20
• The analysis compares QEP and baseline residuals at the same quantized activation, allowing the per-layer QEP advantage to propagate through later layers.The comparison is made before applying the remaining layerwise Lipschitz factors.Source: Define the per-layer pre-activation… · pp. 19–20
• QEP constructs corrected weights by adding a projected correction proportional to the activation mismatch and propagation strength.The correction uses the orthogonal projection Pl and the factor αl.Source: Define the per-layer pre-activation… · pp. 19–21
• The final quantization error decreases monotonically as each propagation-strength parameter αl approaches 1.This result follows from the projection structure of the correction and the monotonic residual bound.Source: Define the per-layer pre-activation… · p. 20
• The QEP correction parameter αl is strictly decreasing in ridge regularization λ, so reducing λ corresponds to increasing propagation strength.The mapping decreases from 1 to 0 as λ ranges from 0 to infinity.Source: §B.4 · p. 21, The unique minimizer of the ridge… · pp. 21–22

• A diagonal damping term is added to the Hessian when it is ill-conditioned or singular, ensuring positive definiteness for inversion.The damping value is set to the mean of the Hessian’s diagonal elements.Source: §C.1 · p. 22

• QEP consistently improves PTQ performance in supplementary perplexity and accuracy evaluations, particularly under low-bit quantization.Additional results cover PTB, C4, WikiText-2, PIQA, StoryCloze, and ARC-Easy.Source: §D.1 · p. 22, §D.2 · p. 23
• Figure 3 compares INT4, INT3, and INT2 across 7B, 13B, and 70B models using WikiText-2 perplexity and averaged normalized zero-shot accuracy.Error bars show the standard error of the mean.Source: Figure 3 · p. 23
• QEP-QuIP retains its advantage across five-seed averages, with especially clear gains at INT3 and INT2.Small seed-to-seed variation does not alter the main conclusions.Source: §D.3 · p. 23
• QuIP+QEP achieves the lowest reported perplexity at INT4 and INT3 against OmniQuant and remains stable at INT2, where OmniQuant diverges.The comparison is on LLaMA-2-7B using WikiText-2 perplexity.Source: §D.4 · p. 23, Table 11 · p. 23