PaperlayerView exact revision on arXiv

Source-linked AI summary

Why Does Post-Training Quantization Work?

Yuxiang Chen, Michael Beyer, Jun Zhu, Jianfei Chen

arXiv:2609.11716v1cs.LGcs.CL

TL;DR

Post-training quantization preserves pretrained LLM predictions despite layerwise errors, but the mechanisms behind this robustness remain unclear. By comparing full-precision and quantized forward passes, the paper finds that pretrained models counteract inherited and newly introduced errors, while LM-head geometry preserves high-ranked token outputs.

  • Problem

    The paper asks why pretrained models accumulate far less quantization-induced hidden-state error than randomly initialized models despite similar weight reconstruction error.

  • Method

    The paper compares full-precision and quantized forward passes block by block to analyze hidden-error growth and its effect on output predictions.

  • Results

    50.2% of block-update error is cumulatively canceled across layers in pretrained Qwen3-32B, while LM-head geometry preserves high-ranked token scores and probabilities.

  • Takeaways & Limitations

    Pretraining develops counteracting residual interactions, and together with LM-head geometry they explain why quantization produces small output changes despite deep error propagation.

  • Takeaways & Limitations

    The analysis covers individual next-token predictions rather than multi-token generation and relies on aggregate statistics plus a uniform-direction approximation for LM-head theory.

Abstract

from arXiv · show

Post-training quantization compresses large language models (LLMs) by storing their weights at reduced precision, and each quantized weight introduces an error into the hidden states. Naively, these errors should accumulate with depth and corrupt next-token prediction; randomly initialized models accumulate these discrepancies rapidly, whereas quantized pretrained models accumulate much less hidden-state error and largely maintain downstream task performance, even though they were never trained with quantization noise. This raises the question we address: why does post-training quantization work? Comparing full-precision and quantized forward passes, we identify two mechanisms that characterize pretrained quantization robustness. First, the error a layer newly introduces tends to oppose the error it inherits from the layer's input. The two cancel partially such that the discrepancy between full-precision and quantized passes grows slowly. This counteracting residual interaction develops during pretraining. Our quantitative analysis identifies it as a major factor slowing hidden-error growth. Second, LM-head geometry preferentially preserves the scores and probabilities of high-ranked tokens, which typically represent the model's most confident predictions. Together, these mechanisms explain why quantization error that passes through numerous layers can still produce only small output changes, and we verify the findings across models and quantization settings.

1 INTRODUCTION

The introduction frames post-training quantization robustness as a property acquired during pretraining, not merely a consequence of small weight-reconstruction errors. It identifies residual-error cancellation across layers and output geometry favoring high-ranked tokens as complementary mechanisms preserving predictions.

  • Introduction: Because quantization errors should naively accumulate across layers, the introduction treats their slow growth and limited output impact as the central problem requiring a mechanistic explanation.Each reduced-precision weight differs slightly from its full-precision value, creating hidden-state discrepancies at every layer.
  • Introduction: Pretrained models show 5.5× smaller hidden-state discrepancy than randomly initialized models despite nearly identical weight-reconstruction errors, indicating robustness is acquired during pretraining.Quantized weights have cosine similarity ∼0.996 to NVFP4 reconstructions, but this similarity alone does not explain hidden-error growth.
  • Introduction: Layer-introduced quantization error tends to oppose inherited error, producing partial cancellation that slows hidden-error growth with depth.The analysis compares quantized and full-precision models on identical inputs and tracks hidden-state error block by block.
  • Introduction: LM-head geometry preferentially preserves the scores and probabilities of high-ranked tokens, helping maintain the model’s most confident predictions after quantization.This mechanism acts after the slow accumulation of hidden-state error through the network.

2 SETUP: COMPARING FULL-PRECISION AND QUANTIZED MODELS

The study compares full-precision and quantized models on identical next-token prediction inputs, tracking hidden-state discrepancies through Transformer blocks and their effects on final logits and probabilities. Quantization is applied primarily to Attention and MLP linear projections, while other operations remain in full precision.

  • Output metrics: Output impact is measured through the LM-head score difference Δz = WLMΔhLM, probability difference, cross-entropy, and DKL(p∥p̂).The next-token distribution is computed as softmax(z/T), with T = 1 unless stated otherwise.
  • Quantization setup: The study quantizes standard Attention and MLP linear projections to NVFP4 with round-to-nearest, while retaining other operations, including the LM head, in BF16.For a linear layer, quantization replaces W with a low-precision approximation Q(W), producing an approximate output XQ(W)^⊤.
  • Comparison protocol: The original and quantized models receive the same prefix at each decoding position, enabling position-wise comparison of their forward passes.Expectations average first across positions within each sequence and then equally across sequences.
  • Hidden-state comparison: At every decoding position, both models start from the same initial hidden state, and their block-level residual updates determine the resulting hidden-state difference.The full-precision model is denoted M, while the quantized model is denoted M̂.
  • Error measurement: The analysis defines hidden error as the quantization-induced difference between corresponding layer representations and propagates it to the LM-head input.The LM-head input is obtained after normalization of the final hidden state.

3 THE GROWTH OF HIDDEN-ERROR NORM ACROSS LAYERS

Pretrained Qwen3-32B accumulates substantially less hidden-state error under NVFP4 than a randomly initialized model, despite similar weight-reconstruction similarity. This robustness is driven largely by counteraction between inherited input error and newly introduced block-update error, which slows error growth across depth.

  • 3 THE GROWTH OF HIDDEN-ERROR NORM ACROSS LAYERS: NVFP4 causes randomly initialized Qwen3-32B to reach a 5.5× larger final absolute hidden-error norm than the pretrained model.The two models have nearly the same weight-reconstruction cosine similarity, showing that weight-level similarity alone does not explain pretrained robustness.
  • Counteraction: Counteraction occurs when block-update error points against inherited block-input error, partially canceling newly introduced error and slowing hidden-error norm growth.For pretrained Qwen3-32B, the interaction term cumulatively cancels 50.2% of the block-update error contribution over layers 1–48, whereas random initialization shows near-zero alignment.
  • Relative hidden error: Tinter and Talign cumulatively cancel 81.8% of Tadd in pretrained Qwen3-32B, compared with only 0.09% at random initialization.Tinter contributes 63.4% cancellation and Talign contributes 18.4%, with most Talign cancellation occurring in later blocks.
  • Counteraction interventions: Removing counteraction raises final relative hidden error by 2.94×, while reversing it raises error by 8.41× and both interventions sharply increase KL divergence.The interventions target blocks 17–48, where counteraction is strongest; removal also increases cumulative Tadd by 4.3×, providing causal evidence for counteraction’s role.

4 THE EFFECT OF FINAL HIDDEN ERROR ON OUTPUT QUALITY

Despite a mean final hidden-state relative error of approximately 0.245, W4 quantization largely preserves output quality, with only a 0.43-point average accuracy drop across six benchmarks. This robustness arises because hidden-state error is mainly angular and LM-head geometry preferentially preserves high-ranked tokens.

  • 4 THE EFFECT OF FINAL HIDDEN ERROR ON OUTPUT QUALITY: 88.9–98.7%: angular changes account for this share of squared relative hidden error across depth, producing an average 12.47° rotation after final normalization.The mean relative final hidden-state error is approximately 0.245, while changes in hidden-state norm remain small.
  • 4.1 Output quality: 0.43 percentage points: W4 accuracy drops only this much on average across six benchmarks, while output quality is largely preserved across additional evaluated models.Loss, relative cross-entropy change, and forward KL divergence remain small; Figure 4(A) reports the primary metrics.
  • 4.1 Output quality: Top-ranked tokens remain substantially more stable: Ret@10 and Ret@20 are both about 85% on average, whereas Flip@1 is 8.3%–12.7%.For the displayed examples, average relative probability changes at ranks 29–30 are 2.5–4.4× those at ranks 1–5.
  • 4.2 Top-ranked token preservation: A 12.5° LM-head-input rotation becomes only a 0.159° vocabulary-mean projection-angle change, a 78.6× attenuation that limits score perturbations.In high dimension, only a small component of the rotation affects projection onto any fixed LM-head weight vector.
  • 4.2 Top-ranked token preservation: Higher-ranked tokens have smaller relative score and log-probability changes because their smaller projection angles make scores less sensitive to the same hidden-state rotation.KL remains small because it weights tokens by original probabilities, while cross-entropy remains small because ground-truth-token probabilities change little on average.

5 DISCUSSION

The discussion shows that pretrained models robustly limit quantization error through input-error counteraction and preferential preservation of high-ranked output scores, while identifying rare-position failures and broader analytical limitations.

  • Counteraction persists across pretrained Qwen, OLMo, and Gemma models, including dense and MoE architectures, quantization components, and PTQ algorithms.The negative layerwise interaction remains in most blocks, indicating calibration-based PTQ changes the counteraction geometry little.
  • Preferential preservation of top-ranked scores and probabilities transfers across models, with higher-ranked tokens showing smaller projection angles and relative score errors across temperatures.
  • Error interactions are negative in 82.5% of blocks, unlike native hidden-state/update interactions, which are positive in 65.6% and 64.1%.Thus, opposing updates are not a general property of block inputs; they emerge more consistently for accumulated hidden errors.
  • Input-error response contributes 99.9% of the negative cumulative interaction, versus less than 0.1% from direct weight effects.The two components have comparable magnitudes, but counteraction mainly arises from the block response to its input error, which opposes accumulated hidden error.
  • Counteraction usually slows hidden-error growth, but rare decoding positions can produce extreme hidden-norm mismatches that dominate averages.These failures mark cases where counteraction no longer keeps the quantized hidden-state trajectory close to the original model.
  • The analysis does not establish how these mechanisms extend to multi-token generation, fully characterize rare trajectories, or remove the LM-head theory’s uniform-direction assumption.

6 CONCLUSION … A.2 TRANSFORMER BLOCK RESPONSES TO QUANTIZATION

The paper explains pretrained models’ quantization robustness through slower hidden-error accumulation despite similar weight reconstruction quality, driven by block-update errors that oppose errors already present at block inputs. It situates this mechanism within prior work on PTQ optimization, quantization-error propagation, and opposing residual contributions.

  • 6 CONCLUSION: Pretrained models accumulate much less hidden error than randomly initialized models under NVFP4 quantization, despite nearly identical weight reconstruction cosine similarities.The conclusion attributes this difference to a pretrained-specific block-update error that tends to oppose the hidden error entering each block.
  • A RELATED WORK: The related-work discussion organizes quantization error into its introduction, propagation through Transformer blocks, and effects on model outputs.This framework connects PTQ optimization methods with mechanistic analyses of hidden-state and output behavior.
  • A.1 PTQ OPTIMIZATION AND LOW-PRECISION FORMATS: Prior PTQ research mainly frames weight-only quantization as an optimization problem, including second-order reconstruction in GPTQ and activation-aware scaling in AWQ.Successors extend calibration through learned clipping, equivalent transformations, block-level and cross-block reconstruction, asymmetric full-precision-output targets, and alternating error correction.
  • A.1 PTQ OPTIMIZATION AND LOW-PRECISION FORMATS: Microscaling research finds that scale construction and format–algorithm compatibility are critical at 4-bit precision, while NVFP4 combines fine-grained scaling with methods for stable end-to-end low-precision training (Zhang et al., 2026a; Meng et al., 2026; NVIDIA et al., 2025; Chen et al., 2026; Panferov et al., 2026).The paper uses NVFP4 round-to-nearest quantization, though the supplied passage truncates before further details.
  • A.2 TRANSFORMER BLOCK RESPONSES TO QUANTIZATION: Prior work links quantization failures to residual magnitudes, late-layer activations, and MLP gates, and links training-trajectory robustness to learning-rate dynamics and other hyperparameters (Catalan-Tatjer et al., 2026).Broader accounts connect parameter-space robustness to training noise and flat minima.
  • A.2 TRANSFORMER BLOCK RESPONSES TO QUANTIZATION: Quantization Error Propagation (QEP) (Arai & Ichikawa, 2025) carries upstream quantization error into layer calibration, while Model-Preserving Adaptive Rounding (Tseng et al., 2026) targets approximate end-to-end output error.These approaches address error propagation during optimization because local activation error can poorly proxy the final distribution.
  • A.2 TRANSFORMER BLOCK RESPONSES TO QUANTIZATION: Under quantization, block-update errors tend to point against the hidden error already present at the block input, producing a stronger negative interaction than that between the hidden state and block update.This finding is presented as a mechanistic parallel to opposing residual contributions across layers in full-precision Transformers (Patrawala et al., 2025).

A.3 OUTPUT ROBUSTNESS UNDER QUANTIZATION … C.3 LM-HEAD SCORE DECOMPOSITION

The paper explains quantization robustness by tracing hidden-state and output changes on shared prefixes, formalizing hidden-error growth and LM-head score changes. Its analysis highlights residual-error counteraction and the separate effects of input length and projection angle on token scores.

  • A.3 OUTPUT ROBUSTNESS UNDER QUANTIZATION: Shared-prefix comparisons trace how quantization changes hidden states, token scores, and output probabilities at each next-token prediction.This isolates quantization effects without allowing the original and quantized models to follow different generated histories.
  • A.3 OUTPUT ROBUSTNESS UNDER QUANTIZATION: The LM head maps final hidden states to token scores by projecting them onto rows that function as output word embeddings.This output geometry motivates analyzing how hidden-state perturbations affect logits and probabilities.
  • B NOTATION: The analysis defines BF16 and NVFP4 model passes, hidden states, block updates, hidden errors, and relative hidden-error norms for quantization comparisons.The recurrence tracks the hidden error entering each subsequent decoder block and distinguishes added-error, interaction, and alignment terms.
  • C.1 HIDDEN-ERROR RECURRENCE PROOFS: Residual hidden error follows ∆h(ℓ) = ∆h(ℓ−1) + ∆u(ℓ), so its squared norm contains an interaction term 2⟨∆h(ℓ−1), ∆u(ℓ)⟩.The theorem decomposes relative error growth into added-error, interaction, and clean-input–update alignment contributions.
  • C.2 LENGTH–ANGLE DECOMPOSITION: The theoretical analysis decomposes hidden-state discrepancies into norm and angular components, including an equal-length identity when the two hidden states have equal norms.The length–angle decomposition follows directly from the cosine definition and separates magnitude from directional effects.
  • C.3 LM-HEAD SCORE DECOMPOSITION: LM-head score changes split exactly into a length part and an angle part determined by the original and quantized LM-head input geometry.This decomposition clarifies what analytic references assuming ρ = 1 omit.
  • C.3 LM-HEAD SCORE DECOMPOSITION: If the quantized LM-head input is a positive scalar multiple of the original, every token score is scaled equally and pairwise score ordering is unchanged.Specifically, if bhLM = chLM with c > 0, then bz = cz.

C.4 HIGH-DIMENSIONAL ROTATION ANALYSIS … D.1 NVFP4 QUANTIZATION FORMAT

The appendix formalizes how high-dimensional LM-head rotations attenuate score changes, how log-probability perturbations are approximated, and when token rankings remain stable. It also specifies the NVFP4 quantization procedure used in experiments.

  • C.4 HIGH-DIMENSIONAL ROTATION ANALYSIS: In high dimensions, the directional coordinate q_k has typical magnitude O(d^-1/2), producing attenuation in the LM-head projection-angle change.The exact identity is cos bθ_k = cos θ_k cos α + sin θ_k sin α q_k.
  • C.4 HIGH-DIMENSIONAL ROTATION ANALYSIS: The rotation analysis derives exact and finite-angle expressions for expected absolute relative score changes, with second-order argument approximations incurring O(α^3) error.These results follow from the symmetric distribution of q_k and its beta-function characterization.
  • C.5 LOG-PROBABILITY APPROXIMATION PROOF: The log-probability proof expands centered logit perturbations to obtain second-order and first-order approximations under uniformly small perturbations.Centering uses m := E_j∼p Δz_j and x_j := Δz_j − m, with E_j∼p x_j = 0.
  • C.6 SOFTMAX AND TOKEN-RANKING STABILITY: Token-preservation conditions reduce top-1 and top-K stability to comparisons between original score margins and differential quantization errors.The top-K condition requires z_a − z_j > Δz_j − Δz_a for every a ∈ S_K and j ∉ S_K.
  • C.6 SOFTMAX AND TOKEN-RANKING STABILITY: For Qwen3-32B under NVFP4, Flip@1 is 10.7% on C4, 12.7% on WikiText-103, and 8.3% on GSM8K text.Flip@1 measures changes in relative scores sufficient to overturn the original top-1 prediction, not arbitrary score changes.
  • D.1 NVFP4 QUANTIZATION FORMAT: NVFP4 uses 4-bit E2M1 codes with tensor-level and 16-entry group-level E4M3 scales, clipping reconstructed values to the E2M1 range [−6, 6].The format has qmax = 6 and fmax = 448; final groups may be zero-padded before quantization.
  • D.1 NVFP4 QUANTIZATION FORMAT: Experiments apply round-to-nearest NVFP4 to Transformer linear weights while retaining embeddings, normalization layers, activations, and the LM head at original precision.For MoE models, expert gate, up, and down projections are quantized but routers are not; reconstructed weights are recast before each linear operation.

D.2 NVFP4 WEIGHT RECONSTRUCTION ERROR · D.3 REPRODUCTION DETAILS · E ADDITIONAL EXPERIMENTAL ANALYSIS

Pretrained and randomly initialized models have nearly identical NVFP4 weight reconstruction errors, so matrix-level reconstruction alone does not explain their different hidden-error growth. The experiments use broad model, dataset, quantization, and evaluation protocols detailed in the reproduction settings.

  • D.2 NVFP4 WEIGHT RECONSTRUCTION ERROR: Both pretrained and randomly initialized weights have reconstruction cosines of about 0.9955 and relative errors of about 9.5%, despite differing hidden-error growth.These nearly identical matrix-level errors indicate that weight reconstruction error alone does not account for the models’ different hidden-state behavior.
  • D.3 REPRODUCTION DETAILS: Token-level experiments use C4, WikiText-103, and GSM8K text, typically with 64 sequences and 512 next-token positions per sequence.Cross-model recurrence analyses use 128 sequences per model and dataset, while PTQ comparisons use eight C4 inputs.
  • D.3 REPRODUCTION DETAILS: Benchmark evaluation covers ARC-Challenge, ARC-Easy, MMLU, HellaSwag, WinoGrande, and TruthfulQA using full test or validation splits in zero-shot settings without chat templates.Scoring uses task-specific likelihood or answer-selection procedures described in the reproduction protocol.
  • D.3 REPRODUCTION DETAILS: The study evaluates Qwen3, OLMo3, OLMoE, Gemma3, and Pythia models, using Qwen3-32B as the primary model for hidden-state and output analyses.The model set includes pretrained checkpoints and, for selected comparisons, random initializations and released OLMo3-7B Stage-1 checkpoints.
  • D.3 REPRODUCTION DETAILS: Unless stated otherwise, post-training quantization uses weight-only NVFP4 RTN for attention and MLP projections, while embeddings, normalization, activations, and the LM head retain original precision.GPTQ and AWQ use 64 disjoint C4 calibration inputs, whereas RTN uses no calibration data.
  • D.3 REPRODUCTION DETAILS: Activation quantization experiments retain full-precision weights and quantize each projection input using E2M1 values, E4M3 block scales, and contiguous groups of 16 values.The tensor-wide scale is recomputed on every forward pass before applying one E4M3 scale per group.
  • D.3 REPRODUCTION DETAILS: Recurrence analyses exclude the one-token-prefix boundary and filter rare extreme hidden-norm differences, retaining 99.20–99.97% of positions across cross-model and cross-dataset settings.The filtering removes 0.08% of Qwen3-32B C4 positions; plotted standard deviations are computed separately for each recurrence term.
  • D.3 REPRODUCTION DETAILS: Figures 3–5 compute position-level decompositions and LM-head geometry statistics over complete inputs, with bootstrap benchmark errors and analytic first- and second-order references evaluated at observed quantities.The random reference uses the same aggregation over initialization–input pairs, and the analytic curves are not fitted to W4 measurements.

E.1 HIDDEN-ERROR GROWTH AT INITIALIZATION AND AFTER PRETRAINING · E.2 HIDDEN-ERROR RECURRENCE ACROSS MODELS · E.3 HIDDEN-ERROR ANALYSIS: ABNORMAL TRAJECTORY FILTERING

Pretraining substantially slows hidden-error growth and induces recurrence terms that counteract newly introduced quantization error across models and datasets. Rare hidden-state norm divergences can dominate unfiltered averages, but filtering them preserves the main statistics while revealing weaker counteraction in those failure cases.

  • E.1 HIDDEN-ERROR GROWTH AT INITIALIZATION AND AFTER PRETRAINING: Pretraining makes second-half relative hidden-error growth 3.1× slower than random initialization in Qwen3-32B, with final hidden-error magnitude and relative error 5.5× and 6.7× smaller.Over blocks 33–64, the relative-error increase is 1.46 × 10^-3 after pretraining versus 4.45 × 10^-3 at random initialization.
  • E.1 HIDDEN-ERROR GROWTH AT INITIALIZATION AND AFTER PRETRAINING: Random initialization increases final relative hidden error by 7.9× in Qwen3-8B, 5.9× in OLMo3-32B, and 4.0× in Gemma3-4B under the same NVFP4 conversion.The comparison spans three architecture families and uses the precision settings specified for each model family in Fig. 9.
  • E.2 HIDDEN-ERROR RECURRENCE ACROSS MODELS: The recurrence generalizes across five pretrained dense and mixture-of-experts models on C4, WikiText-103, and GSM8K: negative T_inter offsets positive T_add, while T_align provides further reduction.The signs T_add > 0 and T_inter < 0 remain unchanged across tested trajectory-filter thresholds, although contribution sizes vary by model and dataset.
  • E.3 HIDDEN-ERROR ANALYSIS: ABNORMAL TRAJECTORY FILTERING: Without filtering, the fraction of T_add canceled by T_inter falls from 50.4% to 30.4%, showing that rare trajectories with sharply diverging BF16 and W4 norms weaken counteraction.These abnormal trajectories are uncommon but can dominate averages of signed recurrence terms despite the recurrence remaining exact at every position.
  • E.3 HIDDEN-ERROR ANALYSIS: ABNORMAL TRAJECTORY FILTERING: A default hidden-norm-gap filter retains 99.72% of positions and changes mean final relative hidden error only from 0.148 to 0.147, removing rare scale failures without changing the main statistic.The filter retains positions with gmax_i ≤ 0.5 and applies the same retained positions at every layer.
  • E.3 HIDDEN-ERROR ANALYSIS: ABNORMAL TRAJECTORY FILTERING: The LM-head-input norm remains statistically stable without filtering: the reported values are only 0.31% and 1.27% higher, supporting the approximation ∥bh_LM∥2/∥h_LM∥2 ≈ 1.The values are averaged equally over C4, WikiText-103, and GSM8K text.

E.4 HIDDEN-ERROR GROWTH: LATE-LAYER DIFFERENCES ACROSS DATA · E.5 COUNTERACTION EMERGENCE DURING PRETRAINING

Data-dependent update-to-hidden norm ratios explain much of Qwen3-32B’s faster late-layer hidden-error growth on GSM8K text, while oracle coefficient matching reduces that growth. Across OLMo3-7B and Pythia checkpoints, counteraction strengthens during pretraining from near-zero to broadly negative error interactions.

  • E.4 HIDDEN-ERROR GROWTH: LATE-LAYER DIFFERENCES ACROSS DATA: Qwen3-32B’s GSM8K and C4 hidden-error trajectories remain close early but separate sharply in later blocks, with faster growth on GSM8K.The difference occurs over blocks 36–64, where the relevant coefficient is larger on GSM8K.
  • E.4 HIDDEN-ERROR GROWTH: LATE-LAYER DIFFERENCES ACROSS DATA: The update-to-hidden norm ratio is larger on GSM8K because comparable block updates act on smaller hidden states, increasing relative hidden-error growth when the recurrence term is positive.Figure 13 links the larger ratio, faster relative hidden-error growth, and its reduction after coefficient matching.
  • E.4 HIDDEN-ERROR GROWTH: LATE-LAYER DIFFERENCES ACROSS DATA: The GSM8K intervention isolates the coefficient by rescaling only residual additions to C4 values while keeping inputs and internal attention and MLP computations fixed.This oracle intervention tests the coefficient’s contribution rather than output quality.
  • E.4 HIDDEN-ERROR GROWTH: LATE-LAYER DIFFERENCES ACROSS DATA: 48.8%: Matching GSM8K’s late-layer coefficient to C4 values reduces final relative hidden error from 0.121 to 0.062.The intervention matches coefficients over blocks 36–64 while leaving internal attention and MLP computations unchanged.
  • E.5 COUNTERACTION EMERGENCE DURING PRETRAINING: At random initialization, block-input and block-update hidden-error interactions are nearly zero, whereas pretrained Qwen3-32B exhibits negative interactions that offset update error and slow hidden-error growth.The OLMo3-7B Stage-1 checkpoints show this interaction changing from weak at step 0 toward negative during pretraining.
  • E.5 COUNTERACTION EMERGENCE DURING PRETRAINING: In Pythia-1.4B and Pythia-2.8B, the cosine between block-input and block-update errors changes from near-zero at step 0 to broadly negative at the final checkpoint.Together with OLMo3, the curves show counteraction becoming more pronounced at later checkpoints.

E.6 COUNTERACTION SOURCE: BLOCK-INPUT-ERROR RESPONSE … E.9 COUNTERACTION ACROSS PTQ ALGORITHMS AND WEIGHT FORMATS

Across models, quantization counteraction arises mainly from blocks responding oppositely to inherited hidden error, not from direct weight perturbations. Interventions and cross-quantization comparisons show that counteraction limits hidden-error growth broadly, while final output quality also depends on newly introduced error and LM-head effects.

  • E.6 COUNTERACTION SOURCE: BLOCK-INPUT-ERROR RESPONSE: Counteraction is specific to quantization-error dynamics: cos ∠(∆h(ℓ−1), ∆u(ℓ)) is negative in 82.5% of blocks, versus positive native hidden-state/update cosines in 65.6% and 64.1%.The negative interaction therefore does not reflect a general property of residual updates.
  • E.6 COUNTERACTION SOURCE: BLOCK-INPUT-ERROR RESPONSE: Under either exact decomposition, counteraction is carried mainly by the block’s response to input hidden error, which opposes accumulated error and slows its propagation.The direct weight effect contributes a nonzero update error but has no consistent direction relative to inherited hidden error.
  • E.7 COUNTERACTION INTERVENTION: REMOVAL AND REVERSAL: The intervention preserves each intervened block’s ∥∆u(ℓ)∥2 while removing makes the applied error orthogonal to input error and reversal flips only negative interactions over the selected block intervals.Qwen3-32B interventions cover blocks 17–48, whereas Qwen3-8B interventions cover blocks 6–23.
  • E.7 COUNTERACTION INTERVENTION: REMOVAL AND REVERSAL: Removing counteraction nearly eliminates cumulative Tinter, increases Tadd by 4.3×, and raises final E[(R(L))2] by 8.4× in Qwen3-32B.The same intervention in Qwen3-8B increases Tadd by 2.7× and final E[(R(L))2] by 4.8×, showing that counteraction limits error growth through direct interaction and by preventing larger subsequent Tadd.
  • E.8 COUNTERACTION ACROSS WEIGHT AND ACTIVATION QUANTIZATION: Across W4, A4, and W4A4, cumulative Tinter is negative in every setting, showing that block-update error counteracts inherited hidden error for weight, activation, and joint quantization.Despite similar counteraction curves, output errors differ because settings introduce different amounts of error and propagate it differently to the LM head.
  • E.9 COUNTERACTION ACROSS PTQ ALGORITHMS AND WEIGHT FORMATS: Across RTN, GPTQ, and AWQ with NVFP4 and asymmetric INT4, Tinter remains negative and blockwise geometry changes little, while Tadd, final relative error, and output metrics vary.For NVFP4, GPTQ and AWQ interaction-cosine curves correlate above 0.998 with RTN; stronger cumulative Tinter does not imply better PTQ quality.

E.10 OUTPUT ROBUSTNESS TO QUANTIZATION ACROSS MODELS · E.11 OUTPUT PROBABILITY SENSITIVITY TO QUANTIZATION ACROSS TEMPERATURES

Across pretrained Qwen, OLMo, and Gemma models, W4 quantization causes modest output changes while preferentially preserving top-ranked token scores and probabilities. This rank-dependent robustness persists across temperatures because score rankings and LM-head geometry remain fixed, although probability metrics vary.

  • E.10 OUTPUT ROBUSTNESS TO QUANTIZATION ACROSS MODELS: Higher-ranked tokens have larger LM-head cosine values, and their relative score and absolute log-probability changes are generally smaller than those of lower-ranked tokens.The rank trend is observed across models and datasets, with local fluctuations and magnitudes varying by model.
  • E.10 OUTPUT ROBUSTNESS TO QUANTIZATION ACROSS MODELS: Across representative pretrained models, macro-mean ΔCE is 0.006–0.017, relative ΔCE is 0.24%–0.57%, and forward KL is 0.019–0.060.These values are macro-means over C4, WikiText-103, and GSM8K text, comparing W4 with BF16.
  • E.10 OUTPUT ROBUSTNESS TO QUANTIZATION ACROSS MODELS: Across six dense and mixture-of-experts models, LM-head input rotations of 5.80°–21.71° produce vocabulary-mean projection-angle changes of only 0.093°–0.391°.The attenuation observed for Qwen3-32B appears in every tested model across C4, WikiText-103, and GSM8K.
  • E.10 OUTPUT ROBUSTNESS TO QUANTIZATION ACROSS MODELS: The measured rank trends are reproduced by directional and probability-space approximations, while the ρ = 1 references isolate measured LM-head input rotation from norm change.The approximations follow the transition from top ranks to the rest of the vocabulary, despite model-dependent fluctuations.
  • E.11 OUTPUT PROBABILITY SENSITIVITY TO QUANTIZATION ACROSS TEMPERATURES: With paired BF16/W4 score vectors fixed, changing temperature preserves score rankings, top-K sets, Flip@1, and LM-head angles, but changes probability metrics.The analysis varies only the softmax temperature over T ∈ {0.5, 0.75, 1, 1.5, 2}, without recomputing hidden states or LM-head projections.
  • E.11 OUTPUT PROBABILITY SENSITIVITY TO QUANTIZATION ACROSS TEMPERATURES: Mean forward KL across tested temperatures spans 0.0247–0.0661 on C4 and 0.1514–0.3253 on GSM8K text.Figure 26 also reports absolute log-probability changes by BF16-model rank and temperature.
Loading 2609.11716v1…