PaperlayerView exact revision on arXiv

Source-linked AI summary

Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantization

Samy Houache, Yann Traonmilin, Jean-François Aujol

arXiv:2610.09877v1cs.LG

TL;DR

Mixed-precision post-training quantization must choose layerwise precisions under a memory budget despite combinatorial allocation and calibration-data sensitivity. The paper uses probabilistic layerwise error analysis to derive local scores for solver-free greedy allocation. On DRUNet it reports competitive clean-calibration quality, gains up to 7.5 dB under tested corruptions, and allocation speed-ups of 28× to 2,570×; it also reports results on latent diffusion.

  • Problem

    Mixed-precision allocation is combinatorial, and calibration-set quality and composition can affect the quantized model.

    Source: §1 · p. 1, §3 · p. 4

  • Method

    The method separates propagated and local quantization error probabilistically, then uses the local term for separable scores and solver-free greedy bit-width allocation.

    Source: §1 · p. 2

  • Results

    On DRUNet, allocation is 28× to 2,570× faster than studied baselines and gains reach 7.5 dB under tested corruptions; latent diffusion results are also reported.

    Source: §1 · p. 2, §7 · p. 10

  • Takeaways & Limitations

    The same local score and greedy rule provide competitive quality under clean calibration, robustness under tested corruptions, and application to latent diffusion.

    Source: §1 · p. 2, §7 · p. 10

    The layerwise recursion assumes a probabilistic upper bound for the previous quantized activations, justified when the algorithm selects configurations with low quantization error.

    Source: §5 · p. 6

Abstract

from arXiv · show

Mixed-precision post-training quantization is a network compression method that assigns bits layer by layer, under a global memory budget using a small calibration set. The main difficulties are to overcome the combinatorial nature of the allocation problem and to manage the sensitivity to small, potentially corrupted databases. Hence, an efficient allocation method should be fast to compute and preserve model quality when calibration data are corrupted. To design such a method, we derive a layerwise probabilistic analysis of the quantization error that separates propagated error from the local perturbation introduced at a given layer. We use this local term to build a separable score for a simple allocation algorithm, that requires no external solver. The probabilistic nature of our approach brings robustness to corrupted data. On denoising tasks with DRUNet, with an average budget of 4 bits per weight, our method matches or improves state-of-the-art mixed-precision baselines under clean calibration, and is more robust to corrupted calibration, with PSNR gains of up to 7.5 dB under the tested corruptions. Experiments show bit-allocation speed-ups from 28x to 2,570x over the studied baselines. For quantized diffusion models, our experiments show that a direct application of our framework also improves the state-of-the-art.

1 INTRODUCTION

Mixed-precision PTQ must balance model quality against memory and energy budgets while choosing layerwise bit-widths efficiently and robustly to degraded calibration data. The paper develops a probabilistic layerwise error analysis and a solver-free allocation method, reporting competitive quality, corruption robustness, and faster allocation.

Motivation

  • Mixed-precision PTQ assigns varying layer bit-widths under a memory budget because layers have different quantization sensitivities.The flexibility can improve the quality–memory trade-off at a fixed budget.Source: §1 · p. 1
  • With 36 layers and six candidate bit-widths per layer, there are about 10^28 assignments before memory constraints, making exhaustive evaluation impractical.Allocation must also be fast to repeat and preserve model quality when calibration inputs are degraded.Source: §1 · p. 2

Contributions

  • The analysis separates error propagated from earlier layers from the perturbation introduced locally, and also derives a global bound for modern architectures including self-attention.Its probabilistic bounds generalize prior worst-case deterministic analysis.Source: §1 · p. 2
  • A separable score from one full-precision pass per calibration image feeds a simple greedy bit-width assignment without an external solver.The score is based on the local term of the layerwise analysis.Source: §1 · p. 2
  • On DRUNet image denoising, the method achieves competitive quality under clean calibration and stable PSNR under both tested calibration corruptions at target budgets, while allocation is 28× to 2,570× faster than studied baselines.The same score and greedy rule transfer to latent diffusion, yielding competitive FID and the highest mean PSNR to FP32 in the paired comparison.Source: §1 · p. 2

2 RELATED WORK

Prior work analyzes quantization error, estimates layer sensitivity, and allocates bit-widths using varied scores and search procedures. This paper distinguishes its approach through probabilistic local error scores, fixed across budgets, and a solver-free greedy allocation.

Error analysis for quantized networks

  • Probabilistic rounding-error analyses use moments rather than worst-case accumulation; this work retains separate global and layerwise bounds, with the latter exposing local error for allocation.Gonon et al. (2023) use a global worst-case operator norm, while Houache et al. (2026) use layer-dependent norms; QEP reduces earlier-layer error by correcting weights before quantization.Source: §2 · pp. 2–3

Sensitivity-driven mixed precision

  • Existing mixed-precision methods vary in sensitivity score and budget solver, while this method uses local error scores and a solver-free greedy rule with fixed, reusable scores.Earlier greedy PTQ updates sensitivity after each decision; matched measurements report speed-ups of 28× to 2,570× over studied baselines.Source: §2 · p. 3

Calibration and quantization backends

  • PTQ depends on a small calibration set whose size and composition can affect the model; DRUNet comparisons use RTN or TF-Enhanced backends without AdaRound and corrupt only calibration inputs for stability tests.AdaRound is used in the diffusion experiment.Source: §2 · p. 3

Quantization of diffusion models

  • Diffusion PTQ must account for changing distributions across denoising steps; this work applies its DRUNet local score and greedy rule to a pretrained latent diffusion model using the Q-Diffusion/AdaRound backend.Q-Diffusion and PTQ4DM adapt calibration and reconstruction, PTQD models quantization noise, TFMQ-DM preserves temporal features, and Qua2SeDiMo learns sensitivity estimates.Source: §2 · p. 3

3 SETUP AND NOTATION

The setup defines quantization error layer by layer and formulates robust mixed-precision allocation as minimizing a high quantile of final output error under a memory budget. The calibration-set objective is difficult to solve directly because it depends on the full bit-width configuration and the number of configurations grows exponentially.

Setup and notation

  • The output error decomposes into propagated error from earlier layers and local quantization error introduced at the current layer.The network uses full-precision activations Sℓ and quantized activations S′ℓ, with quantized weights W′ℓ obtained using the selected quantizer.Source: §3 · p. 3
  • The random amplification ratios track how much propagated and local error are amplified on the current input, while operator norms bound their worst-case amplification.The analysis uses moments of these ratios for a probabilistic approach.Source: §3 · p. 4

Mixed-precision allocation problem

  • The allocation objective minimizes Qτ(b), a τ-quantile of final output error, over configurations whose weighted bit-width cost fits the memory budget.The target average bits per weight B gives a budget equivalent to uniform B-bit PTQ.Source: §3 · p. 4
  • Using a quantile rather than worst-case error permits greater robustness to outliers in available datasets.Here τ is close to one, and δ = 1 − τ is the corresponding tail level.Source: §3 · p. 4
  • The ideal objective depends on the full bit-width configuration, each test requires a full-network evaluation, and configuration counts grow exponentially with layer count.The framework first analyzes fixed-configuration output error, then tracks error by layer to produce a practical allocation score.Source: §3 · p. 4

4 PROBABILISTIC ERROR BOUNDS FOR MIXED-PRECISION PTQ

The section first bounds output error for a fixed configuration, then develops a layerwise probabilistic bound that preserves where errors arise and how they propagate.

4.1 A GLOBAL BOUND FOR A FIXED CONFIGURATION

  • The global bound estimates final output error for a fixed configuration, but does not identify which layers should receive higher bit-widths.Its probabilistic step uses the one-sided Chebyshev inequality; calibration estimates of the bound’s moments do not automatically inherit the theorem’s guarantee.Source: §4 · p. 4, §4.1 · pp. 4–5

4.2 A LAYERWISE BOUND FOR MIXED-PRECISION QUANTIZATION

  • The layerwise bound keeps propagated error separate from each layer’s local perturbation, yielding a probabilistic upper bound on final output error.It uses conditional concentration at each layer and combines the bounds through a union bound.Source: §4.2 · p. 5
  • Exact propagated contributions across joint bit-width configurations remain expensive to compute, so the allocation algorithm retains the local perturbation identifying each source layer.The section uses this local quantity to connect the layerwise analysis to practical allocation.Source: §4.2 · p. 5

5 AN ALGORITHM FOR FAST AND ROBUST MIXED-PRECISION PTQ

The algorithm scores candidate bit-widths using local quantization perturbations evaluated on full-precision activations, then greedily reduces precision to meet the budget.

Local score construction

  • One full-precision pass per calibration image supplies layer inputs for evaluating each block’s local perturbation at every candidate bit-width.Using full-precision inputs approximates quantized inputs along the low-error optimization path and makes each score independent of other blocks’ bit-widths.Source: §5 · pp. 6–7
  • Layers are grouped into allocation blocks that share a bit-width, reducing the number of decisions and defining the practical memory cost.Each block’s weight count contributes to the total cost as its weight count multiplied by its assigned bit-width.Source: §5 · p. 6
  • Each block-and-bit-width score combines the calibration-set mean local perturbation with a standard-deviation penalty for variation across images.The parameter δscore sets the dispersion term’s weight; it is not a confidence level for the selected allocation.Source: §5 · p. 6

Greedy allocation

  • Starting at the largest candidate bit-widths, the greedy allocator chooses the block with the smallest score increase per weight bit saved until the budget is met.The allocation stage reuses precomputed scores and requires no new network evaluation or external solver.Source: §5 · p. 7

6 EXPERIMENTS

Experiments test DRUNet denoising and latent-diffusion generation, including clean and corrupted calibration, allocation speed, and transfer. The method is competitive under clean calibration, more stable under tested corruptions, and faster than the studied baselines.

6.1 EXPERIMENTAL SETUP

  • The experiments cover DRUNet denoising and latent-diffusion generation, with clean calibration, calibration-only corruption, runtime, and transfer evaluations.DRUNet uses three calibration sets of 256 images and the same 100 clean test images; corruption tests keep the test set unchanged.Source: §6 · p. 7, §6.1 · pp. 7–8

6.2 EXPERIMENTS WITH CLEAN CALIBRATION DATASET

  • At B = 4, our method improves OMPQ-ORM by 1.814 dB and is 0.008 dB below CLADO on clean DRUNet denoising.Table 1 reports clean RTN results averaged over three calibration sets and the same 100 test images.Source: §6.2 · p. 8, Table 1 · p. 8
  • With the TF-Enhanced backend, our method is 0.010 dB higher than AIMET at B = 5, with a 95% confidence interval of [0.007, 0.013] dB.At B = 7, the difference is −0.0003 dB, with interval [−0.0011, 0.0005] dB.Source: §6.2 · p. 8, Table 2 · p. 8

6.3 STABILITY UNDER CALIBRATION CORRUPTION

  • Under the tested calibration corruptions, our method gains up to 7.509 dB over CLADO at B = 5 with shared corruption.At B = 4, clean-to-corrupted bit-width rank correlations are 0.961 and 0.987 for our method, versus 0.060 and 0.313 for CLADO under shared and per-channel corruption.Source: §6.3 · p. 9, Table 3 · p. 9

6.4 MEASURED COST

  • Allocation is 28× to 2,570× faster than the studied baselines in matched DRUNet timing comparisons.With the same W3/W4/W8 candidates, it is 2,570× faster than CLADO; timing covers score computation and allocation.Source: §6.4 · p. 9

6.5 LATENT-DIFFUSION TRANSFER

  • For latent diffusion, our model obtains FID 20.853 versus Q-Diffusion’s 22.626 and the highest mean PSNR to FP32, 34.644 dB.The PSNR comparison uses 50,000 outputs paired by generation seed; uniform-W4 TFMQ-DM has mean PSNR to FP32 of 24.545 dB.Source: §6.5 · p. 9

7 DISCUSSION AND CONCLUSION

The conclusion links local probabilistic error scores and greedy allocation to fast, corruption-robust mixed-precision PTQ. It also identifies software support for embedded deployment and tighter probabilistic bounds as open issues.

7 DISCUSSION AND CONCLUSION

  • The method separates propagated error from local perturbation and turns local terms into reusable block-and-bit scores for greedy allocation.The conclusion reports competitive clean-calibration quality, gains up to 7.5 dB under tested corruptions, and allocation speed-ups from 28× to 2,570×.Source: §7 · p. 10
  • Embedded implementation remains a major issue because current software has limitations in managing mixed-precision models.Improving the layerwise probabilistic bounds is also left open.Source: §7 · p. 10

AI USE STATEMENT

  • The authors state that generative AI assisted English editing and literature/code discovery, while the mathematical framework, proofs, algorithm, and core pipeline were authored without AI implementation assistance.AI code assistance was limited to general-purpose utilities such as retrieving layer names and counting parameters.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The paper frames ethical implications as dependent on the underlying models, datasets, and applications, requiring safeguards for each application.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • Appendix A provides proofs; Algorithm 1 describes allocation, and Appendix B reports datasets, protocols, quantization settings, and runtimes.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 PROOF OF THEOREM 1 (LAYERWISE BOUND)

  • The layerwise-bound proof invokes a bounded input domain and proceeds through conditional moments and induction across layers.The assumption states that B0 is deterministic and P(∥x∥2 ≤ B0) = 1.Source: A PROOFS · p. 14, §A.1 · pp. 14–15

A.2 PROOF OF LEMMA 1 (GLOBAL BOUND)

  • The global-bound proof combines deterministic domination with one-sided Chebyshev; its strict inequality handles the zero-variance case.Source: §A.2 · p. 16

A.3 PROOF OF THE PROPAGATION RULES OF TABLE 4

The appendix proves propagation rules for deterministic and probabilistic error bounds on computation graphs, then examines how calibration-sample changes affect empirical score functionals. The DAG treatment has a stated self-attention scope boundary, and the one-sample sensitivity calculation does not guarantee allocation stability.

A.3 PROOF OF THE PROPAGATION RULES OF TABLE 4

  • Table 4 rules propagate parent error bounds through DAG operations, and topological induction establishes ∥∆v∥2 ≤ Gv; attention uses an exact error decomposition without a global Lipschitz property.The listed operations include residual addition, nearest-neighbor upsampling, and self-attention.Source: Table 4 · p. 17, §A.3 · pp. 17–18

A.4 A LAYERWISE BOUND ON DAGS

  • The DAG extension establishes nodewise probabilistic bounds by topological induction with deterministic output bounds; shared ancestors appear only once when combining parents, while Lemma 2 excludes self-attention and experiments handle attention with Lemma 1.Source: §A.4 · pp. 18–20

A.5 SENSITIVITY OF THE EMPIRICAL FUNCTIONALS

  • At n=256 and δ_score=0.05, one-sample score displacement is about 0.49M versus M for the maximum, but this is not a stability guarantee for selected allocations.The score displacement is dominated by the standard-deviation term scaled by 1/√δ_score.Source: §A.5 · p. 20

B.1 DRUNET IMPLEMENTATION DETAILS

The DRUNet appendix reports matched implementation and timing controls alongside comparisons of quality and allocation stability under calibration corruption. Its controls use shared candidates to test whether the observed robustness depends on the candidate bit-width set.

B.1 DRUNET IMPLEMENTATION DETAILS

  • The DRUNet setup uses 36 allocation blocks containing 64 convolutions, three calibration sets of 256 images, and a fixed 100-image test set.Activations remain FP32, and each baseline comparison shares its block partition and parameter-weighted budget.Source: §B.1 · p. 21, Table 5 · p. 21

B.2 CLEAN OUTPUT-SQNR CONTROL FOR DIFFERENT BUDGETS

  • The output-SQNR control quantizes one block at a time, compares the final network output with FP32, and passes negative SQNR to the shared greedy allocation rule.The control adapts Pandey et al. (2023) to the DRUNet blocks and weight-only RTN backend.Source: §B.2 · p. 21

B Ours Output-SQNR OMPQ-ORM CLADO

  • At target B = 4, the same-candidate CLADO control finds nearly identical clean PSNR, while our allocation keeps its PSNR under corruption and CLADO loses more than 5 dB.The control tests whether the corruption result comes from allowing intermediate bit-widths.Source: B Ours Output-SQNR OMPQ-ORM CLADO · p. 22, Table 8 · p. 22

B.4 MEASURED SELECTION TIME

  • The selection-time benchmark measures the complete mixed-precision selection stage, excluding loading, data preparation, quantization application, and PSNR evaluation.A separate layerwise timing totals 24.32 seconds on one A100, using the median over three calibration seeds.Source: §B.4 · p. 22

B.5 COMPLETE DRUNET CURVES AND ALLOCATION STABILITY

  • With identical W3/W4/W8 candidates, our method achieves much higher PSNR than CLADO under both corruptions and changes fewer blocks while better preserving their ordering.The stability measures are averaged over three calibration seeds.Source: Figure 1 · p. 23, Figure 2 · p. 23
  • The adapted OMPQ-ORM comparison shows higher PSNR for our method at intermediate budgets; against AIMET, our method has an advantage at intermediate corrupted-calibration budgets.OMPQ-ORM often changes fewer block widths, but its ordering is less stable under per-channel corruption at higher budgets; our allocation is more stable than AIMET under both corruptions.Source: Figure 3 · p. 24, Figure 4 · p. 24, Figure 5 · p. 25

B.6 FIXED-RANDOM QUALITATIVE EXAMPLES

  • Fixed-random examples show our outputs close to FP32 in the CLADO and OMPQ-ORM comparisons, while both methods remain visually close to FP32 in the AIMET examples.The CLADO examples use B = 4.5 with shared corruption; the OMPQ-ORM examples use clean calibration at B = 4, and the AIMET examples use shared corruption at B = 5.Source: Figure 7 · p. 26, Figure 8 · p. 27, Figure 9 · p. 27

B.7 LATENT-DIFFUSION PROTOCOL AND METRIC SCOPE

The latent-diffusion appendix applies the DRUNet allocation score and greedy rule to a CelebA-HQ model and describes its calibration and evaluation protocol. It distinguishes distribution-level FID from paired fidelity to FP32 outputs.

B.7 LATENT-DIFFUSION PROTOCOL AND METRIC SCOPE

  • The pretrained CompVis CelebA-HQ latent diffusion model uses the Q-Diffusion/AdaRound reconstruction backend, with the local score and greedy allocation rule unchanged from DRUNet.The U-Net has 123 quantized weight tensors grouped into 48 allocation blocks; allocation uses W3–W8 and reaches a parameter-weighted average of 3.9932 bits.Source: §B.7 · p. 28
  • The evaluation generates 50,000 images per method with DDIM-200 and η = 0, using seeds 40 and 41 and the same historical 25,000-image reference.The methods share 5,120 calibration timestep–latent pairs, comprising 256 pairs at each of 20 timesteps.Source: §B.7 · p. 28
  • FID compares generated distributions with the CelebA-HQ reference, whereas paired PSNR measures each quantized output against its corresponding FP32 output.The paired PSNR distributions shift toward higher values, indicating closer similarity to paired FP32 outputs; the displayed samples were selected by a fixed random rule.Source: §B.7 · p. 28, Figure 10 · p. 29, Figure 11 · p. 30

B.8 ADDITIONAL NATURAL-IMAGE RESULTS

The natural-image experiments repeat DRUNet allocation comparisons on BSDS500 and its CBSD68 subset under clean and corrupted calibration. The reported comparisons show strong quality under clean calibration and an advantage over CLADO under both tested corruptions.

B.8 ADDITIONAL NATURAL-IMAGE RESULTS

  • The experiment draws calibration images from the official BSDS500 training and test splits and evaluates on 100 validation images with fixed evaluation images and noise.CBSD68 is a subset of those validation images, not an independent test set or the native-resolution CBSD68 protocol.Source: §B.8 · p. 31
  • Our W3–W8 allocation remains close to CLADO under clean calibration and outperforms OMPQ-ORM; with common W3/W4/W8 candidates, it matches CLADO at B = 4 and performs better under both corruptions.CLADO has higher clean PSNR at B = 3.5, and the same pattern is reported on the CBSD68 subset.Source: §B.8 · p. 31, Table 11 · p. 31, Table 12 · p. 31
  • At B = 4, the allocations remain close to FP32 across all three calibration conditions, and the corruption effect is negligible; the effect is larger at B = 3.5.The corruption comparison keeps the test set fixed; error bars show standard deviation over three paired calibration-seed differences.Source: Figure 12 · p. 32, Figure 13 · p. 32
  • Fixed-random BSDS500 examples and central zooms show mixed-precision outputs remaining similar across calibration conditions at B = 4.PSNR labels refer to full crops, while the zooms display the same 96 × 96 region for every method.Source: Figure 14 · p. 33, Figure 15 · p. 33

B.9 EFFECT OF ALLOCATION GRANULARITY

The granularity experiment compares 36-block allocation with finer allocation across 64 independent convolutional layers, keeping the local score and greedy rule unchanged. Finer allocation improves quality at tighter budgets in the reported CelebA and BSDS500 comparisons.

B.9 EFFECT OF ALLOCATION GRANULARITY

  • The 64-layer variant retains W3–W8, the local score, and the greedy rule, and both variants reuse the same FP32 calibration passes.Each experiment uses three calibration sets of 256 images and a fixed test set of 100 images.Source: §B.9 · p. 34
  • On CelebA, layerwise allocation improves PSNR by 0.78 dB at B = 3.5 and reaches CLADO quality.The CLADO values are a quality reference with its original groups and candidates; the block–layer comparison uses its own paired block control.Source: §B.9 · p. 34, Table 13 · p. 34
  • On BSDS500, finer allocation improves PSNR at B = 3.5 in all three calibration conditions, while both granularities give nearly identical quality at B = 4.Both variants use W3–W8; the comparison changes grouping, which determines how local perturbations are summed before their mean and dispersion are computed.Source: §B.9 · p. 34, Table 14 · p. 34
Loading 2610.09877v1…