PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming

Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, Daniel Soudry

arXiv:2006.10518v2cs.LGstat.ML

TL;DR

Post-training quantization below 8 bits often causes significant accuracy degradation, despite using only a small calibration set. The paper combines layer-wise calibration optimization, integer-programming bit allocation, batch-normalization tuning, and deployment pipelines to improve low-bit quantization. These methods achieve state-of-the-art results across vision and language models, including less than 1% degradation in a challenging compression setting.

  • Problem

    Post-training quantization below 8 bits often incurs significant accuracy degradation, while small calibration sets cannot support fine-tuning without overfitting.

    Source: §1 · pp. 1–2

  • Method

    The paper introduces layer-wise AdaQuant optimization, integer programming for per-layer bit allocation, batch-normalization tuning, and light or advanced calibration pipelines.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    The methods define state-of-the-art post-training quantization results across vision and language tasks, including staying within 1% of full-precision accuracy while converting 96% of the model to 4-bit.

    Source: §1 · p. 2, §5.4 · p. 9

  • Takeaways & Limitations

    The proposed pipelines support practical low-bit deployment while balancing accuracy against performance, power, or compression constraints.

    Source: §1 · p. 2, §3.2 · p. 4, §4 · p. 6

    AdaQuant’s calibration set should roughly scale with layer input dimensions, with convolutional layers requiring B ≥ C_i·k^2.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

Lately, post-training quantization methods have gained considerable attention, as they are simple to use, and require only a small unlabeled calibration set. This small dataset cannot be used to fine-tune the model without significant over-fitting. Instead, these methods only use the calibration set to set the activations' dynamic ranges. However, such methods always resulted in significant accuracy degradation, when used below 8-bits (except on small datasets). Here we aim to break the 8-bit barrier. To this end, we minimize the quantization errors of each layer separately by optimizing its parameters over the calibration set. We empirically demonstrate that this approach is: (1) much less susceptible to over-fitting than the standard fine-tuning approaches, and can be used even on a very small calibration set; and (2) more powerful than previous methods, which only set the activations' dynamic ranges. Furthermore, we demonstrate how to optimally allocate the bit-widths for each layer, while constraining accuracy degradation or model compression by proposing a novel integer programming formulation. Finally, we suggest model global statistics tuning, to correct biases introduced during quantization. Together, these methods yield state-of-the-art results for both vision and text models. For instance, on ResNet50, we obtain less than 1\% accuracy degradation --- with 4-bit weights and activations in all layers, but the smallest two. We open-sourced our code.

1 Introduction

The paper targets the accuracy degradation of sub-8-bit post-training quantization using calibration-only layer optimization, mixed-precision allocation, batch-normalization tuning, and deployment-oriented pipelines.

Motivation

  • Post-training quantization is practical with a small unlabeled calibration set, but sub-8-bit methods incur significant accuracy degradation.It is useful when training data is sensitive or unavailable, including legacy models and medical records.Source: §1 · pp. 1–2

Contributions

  • Integer programming selects layer precisions to maximize speedup or power savings under predefined accuracy-degradation or compression constraints.The formulation supports mixed-precision allocation because different network parts may tolerate different precision levels.Source: §1 · p. 2
  • Batch-normalization tuning re-estimates statistics after quantization to recover much of the degradation caused by biased means and variances.The method targets quantization-induced bias in internal batch-normalization statistics.Source: §1 · p. 2
  • The proposed light and advanced pipelines trade optimization complexity against accuracy, with the light pipeline avoiding a backward pass.The advanced pipeline includes AdaQuant and bias tuning, while the light pipeline can run on inference-only hardware.Source: §1 · p. 2

2 Related work

The related work covers post-training quantization, layerwise rounding and output optimization, integer-programming allocation, and batch-normalization tuning as complementary approaches to quantized inference.

Post-training quantization

  • Quantization accelerates inference, while post-training methods avoid retraining but commonly suffer substantial accuracy loss at INT4 or lower.Post-training quantization is simpler to apply because it operates after model training.Source: §2 · p. 2

AdaQuant

  • AdaQuant extends AdaRound by optimizing weights and quantization parameters for both weights and activations over a calibration set.Unlike AdaRound, AdaQuant relaxes the constraint keeping quantized weights within ±1 of round-to-nearest values.Source: §2 · p. 2

Integer programming

  • Prior allocation methods include greedy compression, pruning-oriented knapsack optimization, and Pareto-efficient mixed precision, whereas this work formulates a generic integer program with explicit objectives and constraints.The proposed formulation is intended to adapt across model types and requirements.Source: §2 · p. 3

Batch-normalization tuning

  • Prior batch-normalization work addressed quantization bias through bias updates or range equalization, while this paper details tuning for per-channel quantized models with fused batch-normalization layers.The procedure uses a few calibration-set invocations and can reabsorb retuned batch-normalization layers to reduce inference complexity.Source: §2 · p. 3

3 Optimizing The Quantization Pipeline

The pipeline uses calibration data for layerwise optimization, mixed-precision allocation, and batch-normalization tuning. These components target lower quantization error while balancing accuracy against performance and avoiding overfitting.

3.1 AdaQuant

  • AdaQuant jointly optimizes each layer’s weights and quantization parameters against layer outputs, rather than minimizing tensor-wide quantization error alone.The method allows quantized weights to move beyond the nearest-rounding constraint and is designed to be less prone to overfitting.Source: §3.1 · p. 3
  • Sequential AdaQuant propagates predecessor-layer quantization errors forward, but should be applied only after bit allocation has been selected.The sequential variant optimizes later layers using inputs produced by already quantized predecessors.Source: §3.1 · p. 4
  • AdaQuant can operate on very small calibration sets without observed overfitting, and its accuracy improves as calibration-set size increases.Figure 1 reports rapid variance reduction as calibration-set size grows; experiments compare methods using mean and standard deviation over five runs.Source: §3.1 · p. 4, Figure 1 · p. 4

3.2 Per-layer bit allocations with integer programming

  • Integer programming selects one weight-and-activation precision configuration per layer to maximize performance improvement under a total degradation constraint.The performance metric can represent execution time or power consumption, while quantization error contributes to the loss penalty.Source: §3.2 · pp. 4–5

3.3 Batch Normalization Tuning

  • Batch-normalization tuning reconstructs BN layers, re-estimates running statistics through forward iterations, and re-fuses them into quantized weights.The procedure compensates for internal-statistics deviations caused by reduced bit-width and requires per-channel weight quantization for re-fusion.Source: §3.3 · p. 5

4 Quantization Flow

The proposed deployment flow offers light and advanced pipelines for per-channel quantization with a small calibration set. The light pipeline prioritizes speed and inference-only compatibility, while the advanced pipeline adds AdaQuant and global tuning.

4 Quantization Flow

  • Both pipelines target the common setting of per-channel quantization with a small calibration set.The design accommodates different deployment constraints, including hardware limits, deployment time, and task or dataset availability.Source: §4 · p. 6

Light pipeline

  • The light pipeline fuses layers, uses integer programming for mixed-precision selection, and applies batch-normalization tuning without back-propagation.Its three steps are intended to be light and fast and can be invoked on inference-only hardware.Source: §4 · p. 6

Advanced pipeline

  • The advanced pipeline adds AdaQuant before integer-programming allocation, then permits global batch-normalization and bias tuning to reduce knowledge-distillation loss.Models optimized by AdaQuant at different bit-widths can be combined into mixed-precision configurations.Source: §4 · p. 6

5 Experiments

Experiments evaluate AdaQuant, integer-programming mixed precision, batch-normalization tuning, and light or advanced pipelines across vision and language models. These methods improve compression–accuracy trade-offs, including challenging low-precision settings.

Experimental scope

  • Experiments cover ResNet18/50, MobileNet-V2, and BERT on ImageNet and SQuAD1.1.The study uses these models to evaluate robustness across vision and text tasks.Source: §5 · p. 6

AdaQuant

  • AdaQuant jointly optimizes weights and quantization parameters, unlike AdaRound’s restricted weight-only optimization.The relaxed weight updates can exceed one quantization step.Source: §5.1 · p. 7, Figure 2 · p. 8

Integer programming

  • Integer programming found that one 2-bit layer plus eight 8-bit layers improved accuracy by over 5.5% versus uniform 4-bit quantization at 12.5% compression.For compression ratios above 12.5%, the method selected mixtures of 4- and 8-bit layers.Source: §5.2 · p. 8

Batch-Norm tuning

  • Batch-normalization tuning recovered more than 1.5% accuracy using only a few inference-mode iterations over the calibration set.The reported comparison is 73.7% versus 75.37%.Source: §5.3 · p. 8, §5.4 · p. 9

Baselines

  • Integer programming outperformed the two greedy baselines in bit-width configuration quality.The two greedy methods produced entirely different configurations despite expected correlations between layer size and sensitivity.Source: §5.4 · p. 8

Full pipeline

  • At 0.13% compression, the advanced pipeline stayed within 1% of full-precision accuracy while converting 96% of the model to 4-bit.For MobileNet-V2, switching 25% of layers to 4-bit maintained less than 2% degradation.Source: §5.4 · p. 9

A Size of calibration set

The calibration-set analysis formulates layerwise quantization fitting as separate regression problems. The required sample count depends on the number of parameters contributing to each output.

Calibration formulation

  • The layerwise setup uses B unique calibration samples and quantized inputs and weights to construct the optimization problem.The input quantization step is fixed in this formulation.Source: A Size of calibration set · p. 12

Fully connected layers

  • For a fully connected layer, each output can be examined independently as scalar linear regression with N parameters and B equations.The problem is under-parameterized when B ≥ N and over-parameterized when B < N.Source: A Size of calibration set · p. 12

Convolution layers

  • For convolution layers, each output element depends on Ci·k^2 parameters, so the analysis requires B ≥ Ci·k^2 samples.The layer has Co×H×W output elements.Source: A Size of calibration set · p. 12

B Reconstruction and re-fusing of Batch Normalization

Batch-normalization reconstruction initializes a near-identity layer, updates running statistics using calibration data, and then re-fuses the layer into the preceding convolution. The tuning procedure relies on forward propagation rather than changing reconstructed BN parameters.

Reconstruction

Statistics tuning

Re-fusing

  • After tuning, batch normalization is re-fused into convolution weights and biases using the per-channel quantization setting.Regular fusing can degrade performance because the weights are quantized.Source: B Reconstruction and re-fusing of… · p. 13

Quantization-error analysis

  • For sufficiently small quantization error, total loss degradation can be approximated as a sum of independent degradation processes by neglecting quadratic error terms.The paper notes that related work uses a similar additivity assumption.Source: C Additive loss assumption for… · pp. 13–14

D Experimental Details

All experiments use very small calibration sets: 1,000 unlabeled ImageNet training images for vision models and one training paragraph for BERT. The methods reportedly perform well under these conditions.

Calibration data

  • Vision experiments use 1,000 unlabeled ImageNet training images, with one image per class.BERT experiments use one paragraph from its training set.Source: D Experimental Details · p. 14

Reported performance

  • AdaQuant, BNT, BT, and IP all performed well on these small calibration sets and produced state-of-the-art results.The reported setting uses subsets of the training data rather than labeled deployment data.Source: D Experimental Details · p. 14

D.1 AdaQuant

AdaQuant optimizes quantization parameters layer by layer using a small calibration set, with experiments comparing its robustness and accuracy against other methods.

  • AdaQuant optimizes each layer’s quantization parameters to reduce output error relative to the full-precision layer.Source: §D.1 · p. 14
  • The implementation uses Adam with separate learning rates for weights, biases, and input and weight quantization parameters.The learning rates are 1e−5, 1e−3, 1e−1, and 1e−3, respectively.Source: §D.1 · p. 14
  • Vision experiments use 1000 unlabeled ImageNet images, while BERT uses one training paragraph for calibration.Adam runs for 100 iterations on vision models and 50–100 iterations for BERT layers.Source: §D.1 · p. 14
  • AdaQuant is evaluated against QAT-KLD and other methods across calibration-set sizes, with repeated runs reporting mean and standard deviation.An additional early-stop comparison evaluates QAT-KLD using split calibration data.Source: §D.1 · p. 14

D.2 Integer Programming

The integer-programming method measures layer sensitivity and compression benefits, then selects a mixed-precision configuration under user-defined constraints.

  • The integer-programming procedure first measures layer properties and then applies the resulting program under a user-defined constraint.Measurements include the base-precision model loss and each layer’s sensitivity to lower precision.Source: §D.2 · p. 14
  • A small loss difference indicates a quantization-robust layer, whereas a large difference indicates a sensitive layer.The loss difference is defined between the reference model and the model with the examined layer quantized at lower precision.Source: §D.2 · p. 15
  • Compression benefit is measured by the model-parameter size reduction obtained when lowering the examined layer’s precision.Source: §D.2 · p. 15

D.3 Batch Normalization and Bias Tuning

The tuning stages correct quantization-induced statistical and bias errors using lightweight batch-normalization updates followed by bias fine-tuning.

  • Batch-normalization tuning updates statistics for fewer than ten iterations, although higher compression requires more iterations.Source: §D.3 · p. 15
  • Bias tuning performs 200 fine-tuning iterations with a learning rate of 0.1.Source: §D.3 · p. 15
  • The experiments use torchvision pretrained vision models and a BERT-base model fine-tuned on SQuAD1.1.The authors provide the BERT fine-tuning script in their repository.Source: E Code · p. 15

Related papers

Loading 2006.10518v2…