PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming

Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, Daniel Soudry

arXiv:2006.10518v2cs.LGstat.ML

TL;DR

Post-training quantization below 8 bits often causes significant accuracy degradation, despite using only a small calibration set. The paper combines layer-wise calibration optimization, integer-programming bit allocation, batch-normalization tuning, and deployment pipelines to improve low-bit quantization. These methods achieve state-of-the-art results across vision and language models, including less than 1% degradation in a challenging compression setting.

  • Problem

    Post-training quantization below 8 bits often incurs significant accuracy degradation, while small calibration sets cannot support fine-tuning without overfitting.

  • Method

    The paper introduces layer-wise AdaQuant optimization, integer programming for per-layer bit allocation, batch-normalization tuning, and light or advanced calibration pipelines.

  • Results

    The methods define state-of-the-art post-training quantization results across vision and language tasks, including staying within 1% of full-precision accuracy while converting 96% of the model to 4-bit.

  • Takeaways & Limitations

    The proposed pipelines support practical low-bit deployment while balancing accuracy against performance, power, or compression constraints.

  • Takeaways & Limitations

    AdaQuant’s calibration set should roughly scale with layer input dimensions, with convolutional layers requiring B ≥ C_i·k^2.

Abstract

from arXiv · show

Lately, post-training quantization methods have gained considerable attention, as they are simple to use, and require only a small unlabeled calibration set. This small dataset cannot be used to fine-tune the model without significant over-fitting. Instead, these methods only use the calibration set to set the activations' dynamic ranges. However, such methods always resulted in significant accuracy degradation, when used below 8-bits (except on small datasets). Here we aim to break the 8-bit barrier. To this end, we minimize the quantization errors of each layer separately by optimizing its parameters over the calibration set. We empirically demonstrate that this approach is: (1) much less susceptible to over-fitting than the standard fine-tuning approaches, and can be used even on a very small calibration set; and (2) more powerful than previous methods, which only set the activations' dynamic ranges. Furthermore, we demonstrate how to optimally allocate the bit-widths for each layer, while constraining accuracy degradation or model compression by proposing a novel integer programming formulation. Finally, we suggest model global statistics tuning, to correct biases introduced during quantization. Together, these methods yield state-of-the-art results for both vision and text models. For instance, on ResNet50, we obtain less than 1\% accuracy degradation --- with 4-bit weights and activations in all layers, but the smallest two. We open-sourced our code.

1 Introduction

The paper targets the accuracy degradation of sub-8-bit post-training quantization using calibration-only layer optimization, mixed-precision allocation, batch-normalization tuning, and deployment-oriented pipelines.

  • Motivation: Post-training quantization is practical with a small unlabeled calibration set, but sub-8-bit methods incur significant accuracy degradation.It is useful when training data is sensitive or unavailable, including legacy models and medical records.
  • Contributions: Integer programming selects layer precisions to maximize speedup or power savings under predefined accuracy-degradation or compression constraints.The formulation supports mixed-precision allocation because different network parts may tolerate different precision levels.
  • Contributions: Batch-normalization tuning re-estimates statistics after quantization to recover much of the degradation caused by biased means and variances.The method targets quantization-induced bias in internal batch-normalization statistics.
  • Contributions: The proposed light and advanced pipelines trade optimization complexity against accuracy, with the light pipeline avoiding a backward pass.The advanced pipeline includes AdaQuant and bias tuning, while the light pipeline can run on inference-only hardware.

2 Related work

The related work covers post-training quantization, layerwise rounding and output optimization, integer-programming allocation, and batch-normalization tuning as complementary approaches to quantized inference.

  • Post-training quantization: Quantization accelerates inference, while post-training methods avoid retraining but commonly suffer substantial accuracy loss at INT4 or lower.Post-training quantization is simpler to apply because it operates after model training.
  • AdaQuant: AdaQuant extends AdaRound by optimizing weights and quantization parameters for both weights and activations over a calibration set.Unlike AdaRound, AdaQuant relaxes the constraint keeping quantized weights within ±1 of round-to-nearest values.
  • Integer programming: Prior allocation methods include greedy compression, pruning-oriented knapsack optimization, and Pareto-efficient mixed precision, whereas this work formulates a generic integer program with explicit objectives and constraints.The proposed formulation is intended to adapt across model types and requirements.
  • Batch-normalization tuning: Prior batch-normalization work addressed quantization bias through bias updates or range equalization, while this paper details tuning for per-channel quantized models with fused batch-normalization layers.The procedure uses a few calibration-set invocations and can reabsorb retuned batch-normalization layers to reduce inference complexity.

3 Optimizing The Quantization Pipeline

The pipeline uses calibration data for layerwise optimization, mixed-precision allocation, and batch-normalization tuning. These components target lower quantization error while balancing accuracy against performance and avoiding overfitting.

  • 3.1 AdaQuant: AdaQuant jointly optimizes each layer’s weights and quantization parameters against layer outputs, rather than minimizing tensor-wide quantization error alone.The method allows quantized weights to move beyond the nearest-rounding constraint and is designed to be less prone to overfitting.
  • 3.1 AdaQuant: Sequential AdaQuant propagates predecessor-layer quantization errors forward, but should be applied only after bit allocation has been selected.The sequential variant optimizes later layers using inputs produced by already quantized predecessors.
  • 3.1 AdaQuant: AdaQuant can operate on very small calibration sets without observed overfitting, and its accuracy improves as calibration-set size increases.Figure 1 reports rapid variance reduction as calibration-set size grows; experiments compare methods using mean and standard deviation over five runs.
  • 3.2 Per-layer bit allocations with integer programming: Integer programming selects one weight-and-activation precision configuration per layer to maximize performance improvement under a total degradation constraint.The performance metric can represent execution time or power consumption, while quantization error contributes to the loss penalty.
  • 3.3 Batch Normalization Tuning: Batch-normalization tuning reconstructs BN layers, re-estimates running statistics through forward iterations, and re-fuses them into quantized weights.The procedure compensates for internal-statistics deviations caused by reduced bit-width and requires per-channel weight quantization for re-fusion.

4 Quantization Flow

The proposed deployment flow offers light and advanced pipelines for per-channel quantization with a small calibration set. The light pipeline prioritizes speed and inference-only compatibility, while the advanced pipeline adds AdaQuant and global tuning.

  • 4 Quantization Flow: Both pipelines target the common setting of per-channel quantization with a small calibration set.The design accommodates different deployment constraints, including hardware limits, deployment time, and task or dataset availability.
  • Light pipeline: The light pipeline fuses layers, uses integer programming for mixed-precision selection, and applies batch-normalization tuning without back-propagation.Its three steps are intended to be light and fast and can be invoked on inference-only hardware.
  • Advanced pipeline: The advanced pipeline adds AdaQuant before integer-programming allocation, then permits global batch-normalization and bias tuning to reduce knowledge-distillation loss.Models optimized by AdaQuant at different bit-widths can be combined into mixed-precision configurations.

5 Experiments

Experiments evaluate AdaQuant, integer-programming mixed precision, batch-normalization tuning, and light or advanced pipelines across vision and language models. These methods improve compression–accuracy trade-offs, including challenging low-precision settings.

  • Experimental scope: Experiments cover ResNet18/50, MobileNet-V2, and BERT on ImageNet and SQuAD1.1.The study uses these models to evaluate robustness across vision and text tasks.
  • AdaQuant: AdaQuant jointly optimizes weights and quantization parameters, unlike AdaRound’s restricted weight-only optimization.The relaxed weight updates can exceed one quantization step.
  • Integer programming: Integer programming found that one 2-bit layer plus eight 8-bit layers improved accuracy by over 5.5% versus uniform 4-bit quantization at 12.5% compression.For compression ratios above 12.5%, the method selected mixtures of 4- and 8-bit layers.
  • Batch-Norm tuning: Batch-normalization tuning recovered more than 1.5% accuracy using only a few inference-mode iterations over the calibration set.The reported comparison is 73.7% versus 75.37%.
  • Baselines: Integer programming outperformed the two greedy baselines in bit-width configuration quality.The two greedy methods produced entirely different configurations despite expected correlations between layer size and sensitivity.
  • Full pipeline: At 0.13% compression, the advanced pipeline stayed within 1% of full-precision accuracy while converting 96% of the model to 4-bit.For MobileNet-V2, switching 25% of layers to 4-bit maintained less than 2% degradation.

A Size of calibration set

The calibration-set analysis formulates layerwise quantization fitting as separate regression problems. The required sample count depends on the number of parameters contributing to each output.

  • Calibration formulation: The layerwise setup uses B unique calibration samples and quantized inputs and weights to construct the optimization problem.The input quantization step is fixed in this formulation.
  • Fully connected layers: For a fully connected layer, each output can be examined independently as scalar linear regression with N parameters and B equations.The problem is under-parameterized when B ≥ N and over-parameterized when B < N.
  • Convolution layers: For convolution layers, each output element depends on Ci·k^2 parameters, so the analysis requires B ≥ Ci·k^2 samples.The layer has Co×H×W output elements.

B Reconstruction and re-fusing of Batch Normalization

Batch-normalization reconstruction initializes a near-identity layer, updates running statistics using calibration data, and then re-fuses the layer into the preceding convolution. The tuning procedure relies on forward propagation rather than changing reconstructed BN parameters.

  • Reconstruction: The reconstructed batch-normalization layer is initialized so that BNr(x) = x.The initialization includes reconstructed statistics and parameters γr and βr.
  • Statistics tuning: BN tuning updates running mean and variance from calibration data while keeping γr and βr fixed.The procedure uses only forward propagation.
  • Re-fusing: After tuning, batch normalization is re-fused into convolution weights and biases using the per-channel quantization setting.Regular fusing can degrade performance because the weights are quantized.
  • Quantization-error analysis: For sufficiently small quantization error, total loss degradation can be approximated as a sum of independent degradation processes by neglecting quadratic error terms.The paper notes that related work uses a similar additivity assumption.

D Experimental Details

All experiments use very small calibration sets: 1,000 unlabeled ImageNet training images for vision models and one training paragraph for BERT. The methods reportedly perform well under these conditions.

  • Calibration data: Vision experiments use 1,000 unlabeled ImageNet training images, with one image per class.BERT experiments use one paragraph from its training set.
  • Reported performance: AdaQuant, BNT, BT, and IP all performed well on these small calibration sets and produced state-of-the-art results.The reported setting uses subsets of the training data rather than labeled deployment data.

D.1 AdaQuant

AdaQuant optimizes quantization parameters layer by layer using a small calibration set, with experiments comparing its robustness and accuracy against other methods.

  • AdaQuant optimizes each layer’s quantization parameters to reduce output error relative to the full-precision layer.
  • The implementation uses Adam with separate learning rates for weights, biases, and input and weight quantization parameters.The learning rates are 1e−5, 1e−3, 1e−1, and 1e−3, respectively.
  • Vision experiments use 1000 unlabeled ImageNet images, while BERT uses one training paragraph for calibration.Adam runs for 100 iterations on vision models and 50–100 iterations for BERT layers.
  • AdaQuant is evaluated against QAT-KLD and other methods across calibration-set sizes, with repeated runs reporting mean and standard deviation.An additional early-stop comparison evaluates QAT-KLD using split calibration data.

D.2 Integer Programming

The integer-programming method measures layer sensitivity and compression benefits, then selects a mixed-precision configuration under user-defined constraints.

  • The integer-programming procedure first measures layer properties and then applies the resulting program under a user-defined constraint.Measurements include the base-precision model loss and each layer’s sensitivity to lower precision.
  • A small loss difference indicates a quantization-robust layer, whereas a large difference indicates a sensitive layer.The loss difference is defined between the reference model and the model with the examined layer quantized at lower precision.
  • Compression benefit is measured by the model-parameter size reduction obtained when lowering the examined layer’s precision.

D.3 Batch Normalization and Bias Tuning

The tuning stages correct quantization-induced statistical and bias errors using lightweight batch-normalization updates followed by bias fine-tuning.

  • Batch-normalization tuning updates statistics for fewer than ten iterations, although higher compression requires more iterations.
  • Bias tuning performs 200 fine-tuning iterations with a learning rate of 0.1.
  • The experiments use torchvision pretrained vision models and a BERT-base model fine-tuned on SQuAD1.1.The authors provide the BERT fine-tuning script in their repository.
Loading 2006.10518v2…