PaperlayerView exact revision on arXiv

Source-linked AI summary

Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights

Aojun Zhou, Anbang Yao, Yiwen Guo, Lin Xu, Yurong Chen

arXiv:1702.03044v2cs.CVcs.AIcs.NE

TL;DR

Low-precision CNN quantization must avoid accuracy loss and excessive retraining. INQ iteratively partitions weights, quantizes groups to powers of two or zero, and retrains the remainder; on ImageNet, it achieved improved or similar accuracy across tested architectures and bit widths.

  • Problem

    Low-precision CNN quantization remains limited by non-negligible accuracy loss and increased training iterations needed for convergence.

  • Method

    INQ repeatedly partitions each layer’s weights, quantizes one group with variable-length encoding, and retrains the other until all weights become powers of two or zero.

  • Results

    Across ImageNet experiments, 5-bit INQ improved accuracy over full-precision baselines, while ResNet-18 4-bit, 3-bit, and 2-bit ternary models had improved or very similar accuracy.

  • Takeaways & Limitations

    INQ provides a low-precision CNN framework whose power-of-two weights can replace floating-point multiplications with cheaper binary bit shifts on dedicated hardware.

  • Takeaways & Limitations

    The paper’s rationale is conjectural that changing network-weight importance is critical for achieving lossless quantization.

Abstract

from arXiv · show

This paper presents incremental network quantization (INQ), a novel method, targeting to efficiently convert any pre-trained full-precision convolutional neural network (CNN) model into a low-precision version whose weights are constrained to be either powers of two or zero. Unlike existing methods which are struggled in noticeable accuracy loss, our INQ has the potential to resolve this issue, as benefiting from two innovations. On one hand, we introduce three interdependent operations, namely weight partition, group-wise quantization and re-training. A well-proven measure is employed to divide the weights in each layer of a pre-trained CNN model into two disjoint groups. The weights in the first group are responsible to form a low-precision base, thus they are quantized by a variable-length encoding method. The weights in the other group are responsible to compensate for the accuracy loss from the quantization, thus they are the ones to be re-trained. On the other hand, these three operations are repeated on the latest re-trained group in an iterative manner until all the weights are converted into low-precision ones, acting as an incremental network quantization and accuracy enhancement procedure. Extensive experiments on the ImageNet classification task using almost all known deep CNN architectures including AlexNet, VGG-16, GoogleNet and ResNets well testify the efficacy of the proposed method. Specifically, at 5-bit quantization, our models have improved accuracy than the 32-bit floating-point references. Taking ResNet-18 as an example, we further show that our quantized models with 4-bit, 3-bit and 2-bit ternary weights have improved or very similar accuracy against its 32-bit floating-point baseline. Besides, impressive results with the combination of network pruning and INQ are also reported. The code is available at https://github.com/Zhouaojun/Incremental-Network-Quantization.

1 INTRODUCTION

Deep CNNs achieve strong vision results but are difficult to deploy under limited computation and power budgets. INQ addresses low-precision quantization’s accuracy and convergence challenges through iterative partitioning, quantization, and retraining, with strong ImageNet results.

  • Motivation: ResNet-152 requires about 230 MB and 11.3 billion FLOPs to classify a 224 × 224 image crop.These resource demands make deployment on devices with limited computation and power budgets challenging.
  • Problem: CNN quantization remains open because low-precision methods can incur non-negligible accuracy loss and require increased training iterations for convergence.INQ is presented to address both issues.
  • Method: INQ divides weights into groups, quantizes one group, retrains the other, and repeats these operations on the latest retrained group until all weights are quantized.The final weights are constrained to powers of two or zero.
  • Results: At 5-bit quantization, INQ improves top-1 accuracy by 0.13% to 2.28% and top-5 accuracy by 0.23% to 1.65% across several CNN architectures.The reported comparisons use AlexNet, VGG-16, GoogleNet, and ResNets against their full-precision baselines.
  • Results: Less than 8 retraining epochs generally produced lossless 5-bit models in the experiments.The experiments evaluated INQ on ImageNet across AlexNet, VGG-16, GoogleNet, ResNet-18, and ResNet-50.

2 INCREMENTAL NETWORK QUANTIZATION

INQ incrementally converts pre-trained full-precision CNN weights into powers of two or zero by alternating partition, group-wise quantization, and re-training. This strategy targets accuracy loss and convergence costs associated with low-precision quantization.

  • Weight quantization: INQ constrains converted weights to powers of two or zero, enabling floating-point multiplications to be replaced by binary bit shifts.The method targets pre-trained 32-bit floating-point CNNs without assuming a particular architecture.
  • Weight quantization: Weights with absolute values below 2^n2 are pruned to zero, while the remaining values are bounded between powers determined by n1 and n2.The expected bit-width b is fixed beforehand; n1 is selected and n2 is then computed.
  • Weight quantization: The quantization set uses variable-length encoding: one bit represents zero, while the remaining bits represent powers-of-two values.For 5-bit quantization, the scheme supports at most 2^4 + 1 candidate values.
  • Incremental quantization strategy: The partition, quantization, and re-training operations repeat on the latest re-trained group until all weights are quantized.Illustrated iterations increase the accumulated quantized portion from 50% to 75%, 87.5%, and finally 100%.
  • Incremental quantization strategy: INQ partitions each layer into disjoint groups, quantizes one group to form a low-precision base, and re-trains the other group to compensate for accuracy loss.The partition mask fixes already quantized weights during subsequent optimization, so only floating-point weights are updated.
  • Incremental network quantization algorithm: The nested INQ procedure is designed to reduce accuracy loss relative to simultaneous quantization and potentially produce lossless low-precision CNNs.The paper identifies importance-aware partitioning, group-wise quantization, and iterative re-training as the method’s three main merits.

3 EXPERIMENTAL RESULTS

INQ was evaluated on diverse CNN architectures using ImageNet classification, including 5-bit conversion, partition-strategy comparisons, lower-bit models, and compression combinations. Across these experiments, the method maintained or improved accuracy while supporting increasingly compact representations.

  • 3 EXPERIMENTAL RESULTS: ImageNet experiments covered AlexNet, VGG-16, GoogleNet, ResNet-18, and ResNet-50 using top-1 and top-5 error rates.The study used approximately 1.2 million training images, 50,000 validation images, and 1,000 classes.
  • 3.1 RESULTS ON IMAGENET: 5-bit INQ converted diverse CNN models into low-precision versions with consistently improved model accuracy.The evaluated architectures were AlexNet, VGG-16, GoogleNet, ResNet-18, and ResNet-50.
  • 3.2 ANALYSIS OF WEIGHT PARTITION STRATEGIES: Pruning-inspired partition reduced ResNet-18 top-1 and top-5 error rates by 1.09% and 0.83%, respectively, compared with random partition.With random partition, the reported top-1 and top-5 error rates were 32.11% and 11.73%.
  • 3.3 THE TRADE-OFF BETWEEN EXPECTED BIT-WIDTH AND MODEL ACCURACY: 3-bit INQ incurred only 0.19% and 0.33% losses in top-1 and top-5 recognition rates, respectively, relative to the full-precision reference.The experiments also generated 4-bit and 2-bit ternary models.
  • 3.3 THE TRADE-OFF BETWEEN EXPECTED BIT-WIDTH AND MODEL ACCURACY: 2-bit ternary INQ decreased top-1 and top-5 error rates by 2.25% and 1.56%, respectively, versus the pre-trained full-precision reference.Its accuracy was reported as considerably better than state-of-the-art binary-weight and ternary-weight network results.
  • 3.4 LOW-BIT DEEP COMPRESSION: Combining INQ with DNS was evaluated against deep compression on AlexNet across convolutional and fully connected layers using pruning, quantization, and Huffman coding components.The comparison included layer types and abbreviated pipeline components defined in the table caption.

4 CONCLUSIONS

INQ converts pre-trained full-precision CNNs into low-precision models using complementary quantization and re-training operations applied incrementally until all weights are quantized.

  • INQ partitions each layer’s weights into two groups, quantizing one as a low-precision base and re-training the other to compensate for accuracy loss.The quantized weights are constrained to powers of two or zero, while the re-trained group remains adjustable until later iterations.
  • The method repeats weight partition, group-wise quantization, and re-training on the latest re-trained group until all weights are quantized.

A APPENDIX 1: STATISTICAL ANALYSIS OF THE QUANTIZED WEIGHTS

The appendix analyzes the distribution of quantized weights in a 5-bit AlexNet model, finding that a few power-of-two values dominate several convolutional layers.

  • In AlexNet’s first convolutional layer, six values—{−2−6, −2−5, −2−4, 2−6, 2−5, 2−4}—occupy over 60% of quantized weights.
  • In the second convolutional layer, nine values including zero—{−2−8, −2−7, −2−6, −2−5, 0, 2−8, 2−7, 2−6, 2−5}—occupy over 94% of quantized weights.
  • The quantized-weight distributions in the third, fourth, and fifth convolutional layers are similar to that of the second layer.

B APPENDIX 2: LOSSLESS CNNS WITH LOW-PRECISION WEIGHTS AND LOW-PRECISION ACTIVATIONS

The appendix reports INQ results with both low-precision weights and activations. A VGG-16 model using 5-bit weights and 4-bit activations improves recognition rates over a 32-bit floating-point reference.

  • The VGG-16 model uses 5-bit weights and 4-bit activations, compared with a pretrained reference using 32-bit floating-point weights and activations.
  • The low-precision VGG-16 model shows improved top-5 and top-1 recognition rates compared with the pretrained 32-bit floating-point reference.
Loading 1702.03044v2…