PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Neural Network Compression with Single and Multiple Level Quantization

Yuhui Xu, Yongzhuang Wang, Aojun Zhou, Weiyao Lin, Hongkai Xiong

arXiv:1803.03289v2cs.LGcs.AIstat.ML

TL;DR

Large DNNs create storage and computation burdens, while existing quantization methods do not sufficiently use depth information for low-bit compression. The paper proposes SLQ and MLQ using width- and depth-level quantization, and reports accuracy gains for high-bit quantization and impressive ternary results without changing network architecture.

  • Problem

    Existing network quantization methods do not sufficiently exploit depth information to produce low-bit compressed networks, while large DNNs burden storage and computation.

    Source: Introduction · p. 1

  • Method

    SLQ uses parameter distributions for high-bit quantization, while MLQ combines width- and depth-level partitioning with iterative layer compensation for ternary quantization.

    Source: Introduction · pp. 1–2

  • Results

    SLQ obtains accuracy gains in high-bit quantization on two datasets, and MLQ achieves impressive ternary quantization results without changing network architecture.

    Source: Conclusion · p. 8

  • Takeaways & Limitations

    The paper presents quantization approaches spanning high-bit and ternary settings while retaining the network architecture for MLQ.

    Source: Introduction · p. 2, Conclusion · p. 8

Abstract

from arXiv · show

Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufficiently exploit the depth information to generate low-bit compressed network. In this paper, we propose two novel network quantization approaches, single-level network quantization (SLQ) for high-bit quantization and multi-level network quantization (MLQ) for extremely low-bit quantization (ternary).We are the first to consider the network quantization from both width and depth level. In the width level, parameters are divided into two parts: one for quantization and the other for re-training to eliminate the quantization loss. SLQ leverages the distribution of the parameters to improve the width level. In the depth level, we introduce incremental layer compensation to quantize layers iteratively which decreases the quantization loss in each iteration. The proposed approaches are validated with extensive experiments based on the state-of-the-art neural networks including AlexNet, VGG-16, GoogleNet and ResNet-18. Both SLQ and MLQ achieve impressive results.

Introduction

Large DNNs burden storage and computation, making deployment on resource-constrained devices difficult. The paper proposes SLQ for high-bit quantization and MLQ for ternary quantization by addressing both parameter partitions and layer partitions.

Introduction

  • Large DNNs impose storage and computation burdens that make deployment on resource-constrained devices such as phones and drones more difficult.The passage identifies network compression as a means to reduce these costs.Source: Introduction · p. 1
  • Figure 1 contrasts single-level and multi-level quantization, with blue indicating full-precision weights or layers and orange indicating quantized ones.The paper identifies SLQ and MLQ as its high-bit and ternary approaches, respectively.Source: Figure 1 · p. 1, Introduction · p. 2
  • Network compression methods face an accuracy–complexity tradeoff, and the cited prior methods often reduce accuracy to some degree.The paper notes that incremental network quantization retrains unquantized parameters to compensate for quantization loss, but treats layers equally and does not account for parameter distributions.Source: Introduction · p. 1
  • For ternary quantization, width-level retraining alone cannot compensate for the large quantization loss; MLQ iteratively quantizes layers and retrains others to recover accuracy.The paper describes this as incremental layer compensation.Source: Introduction · p. 2
  • The paper proposes SLQ for high-bit quantization and MLQ for ternary quantization, considering both width-level parameter partitions and depth-level layer partitions.SLQ uses weight distributions; MLQ applies incremental layer compensation to quantize layers iteratively.Source: Introduction · pp. 1–2

Related Work

Related work covers compression through low-rank decomposition, pruning, quantization, and other strategies, including knowledge transfer and mobile-oriented architectures.

Related Work

  • Low-rank decomposition reduces parameter dimensions, works well on fully connected layers, and can achieve 3× compression, though accuracy can suffer for some models.Yu et al. 2017 extend the idea to convolutional layers using smooth components in a low-rank subspace and weights sparsely scattered outside it.Source: Related Work · p. 2
  • Pruning removes unimportant parameters or filters; Han et al. 2015 report 9× AlexNet and 16× VGG-16 model-size reductions, while Li et al. 2016 report computation-cost reductions up to 34% on VGG-16 and 38% on ResNet-110.The passage also describes later heuristic filter pruning and channel pruning formulated as data recovery.Source: Related Work · p. 2
  • Quantization groups similar weights to reduce free parameters; deep compression combines pruning, vector quantization, and Huffman coding, reporting 35× compression on AlexNet and 49× on VGG-16.The passage describes incremental network quantization as partitioning weights between quantization and retraining, while noting that its handcrafted codebook does not control quantization loss and that it performs poorly at extremely low bitwidth.Source: Related Work · p. 2
  • Other strategies include binary or ternary weights, low-bit activations, knowledge transfer, and mobile-specific architectures such as MobileNets and ShuffleNet.ShuffleNet applies group convolutions and shuffle operations and reports 13× speed up in AlexNet.Source: Related Work · p. 2

Overview

The overview presents an iterative framework of clustering, loss-based partition, weight-sharing, and retraining. SLQ partitions clusters, whereas MLQ partitions both clusters and layers.

Overview

  • The framework iterates clustering, loss-based partition, weight-sharing, and retraining until all weights are quantized.Clustering uses layer-wise k-means; one group is quantized to cluster centroids while the other is retrained.Source: Overview · p. 2, Figure 2 · p. 3
  • SLQ partitions clusters only, while MLQ partitions both clusters and layers; SLQ is a particular case of MLQ.Figure 2 is identified as the framework diagram.Source: Overview · p. 2, Figure 2 · p. 3
  • Figure 3 compares accuracy as the quantization value changes for two AlexNet clusters and reports test accuracy for ten clusters sorted in descending order.Q/C denotes the quantization value Q divided by the cluster centroid C.Source: Figure 3 · p. 3

Single-level Quantization

Single-level quantization clusters weights, quantizes higher-loss clusters first, and retrains the rest to compensate for quantization loss. Its extended form constrains cluster centroids to desired value types.

Clustering

  • K-means clusters weights layer-wise, and quantizing them to cluster centroids yields low quantization loss.The approach uses data-driven clustering rather than handcrafted mapping rules.Source: Clustering · p. 3

Loss based Partition

  • Higher-loss clusters are quantized first, while lower-loss clusters are assigned to retraining.The partition separates clusters so the minimum quantization loss in the quantized group exceeds the maximum in the retrained group.Source: Loss based Partition · p. 3

Weight-sharing

Re-training

  • After fixing quantized weights, the network retrains the other weights and repeats clustering until all weights are quantized.The process iterates over the weights left unquantized.Source: Re-training · pp. 3–4, Figure 4 · p. 4

Extended Approach

  • Extended SLQ constrains cluster centroids toward or to a desired value type, such as powers of two for FPGA deployment.It adds an L1 regularization term to the k-means loss and reclusters before weight-sharing and retraining.Source: Extended Approach · p. 4

Multi-Level Quantization

Multi-Level Quantization (MLQ) extends single-level quantization with incremental layer compensation for ternary networks. It quantizes more-impactful Boundary centroids iteratively before quantizing the Hearts, retraining the remaining weights along the way.

Multi-Level Quantization

  • Figure 5 depicts full-precision, retrained, and quantized layers in blue, green, and orange, respectively.The depicted order is Boundaries first, Hearts second.Source: Figure 5 · p. 5
  • MLQ partitions both layers and within-layer parameters, addressing the large quantization loss that makes SLQ unsuitable for ternary networks.SLQ’s small number of clusters at low bit-width leaves per-iteration loss too large to eliminate through retraining alone.Source: Multi-Level Quantization · pp. 4–5
  • Each layer’s three ternary centroids are split into Boundaries (a_i and c_i), which affect performance more, and the less influential Heart (b_i).MLQ quantizes the Boundaries first, then the Hearts.Source: Multi-Level Quantization · p. 5
  • Within each phase, ILC iteratively quantizes a subset of layers while retraining the remaining weights, then repeats until the group is quantized.The procedure handles the Boundaries first and the Hearts afterward.Source: Multi-Level Quantization · p. 5

Experiments

The experiments use CIFAR-10 and ImageNet to evaluate SLQ and MLQ. The supplied passages identify the datasets and the 5-bit SLQ CIFAR-10 results table, but do not provide its cell values.

Experiments

  • The experiments evaluate SLQ and MLQ on CIFAR-10 and ImageNet.CIFAR-10 has 60,000 32×32 color images across 10 classes; ImageNet has 1,000 classes, nearly 1.2 million training images, and 50,000 validation images.Source: Experiments · p. 5
  • Figure 6 contrasts light-CNN training curves from scratch with curves across five SLQ quantization iterations.The supplied caption does not report curve values or outcomes.Source: Figure 6 · p. 5
  • For bit-width b, the method stores one bit for zero and uses the other b−1 bits to code non-zero centroids, giving each layer 2b−1 + 1 centroids.The passage describes this coding scheme as a fairness measure for comparison with other methods.Source: Experiments · p. 5
  • Table 1 reports 5-bit SLQ experiment results on CIFAR-10.The supplied table passage provides its scope, not its cell values.Source: Table 1 · p. 5

Results for SLQ

On CIFAR-10, SLQ improves accuracy after 5-bit quantization for both tested networks. On ImageNet, the 5-bit models outperform full-precision references in Top1 and Top5 accuracy, and improve on INQ in Top1 and most Top5 comparisons.

SLQ Results on CIFAR-10

  • Both light CNN and ResNet20 increase accuracy after 5-bit SLQ quantization on CIFAR-10.The light CNN is trained from scratch, and its reported quantization loss decreases over five iterations.Source: SLQ Results on CIFAR-10 · p. 5

SLQ Results on ImageNet

  • The ImageNet evaluation applies 5-bit SLQ to AlexNet, VGG16, GoogleNet, and Resinet-18, using the same cluster-partition approach across these architectures.The paper describes this shared partitioning as easier to implement and robust across architectures.Source: SLQ Results on ImageNet · p. 5, Table 2 · p. 6
  • Tables 5 and 6 are identified as results for centroid initialization and ESLQ, respectively, but the supplied passages do not state their comparison values.The ImageNet results section also identifies Table 3 as the SLQ results table.Source: Table 5 · p. 6, Table 6 · p. 6, Table 3 · p. 6
  • 5-bit SLQ models outperform full-precision references in both Top1 and Top5 accuracy on ImageNet, improving on INQ in Top1 and most Top5 comparisons.The authors attribute the gains to considering weight distributions and to loss-based partitioning.Source: Table 3 · p. 6, SLQ Results on ImageNet · p. 7

Results for SLQ with Low-bit Setting

SLQ retains good Top-1 and Top-5 performance at 4 bits, while the 3-bit VGG-16 model falls slightly below its full-precision reference. Non-linear centroid initialization performs better than linear initialization in the reported VGG-16 experiment.

Results for SLQ with Low-bit Setting

  • At 3 bits, VGG-16 accuracy drops slightly below the full-precision reference, while 4-bit SLQ retains good Top-1 and Top-5 performance.With five centroids at 3 bits, fewer iterative quantization steps are possible and the final step incurs larger quantization loss.Source: Results for SLQ with Low-bit Setting · p. 7
  • The reported cluster partitions are {5, 4, 4, 2, 2} at 5 bits, {3, 2, 2, 2} at 4 bits, and {2, 2, 1} at 3 bits.These settings accompany the VGG-16 bit-width comparison.Source: Results for SLQ with Low-bit Setting · p. 7

Results for Centroid Initialization

  • Non-linear centroid initialization yields higher accuracy than linear initialization on VGG-16.The authors attribute its smaller final-iteration quantization loss to smaller centroids and fewer weights in that iteration, and use non-linear initialization in all experiments.Source: Results for Centroid Initialization · p. 7

Results for ESLQ

ESLQ reports accuracy increases for both tested high-bit quantization types. MLQ produces ternary CIFAR-10 models with little accuracy loss and comparable ImageNet results while quantizing every AlexNet layer without changing its architecture.

Results for ESLQ

  • Both ESLQ variants—two-significant-figure scientific notation and powers of two or zero—report accuracy increases after quantization.Table 6 identifies the variants as ESLQ1 and ESLQ2.Source: Results for ESLQ · p. 7

Results for MLQ

  • Ternary light CNN and ResNet20 on CIFAR-10 lose little accuracy versus their full-precision versions.The experiments use no data augmentation.Source: Table 7 · p. 7, Results for MLQ · p. 7
  • MLQ achieves comparable ternary AlexNet results on ImageNet while quantizing every layer without adding batch-normalization layers or changing the architecture.It also outperforms FGQ, though it requires more training time.Source: Table 8 · p. 7, Results for MLQ · p. 7

Compression Ratio and Acceleration

The paper reports compression ratios from bitwidth and from combining quantization with pruning, and gives a training-time estimate for 5-bit AlexNet quantization.

Compression Ratio and Acceleration

Conclusion

The paper proposes SLQ for high-bit quantization and MLQ for ternary quantization, considering network quantization across width and depth; it reports accuracy gains for SLQ and impressive MLQ results without architecture changes.

Conclusion

  • SLQ uses parameter distributions for high-bit quantization, while MLQ targets ternary quantization across width and depth without changing network architecture.The conclusion reports accuracy gains for SLQ on state-of-the-art networks across two datasets and impressive MLQ results for ternary quantization.Source: Conclusion · p. 8
Loading 1803.03289v2…