Source-linked AI summary
Compressing Deep Convolutional Networks using Vector Quantization
Yunchao Gong, Liu Liu, Ming Yang, Lubomir Bourdev
TL;DR
Large CNN parameter sets make deployment on resource-limited embedded hardware difficult. The paper studies vector quantization for compressing dense connected layers instead of using matrix factorization, achieving up to 24 times compression with top-five accuracy loss within 1%.
Problem
CNNs achieve strong recognition results but their millions of parameters make model storage extremely large for resource-limited embedded devices.
Method
The paper systematically evaluates information-theoretical vector quantization methods, including scalar and structured quantization, for compressing dense connected layers.
Results
Up to 24 times compression was achieved while keeping top-five accuracy loss within 1%.
Takeaways & Limitations
Structured quantization can compress CNN parameters beyond scalar k-means while retaining recognition accuracy within the reported scope.
Takeaways & Limitations
The paper mainly focuses on compressing dense connected layers, leaving compression of convolutional layers for future investigation.
Abstract
from arXiv · showhide
Deep convolutional neural networks (CNN) has become the most promising method for object recognition, repeatedly demonstrating record breaking results for image classification and object detection in recent years. However, a very deep CNN generally involves many layers with millions of parameters, making the storage of the network model to be extremely large. This prohibits the usage of deep CNNs on resource limited hardware, especially cell phones or other embedded devices. In this paper, we tackle this model storage issue by investigating information theoretical vector quantization methods for compressing the parameters of CNNs. In particular, we have found in terms of compressing the most storage demanding dense connected layers, vector quantization methods have a clear gain over existing matrix factorization methods. Simply applying k-means clustering to the weights or conducting product quantization can lead to a very good balance between model size and recognition accuracy. For the 1000-category classification task in the ImageNet challenge, we are able to achieve 16-24 times compression of the network with only 1% loss of classification accuracy using the state-of-the-art CNN.
1 INTRODUCTION
CNNs deliver strong recognition performance but are difficult to deploy on embedded platforms because their models contain enormous numbers of parameters. This work studies vector quantization as a storage-focused compression strategy and systematically compares such methods.
- CNNs have become a leading architecture for object recognition, image classification, and retrieval.
- Typical CNNs contain eight layers and about 10^8 parameters, motivating investigation of parameter structure for storage compression.
- The paper focuses on reducing model storage rather than speeding up testing time.
- Vector quantization methods are studied as an alternative to matrix factorization for compressing dense connected layers.
- The study evaluates binarization, k-means scalar quantization, product quantization, and residual quantization.
- Experiments also examine tasks such as image retrieval to assess the compressed model's generalization ability.
2 RELATED WORK
Related work established CNN success and parameter redundancy while emphasizing computation-speed improvements; this paper instead treats that redundancy as an opportunity for model-storage compression.
- State-of-the-art CNNs achieve strong results in image classification, object detection, and image retrieval.
- CNN models commonly contain hundreds of millions of parameters, creating substantial storage and testing-speed bottlenecks.
- Prior compression research showed that a small subset of neural-network parameters can accurately predict weights within a layer.
- The paper frames vector quantization as a compression realization of parameter redundancy, achieving about 20 times compression with little performance decrease.
3 COMPRESS DENSE CONNECTED LAYERS
The paper compares matrix factorization with several vector-quantization schemes for compressing dense-layer parameters. These methods represent weights using compact factors, signs, scalar codebooks, or structured codebooks and indices.
- The compression study first considers matrix factorization and then vector quantization for dense connected-layer parameters.
- MATRIX FACTORIZATION METHODS: SVD approximates a parameter matrix by retaining its leading k singular vectors and singular values.
- MATRIX FACTORIZATION METHODS: SVD compression requires storing two low-rank matrices and eigenvalues, with rate mn/[k(m+n+1)].
- BINARIZATION: Binarization represents each neuron with one bit by turning positive parameters on and negative parameters off.
- SCALAR QUANTIZATION USING kMEANS: Scalar k-means quantization clusters all scalar weights, stores cluster indices and a codebook, and reconstructs weights by lookup.
- SCALAR QUANTIZATION USING kMEANS: With k = 256 centers, scalar quantization uses 8 bits per cluster index and has compression rate 32/log2(k), assuming negligible codebook storage.
- PRODUCT QUANTIZATION: Product quantization partitions the vector space into disjoint subspaces and quantizes each subspace separately.
- PRODUCT QUANTIZATION: Product quantization stores subvector cluster indices and a codebook for each subspace, and can be applied along either matrix axis.
4 EXPERIMENTS
Experiments evaluate vector-quantization methods on ILSVRC classification, comparing compression rate and accuracy across product-quantization settings, methods, and individual layers. Product quantization and k-means provide strong storage–accuracy trade-offs, while compression affects layers differently.
- Experimental setup: The experiments evaluate compression methods using validation accuracy@1 and accuracy@5, targeting higher compression at fixed accuracy or higher accuracy at fixed compression.The evaluation uses the ILSVRC2012 benchmark and compares compression methods under storage–accuracy trade-offs.
- Product quantization analysis: Using more PQ centers and smaller segments reduces classification error when segment size is aligned.The analysis varies k = 4, 8, 16 and segment dimensions s = 1, 2, 3, 4 across axis alignments.
- Product quantization analysis: At the same compression rate, more PQ centers are not always helpful because their codebooks consume storage; k = 16 was not clearly better than k = 8.The x-axis alignment was slightly better, and k = 8 was selected as a balance between compression rate and accuracy.
- Overall comparison: SVD performs well for compressing convolutional layers but poorly for dense connected layers, whereas k-means achieves 4-8 times compression with accuracy loss within 1%.Product quantization improves beyond k-means, while residual quantization has lower compression and worse accuracy at the same compression rate.
- Overall comparison: Product quantization outperforms k-means, indicating meaningful local sub-vector structure, while residual quantization performs poorly, suggesting few global structures in the weight vectors.Binarization is comparable to k-means or product quantization at the same compression rate and is suitable for aggressive compression.
- Single-layer error analysis: Compressing the final classification layer decreases accuracy more than compressing the eighth and ninth hidden layers, and compressing all three layers increases error further.The difference is especially pronounced at high compression rates.
5 APPLICATION TO IMAGE RETRIEVAL
The compressed CNN is evaluated for image retrieval on the Holidays dataset, where product quantization generally performs best. The compressed model can process images on a cellphone and upload only features for database-side retrieval.
- Compressed CNNs can process images on a cellphone while uploading only the resulting feature for retrieval against a database.
- The Holidays benchmark contains 1,491 images from 500 instances, with 500 query images and mean average precision used for evaluation.Each compressed CNN generates 2,048-dimensional last-hidden-layer features, compared using cosine distance.
- Product quantization consistently outperformed the other reported quantization methods on retrieval results.The overall trend matched the classification results.
- Binary k-means with two centers achieved unusually high retrieval performance, exceeding the original feature, likely because binary quantization is robust to small image transformations in near-duplicate retrieval.The authors caution that this case is special and is not an accurate approximation of the original weight matrix.
- The application tests whether compressed CNNs generalize from image classification to image retrieval.The paper states that vector-quantized CNNs can be safely applied beyond image classification.
6 DISCUSSION
The paper addresses CNN model storage in embedded systems through vector quantization, especially for dense connected layers. Scalar and structured quantization achieve substantial compression with limited top-five accuracy loss, while convolutional-layer compression remains future work.
- 6 DISCUSSION: The study systematically applies vector quantization to compress the 108 parameters of deep CNNs for storage reduction.It focuses on dense connected layers and contrasts this direction with prior matrix-factorization approaches.
- 6 DISCUSSION: 8-16 compression rate is achieved with k-means scalar quantization while keeping top-five accuracy loss below 0.5%.
- 6 DISCUSSION: Up to 24 times compression is achieved with structured quantization while keeping top-five accuracy loss within 1%.
- 6 DISCUSSION: More than 20 times parameter compression reduces a roughly 200MB state-of-the-art model to less than 10MB for embedded deployment.The paper also reports that this compression empirically corresponds to about 5% useful CNN parameters.
- 6 DISCUSSION: The paper leaves hardware-efficient operations, fine-tuning, and vector-quantization methods for convolutional layers as future research directions.