Source-linked AI summary
Speeding up Convolutional Neural Networks with Low Rank Expansions
Max Jaderberg, Andrea Vedaldi, Andrew Zisserman
TL;DR
CNNs are computationally demanding, limiting their deployability despite strong performance. The paper approximates convolutional filter banks with low-rank, spatially rank-1 bases by exploiting channel and filter redundancy, achieving 2.5× speedup without accuracy loss and 4.5× speedup with a 1% accuracy drop in scene-text character recognition.
Problem
CNNs can deliver strong performance but are computationally costly because convolutional layers consume substantial processing time.
Method
The paper approximates full-rank convolutional filter banks using combinations of rank-1 filter bases, with two schemes and optimization techniques targeting filter or layer-output reconstruction.
Results
2.5× speedup is achieved with no loss in accuracy, while 4.5× speedup yields a 1% classification-accuracy drop to 90.3% on scene-text character recognition.
Takeaways & Limitations
The resulting approximations require significantly fewer operations and provide tunable speedups for convolutional neural networks, including sliding-window applications.
Takeaways & Limitations
Dropout led to under-fitting during the heavily approximated training setup, and Scheme 1 timings may fall short of theoretical speedups because of costly im2col and BLAS calls.
Abstract
from arXiv · showhide
The focus of this paper is speeding up the evaluation of convolutional neural networks. While delivering impressive results across a range of computer vision and machine learning tasks, these networks are computationally demanding, limiting their deployability. Convolutional layers generally consume the bulk of the processing time, and so in this work we present two simple schemes for drastically speeding up these layers. This is achieved by exploiting cross-channel or filter redundancy to construct a low rank basis of filters that are rank-1 in the spatial domain. Our methods are architecture agnostic, and can be easily applied to existing CPU and GPU convolutional frameworks for tuneable speedup performance. We demonstrate this with a real world network designed for scene text character recognition, showing a possible 2.5x speedup with no loss in accuracy, and 4.5x speedup with less than 1% drop in accuracy, still achieving state-of-the-art on standard benchmarks.
1 Introduction
CNNs achieve strong benchmark performance but are computationally costly because they use chains of convolutional layers. Existing acceleration approaches include hardware-specific optimizations, transform methods, quantization, and search-space reduction, while this paper proposes architecture-agnostic redundancy exploitation.
- CNNs deliver state-of-the-art performance but often require GPUs or highly optimized distributed CPU architectures to process large datasets.
- The paper exploits redundancy between feature channels and filters through two approximation schemes and two optimization methods per scheme.
- The proposed methods are orthogonal to architecture-specific optimizations and can be applied to existing CPU and GPU software.
- Prior general methods include low-rank filter approximations, rank-1 filters designed into networks, and 8-bit quantization, each with stated applicability or accuracy limitations.
- Hardware-specific alternatives use optimized CPU/GPU code, FFT-based convolution, fixed-point instructions, aligned memory, or bespoke FPGA implementations.
- Sliding-window inference can also be accelerated through flattened multiscale pyramids and selective search that reduces the number of full CNN evaluations.
2 Filter Approximations
The paper approximates CNN convolutional layers with low-rank filter bases that exploit spatial separability and redundancy across channels or filters. Two schemes reduce computation by replacing expensive full convolutions with separable basis operations and recombination.
- Basis approximation: A smaller basis of M filters can generate approximate feature maps through convolution followed by a cheap linear combination.The basis reduces the number of filters convolved with the input; its complexity is O((d^2M + MN)H′W′).
- Layer formulation: CNN convolutional layers apply full-rank 3D filters across C input channels, costing O(CNd^2H′W′).Each 3D filter comprises 2D filters for the input channels, whose responses are summed to produce an output feature map.
- Design rationale: Both schemes use low-rank bases of spatially separable filters, extending spatial low-rank approximation by exploiting cross-channel redundancy.The approach is presented as architecture agnostic and applicable to existing CPU and GPU software.
- Scheme 2: Scheme 2 factors each convolutional layer into two rectangular-filter convolutional layers, exploiting redundancy across both input and output channels.Unlike Scheme 1, it considers 3D filters throughout and combines input- and output-channel redundancy.
- Optimization caveat: Optimizing approximations within the full network can overfit training data, while regularization such as dropout can lead to under-fitting.The paper reports that this alternative does not improve classification accuracy over L2 data-reconstruction optimization.
3 Experiments & Results
Experiments on a scene-text character CNN evaluate layer-wise reconstruction, implementation effects, and end-to-end accuracy–speed trade-offs for the two approximation schemes. Scheme 2 achieves substantial measured speedups with minimal accuracy loss, while hardware and convolution-routine details affect realized performance.
- Experimental setup: The experiments use a four-layer CNN with a softmax output for case-insensitive scene-text character classification.The model processes normalized 24 × 24 grayscale patches and uses Conv1–Conv4 followed by softmax.
- Datasets & Evaluation: The training set contains 163,222 character samples, while the ICDAR 2003 test set contains 5,379 cropped characters.Evaluation reports case-insensitive accuracy while ignoring the background class.
- Layer-wise Performance: Reconstruction error increases as theoretical and practical speedup increases, while data reconstruction generally yields lower error than filter reconstruction at the same speedup.This pattern also holds for random Gaussian noise passed through the approximated layers.
- Layer-wise Performance: Scheme 1 has a better theoretical reconstruction-error-to-speedup ratio, but Scheme 2 is faster in practice under Caffe’s 3D-convolution implementation.Scheme 2 needs only two im2col and BLAS calls, whereas Scheme 1 requires more costly per-channel calls.
- Full Net Performance: The approximations support fast sliding-window detection maps, with Scheme 2 retaining sufficient quality for text localization at 6.7× speedup.The authors report very little difference even at 3.5× speedup and describe the end-to-end speedup as tunable.
- Full Net Performance: 2.5× speedup under Scheme 2 causes no accuracy loss, while 4.5× speedup causes a 1% drop and yields 90.3% classification accuracy.The 4.5× configuration uses 31 horizontal and 128 vertical filters for Conv2, plus 26 horizontal and 512 vertical filters for Conv3.
- Comparisons: Compared with an FFT-based CNN, Scheme 2 reaches 2.4× actual speedup versus 2.2× under the same layer setup.On OverFeat, Scheme 2 gives a 2× theoretical speedup with a 0.5% top-5 accuracy drop, compared with 1.2% for the reported alternative.
4 Conclusions
The paper exploits redundancy in CNN convolutional layers by approximating full-rank filter banks with rank-1 filter bases, substantially reducing computation. In a scene text recognition CNN, this produced large speedups, while future work targets alternative separable-filter arrangements and discriminative training.
- Conclusions: 4.5× speedup was achieved with only a 1% drop in classification accuracy in a scene text character-recognition CNN.The approximations required significantly fewer operations to compute.
- Conclusions: The proposed approach exploits representational redundancy by approximating learnt full-rank filter banks as combinations of rank-1 filter bases.The paper presented two approximation schemes and two optimization techniques for attaining them.
- Future work: Future work could investigate alternative arrangements of separable filters, including separate horizontal and vertical basis layers followed by linear combination.The paper also suggests exploring connections between the two approximation structures.
- Future work: Further investigation is needed into whether low-rank filter layers can be learned effectively during training in a discriminative manner.