Source-linked AI summary
Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
Vadim Lebedev, Yaroslav Ganin, Maksim Rakhuba, Ivan Oseledets, Victor Lempitsky
TL;DR
CNN convolutions are costly on low-end processors, motivating faster convolutional layers. The paper uses low-rank CP-decomposition with non-linear least squares, replaces a layer with four smaller convolutions, and globally fine-tunes the network. It reports considerable speedups with minimal accuracy loss, while performance is stronger for smaller architectures than for AlexNet.
Problem
CNN convolutional computation can be problematic on low-end architectures, especially when real-time operation is required.
Method
The method decomposes a convolutional kernel with low-rank CP-decomposition, replaces the layer with four smaller convolutions, and fine-tunes the entire network by backpropagation.
Results
The approach achieves considerable speedups with minimal accuracy loss and can provide better speed-accuracy tradeoffs than previous methods.
Takeaways & Limitations
The resulting networks reduce parameters several times while remaining accurate, yielding more compact models with reduced memory footprints.
Takeaways & Limitations
The approach is superior for smaller architectures but is not the best-performing method for large networks such as AlexNet.
Abstract
from arXiv · showhide
We propose a simple two-step approach for speeding up convolution layers within large convolutional neural networks based on tensor decomposition and discriminative fine-tuning. Given a layer, we use non-linear least squares to compute a low-rank CP-decomposition of the 4D convolution kernel tensor into a sum of a small number of rank-one tensors. At the second step, this decomposition is used to replace the original convolutional layer with a sequence of four convolutional layers with small kernels. After such replacement, the entire network is fine-tuned on the training data using standard backpropagation process. We evaluate this approach on two CNNs and show that it is competitive with previous approaches, leading to higher obtained CPU speedups at the cost of lower accuracy drops for the smaller of the two networks. Thus, for the 36-class character classification CNN, our approach obtains a 8.5x CPU speedup of the whole network with only minor accuracy drop (1% from 91% to 90%). For the standard ImageNet architecture (AlexNet), the approach speeds up the second convolution layer by a factor of 4x at the cost of $1\%$ increase of the overall top-5 classification error.
1 INTRODUCTION
CNN deployment on low-end CPUs is constrained by convolutional computation, motivating tensor-decomposition methods that reduce cost while preserving accuracy and memory efficiency.
- Motivation: Convolution dominates CNN computation, making efficient convolution implementations important for training and testing.This concern is especially relevant for low-end CPUs, mobile processors, and robotics platforms requiring real-time operation.
- Approach: The method applies low-rank CP-decomposition to the full convolution kernel tensor using non-linear least squares.The approach investigates tensor decomposition specifically for speeding up CNN convolutions.
- Approach: CP-decomposition replaces a convolution with four smaller standard convolutional layers that can be inserted into existing CNN packages.The resulting layers support straightforward global fine-tuning through standard backpropagation.
- Findings: The combination of full-kernel CP-decomposition and global fine-tuning can yield better speed-accuracy tradeoffs than previous methods.The authors identify efficiency as the principal advantage of their approach.
- Findings: The resulting CNNs remain accurate despite reducing parameter counts several times, producing more compact networks with smaller memory footprints.The authors note particular value for architectures with limited RAM or storage memory.
- Implication: The results support the view that modern CNNs are over-parameterized relative to the information needed for classification.The paper relates the excess parameters to facilitating convergence to good local minima.
2 RELATED WORK
Earlier work accelerated CNN convolutions through separable-filter, two-component, and tensor-part decomposition schemes; the paper contrasts these with its full-kernel CP-decomposition and global fine-tuning approach.
- Prior decompositions: Jaderberg et al. approximated the 4D kernel tensor as a product of two 3D tensors, improving on the earlier filter-decomposition scheme.Their decomposition uses an intermediate map stack with rank R.
- Comparison: The paper’s Figure 1 contrasts full convolution, Jaderberg’s two-mapping decomposition, and the four-convolution CP construction.The CP construction computes target values from one-pixel or one-dimensional spatial inputs.
- Fine-tuning: Jaderberg et al. used local fine-tuning to minimize deviations between full and approximated convolution outputs on training data.The present method instead fine-tunes the entire network using the original discriminative criterion.
- Prior decompositions: Denton et al. applied CP-decomposition to biclustered parts of the kernel tensor and computed those decompositions with a greedy approach.Biclustering splits the two non-spatial dimensions into subgroups and reduces effective CP ranks.
3 METHOD
The method decomposes a convolutional kernel with low-rank CP decomposition, replaces the layer with four smaller convolutions, and fine-tunes the entire network by backpropagation. This reduces convolutional complexity while preserving compatibility with standard CNN layers.
- Kernel decomposition: The approach decomposes a convolutional layer’s 4D kernel tensor using low-rank CP decomposition optimized by non-linear least squares.CP decomposition represents the kernel through four component matrices, with rank R controlling the approximation.
- Layer replacement: A CP-decomposed convolution is implemented as a sequence of four convolutions with smaller kernels, including two 1×1 convolutions.The intermediate tensors are standard CNN map stacks, enabling implementation with existing convolution layers.
- Discriminative fine-tuning: All network layers participate in standard backpropagation fine-tuning after the original layer is replaced.The fine-tuning includes layers above and below the approximated layer and all four inserted convolutional layers.
- Complexity analysis: The original convolution uses d^2ST parameters and operations per pixel, whereas the proposed decomposition uses R(S + 2d + T).The stated complexity applies both to parameter count and to multiplication-plus-addition operations per output pixel.
- Complexity analysis: Compared with the two-component scheme of Jaderberg et al., the proposed complexity can improve by a further factor of d when d ≪ T.The comparison assumes comparable rank and uses the complexity expressions stated for the competing approaches.
- Complexity analysis: A theoretical complexity comparison with Denton et al.’s biclustering is not straightforward because biclustering changes both the number of tensors and the required ranks.The paper therefore restricts this comparison to empirical evaluation rather than assuming equal ranks.
4 EXPERIMENTS
Experiments evaluate CP-decomposition across a small character-classification CNN and AlexNet, measuring approximation quality, accuracy, CPU speed, and parameter reduction. The method achieves substantial speedups, but its relative advantage depends on network size and decomposition quality.
- Character-classification CNN: Layer 3 remained accurate despite tensor approximation error as high as 78%.The experiments indicate that accurate kernel approximation is not always necessary for the network to function properly.
- Character-classification CNN: 8.5x whole-network speedup reduced CharNet accuracy by 1% to 90.2%.The procedure approximated layers 2 and 3 with rank 64, fine-tuning after approximating layer 2.
- AlexNet: 3.6x layer speedup at 0.5% accuracy drop surpassed the compared 2x scheme for AlexNet conv2.A rank-140 approximation provided 4.5x speedup at approximately 1% accuracy loss.
- Comparison and limitations: Data-reconstruction refinement failed because larger learning rates caused exploding gradients and smaller rates did not reduce reconstruction loss effectively.The authors suspect instability of low-rank CP-decomposition and suggest alternating component learning as future work.
- Comparison and limitations: The method was superior for smaller architectures but not the best approach for large networks such as AlexNet.For similar approximately 1% accuracy loss, the compared AlexNet method reached 6.6x speedup versus 4.5x for this approach.
- NLS versus greedy: NLS decomposition produced higher accuracy than greedy decomposition, with the advantage increasing for AlexNet.Fine-tuning quality also depended on the quality of the initial approximation.
5 DISCUSSION
The approach combines NLS-based low-rank CP-decomposition with discriminative fine-tuning to obtain considerable speedups with minimal accuracy loss. Results also suggest regularizing effects, while comparisons remain preliminary and the method's applicability to spatially varying kernels is an open direction.
- NLS-based low-rank CP-decomposition combined with discriminative fine-tuning achieves considerable speedups with minimal accuracy loss.The method replaces convolutional layers using low-rank approximations and fine-tunes the network.
- NLS decomposition yields higher accuracy than greedy decomposition, both with and without fine-tuning, especially for AlexNet.
- Preliminary comparisons outperform Denton et al. and Jaderberg et al. for the character classification CNN, but broader comparisons remain necessary.The authors specifically identify comparison with Denton et al. and the usefulness of biclustering with NLS as unresolved.
- Spatially varying-kernel layers may benefit from parameter reduction and additional tensor dimensions could enable further speedups.
- Low-rank decompositions may slightly improve overall accuracy at higher ranks, consistent with a regularizing effect.
APPENDIX
The appendix compares greedy and NLS optimization for CP-decomposition. NLS obtains substantially better approximations than sequentially adding best rank-one tensors, including for a rank-two tensor example.
- NLS consistently produces better CP-decompositions at smaller ranks than the greedy rank-one addition approach.
- For a rank-two 2 × 2 × 2 tensor, two greedy rank-one approximations have relative error 0.35.
- NLS obtains a rank-two approximation of the same tensor with 10^-7 error.