Source-linked AI summary
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
Yang He, Guoliang Kang, Xuanyi Dong, Yanwei Fu, Yi Yang
TL;DR
Deep CNNs are expensive to deploy because their accuracy often requires deep and wide architectures, while hard filter pruning reduces capacity and depends on pre-trained models. The paper proposes Soft Filter Pruning, which repeatedly zeroes low-norm filters but continues updating them during training. SFP reports competitive or improved performance while accelerating CNN inference, including more than 40% FLOPs reduction with a 0.18% accuracy gain for ResNet-110 on CIFAR-10 using a pre-trained model.
Problem
Deep CNNs can exceed mobile-device limits in storage, memory, and computation, motivating compact models with low cost and high accuracy.
Method
SFP repeatedly selects small-ℓ2-norm filters, sets them to zero, and updates them during subsequent training rather than permanently removing them.
Results
SFP achieves competitive performance without a pre-trained model, while using a pre-trained model yields higher performance and advances the state of the art.
Takeaways & Limitations
SFP preserves model capacity during pruning and can be combined with matrix decomposition and low-precision weights for further performance improvement.
Takeaways & Limitations
FLOPs reduction does not necessarily produce equivalent realistic speedup because non-tensor operations, I/O delay, buffer switching, and BLAS efficiency also affect inference time.
Abstract
from arXiv · showhide
This paper proposed a Soft Filter Pruning (SFP) method to accelerate the inference procedure of deep Convolutional Neural Networks (CNNs). Specifically, the proposed SFP enables the pruned filters to be updated when training the model after pruning. SFP has two advantages over previous works: (1) Larger model capacity. Updating previously pruned filters provides our approach with larger optimization space than fixing the filters to zero. Therefore, the network trained by our method has a larger model capacity to learn from the training data. (2) Less dependence on the pre-trained model. Large capacity enables SFP to train from scratch and prune the model simultaneously. In contrast, previous filter pruning methods should be conducted on the basis of the pre-trained model to guarantee their performance. Empirically, SFP from scratch outperforms the previous filter pruning methods. Moreover, our approach has been demonstrated effective for many advanced CNN architectures. Notably, on ILSCRC-2012, SFP reduces more than 42% FLOPs on ResNet-101 with even 0.2% top-5 accuracy improvement, which has advanced the state-of-the-art. Code is publicly available on GitHub: https://github.com/he-y/soft-filter-pruning
1 Introduction
Deep CNNs deliver strong accuracy through deeper and wider architectures, but their storage, memory, and computation can exceed mobile-device limits. SFP addresses hard-pruning capacity loss and pre-trained-model dependence by updating pruned filters during training.
- Motivation: 11.3 billion FLOPs are required for ResNet-152 to process one image on CPU, alongside 60.2 million parameters and 231MB storage.The model also needs more than 380MB of memory and six seconds per image.
- Motivation: Mobile deployment requires deep CNNs with low computational cost, small size, and high accuracy.
- Prior limitations: Structured filter pruning improves practical efficiency over unstructured weight pruning by reducing memory usage and computational cost.Unstructured models cannot leverage existing high-efficiency BLAS libraries.
- Prior limitations: Hard filter pruning reduces model capacity and commonly depends on fine-tuning a pre-trained model, increasing training time.Discarded filters remain fixed, while prior methods prune filters from pre-trained models and then fine-tune them.
- Soft Filter Pruning: SFP sets small-norm filters to zero, updates them during subsequent training, and repeatedly prunes new filters until convergence.This procedure is intended to preserve larger model capacity than hard pruning.
- Contributions: ResNet-110 was accelerated two times with about 4% relative accuracy improvement on CIFAR-10, while ILSVRC-2012 experiments achieved state-of-the-art results.
2 Related Works
CNN acceleration research includes matrix decomposition, low-precision weights, and pruning. SFP differs from prior soft weight pruning by targeting filters and evaluating compression and acceleration on broader, more advanced CNN settings.
- Acceleration approaches: Matrix decomposition approximates deep CNN tensors with products of low-rank matrices to save computational cost.
- Acceleration approaches: Low-precision weights compress CNNs by representing model parameters with reduced numerical precision.
- Weight pruning: Weight-pruning methods discard small weights or exploit sparsity in feature maps and parameters to reduce model size or training iterations.
- Filter pruning: Filter pruning removes corresponding feature maps, reducing storage and memory consumption while accelerating inference through structured sparsity.
- SFP distinction: SFP focuses on filter pruning for compression and acceleration, whereas prior soft pruning focused on weight compression and was evaluated only on AlexNet.The paper reports extensive experiments on large-scale datasets and state-of-the-art CNN models.
3 Methodology
SFP prunes filters by importance while continuing to update the zeroed filters during training, preserving representational capacity and enabling parallel pruning. Filter removal reduces feature-map dimensions and convolutional computation, although FLOPs do not equal realistic speedup.
- 3.1 Preliminaries: W(i) contains the convolution filters of layer i, while U and V are its input and output tensors with channel and spatial dimensions.The i-th layer has Ni input channels, Ni+1 output channels, and L denotes the number of layers.
- 3.1 Preliminaries: Pruning Ni+1Pi filters reduces the output tensor channels to Ni+1(1 − Pi), also reducing the next layer’s input size.Removing output feature maps decreases the computational cost of the network.
- 3.2 Soft Filter Pruning: Hard pruning permanently removes selected filters and typically alternates layerwise pruning with convergence training on a pre-trained model.Because pruned filters are not updated again, the model’s capacity is reduced.
- 3.2 Soft Filter Pruning: SFP dynamically updates zeroed filters during training, preserving the compressed model’s representative capacity and avoiding greedy layer-by-layer pruning.The algorithm can prune almost all layers at the same time.
- 3.2 Soft Filter Pruning: SFP ranks filters by ℓp-norm and selects the lowest Ni+1Pi filters for pruning, using ℓ2-norm in practice.
- 3.2 Soft Filter Pruning: After pruning, one training epoch reconstructs pruned filters through back-propagation, allowing them to become non-zero again.This gives the pruned model the same training capacity as the original model during training.
- 3.2 Soft Filter Pruning: Pruning can produce a compact inference model because converged zero filters correspond to feature maps that remain zero and can be removed.
- 3.3 Computation Complexity Analysis: 1 − (1 − Pi+1)(1 − Pi) of the original calculation is reduced across adjacent pruned layers, accelerating neural-network inference.The reduction follows from decreased useful output channels in one layer and decreased input channels in the next.
4 Evaluation and Results
Experiments on CIFAR-10 and ILSVRC-2012 evaluate SFP across ResNet architectures, pruning settings, and speedup measures. SFP generally preserves or improves accuracy while reducing computation, with performance depending on pruning rate, interval, and layer selection.
- Experimental setting: SFP is evaluated on CIFAR-10 and ILSVRC-2012 using ResNet architectures, with repeated experiments reporting mean ± standard deviation.CIFAR-10 tests ResNet-20, 32, 56, and 110; ILSVRC-2012 tests ResNet-18, 34, 50, and 101.
- ResNet on CIFAR-10: 40.8% speed-up with only 0.30% accuracy drop is achieved for ResNet-110 on CIFAR-10 without fine-tuning.Using a pre-trained model, SFP exceeds the original model by 0.18% while reducing more than 40% FLOPs.
- ResNet on ILSVRC-2012: More than 40% FLOPs reduction with a 0.2% top-5 accuracy increase is reported for a pruned pre-trained ResNet-101 on ILSVRC-2012.For ResNet-34, SFP without fine-tuning also exceeds the compared hard-pruning model by 2.57% accuracy.
- Ablation study: 93.89±0.19%, 93.93±0.41%, and 93.38±0.30% accuracy are obtained with ℓ2-norm selection at 10%, 20%, and 30% pruning rates.The ℓ2-norm criterion performs slightly better than ℓ1-norm selection in the reported comparisons.
- Ablation study: For pruning rates between 0% and about 23%, SFP accuracy exceeds the baseline model, while higher rates produce an approximately linear decline.The authors attribute the initial improvement to a regularization effect that reduces over-fitting.
- Ablation study: 93.96 ± 0.13% accuracy results from pruning first block layers, compared with 93.38 ± 0.44% when pruning second block layers.Different layers have different sensitivity, so layer selection may improve performance at the cost of additional hyper-parameters.
5 Conclusion and Future Work
SFP accelerates deep CNNs by updating pruned filters during training, preserving model capacity. It performs competitively without pre-training and improves results when using a pre-trained model.
- SFP updates pruned filters during training to maintain model capacity and achieve superior performance.
- SFP achieves competitive performance without relying on a pre-trained model.
- Using a pre-trained model further improves SFP and advances the state of the art in model acceleration.
- SFP can combine with matrix decomposition and low-precision weights to further improve performance.