Source-linked AI summary
Channel Gating Neural Networks
Weizhe Hua, Yuan Zhou, Christopher De Sa, Zhiru Zhang, G. Edward Suh
TL;DR
CNNs have substantial computation and memory requirements, while many existing reduction techniques optimize inference statically rather than exploiting input-specific runtime characteristics. The paper introduces channel gating, a dynamic fine-grained pruning scheme with hardware-efficient execution, and reports reduced computation with minimal accuracy loss across CNN evaluations.
Problem
CNN computation and memory requirements limit deployment on constrained devices, while most existing cost-reduction techniques optimize networks statically and are agnostic to runtime input data.
Method
Channel gating divides layers into base and conditional paths and uses learnable activation-aware gates to skip selected channel computations for ineffective output activations.
Results
2.7-8.0× FLOP reduction is achieved with minimal accuracy degradation on CIFAR-10, while an ASIC accelerator provides 2.4× speed-up for a CGNet with 2.8× theoretical FLOP reduction.
Takeaways & Limitations
Channel gating provides better trade-offs between accuracy and computation cost than existing pruning techniques and can be efficiently realized on dense systolic arrays.
Takeaways & Limitations
Channel gating depends on partial sums being good predictors of final sums, and the reported future work includes applying it to object detection tasks.
Abstract
from arXiv · showhide
This paper introduces channel gating, a dynamic, fine-grained, and hardware-efficient pruning scheme to reduce the computation cost for convolutional neural networks (CNNs). Channel gating identifies regions in the features that contribute less to the classification result, and skips the computation on a subset of the input channels for these ineffective regions. Unlike static network pruning, channel gating optimizes CNN inference at run-time by exploiting input-specific characteristics, which allows substantially reducing the compute cost with almost no accuracy loss. We experimentally show that applying channel gating in state-of-the-art networks achieves 2.7-8.0$\times$ reduction in floating-point operations (FLOPs) and 2.0-4.4$\times$ reduction in off-chip memory accesses with a minimal accuracy loss on CIFAR-10. Combining our method with knowledge distillation reduces the compute cost of ResNet-18 by 2.6$\times$ without accuracy drop on ImageNet. We further demonstrate that channel gating can be realized in hardware efficiently. Our approach exhibits sparsity patterns that are well-suited to dense systolic arrays with minimal additional hardware. We have designed an accelerator for channel gating networks, which can be implemented using either FPGAs or ASICs. Running a quantized ResNet-18 model for ImageNet, our accelerator achieves an encouraging speedup of 2.4$\times$ on average, with a theoretical FLOP reduction of 2.8$\times$.
1 Introduction
CNNs impose substantial computation and memory costs, while most existing reduction methods optimize networks statically without using input-specific runtime characteristics. Channel gating addresses this with dynamic, fine-grained pruning designed for efficient hardware implementation.
- CNN computation and memory requirements limit deployment on constrained mobile and embedded devices.
- Channel gating divides each CNN layer into base and conditional paths, using partial sums to decide whether additional channel computation is needed.
- Channel grouping is introduced to include and select input channels equally, avoiding biased training updates between base and conditional paths.
- 2.4× speed-up is achieved by the ASIC accelerator for a CGNet with a theoretical FLOP reduction of 2.8×.
2 Related Work
Prior pruning methods remove channels or computations using static criteria or adaptive strategies at other network granularities. The paper positions channel gating as a dynamic alternative that operates on fine-grained output activations.
- Static Pruning: Static pruning identifies ineffective channels from weight or activation magnitudes, then retrains the pruned model to reduce accuracy loss.
- Dynamic Pruning: Channel gating generates binary pruning decisions from partial sums for each output activation using base and conditional paths.
- Static Pruning: PerforatedCNN skips output-activation computations at fixed spatial locations, whereas channel gating uses input-dependent decisions.
- Dynamic Pruning: SACT adaptively stops computation in spatial regions reaching a predefined confidence level by adjusting the number of residual units.
3 Channel Gating
Channel gating dynamically skips conditional convolution computation for ineffective output activations, using partial sums and learned gates. Channel grouping reduces training bias while preserving hardware-efficient execution and reducing memory access.
- Channel Gating Block: Channel gating splits each layer into always-executed base and conditionally executed paths, using partial convolutions to decide whether remaining channels are needed.The final output combines the base and conditional paths element-wise.
- Channel Gating Block: 0.86: partial and final sums reach an average Pearson correlation of 0.86 when η is 1/2 across 20 ResNet-18 convolutional layers.The measurements use 1,000 training samples and also report 0.56 and 0.72 for larger partial-channel fractions.
- Channel Gating Block: Partial sums can identify and approximate ineffective output activations even when only 1/8 of the channels compute the partial sum.They do not accurately predict exact values for every output activation.
- Learnable Gate Functions: Per-output-channel learnable thresholds and activation-specific Heaviside gates are trained to route only a small fraction of activations through the conditional path.ReLU uses one threshold, while sigmoid and hyperbolic tangent use upper and lower thresholds for saturated values.
- Learnable Gate Functions: Channel-wise gating skips an entire output channel when fewer than τc of its activations use the conditional path, reducing weight loads with (w_l+1 · h_l+1 + 1)·c_l+1 comparisons per layer.The channel-wise gate adds one threshold and c_l+1 additional comparisons.
- Unbiased Channel Selection with Channel Grouping: Channel grouping makes every input channel serve as a base-path input once and as a conditional-path input G−1 times, equalizing weight-update frequency.An optional channel shuffle can improve cross-group information flow.
4 Training CGNet
CGNet training combines differentiable gate approximation, sparsity-inducing objectives, and knowledge distillation to reduce computation while limiting accuracy loss.
- Differentiable gate training: Separate batch-normalization layers balance the base and conditional paths before their outputs are combined during training.Directly applying batch normalization after combining the paths can diminish the base path's contribution.
- Differentiable gate training: A smooth function approximates the non-differentiable gate during backward propagation, enabling gradients with respect to x and Δ.The approximation is s(x, Δ) = 1/(1+e^(ε·(x−Δ))).
- Inducing sparsity: The sparsity objective targets a threshold Δ because increasing Δ monotonically increases the pruning ratio and reduces FLOPs.The loss includes a squared difference between Δ and the target threshold T.
- Inducing sparsity: The proposed threshold-based objective keeps pruning ratios more balanced across layers than directly optimizing computation cost.The alternative loss preferentially prunes higher-FLOP layers and creates imbalanced layerwise pruning.
- Knowledge distillation: Knowledge distillation trains a student network from softened teacher outputs, using a ResNet-50 teacher for ImageNet ResNet-18 CGNets.The reported setup uses κ = 1 and λkd = 0.5.
5 Experiments
Experiments show that channel gating substantially reduces computation and memory costs across CNN architectures while preserving accuracy, and that its dynamic sparsity can be efficiently exploited by specialized hardware.
- 2.7–8.0× computation reduction is achieved with minimal accuracy degradation across five state-of-the-art CIFAR-10 architectures.
- Channel gating outperforms alternative pruning methods on ResNet and MobileNet, combining smaller accuracy drops with higher FLOP savings.
- On ImageNet ResNet-18, channel grouping raises top-1 accuracy by 0.9% and FLOP reduction by 20% versus ungroupped gating.
- Knowledge distillation increases ImageNet ResNet-18 FLOP saving from 1.93× to 2.55× while improving top-1 accuracy by 1.3%.
- 3.1× and 4.4× weight-access reductions are obtained with 0.6% and 0.3% accuracy degradation at activation thresholds T=1.5 and T=2.0, respectively.
- FLOP reduction is used as a proxy for CPU/GPU speed-up, though sampled dense matrix multiplication may help implement conditional paths efficiently.
- The ASIC accelerator achieves 2.4× actual speed-up versus 2.8× theoretical FLOP reduction, indicating effective hardware exploitation of dynamic sparsity.
6 Conclusions and Future Work
The paper concludes that channel gating provides favorable accuracy–computation trade-offs through learned dynamic pruning, while identifying object detection as future work.
- Channel gating and its from-scratch training method provide better accuracy–computation trade-offs than existing pruning techniques.
- Applying channel gating to object detection is identified as potential future work.