Source-linked AI summary
More is Less: A More Complicated Network with Less Inference Complexity
Xuanyi Dong, Junshi Huang, Yi Yang, Shuicheng Yan
TL;DR
CNN inference is computationally expensive, especially for devices with limited capacity. The paper equips each convolutional layer with a low-cost collaborative layer whose output guides skipped computation, achieving significant speedups across CIFAR and ILSVRC-12 experiments with only slight classification-accuracy loss.
Problem
CNN computation costs remain tremendous, creating a need to accelerate inference for devices with limited computational capacity.
Method
The LCCN equips each original convolutional layer with a low-cost collaborative layer and skips original-layer calculations at zero cells predicted by the LCCL.
Results
34% speedup is reported for LCCN on ResNet-110, while experiments across CIFAR and ILSVRC-12 show significant speed-up with only slight classification-accuracy loss.
Takeaways & Limitations
The data-dependent collaborative structure provides a general way to reduce test-phase CNN computation while retaining near-original classification performance.
Takeaways & Limitations
Realistic speedup is lower than theoretical speedup because data reconstruction is costly and early convolutional layers have less sparsity.
Abstract
from arXiv · showhide
In this paper, we present a novel and general network structure towards accelerating the inference process of convolutional neural networks, which is more complicated in network structure yet with less inference complexity. The core idea is to equip each original convolutional layer with another low-cost collaborative layer (LCCL), and the element-wise multiplication of the ReLU outputs of these two parallel layers produces the layer-wise output. The combined layer is potentially more discriminative than the original convolutional layer, and its inference is faster for two reasons: 1) the zero cells of the LCCL feature maps will remain zero after element-wise multiplication, and thus it is safe to skip the calculation of the corresponding high-cost convolution in the original convolutional layer, 2) LCCL is very fast if it is implemented as a 1*1 convolution or only a single filter shared by all channels. Extensive experiments on the CIFAR-10, CIFAR-100 and ILSCRC-2012 benchmarks show that our proposed network structure can accelerate the inference process by 32\% on average with negligible performance drop.
1. Introduction
The paper targets the high computation costs of CNN inference with a collaborative architecture that uses low-cost layers to skip selected convolution calculations. Experiments report substantial acceleration with limited accuracy loss.
- Motivation: CNN inference remains computationally expensive, motivating acceleration for devices with limited computational capacity.For 224 × 224 images, AlexNet requires 725M FLOPs, VGG-S 2640M FLOPs, and GoogleNet 1566M FLOPs.
- Approach: LCCL response maps predict zero cells after ReLU, allowing corresponding computations in the original convolutional layer to be skipped.The method relies on sparse response maps and computes only non-zero cells in the original layer.
- Approach: The final response maps are produced by element-wise multiplication of the activated outputs from the original layer and LCCL.The architecture can be trained with stochastic gradient descent and backpropagation.
- Approach: The proposed LCCN equips each original convolutional layer with a low-cost collaborative layer to accelerate inference.The architecture is presented as a general approach that applies collaborative layers throughout the network.
- Results: 34% speedup is reported for LCCN on ResNet-110 compared with existing methods.The introduction describes significant improvements across many deep neural networks.
2. Related Work
Prior CNN acceleration methods reduce computation through low-rank decomposition, fixed-point or low-precision representations, quantization and pruning, sparsity constraints, or efficient architecture design.
- Low Rank: Low-rank methods approximate convolutional or fully connected weight tensors to reduce computation complexity.Some approaches decompose an original layer into two approximated layers with reduced computation.
- Fixed Point: Fixed-point approaches accelerate inference by using reduced-precision numerical representations, including binary weights or data tensors.The cited methods use dynamic fixed-point, stochastic rounding, or 1-bit representations.
- Product Quantization: Product-quantization methods compress parameters through quantization, pruning, weight sharing, and low-precision training.One cited approach reduces convolutional parameter precision from 32 to 5 bits after pruning and fine-tuning.
- Sparsity: Sparsity-based methods exploit sparse inputs, neurons, response maps, or convolutional kernels to reduce evaluation cost.The cited methods include sparse constraints, spatially sparse processing, and group-sparsity regularization.
- Architecture: Architecture-based methods design smaller or thinner networks, often using knowledge distillation to preserve performance while reducing parameters and runtime.Examples include student networks distilled from larger teacher models and compact architectures matching AlexNet with fewer parameters.
3. Low-Cost Collaborative Network
The Low-Cost Collaborative Network equips convolutional layers with inexpensive collaborative layers whose outputs guide computation skipping while preserving the original layer’s response structure. Its acceleration depends on sparse LCCL outputs, efficient kernel implementations, activation choices, and the sparsity ratio r.
- Overall structure: The LCCL output matches the corresponding convolutional layer’s spatial and channel dimensions, while its computation cost can be treated as negligible because of its small size.The LCCN therefore retains compatible response-map dimensions while adding a low-cost branch.
- Overall structure: The architecture equips each original convolutional layer with an LCCL and combines their response maps through element-wise multiplication.The resulting response map is collaboratively calculated from the original convolutional output and the LCCL output.
- Overall structure: Zero cells in the LCCL output remain zero after multiplication, allowing corresponding original-convolution calculations to be skipped.This reduces the original layer’s computation according to the LCCL zero-cell positions.
- Kernel selection: The experiments use a k × k × C × 1 LCCL filter shared across original filters because the 1 × 1 × C × T alternative has inefficient GEMV and reconstruction overhead.Both structures are theoretically cheaper than the original convolutional layer, but the 1 × 1 × C × T design is left for future work.
- Activation and integration: Using the same post-BN-and-ReLU input as the original convolutional layer improves zero-cell prediction compared with a pre-activation input strategy.The paper reports that “Bef” inputs are more diverse and therefore less accurate for predicting the original layer’s zero cells.
- Computation complexity: The theoretical speedup ratio depends strongly on sparsity r, while the 1/C term has little effect because CNN input channels are typically wide.The paper reports that high sparsity in certain layers indicates considerable speedup potential.
4. Experiments
Experiments evaluate LCCN across CIFAR-10, CIFAR-100, ILSVRC-12, and several CNN architectures, examining connection strategies, collaborative kernels, sparsity, and speedup. Results generally show meaningful acceleration with slight performance changes, while realistic speedup remains below theoretical estimates.
- Benchmark setup: Experiments validate LCCN on CIFAR-10, CIFAR-100, and ILSVRC-12 using multiple CNN architectures.The study primarily targets deep residual networks and evaluates both benchmark performance and inference acceleration.
- Connection strategies: After-activation connection strategies provide the best performance with a considerable speedup ratio.The authors attribute this to the collaborative layer receiving the same input distribution as the corresponding convolutional layer.
- Collaborative kernels: k × k collaborative kernels significantly improve performance over 1 × 1 kernels with a similar speedup ratio.The authors suggest the larger receptive field of k × k kernels may explain this improvement.
- Sparsity and speedup: More than 50% of computation cost is saved in the first few collaborative layers, while k × k × C × 1 methods achieve more than 30% increase in speed overall.The sparsity statistics are measured across 400 training epochs on ResNet-20 equipped with 1 × 1 × C × T LCCLs.
- Architecture results: LCCN works on deeper networks and can achieve more than a 30% calculation reduction in deep residual networks.On CIFAR-100, LCCN on WRN-52-1 obtains higher accuracy than original WRN-40-1 with about 2% more FLOPs.
- Comparison with existing methods: LCCN achieves meaningful speedup with slight performance drops on ResNet-18 and ResNet-34, and better accuracy than other compared acceleration methods.The comparison reports some performance drop, but the method outperforms the other acceleration methods in accuracy.
5. Conclusion
The paper proposes LCCN, a more complicated CNN structure that reduces inference complexity by using low-cost collaborative layers to predict zero cells and skip corresponding computations.
- LCCN equips each original convolutional layer with a low-cost collaborative layer to accelerate deep CNN inference.
- The collaboration speeds test-phase computation by skipping calculations for zero cells predicted by the collaborative layer.
- ReLU and BN enhance sparsity while helping maintain performance in LCCN structures.
- LCCN provides data-dependent acceleration rather than relying on hard acceleration structures.
- Experiments on CIFAR and ILSVRC-12 report significant speed-up with only slight classification-accuracy loss.
- The approach is presented as applicable to convolutional-network tasks such as detection, segmentation, and identification, and compatible with other acceleration algorithms.