Source-linked AI summary
Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning
Tien-Ju Yang, Yu-Hsin Chen, Vivienne Sze
TL;DR
Battery-powered devices face CNN energy constraints, and model size or operation count alone does not reliably estimate energy consumption. The paper proposes energy-aware pruning guided by modeled computation and memory-access energy, achieving substantial energy reduction while preserving accuracy.
Problem
Battery-powered devices cannot readily run state-of-the-art CNNs because energy consumption is a primary deployment constraint, while weight and operation counts do not directly reflect actual energy use.
Method
The paper estimates CNN energy from computation and memory accesses, then prunes layers by energy impact and output-feature-map changes, followed by global fine-tuning.
Results
1.6× energy reduction is achieved for GoogLeNet with the proposed pruning method.
Takeaways & Limitations
Optimizing CNN energy requires accounting for feature maps and memory accesses rather than minimizing weights or MACs alone; pruning performance also depends on the number of target classes.
Takeaways & Limitations
Prior pruning methods that target model size do not minimize the pruned CNNs' energy consumption because weight count alone does not dictate energy use.
Abstract
from arXiv · showhide
Deep convolutional neural networks (CNNs) are indispensable to state-of-the-art computer vision algorithms. However, they are still rarely deployed on battery-powered mobile devices, such as smartphones and wearable gadgets, where vision algorithms can enable many revolutionary real-world applications. The key limiting factor is the high energy consumption of CNN processing due to its high computational complexity. While there are many previous efforts that try to reduce the CNN model size or amount of computation, we find that they do not necessarily result in lower energy consumption, and therefore do not serve as a good metric for energy cost estimation. To close the gap between CNN design and energy consumption optimization, we propose an energy-aware pruning algorithm for CNNs that directly uses energy consumption estimation of a CNN to guide the pruning process. The energy estimation methodology uses parameters extrapolated from actual hardware measurements that target realistic battery-powered system setups. The proposed layer-by-layer pruning algorithm also prunes more aggressively than previously proposed pruning methods by minimizing the error in output feature maps instead of filter weights. For each layer, the weights are first pruned and then locally fine-tuned with a closed-form least-square solution to quickly restore the accuracy. After all layers are pruned, the entire network is further globally fine-tuned using back-propagation. With the proposed pruning method, the energy consumption of AlexNet and GoogLeNet are reduced by 3.7x and 1.6x, respectively, with less than 1% top-5 accuracy loss. Finally, we show that pruning the AlexNet with a reduced number of target classes can greatly decrease the number of weights but the energy reduction is limited. Energy modeling tool and energy-aware pruned models available at http://eyeriss.mit.edu/energy.html
1. Introduction
CNNs achieve state-of-the-art vision performance but remain difficult to deploy on battery-powered devices because their processing consumes substantial energy. The paper addresses this gap with energy estimation and pruning that directly target energy consumption while limiting accuracy loss.
- Motivation: CNN processing consumes enough energy that battery-powered devices cannot afford real-time state-of-the-art vision workloads.The paper cites smartphones being unable to run AlexNet object classification in real time for more than an hour.
- Motivation: Weight count and operation count do not directly reflect CNN energy consumption because memory accesses can dominate computation.DRAM data movement can consume orders of magnitude more energy than computation, and CNN shape configurations determine memory-access costs.
- Approach: The proposed design methodology uses estimated energy, rather than weights or operations, to guide energy-efficient CNN design.The estimator models computation and memory accesses using parameters extrapolated from hardware measurements and also accounts for sparsity and bitwidth.
- Approach: The pruning algorithm minimizes output-feature-map changes, identifies high-energy layers, and prunes more aggressively while targeting marginal accuracy degradation.It combines layer-wise energy guidance with feature-map-based pruning rather than filter-weight changes.
2. Energy Estimation Methodology
The methodology estimates CNN energy by combining computation and memory-movement costs while accounting for data reuse, memory hierarchy, sparsity, and bitwidth. This enables design-time comparison and energy-aware optimization of CNN models.
- Energy sources: CNN energy arises from MAC computation and memory accesses, whose scaling differs because data reuse changes required movement.MAC operations dominate operation counts, but reused weights and activations prevent data movement from scaling directly with MAC count.
- Estimation framework: The estimator minimizes memory-access energy across hierarchy levels and uses hardware-extrapolated energy values for computation and data movement.The methodology is based on a prior framework and targets realistic battery-powered systems.
- Memory hierarchy: Memory hierarchy makes actual energy depend on where reused data is accessed, not merely on the amount of reuse.Accesses from DRAM and multi-level buffers have different costs, so hierarchy-aware mapping is needed to minimize energy.
- Estimation framework: For each layer, computation energy is based on MAC count and per-MAC cost, while data energy is based on memory-access counts and costs.The framework explicitly separates computation energy, Ecomp, from data-movement energy, Edata.
- Extensions: Sparsity and bitwidth reduction are modeled by skipping zero-related work, compressing sparse data, and scaling component energy costs.Multiplier energy scales quadratically with bitwidth, whereas memory-access energy scales linearly.
- Use: The methodology supports design-time energy comparison and provides feedback for CNN designs with reduced energy consumption.It is used to quantify differences among CNN models and efficiency techniques.
3. CNN Pruning: Related Work
Prior CNN pruning methods primarily reduce model size or independently remove weights, which can limit compression and fail to minimize energy. The paper instead proposes energy-guided pruning that considers joint output effects.
- Weight pruning: Magnitude-based and sensitivity-based methods prune weights to reduce model size while attempting to maintain accuracy.Several approaches iteratively prune and fine-tune networks, while others allow previously pruned weights to return.
- Weight pruning: Independent weight evaluation ignores correlations, so the combined effect of many removed weights can limit achievable compression.This limitation becomes important when the compression ratio is large.
- Filter pruning: Filter pruning removes or merges entire filters using activation sparsity or filter similarity rather than pruning individual weights.The related approaches include removing frequently zero-producing filters and merging similar filters.
- Energy gap: Reducing model size alone does not guarantee minimized CNN energy because weight count does not determine memory-access costs.The paper therefore distinguishes prior compression objectives from energy optimization.
- Proposed direction: The proposed fine-grained method targets high-energy layers and joint weight influence on output feature maps to improve energy efficiency and compression.It combines energy guidance with output-aware pruning.
- Extensions: Energy-aware pruning can be combined with bitwidth reduction, weight sharing, coding, distillation, decomposition, and feature-map pruning.These techniques are presented as complementary ways to further reduce energy consumption.
4. Energy-Aware Pruning
Energy-aware pruning sparsifies CNN filters by prioritizing high-energy layers and minimizing output feature-map error, with local and global fine-tuning to preserve accuracy.
- The method takes a CNN and produces a sparser model with lower energy consumption while targeting minimal accuracy impact.
- Each layer undergoes magnitude-based removal, residual-driven weight restoration, and local fine-tuning while minimizing output feature-map error.Restored weights are selected based on their effect on residual reduction, and local least-square optimization changes retained weight values.
- Pruning order is determined by estimated layer energy, so the highest-energy layers are pruned first to maximize overall energy reduction.The order is redetermined at the start of each outer-loop iteration using updated layer-energy estimates.
- The layer computation is represented as Yi = XiAi + Bi1, with convolution converted to matrix multiplication through a Toeplitz input representation.For fully connected layers, Ai is a filter, Xi contains corresponding input feature maps, and Bi is the filter bias.
- The restoration procedure starts from over-pruned filters and greedily restores weights until the target number of non-zero weights is reached.Weights are restored from the filter with the largest residual, with multiple top-improvement weights optionally restored per iteration.
- Local least-square optimization efficiently adjusts retained weights, followed by global back-propagation fine-tuning with pruned weights fixed at zero.Local fine-tuning restores part of the accuracy before global fine-tuning, allowing more aggressive pruning under a given accuracy-loss tolerance.
5. Experiment Results
Experiments show that energy-aware pruning reduces energy while preserving accuracy, and that energy depends on data movement and layer structure—not only model size or MACs. Reducing target classes shrinks weights and MACs faster than energy, because feature-map movement remains comparatively resistant to pruning.
- Pruning Method Evaluation: All energy-aware pruned networks lose less than 1% accuracy, while AlexNet gains a further 3.2× MAC reduction and 1.7× energy reduction over magnitude-based pruning.On AlexNet, the proposed method also achieves a 15% smaller model size than the comparison method.
- Pruning Method Evaluation: 2.9× model-size, 3.4× MAC, and 1.6× estimated-energy reductions are achieved on GoogLeNet.The method remains effective on compact CNN architectures.
- Energy Consumption Analysis: 72.6% of original AlexNet energy comes from convolutional layers, despite those layers containing only 3.8% of the weights.Convolutional layers consume more energy because feature-map movement and repeated weight use increase their cost.
- Energy Consumption Analysis: Energy-aware pruning provides a better accuracy-versus-energy trade-off than popular CNN models, shown by a steeper accuracy-energy slope.The comparison concerns estimated energy and top-5 accuracy across dense and pruned models.
- Energy Consumption Analysis: Lower MAC count does not necessarily mean lower energy: pruned GoogLeNet uses fewer MACs but consumes more energy than magnitude-pruned SqueezeNet.Different data reuse patterns caused by layer shapes help explain the discrepancy.
- Number of Target Class Reduction: As target classes decrease from 1000 to 100 or 10, weights, MACs, and estimated energy all decline, but model size falls fastest and energy falls slowest.The two 10-class models have similar compression ratios and performance, suggesting preserved-class type is less influential than class count.
- Number of Target Class Reduction: Reducing target classes makes energy less sensitive to filter sparsity because feature-map energy gradually saturates through data reuse and memory hierarchy effects.An activation fetched from DRAM can be reused by several weights, so pruning some weights does not eliminate every fetch.
6. Conclusion
The work introduces energy-aware CNN pruning guided by estimated computation and memory-access energy, rather than model size or MAC count alone. It reduces energy substantially on AlexNet and GoogLeNet, while examining the limited energy benefit of reducing target classes.
- 3.7× and 1.6× energy reductions are achieved for AlexNet and GoogLeNet, respectively, compared with their original dense models.The method estimates CNN energy from computation and memory accesses using values extrapolated from actual hardware measurements.
- Energy consumption directly guides layer-by-layer pruning, beginning with the highest-energy layers and removing weights with the smallest joint impact on output feature maps.
- The energy methodology models computation and memory accesses to improve estimation over using model size or MAC count alone.
- Reducing AlexNet’s target classes greatly decreases the number of weights, but energy consumption does not decline as rapidly.