Source-linked AI summary
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, Song Han
TL;DR
Model compression traditionally depends on expert-crafted policies for a large, difficult design space. AMC uses reinforcement learning to search layer-wise compression policies and supports both hardware-constrained and accuracy-guaranteed objectives. Across several networks, it reports stronger compression or acceleration than heuristic policies, including 1.95× Android speedup for MobileNet and no-accuracy-loss compression for ResNet-50.
Problem
Conventional compression requires domain experts to explore a large size-speed-accuracy design space, and hand-crafted policies are usually suboptimal and labor-consuming.
Method
AMC uses a DDPG agent to search continuous, layer-wise compression ratios, with reward schemes for resource-constrained and accuracy-guaranteed compression.
Results
AMC reports better performance than hand-crafted policies across multiple networks, including increasing ResNet-50 compression from 3.4× to 5× without accuracy loss and achieving 1.95× Android speedup for MobileNet.
Takeaways & Limitations
The automated policy search generalizes across MobileNet, MobileNet-V2, ResNet, and VGG and from classification to detection tasks.
Takeaways & Limitations
The reinforcement-learning reward uses validation accuracy without fine-tuning, which may favor less aggressive compression than a fine-tuned-accuracy reward.
Abstract
from arXiv · showhide
Model compression is a critical technique to efficiently deploy neural network models on mobile devices which have limited computation resources and tight power budgets. Conventional model compression techniques rely on hand-crafted heuristics and rule-based policies that require domain experts to explore the large design space trading off among model size, speed, and accuracy, which is usually sub-optimal and time-consuming. In this paper, we propose AutoML for Model Compression (AMC) which leverage reinforcement learning to provide the model compression policy. This learning-based compression policy outperforms conventional rule-based compression policy by having higher compression ratio, better preserving the accuracy and freeing human labor. Under 4x FLOPs reduction, we achieved 2.7% better accuracy than the handcrafted model compression policy for VGG-16 on ImageNet. We applied this automated, push-the-button compression pipeline to MobileNet and achieved 1.81x speedup of measured inference latency on an Android phone and 1.43x speedup on the Titan XP GPU, with only 0.1% loss of ImageNet Top-1 accuracy.
1 Introduction
AMC replaces hand-crafted, expert-driven compression with reinforcement-learning policies that search layer-wise compression decisions. It supports resource-constrained and accuracy-guaranteed objectives, with experiments showing stronger compression and acceleration across networks.
- Motivation: Hand-crafted policies are often suboptimal and time-consuming because layers differ in redundancy and their compression choices are interdependent.Conventional methods require domain expertise to trade off model size, speed, and accuracy across a large design space.
- AMC approach: AMC uses reinforcement learning to automatically search compression policies instead of relying on rule-based policies and experienced engineers.The method is designed to find layer-wise sparsity decisions for arbitrary networks while improving compression quality.
- AMC approach: DDPG processes layers sequentially, outputs a continuous compression ratio for each layer, evaluates the compressed model, and receives a reward based on accuracy and hardware cost.The search uses validation accuracy without fine-tuning, then fine-tunes the best-explored model after policy search.
- Compression protocols: AMC offers resource-constrained compression for maximizing accuracy under hardware budgets and accuracy-guaranteed compression for minimizing model size without accuracy loss.The resource constraints can involve FLOPs, latency, or model size.
- Results: 3.4× to 5×: AMC increases ResNet-50's expert-tuned compression ratio with no accuracy loss, while 2× FLOPs reduction in MobileNet yields 70.2% top-one accuracy.The MobileNet model achieves 1.53× speedup on Titan XP and 1.95× on an Android phone.
2 Related Work
Prior model-search and compression approaches use varied strategies, while AMC combines continuous compression control with accuracy and latency optimization and hardware-aware objectives.
- CNN compression and acceleration: CNN acceleration methods include pruning, quantization, special convolution implementations, and tensor factorization.Tensor factorization includes decomposing weights or convolutional layers into lighter components.
- Reinforcement-learning model search: AMC can obtain rewards without fine-tuning and produce both accuracy-guaranteed and hardware resource-constrained models.These capabilities distinguish its search procedure from the other compared reinforcement-learning approaches.
- Reinforcement-learning model search: AMC differs from NAS, network transformation, and N2N by optimizing accuracy and latency with a simple non-RNN controller and continuous action space.The comparison also highlights faster exploration with fewer GPU hours.
3 Methodology
AMC automates layer-wise compression-policy search with reinforcement learning, replacing hand-crafted heuristics with continuous sparsity control and resource-aware protocols. It evaluates compressed models before fine-tuning to accelerate search and supports both resource-constrained and accuracy-guaranteed objectives.
- Automated Compression with Reinforcement Learning: AMC trains a reinforcement-learning agent to predict each layer’s sparsity and quickly evaluates accuracy after pruning before fine-tuning.The pre-fine-tuning validation accuracy serves as an efficient delegate of fine-tuned accuracy during search.
- Automated Compression with Reinforcement Learning: The layer state includes architecture, computation, and pruning-history features, including dimensions, kernel size, FLOPs, reduced FLOPs, remaining FLOPs, and the previous action.These features are scaled within [0, 1] before being passed to the agent.
- Automated Compression with Reinforcement Learning: DDPG provides continuous compression-ratio control because compression quality is sensitive to each layer’s sparsity and discrete action spaces become difficult to explore.The agent processes layers sequentially, outputs a sparsity ratio, and advances after compressing each layer.
- Search Protocols: Resource-constrained compression bounds layer actions so the compressed model meets a target resource budget, while alternative resources can include model size, FLOPs, or mobile-device inference time.The procedure adjusts actions using current-layer and remaining-layer resource quantities to reach the target compression ratio.
- Search Protocols: The resource-constrained procedure can precisely reach the target compression ratio because the agent receives no incentive to go below the budget.The same framework can be applied to constraints such as actual mobile-device inference time.
- Search Protocols: Accuracy-guaranteed compression modifies the reward to remain sensitive to error while providing a small incentive to reduce FLOPs or model size.Experiments report that the agent automatically finds the compression limit under this reward.
4 Experimental Results
AMC uses reinforcement learning to search layer-wise compression policies across networks and constraints, outperforming hand-crafted policies while preserving accuracy and improving compression or speed.
- CIFAR-10 and Analysis: AMC outperforms hand-crafted policies across Plain-20 and ResNet experiments on CIFAR-10 under FLOPs-constrained and accuracy-guaranteed compression.The search uses channel pruning for FLOPs-constrained compression and fine-grained pruning for accuracy-guaranteed compression.
- CIFAR-10 and Analysis: 60% compression ratio was achieved for ResNet-50 on CIFAR-10 with slightly higher validation and test accuracy.The result uses the RParam reward for accuracy-guaranteed fine-grained pruning.
- ImageNet: AMC consistently outperforms uniform baselines on MobileNet and MobileNet-V2, including a 1.0% accuracy improvement for MobileNet-V2 at the same computation.The method improves the accuracy-computation trade-off of already compact networks.
- ImageNet: 0.7% better mAP[0.5, 0.95] than the best hand-crafted pruning methods was obtained for PASCAL VOC object detection at the same compression ratio.AMC also surpassed the baseline by 0.5% mAP when compressed VGG-16 was used as the Faster R-CNN backbone.
5 Conclusion
AMC uses reinforcement learning to automate model-compression policy search, improving compression quality across architectures and tasks while reducing reliance on labor-intensive expert heuristics.
- AMC automatically searches the model-compression design space with reinforcement learning instead of hand-crafted features and domain-expert heuristics.The approach targets trade-offs among model size, speed, and accuracy.
- Compelling results are demonstrated for MobileNet, MobileNet-V2, ResNet, and VGG on CIFAR and ImageNet.The compressed model also generalizes from classification to detection tasks.
- MobileNet inference speed on Google Pixel 1 increases from 8.1 fps to 16.0 fps.