Source-linked AI summary
ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
Xiaoliang Dai, Peizhao Zhang, Bichen Wu, Hongxu Yin, Fei Sun, Yanghan Wang, Marat Dukhan, Yunqing Hu, Yiming Wu, Yangqing Jia, Peter Vajda, Matt Uyttendaele, Niraj K. Jha
TL;DR
Resource-constrained deployment makes compact neural-network design difficult because FLOPs may miss hardware-specific latency and direct measurements are costly. Chameleon adapts existing building blocks to platform and resource constraints using predictive models and efficient evolutionary search, achieving significant accuracy gains at matched latency or energy with searches taking CPU minutes.
Problem
Compact neural networks remain difficult to deploy because FLOPs may not capture hardware characteristics, while direct platform-specific latency measurements are slow and expensive.
Method
Chameleon uses hardware-aware adaptation of existing building blocks, predictive accuracy, latency, and energy models, and efficient evolutionary search under resource constraints.
Results
Chameleon achieves significant accuracy gains over handcrafted and automatically searched models at the same latency or energy, with adaptation searches taking only CPU minutes.
Takeaways & Limitations
Computation distribution across efficient building blocks is critical to model performance across devices and resource budgets.
Abstract
from arXiv · showhide
This paper proposes an efficient neural network (NN) architecture design methodology called Chameleon that honors given resource constraints. Instead of developing new building blocks or using computationally-intensive reinforcement learning algorithms, our approach leverages existing efficient network building blocks and focuses on exploiting hardware traits and adapting computation resources to fit target latency and/or energy constraints. We formulate platform-aware NN architecture search in an optimization framework and propose a novel algorithm to search for optimal architectures aided by efficient accuracy and resource (latency and/or energy) predictors. At the core of our algorithm lies an accuracy predictor built atop Gaussian Process with Bayesian optimization for iterative sampling. With a one-time building cost for the predictors, our algorithm produces state-of-the-art model architectures on different platforms under given constraints in just minutes. Our results show that adapting computation resources to building blocks is critical to model performance. Without the addition of any bells and whistles, our models achieve significant accuracy improvements against state-of-the-art hand-crafted and automatically designed architectures. We achieve 73.8% and 75.3% top-1 accuracy on ImageNet at 20ms latency on a mobile CPU and DSP. At reduced latency, our models achieve up to 8.5% (4.8%) and 6.6% (9.3%) absolute top-1 accuracy improvements compared to MobileNetV2 and MnasNet, respectively, on a mobile CPU (DSP), and 2.7% (4.6%) and 5.6% (2.6%) accuracy gains over ResNet-101 and ResNet-152, respectively, on an Nvidia GPU (Intel CPU).
1. Introduction
Chameleon addresses the difficulty of deploying efficient neural networks across platforms with divergent hardware traits and resource constraints. It adapts existing building blocks using predictive, platform-aware search to produce architectures efficiently.
- Deployment challenges: Platform-specific hardware characteristics make it difficult for one neural network architecture to run optimally across devices.For example, a Hexagon v62 DSP favors convolution channel sizes that are multiples of 32.
- Deployment challenges: Different applications impose incompatible latency and energy requirements, so architectures must be adapted to specific use scenarios.Real-time video analysis may prioritize latency, whereas IoT devices may prioritize energy consumption.
- Limits of existing approaches: Existing NAS and SMBO approaches require expensive network training and performance measurements throughout search, limiting large-scale cross-platform deployment.The MNAS architecture reportedly required hundreds of GPU hours to develop.
- Chameleon methodology: Chameleon allocates computation resources using hardware traits and existing efficient building blocks instead of new cell structures or computationally intensive reinforcement learning.Its predictive models estimate accuracy, latency, and energy to accelerate search.
- Chameleon methodology: The method uses an accuracy predictor based on Gaussian processes and Bayesian optimization to search efficiently over architectures.The algorithm uses efficient accuracy and resource predictors to search a space comparable to RL-based NAS.
- Efficiency and scalability: With a one-time predictor-building cost, Chameleon searches for models under different platforms and constraints in minutes, supporting heterogeneous deployment.The approach is presented as efficient and scalable across deployment settings.
2. Related work
Related work improves neural-network efficiency through model simplification, compact architectures, and automated platform-aware search. These approaches trade computation against accuracy using pruning, quantization, specialized operators, scaling, or direct metric optimization.
- Model simplification: Model simplification reduces neural-network computation through pruning and quantization, often while preserving accuracy or improving runtime.Examples include hardware-aware pruning and quantization with little or no accuracy loss.
- Compact architecture: Handcrafted compact architectures improve accuracy-efficiency trade-offs by using efficient operators such as depthwise and low-cost group convolutions.MobileNet and ShuffleNet reduce computation relative to ResNet, while later versions further shrink model size.
- Compact architecture: Linear scaling accommodates latency constraints by uniformly thinning layers or reducing input resolution.This method is simple and widely used for deploying compact models across settings.
- NAS and SMBO: Platform-aware NAS and SMBO automate model synthesis using direct deployment metrics such as latency and energy.MnasNet reported a 2% absolute top-1 accuracy gain over MobileNetV2 1.0x with a 1.3% latency overhead on Google Pixel 1.
3. Methodology
Chameleon adapts an existing neural-network architecture to a target platform and resource budget by searching over architecture variants with predictive accuracy, latency, and energy models. Its efficient evolutionary search combines hardware-aware resource constraints with Gaussian-process accuracy prediction and Bayesian-optimization-based sample selection.
- Platform-aware model adaptation: Chameleon takes a default architecture and a platform-specific resource budget, then searches for an adapted architecture using efficient evolutionary search.Architecture genes encode hyperparameters such as the number of filters and bottlenecks.
- Platform-aware model adaptation: The search maximizes predicted accuracy while incorporating latency or energy constraints as a regularization term in the fitness function.The resource metric is evaluated for a target platform against a scenario-specific threshold.
- Predictive models: Accuracy, latency, and energy predictors estimate candidate performance in less than one CPU second, avoiding repeated network training and hardware measurement during search.The predictors replace time-consuming direct evaluation of every candidate.
- Accuracy predictor: A Gaussian-process accuracy predictor supports reliable prediction with scarce training data and provides uncertainty estimates for selecting additional samples.Among six models evaluated using 240 pretrained models, the GP regressor has the lowest leave-one-out MSE.
- Iterative sample selection: Bayesian optimization selects both high-accuracy-per-FLOPs exploitation samples and high-uncertainty exploration samples to improve sample efficiency.The resulting sampling density is higher in the area of “samples of interest,” where adaptation typically rests.
4. Experiments
Experiments adapt MobileNetV2- and ResNet-based architectures across mobile, server, and energy-constrained platforms. Chameleon consistently improves accuracy–resource trade-offs, including gains at matched latency or energy.
- Mobile models: 8.5% absolute accuracy gain over MobileNetV2 0.5x is achieved at the same 4.0ms latency on a Snapdragon 835 mobile CPU.The comparison uses 96×96 input resolution.
- Energy-constrained adaptation: 60.0% accuracy at 14mJ improves on MobileNetV2 0.75x’s 58.8% at 19mJ, reducing energy by 26% while increasing accuracy by 1.2%.The comparison is measured on a Snapdragon 835 smartphone CPU.
- Resource utilization: ChamNet redistributes FLOPs from early to late stages, where smaller feature maps can support more filters or larger expansion factors.Early stages have lower CPU FLOPs-per-second, plausibly due to cache incompatibility with large image sizes.
- Resource utilization: 2.1% accuracy gain with 5% lower latency results from better FLOPs redistribution than baseline MobileNetV2.ChamNet achieves 71.9% accuracy at 15.0ms versus MobileNetV2’s 69.8% at 16.6ms.
5. Conclusions
Chameleon is a platform-aware model adaptation framework that uses efficient building blocks and predictive models to target different platforms and use scenarios. It improves accuracy at matched latency or energy while requiring only CPU minutes for adaptation searches.
- Conclusion: Chameleon adapts models to real-world platforms and use scenarios using efficient building blocks and predictive models.The framework bypasses expensive training and measurement processes during adaptation.
- Conclusion: Chameleon significantly improves accuracy without latency or energy overhead and completes adaptation searches in CPU minutes.The conclusion reports gains relative to both handcrafted and automatically searched models at the same latency or energy.