MnasNet: Platform-Aware Neural Architecture Search for Mobile

Mingxing Tan; Bo Chen; Ruoming Pang; Vijay Vasudevan; Mark Sandler; Andrew Howard; Quoc V. Le

Source-linked AI summary

MnasNet: Platform-Aware Neural Architecture Search for Mobile

Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, Quoc V. Le

arXiv:1807.11626v3Jul 31, 2018cs.CVcs.LG

TL;DR

Designing mobile CNNs requires balancing accuracy with latency across a large architectural design space. MnasNet automates this trade-off using real-device latency and a factorized hierarchical search space, achieving state-of-the-art results across ImageNet classification and COCO object detection under typical mobile latency constraints.

Problem
Designing resource-constrained mobile CNNs requires balancing accuracy and resource efficiency across a large architectural design space.
Method
MnasNet uses multi-objective neural architecture search with directly measured mobile-device latency and a factorized hierarchical search space that permits layer diversity.
Results
MnasNet achieves state-of-the-art accuracy on ImageNet classification and COCO object detection under typical mobile latency constraints, including 75.2% top-1 accuracy at 78ms on a Pixel phone.
Takeaways & Limitations
Platform-aware architecture search can automatically find mobile CNNs with improved accuracy-latency trade-offs across classification and detection tasks.
Takeaways & Limitations
The search uses a weighted-product reward, although the authors expect weighted-sum methods would also work.

Abstract

from arXiv · show

Designing convolutional neural networks (CNN) for mobile devices is challenging because mobile models need to be small and fast, yet still accurate. Although significant efforts have been dedicated to design and improve mobile CNNs on all dimensions, it is very difficult to manually balance these trade-offs when there are so many architectural possibilities to consider. In this paper, we propose an automated mobile neural architecture search (MNAS) approach, which explicitly incorporate model latency into the main objective so that the search can identify a model that achieves a good trade-off between accuracy and latency. Unlike previous work, where latency is considered via another, often inaccurate proxy (e.g., FLOPS), our approach directly measures real-world inference latency by executing the model on mobile phones. To further strike the right balance between flexibility and search space size, we propose a novel factorized hierarchical search space that encourages layer diversity throughout the network. Experimental results show that our approach consistently outperforms state-of-the-art mobile CNN models across multiple vision tasks. On the ImageNet classification task, our MnasNet achieves 75.2% top-1 accuracy with 78ms latency on a Pixel phone, which is 1.8x faster than MobileNetV2 [29] with 0.5% higher accuracy and 2.3x faster than NASNet [36] with 1.2% higher accuracy. Our MnasNet also achieves better mAP quality than MobileNets for COCO object detection. Code is at https://github.com/tensorflow/tpu/tree/master/models/official/mnasnet

1. Introduction

MnasNet frames mobile CNN design as a difficult accuracy–efficiency trade-off and addresses it with architecture search that jointly considers accuracy, real-world latency, and a factorized hierarchical search space. The approach achieves state-of-the-art results on ImageNet classification and COCO object detection under mobile latency constraints.

Motivation: Mobile CNN design must balance accuracy and resource efficiency across a significantly large architectural design space.Mobile devices have restricted computational resources, motivating shallower networks and less expensive operations such as depthwise and group convolution.
Results: 3.0% higher ImageNet accuracy is achieved than MobileNetV2 with similar latency on a Google Pixel phone.The comparison is made using MnasNet models evaluated on ImageNet classification.
Results: 1.8× faster than MobileNetV2 and 2.3× faster than NASNet, with better accuracy, are MnasNet results when target accuracy is constrained.These comparisons are summarized for state-of-the-art mobile models and are reported alongside ImageNet results.
Approach: MnasNet introduces multi-objective neural architecture search that optimizes both accuracy and real-world latency on mobile devices.The approach explicitly targets inference latency rather than relying only on indirect proxies.
Approach: The factorized hierarchical search space enables layer diversity while balancing flexibility against search space size.This search-space design is one of the paper’s two main ideas for mobile architecture search.
Results: MnasNet demonstrates new state-of-the-art accuracy on both ImageNet classification and COCO object detection under typical mobile latency constraints.The paper applies the approach to ImageNet classification and COCO object detection.

2. Related Work

Prior mobile CNN efficiency work relies on compression techniques or manually designed architectures, while neural architecture search automates model design but generally omits mobile platform constraints.

Resource-efficient CNNs: Compression methods quantize weights or activations and prune filters using FLOPs or platform-aware latency, but remain tied to a baseline model.These approaches do not learn novel CNN compositions.
Hand-crafted mobile architectures: Hand-crafted mobile architectures improve efficiency through lower-cost convolutions, reduced filter sizes, depthwise separable convolution, group convolution, channel shuffle, or learned inter-layer connections.Examples include SqueezeNet, MobileNet, ShuffleNets, and CondenseNet.
Neural architecture search: Neural architecture search uses reinforcement learning, evolutionary search, differentiable search, and other learning algorithms to automate model design.These methods can generate mobile-size models by repeatedly stacking a few searched cells, but do not incorporate mobile platform constraints into the search process or search space.

3. Problem Formulation

The design problem is formulated as a multi-objective search for accurate CNNs with low real-world mobile inference latency. A customized weighted-product objective approximates multiple Pareto-optimal solutions while supporting application-specific accuracy–latency trade-offs.

Multi-objective design goal: The search jointly targets high accuracy and low inference latency, measuring latency by executing CNNs on real mobile devices rather than using indirect metrics such as FLOPS.This incorporates real-world inference latency directly into the optimization objective.
Multi-objective design goal: A hard latency constraint maximizes accuracy under a target latency but yields only a single optimized metric rather than multiple Pareto-optimal solutions.Pareto-optimal models improve accuracy without increasing latency or reduce latency without decreasing accuracy.
Pareto optimization: The method uses a customized weighted-product objective to approximate multiple Pareto-optimal solutions within one architecture search.The weighted product is selected because it is easy to customize, while weighted sums are also expected to work.
Pareto optimization: Doubling latency is empirically associated with a 5% relative accuracy gain, guiding application-specific α and β choices so such trade-offs receive similar reward.The parameters are chosen so Pareto-optimal solutions have similar rewards under different accuracy–latency trade-offs.
Constraint handling: With (α = 0, β = −1), latency is a hard constraint, whereas (α = β = −0.07) makes the target latency T a soft constraint with smoothly adjusted objective values.The hard-constraint setting penalizes models that exceed T; the soft-constraint setting adjusts the objective according to measured latency.

4. Mobile Neural Architecture Search

MnasNet uses a factorized hierarchical search space that enables layer diversity while controlling search-space size, then applies reinforcement learning to optimize accuracy and real-phone latency. The search samples models, evaluates their task accuracy and measured mobile inference latency, and updates an RNN controller with PPO.

Factorized hierarchical search space: Layer diversity is critical for achieving both high accuracy and lower latency, unlike approaches that repeatedly stack a few searched cells.Previous methods search for only a few complex cells, preventing architectural variation across layers.
Factorized hierarchical search space: The search space factorizes CNNs into unique blocks and separately searches each block’s operations and connections, allowing different architectures in different blocks.Blocks are organized by input resolutions and filter sizes, with repeated layers sharing the searched block-level design.
Factorized hierarchical search space: The factorized hierarchy balances layer diversity with search-space size by searching one layer per block and repeating it, rather than independently searching every layer.For B blocks with sub-search space size S and average N layers per block, the hierarchical space is described as S^B versus S^(B*N) for a flat per-layer space.
Reinforcement-learning search algorithm: Reinforcement learning searches for Pareto-optimal models by having an RNN controller sample tokenized architectures, evaluating accuracy and real-phone latency, and updating its parameters with PPO.The framework combines the controller, a trainer for model accuracy, and a mobile-phone inference engine for latency measurement in a repeated sample-evaluate-update loop.

5. Experimental Setup

The experimental setup uses a large-scale architecture search with real-device latency measurements, followed by selective transfer of top models to ImageNet and COCO. Full ImageNet training uses specified optimization, regularization, learning-rate, batching, and preprocessing settings.

Architecture search: Each architecture search takes 4.5 days on 64 TPUv2 devices, using the same RNN controller as NASNet to isolate search-space effects.The controller samples approximately 8K models during search.
Architecture search: Real-world latency is measured for every sampled model on the single-thread big CPU core of Pixel 1 phones.Only 15 top-performing models transfer to full ImageNet, while 1 model transfers to COCO.
ImageNet training: ImageNet training uses RMSProp with decay 0.9, momentum 0.9, batch size 4K, 224×224 images, and a learning rate rising to 0.256 before scheduled decay.Batch normalization follows every convolution, weight decay is 1e-5, and dropout rate 0.2 is applied to the last layer.

6. Results

The results show that MnasNet delivers strong accuracy-latency trade-offs on ImageNet, remains robust across model scaling choices, and transfers effectively to COCO object detection. Targeted architecture search further improves performance under tight latency constraints.

ImageNet classification: 75.2% top-1 and 92.5% top-5 accuracy are achieved at 78ms latency with 3.9M parameters and 312M multiply-adds.MnasNet A1 establishes state-of-the-art accuracy for the stated mobile latency constraint.
ImageNet classification: 1.8× faster than MobileNetV2 (1.4) with 0.5% higher accuracy, MnasNet also runs 2.3× faster than mobile-size NASNet-A with 1.2% higher top-1 accuracy.These comparisons use the same Pixel phone for MobileNetV2 and automatically searched mobile-size NASNet-A as the searched-CNN baseline.
Model scaling: 20ms to 160ms inference latency is covered as the depth multiplier varies from 0.35 to 1.4, while MnasNet consistently outperforms MobileNetV2 across depth multipliers and input sizes.Input-size changes improve accuracy by up to 4.1% relative to MobileNetV2.
Model search: 22ms latency is targeted by a new architecture search, which further improves accuracy beyond scaling baseline models with the same parameters.The comparison uses 224x224 image sizes for all models.
COCO object detection: 23.0 vs 23.2 mAP is achieved relative to standard SSD300, while MnasNet significantly improves accuracy over MobileNet V1 and V2.The MnasNet models are used as SSDLite feature extractors and evaluated on COCO test-dev2017.

7. Ablation Study and Discussion

The ablation study examines latency constraints, search-space design, and layer diversity in MnasNet. Results indicate that flexible latency objectives and diverse layer architectures are central to achieving favorable accuracy–latency trade-offs.

Latency Constraint: Setting α=0 and β=−1 treats latency as a hard constraint, whereas α=β=−0.07 treats the target latency as a soft constraint.The controller focuses on faster models under the hard constraint and balances the target latency under the soft constraint.
Reward and Search Space: Adding the multi-objective reward to NASNet’s cell-based search space produces a faster model, while combining it with the factorized search space achieves higher accuracy andThe comparison disentangles the contributions of the reward design and the new search space.
Layer Diversity: MnasNet-A1 uses diverse layer architectures throughout the network, including both 3x3 and 5x5 convolutions.This differs from previous mobile models described as using only 3x3 convolutions.
Layer Diversity: MnasNet has better accuracy–latency trade-offs than variants repeating a single layer type, highlighting the importance of layer diversity in resource-constrained CNNs.The variants fix both kernel size and expansion ratio.

8. Conclusion

The paper presents an automated reinforcement-learning approach for designing resource-efficient mobile CNNs. It combines real-world platform latency with a factorized hierarchical search space to find better accuracy–latency trade-offs than existing mobile models.

Conclusion: The approach uses reinforcement learning to automate the design of resource-efficient mobile CNN models.This frames mobile architecture design as an automated neural architecture search problem.
Conclusion: The search incorporates platform-aware real-world latency information as a primary optimization signal.Using measured latency targets deployment-relevant accuracy–latency trade-offs rather than an abstract efficiency proxy.
Conclusion: A factorized hierarchical search space enables searching for mobile models with improved accuracy–latency trade-offs.The search space is the paper’s second main design idea alongside platform-aware latency optimization.
Conclusion: The experiments demonstrate that the approach can automatically find significantly better mobile models than existing models.The supplied conclusion passage states this result without specifying the comparison metrics or tasks.