PaperlayerView exact revision on arXiv

Source-linked AI summary

Searching for MobileNetV3

Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V. Le, Hartwig Adam

arXiv:1905.02244v5cs.CV

TL;DR

Mobile computer vision needs architectures that balance accuracy and latency on mobile devices. The paper combines hardware-aware search, NetAdapt, and efficient design innovations to produce MobileNetV3 models and reports new state-of-the-art results across mobile classification, detection, and segmentation.

  • Problem

    The paper addresses the need for mobile computer vision architectures that improve the accuracy–latency trade-off across on-device applications.

  • Method

    MobileNetV3 combines hardware-aware neural architecture search, layer-wise NetAdapt, efficient network design, and the LR-ASPP segmentation decoder.

  • Results

    MobileNetV3 achieves new state-of-the-art results in mobile classification, object detection, and semantic segmentation.

  • Takeaways & Limitations

    The results provide positive evidence that automated search techniques and human-designed architectural advances can jointly improve mobile models.

  • Takeaways & Limitations

    How best to combine automatic search techniques with human intuition remains an open question.

Abstract

from arXiv · show

We present the next generation of MobileNets based on a combination of complementary search techniques as well as a novel architecture design. MobileNetV3 is tuned to mobile phone CPUs through a combination of hardware-aware network architecture search (NAS) complemented by the NetAdapt algorithm and then subsequently improved through novel architecture advances. This paper starts the exploration of how automated search algorithms and network design can work together to harness complementary approaches improving the overall state of the art. Through this process we create two new MobileNet models for release: MobileNetV3-Large and MobileNetV3-Small which are targeted for high and low resource use cases. These models are then adapted and applied to the tasks of object detection and semantic segmentation. For the task of semantic segmentation (or any dense pixel prediction), we propose a new efficient segmentation decoder Lite Reduced Atrous Spatial Pyramid Pooling (LR-ASPP). We achieve new state of the art results for mobile classification, detection and segmentation. MobileNetV3-Large is 3.2\% more accurate on ImageNet classification while reducing latency by 15\% compared to MobileNetV2. MobileNetV3-Small is 4.6\% more accurate while reducing latency by 5\% compared to MobileNetV2. MobileNetV3-Large detection is 25\% faster at roughly the same accuracy as MobileNetV2 on COCO detection. MobileNetV3-Large LR-ASPP is 30\% faster than MobileNetV2 R-ASPP at similar accuracy for Cityscapes segmentation.

1. Introduction

The paper develops MobileNetV3 Large and Small to optimize mobile computer-vision accuracy–latency trade-offs, combining automated search with novel architecture advances and an efficient segmentation decoder.

  • 1. Introduction: Efficient neural networks improve mobile user experience through higher accuracy and lower latency while supporting privacy and battery preservation.On-device evaluation avoids sending user data to servers, and reduced power consumption helps preserve battery life.
  • 1. Introduction: MobileNetV3 Large and Small target high-accuracy, efficient neural networks for on-device computer vision.The models are designed to advance the state of the art while supporting mobile applications.
  • 1. Introduction: The approach combines complementary search techniques, mobile-efficient nonlinearities, network design, and a segmentation decoder.The paper evaluates each technique across classification, detection, and segmentation use cases.

2. Related Work

Related work improves neural-network efficiency through handcrafted architectures, automated architecture search and network simplification, reduced-precision quantization, and knowledge distillation. MobileNet progresses from depthwise separable convolutions to inverted residuals and linear bottlenecks while emphasizing computation and measured latency.

  • Efficient architecture design: Efficient architecture research combines handcrafted structures with algorithmic neural architecture search to balance accuracy and efficiency.The section frames both approaches as important advances in efficient deep-network design.
  • Handcrafted mobile architectures: MobileNetV1 uses depthwise separable convolution, while MobileNetV2 adds inverted residuals and linear bottlenecks in a resource-efficient block.Earlier SqueezeNet primarily reduced parameters with 1x1 convolutions and squeeze-and-expand modules; later work emphasizes MAdds and measured latency.
  • Automated architecture search: Architecture search evolved from reinforcement-learning searches toward cell-level and block-level methods, differentiable gradient-based optimization, and automated network simplification for constrained mobile platforms.Fully configurable search spaces can become exponentially large and intractable, motivating restricted or more efficient search strategies.
  • Complementary efficiency methods: Quantization improves network efficiency through reduced-precision arithmetic.The section presents quantization as a complementary efficiency method.
  • Complementary efficiency methods: Knowledge distillation generates small accurate student networks guided by a large teacher network.Distillation is described as an additional complementary method for producing compact accurate models.

3. Efficient Mobile Building Blocks

MobileNetV3 combines efficient building blocks from MobileNetV1, MobileNetV2, and MnasNet with modified swish nonlinearities and hard sigmoid replacements. Its blocks integrate depthwise separable convolutions, inverted residuals, linear bottlenecks, and squeeze-and-excitation modules.

  • MobileNetV1: MobileNetV1 introduced depthwise separable convolutions, separating spatial filtering from feature generation through depthwise and 1x1 pointwise layers.This replaced traditional convolution layers with a lighter spatial-filtering operation and a heavier feature-generation operation.
  • MobileNetV2: MobileNetV2 introduced inverted residuals and linear bottlenecks, using 1x1 expansion, depthwise convolution, and 1x1 projection layers.Residual connections are used only when input and output channel counts match.
  • MnasNet: MnasNet added squeeze-and-excitation attention modules to the MobileNetV2 bottleneck after depthwise filtering, where attention acts on the largest representation.The module placement differs from the ResNet-based arrangement described in the passage.
  • MobileNetV3: MobileNetV3 combines these layers as building blocks and upgrades them with modified swish nonlinearities.The model uses the preceding efficient structures together rather than introducing a single replacement block.
  • MobileNetV3: MobileNetV3 replaces sigmoid with hard sigmoid in squeeze-and-excitation and swish components because sigmoid can be inefficient and difficult for accurate fixed-point arithmetic.The replacement addresses both computation efficiency and fixed-point accuracy challenges.

4. Network Search

MobileNetV3 combines platform-aware NAS for global network structures with NetAdapt for sequential per-layer filter optimization. The search modifies the reward for small models and selects discrete proposals by their latency–accuracy trade-off until the target latency is reached.

  • Search strategy: Platform-aware NAS searches global network structures by optimizing network blocks, while NetAdapt searches per layer for filter counts.The two techniques are complementary and target hardware-specific optimization.
  • Search strategy: For Large mobile models, the method reuses MnasNet-A1 as the initial model before applying NetAdapt and other optimizations.The reuse follows similar results from the same RNN-based controller and factorized hierarchical search space at target latency around 80ms.
  • MobileNetV3-Small search: w = −0.15, versus w = −0.07 in [43], compensates for larger accuracy changes across latencies in small models.The revised reward initializes a new architecture search for MobileNetV3-Small before NetAdapt and further optimizations.
  • NetAdapt: NetAdapt starts from a NAS-derived seed and sequentially generates, fine-tunes, evaluates, and selects layer-level proposals that reduce latency.The process iterates until the target latency is reached, then retrains the resulting architecture from scratch.
  • NetAdapt: ∆Acc / |∆latency| selects proposals maximizing the discrete latency–accuracy trade-off slope, subject to each proposal providing at least δ latency reduction.The experiments use T = 10000 and set δ = 0.01|L|, where L is the seed model latency.

5. Network Improvements

The network improvements redesign expensive early and late layers, introduce h-swish, and refine nonlinearities and bottleneck structure to reduce latency while preserving accuracy. These changes underpin the MobileNetV3-Large and MobileNetV3-Small models for high- and low-resource use cases.

  • H-swish: H-swish replaces costly swish computation with a faster, more quantization-friendly nonlinearity based on a piece-wise linear hard analog.The architecture uses h-swish only in the second half of the model, where most swish benefits are realized and activation memory is lower.
  • Network redesign: The authors redesign expensive early and late layers outside the search space to reduce latency while maintaining accuracy.These architectural modifications complement the architecture search process.
  • Efficient last stage: 7 milliseconds and 11% of running time are saved by the efficient last stage after moving feature generation to 1x1 spatial resolution and removing redundant layers.The optimized design drops three expensive layers at the end without accuracy loss.
  • Efficient initial filters: 16 filters preserve the same accuracy as 32 filters with ReLU or swish while saving 2 milliseconds and 10 million MAdds.The initial 3x3 convolution uses hard swish to reduce redundancy without sacrificing performance.
  • H-swish: The net effect of h-swish on accuracy and latency is positive without optimizations and substantial with an optimized piece-wise implementation.Although h-swish still adds latency, optimized ReLU6-based implementations reduce memory-access costs.
  • MobileNetV3 models: MobileNetV3-Large and MobileNetV3-Small combine platform-aware NAS, NetAdapt, and these network improvements for high- and low-resource use cases.Full network specifications are provided in tables 1 and 2.

6. Experiments

The experiments evaluate MobileNetV3 on classification, detection, and segmentation, alongside ablations of architectural choices. MobileNetV3 generally improves accuracy–latency trade-offs, with especially strong gains from optimized h-swish, channel reduction, and LR-ASPP.

  • Classification: MobileNetV3 outperforms MnasNet, ProxylessNas, and MobileNetV2 in the reported floating-point comparisons across Pixel phones.The paper also reports quantized performance separately.
  • Classification: 3%: MobileNetV3-Small nearly outperforms multiplier-scaled MobileNetV3-Large at matched performance, while resolution offers better trade-offs than multiplier.Resolution may be constrained by segmentation and detection requirements.
  • Ablations: 6ms: optimized h-swish saves more than 10% of runtime, while adding only 1ms over traditional ReLU.MobileNetV3 uses h-swish in the middle of the network and dominates ReLU on the latency–accuracy frontier.
  • Object detection: 27% faster: MobileNetV3-Large with channel reduction matches MobileNetV2 at near-identical mAP; MobileNetV3-Small is 2.4 and 0.5 mAP higher than MobileNetV2 and MnasNet while being 35% faster.For both MobileNetV3 models, channel reduction contributes approximately 15% latency reduction with no mAP loss.
  • Semantic segmentation: 6.4%, 10.6%, 12.3%: MobileNetV3 segmentation models outperform ESPNetv2, CCC2, and ESPNetv1, respectively, while being faster in MAdds.Removing atrous convolution reduces performance by 0.6% but improves speed to 1.98B MAdds for half-resolution inputs.

7. Conclusions and future work

The paper introduces MobileNetV3-Large and MobileNetV3-Small, combining multiple architecture-search algorithms with network-design advances to achieve state-of-the-art mobile classification, detection, and segmentation. It also presents LR-ASPP and identifies integrating automatic search with human intuition as future work.

  • Contributions: MobileNetV3-Large and MobileNetV3-Small achieve state-of-the-art results in mobile classification, detection, and segmentation.These models represent the paper’s next generation of mobile models.
  • Contributions: The models combine multiple network architecture search algorithms with advances in network design.The authors describe this combination as a way to deliver the next generation of mobile models.
  • Contributions: Swish nonlinearities and squeeze-and-excite mechanisms are adapted for efficient, quantization-friendly use in mobile models.The paper presents these techniques as effective tools for the mobile model domain.
  • Contributions: LR-ASPP is introduced as a new lightweight segmentation decoder.The decoder is designed for the mobile model domain.
  • Future work: Future work will refine how automatic search techniques are blended with human intuition.The authors describe this integration as an open question despite their initial positive results.

A. Performance table for different resolutions and multipliers

Table 9 provides a detailed performance comparison across the evaluated resolutions and multipliers, reporting computational cost, accuracy, model size, and latency.

  • A. Performance table for different resolutions and multipliers: Table 9 reports multiply-adds, accuracy, parameter count, and latency for the evaluated configurations.The table is described as containing detailed measurements for these four metrics.
Loading 1905.02244v5…