PaperlayerView exact revision on arXiv

Source-linked AI summary

MixConv: Mixed Depthwise Convolutional Kernels

Mingxing Tan, Quoc V. Le

arXiv:1907.09595v3cs.CVcs.LG

TL;DR

Depthwise convolution widely uses kernel sizes that are not systematically examined. The paper studies this choice, introduces MixConv to combine multiple kernel sizes, and develops MixNets that outperform prior mobile ConvNets, with MixNet-L reaching 78.9% ImageNet top-1 accuracy under typical mobile constraints.

  • Problem

    Kernel size is often overlooked in depthwise convolution, despite larger kernels potentially improving accuracy and efficiency.

  • Method

    MixConv partitions channels into groups using different kernel sizes, and neural architecture search uses this operation to develop MixNets.

  • Results

    MixNets outperform prior mobile ConvNets; MixNet-L achieves 78.9% ImageNet top-1 accuracy under typical mobile FLOPS below 600M.

  • Takeaways & Limitations

    MixConv improves MobileNets accuracy and efficiency on both ImageNet classification and COCO object detection as a drop-in replacement for vanilla depthwise convolution.

  • Takeaways & Limitations

    The design restricts group kernel sizes to distinct values starting at 3x3 and increasing by 2 per group, and omits exponential channel partition and dilated convolutions from search.

Abstract

from arXiv · show

Depthwise convolution is becoming increasingly popular in modern efficient ConvNets, but its kernel size is often overlooked. In this paper, we systematically study the impact of different kernel sizes, and observe that combining the benefits of multiple kernel sizes can lead to better accuracy and efficiency. Based on this observation, we propose a new mixed depthwise convolution (MixConv), which naturally mixes up multiple kernel sizes in a single convolution. As a simple drop-in replacement of vanilla depthwise convolution, our MixConv improves the accuracy and efficiency for existing MobileNets on both ImageNet classification and COCO object detection. To demonstrate the effectiveness of MixConv, we integrate it into AutoML search space and develop a new family of models, named as MixNets, which outperform previous mobile models including MobileNetV2 [20] (ImageNet top-1 accuracy +4.2%), ShuffleNetV2 [16] (+3.5%), MnasNet [26] (+1.3%), ProxylessNAS [2] (+2.2%), and FBNet [27] (+2.0%). In particular, our MixNet-L achieves a new state-of-the-art 78.9% ImageNet top-1 accuracy under typical mobile settings (<600M FLOPS). Code is at https://github.com/ tensorflow/tpu/tree/master/models/official/mnasnet/mixnet

1 Introduction

The paper revisits kernel-size choices in depthwise convolution and proposes MixConv, which combines multiple kernel sizes to improve accuracy and efficiency. Integrated into searched MixNets, the approach outperforms prior mobile ConvNets and reaches 78.9% ImageNet top-1 accuracy.

  • Depthwise convolution reduces computational cost by applying separate kernels to individual channels, but kernel size remains an overlooked design factor.
  • Larger kernels increase parameters, while accuracy rises from 3x3 to 7x7 and then drops quickly beyond 9x9.
  • MixConv partitions channels into groups and applies different kernel sizes within one depthwise convolution.
  • MixConv serves as a drop-in replacement for vanilla depthwise convolution and improves MobileNets accuracy and efficiency on ImageNet classification and COCO object detection.
  • 78.9% ImageNet top-1 accuracy is achieved by MixNet-L under typical mobile model-size and FLOPS settings.

2 Related Work

Prior work improves ConvNet efficiency through efficient operations and architectures, uses multiple branches or scales, and applies neural architecture search to automate design choices.

  • Efficient ConvNets: Depthwise convolution is widely used in mobile-size ConvNets, alongside efficient operations, bottleneck layers, and efficient architectures.Examples include MobileNets, ShuffleNets, and MnasNet.
  • Multi-Scale Networks and Features: Multibranch ConvNets combine different operations within a layer, while other architectures combine multi-scale feature maps from different layers.The related approaches include Inceptions, Inception-ResNet, ResNeXt, NASNet, DenseNet, and feature pyramid networks.
  • Neural Architecture Search: Neural architecture search automates model design and has achieved better performance than hand-crafted models.Search methods are used to learn design choices and construct neural architectures.
  • Neural Architecture Search: Because MixConv offers many design choices, existing architecture search methods can incorporate it into the search space for developing MixNets.This connects the flexible operation to automated model-family design.

3 MixConv

MixConv mixes different kernel sizes across channel groups within one depthwise convolution, aiming to capture patterns at varied resolutions while improving efficiency and accuracy. Experiments show benefits over vanilla depthwise convolution on ImageNet and COCO, with design studies covering grouping, channel partitioning, and dilation.

  • MixConv Feature Map: MixConv partitions channels into groups and applies a distinct kernel size to each group, then concatenates the resulting outputs.The groups preserve the original spatial dimensions while dividing the channel computation across kernels.
  • MixConv Feature Map: MixConv can serve as a drop-in replacement for vanilla depthwise convolution and may be implemented as a single optimized operation.The paper describes group-convolution implementation as a possible platform optimization.
  • MixConv Design Choices: MixConv uses flexible group sizes; g = 1 recovers vanilla depthwise convolution, while g = 4 is generally a safe MobileNet choice and search explores g from 1 to 5.Group size controls how many kernel types are used for one input tensor.
  • MixConv Design Choices: Kernel sizes are restricted to distinct odd values beginning at 3x3 and increasing by 2 per group, while channel allocation can be equal or exponential.A four-group example uses 3x3, 5x5, 7x7, and 9x9 kernels; exponential allocation assigns more channels to smaller kernels.
  • MixConv Performance on MobileNets: MixConv generally uses fewer parameters and FLOPS with similar or better ImageNet accuracy, and is less sensitive than vanilla depthwise convolution to very large kernels.The evaluated kernel range extends from 3x3 to 13x13.
  • MixConv Performance on MobileNets: On COCO detection, MixConv consistently improves efficiency and accuracy; MixConv357 raises mAP by 0.6% on MobileNetV1 and 1.1% on MobileNetV2 versus vanilla depthwise7x7.These gains use fewer parameters and FLOPS than the comparison baseline.
  • Ablation Study: In single-layer MobileNetV2 tests, stride-2 layers benefit most from larger kernels, while MixConv3579 uses half the parameters and FLOPS of DepthwiseConv9x9 with similar or slightly better performance.Most other layers show little accuracy change from kernel size.
  • Ablation Study: Exponential partition reduces parameters and FLOPS but has no clear overall winner across MobileNetV1 and V2, while dilation loses accuracy for large kernels.The paper attributes the dilation result to skipping local information at large dilation rates.

4 MixNet

MixNets incorporate MixConv into neural architecture search, using multiple kernel sizes and other mobile-model design choices. They outperform prior mobile ConvNets on ImageNet and transfer learning under comparable efficiency constraints.

  • Architecture Search: MixNets add MixConv to a neural architecture search space based on MobileNetV2, alongside kernel size, expansion ratio, channel size, and other choices.The search space also includes swish activation, squeeze-and-excitation, and grouped 1x1 convolutions.
  • Architecture Search: MixNet-S and MixNet-M are obtained through neural architecture search, while MixNet-L scales MixNet-M with depth multiplier 1.3.All models use the same training settings as MnasNet.
  • ImageNet Performance: 4.2% higher ImageNet top-1 accuracy than MobileNetV2 and 3.5% higher than ShuffleNetV2 are achieved under the same FLOPS constraint.Against automated models under similar FLOPS constraints, MixNets improve over MnasNet by 1.3%, FBNet by 2.0%, and ProxylessNAS by 2.2%.
  • ImageNet Performance: 78.9% ImageNet top-1 accuracy is achieved by MixNet-L under a typical mobile constraint of less than 600M FLOPS.The result is reported as a new state-of-the-art under these settings.
  • Architecture Search: MixNet architectures use smaller kernels more often early and larger kernels later, while the larger MixNet-M uses more large kernels and layers at higher computational cost.This design places computationally cheaper kernels in early stages and larger kernels in later stages.
  • Transfer Learning Performance: 11.4x greater efficiency and 1% higher accuracy than ResNet-50 are reported for MixNet-M on transfer learning, with 97.92% accuracy, 3.49M parameters, and 352M FLOPS.MixNets also outperform previous models across four transfer learning datasets.

5 Conclusions

The paper identifies limitations of single-kernel depthwise convolution and proposes MixConv to combine multiple kernel sizes. MixConv improves MobileNets, while MixNets extend the approach through neural architecture search and outperform recent mobile ConvNets.

  • Conclusions: MixConv mixes multiple kernels in one depthwise-convolution operation to address the limitations of using a single kernel size.It is presented as a drop-in replacement for vanilla depthwise convolution.
  • Conclusions: MixConv improves MobileNet accuracy and efficiency on both image classification and object detection tasks.The reported classification and detection evaluations include ImageNet and COCO.
  • Conclusions: MixNets use neural architecture search with MixConv and achieve better accuracy and efficiency than recent mobile ConvNets on ImageNet and four transfer learning datasets.The conclusion states this as the paper's broader empirical outcome.
Loading 1907.09595v3…