PaperlayerView exact revision on arXiv

Source-linked AI summary

Pelee: A Real-Time Object Detection System on Mobile Devices

Robert J. Wang, Xiang Li, Charles X. Ling

arXiv:1804.06882v3cs.CV

TL;DR

Mobile CNNs need efficient architectures, but prevalent depthwise separable convolutions lack efficient implementation. The paper proposes conventional-convolution PeleeNet and an SSD-based Pelee detector, achieving competitive accuracy, compact models, and real-time mobile inference.

  • Problem

    Efficient mobile CNN design is needed under limited memory and computation, while depthwise separable convolution lacks efficient implementation and efficient models are rarely combined with fast object detection.

  • Method

    The paper proposes conventional-convolution PeleeNet and combines it with an SSD architecture optimized through feature-map selection, residual prediction blocks, and small prediction kernels.

  • Results

    PeleeNet and Pelee achieve compelling classification and detection results, including 23.6 FPS on iPhone 8 and 125 FPS on NVIDIA TX2, while Pelee reaches 76.4% mAP on VOC2007 and 22.4 mAP on COCO.

  • Takeaways & Limitations

    Conventional convolution can support efficient mobile classification and real-time object detection without relying on depthwise separable convolution.

  • Takeaways & Limitations

    On iPhone 8, PeleeNet is slower than MobileNet for small input dimensions, partly because the platform favors separable convolution and PeleeNet uses many narrow multi-branch layers.

Abstract

from arXiv · show

An increasing need of running Convolutional Neural Network (CNN) models on mobile devices with limited computing power and memory resource encourages studies on efficient model design. A number of efficient architectures have been proposed in recent years, for example, MobileNet, ShuffleNet, and MobileNetV2. However, all these models are heavily dependent on depthwise separable convolution which lacks efficient implementation in most deep learning frameworks. In this study, we propose an efficient architecture named PeleeNet, which is built with conventional convolution instead. On ImageNet ILSVRC 2012 dataset, our proposed PeleeNet achieves a higher accuracy and over 1.8 times faster speed than MobileNet and MobileNetV2 on NVIDIA TX2. Meanwhile, PeleeNet is only 66% of the model size of MobileNet. We then propose a real-time object detection system by combining PeleeNet with Single Shot MultiBox Detector (SSD) method and optimizing the architecture for fast speed. Our proposed detection system2, named Pelee, achieves 76.4% mAP (mean average precision) on PASCAL VOC2007 and 22.4 mAP on MS COCO dataset at the speed of 23.6 FPS on iPhone 8 and 125 FPS on NVIDIA TX2. The result on COCO outperforms YOLOv2 in consideration of a higher precision, 13.6 times lower computational cost and 11.3 times smaller model size.

1 Introduction

The paper introduces PeleeNet, a conventional-convolution architecture for efficient mobile image classification, and combines it with an optimized SSD detector to target fast object detection under tight resource constraints.

  • Motivation: Mobile CNN architectures face strict memory and computational constraints, while depthwise separable convolution lacks efficient implementation and few studies combine efficient models with fast detection.The research therefore addresses both efficient image classification and object detection.
  • PeleeNet: PeleeNet is a DenseNet variant designed for mobile devices and constrained memory and computation budgets.It follows DenseNet’s connectivity pattern and selected design principles.
  • Classification results: PeleeNet reaches 72.6% top-1 accuracy on ImageNet, 2.1% above MobileNet, while using 66% of MobileNet’s model size.On Stanford Dogs, it is 5.05% above the original-DenseNet model and 6.53% above MobileNet.
  • PeleeNet: A 2-way dense layer uses parallel 3x3 convolution paths to obtain different receptive-field scales for visual patterns of different object sizes.One path uses a single 3x3 convolution, while the other stacks two 3x3 convolutions.
  • PeleeNet: A cost-efficient stem block improves feature-expression ability without adding much computational cost.The design is presented as preferable to more expensive alternatives such as increasing initial channels or growth rate.
  • PeleeNet: The bottleneck layer dynamically varies its channel count with input shape rather than using DenseNet’s fixed 4-times-growth-rate setting.This avoids bottleneck layers increasing computation when their channels exceed the number of input channels, while retaining bottlenecks across dense layers.
  • Object detection system: The optimized Pelee detector selects five scale feature maps and uses residual prediction blocks and 1x1 prediction kernels to reduce detection cost.The 1x1 kernels preserve nearly the accuracy of 3x3 kernels while reducing computational cost by 21.5%.
  • Evaluation: The benchmark evaluates efficient classification models and one-stage detectors on NVIDIA TX2 and iPhone 8.These embedded-platform tests assess both model efficiency and real-device performance.

2 PeleeNet: An Efficient Feature Extraction Network

PeleeNet is a four-stage, DenseNet-inspired feature extractor designed for mobile constraints, using architectural choices evaluated against efficiency and accuracy targets. Its results show strong accuracy and speed on embedded platforms, while device-specific behavior varies.

  • Architecture: PeleeNet retains a four-stage feature-extractor structure because early feature maps are considered important for representational ability.The architecture uses a stem block and four stages, with average pooling at the ends of most stages.
  • Design evaluation: 79.25% accuracy: combining the design choices improved Stanford Dogs accuracy by 4.23% over DenseNet-41 at less computational cost.The models were trained for 120 epochs with mini-batch size 256 in this experiment.
  • ImageNet results: 66% model size: PeleeNet achieved higher accuracy than MobileNet and ShuffleNet with no more than 66% of their model size and lower computational cost.PeleeNet was also reported as 1/49 the model size of VGG16.
  • Real-device speed: 1.8 times faster in FP16 than FP32: PeleeNet benefits substantially from TX2 half-precision inference, unlike MobileNet and MobileNetV2.The depthwise-separable-convolution networks showed nearly unchanged speed between FP16 and FP32 modes.
  • Real-device speed: On iPhone 8, PeleeNet is slower than MobileNet for small inputs but faster for large inputs, partly because CoreML and the network’s multi-branch narrow-channel design favor separable convolution.The authors state that the original design was misled by FLOPs and included unnecessary complexity.

3 Pelee: A Real-Time Object Detection System

Pelee combines an SSD detector optimized for speed with PeleeNet, using selected feature maps, residual prediction blocks, and small prediction kernels. It achieves strong VOC and COCO accuracy while reducing computational cost and model size relative to competing detectors.

  • System overview: Pelee combines a speed-optimized SSD architecture with PeleeNet for mobile object detection.The system uses selected feature maps, residual prediction blocks, and small convolutional kernels for prediction.
  • Feature map selection: Five feature-map scales—19 x 19, 10 x 10, 5 x 5, 3 x 3, and 1 x 1—are used, while the 38 x 38 map is omitted to balance speed and accuracy.The 19 x 19 map uses two default-box scales; each remaining map uses one.
  • Prediction design: 21.5% lower computational cost and 33.9% smaller model size result from using 1x1 rather than 3x3 prediction kernels, with almost the same accuracy.A residual prediction block also improves accuracy by 2.2% over a model without that block.
  • Results on PASCAL VOC2007: 76.4% mAP is achieved on PASCAL VOC2007 after fine-tuning a COCO-trained model on the 07+12 dataset.The reported training setup uses COCO trainval35k followed by fine-tuning on VOC 07+12.
  • Real-device speed: Pelee runs faster than SSD+MobileNet on iPhone and TX2 in FP32 mode, with a greater speed advantage over SSD+MobileNet and SSDLite+MobileNetV2 in FP16 mode.Inference-speed measurements include image preprocessing but exclude CPU post-processing such as decoding and non-maximum suppression.
  • Results on COCO: 22.4 mAP is achieved on COCO, with higher accuracy than YOLOv2 in both mAP@[0.5:0.95] and mAP@0.75.On COCO, Pelee is also reported as 3.7 times faster and 11.3 times smaller than YOLOv2.

4 Conclusion

The paper concludes that efficient mobile models need not rely on depthwise separable convolution. By combining conventional-convolution architecture design with mobile GPU and hardware-optimized runtime libraries, Pelee supports real-time image classification and object detection.

  • Conclusion: PeleeNet and Pelee use conventional convolution instead of depthwise separable convolution and achieve compelling results on ILSVRC 2012, VOC 2007, and COCO.The conclusion frames depthwise separable convolution as one option rather than the only route to efficient models.
  • Conclusion: 23.6 FPS on iPhone 8 and 125 FPS on NVIDIA TX2 demonstrate real-time prediction for the proposed Pelee system.The conclusion attributes this capability to combining efficient architecture design with mobile GPU and hardware-specified optimized runtime libraries.
Loading 1804.06882v3…