PaperlayerView exact revision on arXiv

Source-linked AI summary

GhostNets on Heterogeneous Devices via Cheap Operations

Kai Han, Yunhe Wang, Chang Xu, Jianyuan Guo, Chunjing Xu, Enhua Wu, Qi Tian

arXiv:2201.03297v1cs.CV

TL;DR

Mobile deployment is constrained by limited memory and computation, while feature-map redundancy remains underused in neural architecture design. The paper introduces C-Ghost modules and C-GhostNet for CPUs, plus G-Ghost stages and G-GhostNet for GPUs, using cheap operations to generate redundant features efficiently. The proposed models achieve favorable accuracy–latency trade-offs across heterogeneous devices, while computational cost and performance remain a trade-off.

  • Problem

    Limited memory and computation complicate CNN deployment on mobile devices, and feature-map redundancy has rarely been exploited in neural architecture design.

  • Method

    The paper uses C-Ghost modules and G-Ghost stages to generate intrinsic and redundant feature maps with cheap operations for CPU- and GPU-efficient CNNs.

  • Results

    C-GhostNet and G-GhostNet achieve favorable accuracy–latency trade-offs for CPU and GPU deployment, respectively.

  • Takeaways & Limitations

    Ghost modules can serve as plug-and-play components for compact CNNs, while Ghost stages extend redundancy exploitation across layers for GPU efficiency.

  • Takeaways & Limitations

    The ghost ratio requires a trade-off because increasing it decreases FLOPs while gradually reducing accuracy.

Abstract

from arXiv · show

Deploying convolutional neural networks (CNNs) on mobile devices is difficult due to the limited memory and computation resources. We aim to design efficient neural networks for heterogeneous devices including CPU and GPU, by exploiting the redundancy in feature maps, which has rarely been investigated in neural architecture design. For CPU-like devices, we propose a novel CPU-efficient Ghost (C-Ghost) module to generate more feature maps from cheap operations. Based on a set of intrinsic feature maps, we apply a series of linear transformations with cheap cost to generate many ghost feature maps that could fully reveal information underlying intrinsic features. The proposed C-Ghost module can be taken as a plug-and-play component to upgrade existing convolutional neural networks. C-Ghost bottlenecks are designed to stack C-Ghost modules, and then the lightweight C-GhostNet can be easily established. We further consider the efficient networks for GPU devices. Without involving too many GPU-inefficient operations (e.g.,, depth-wise convolution) in a building stage, we propose to utilize the stage-wise feature redundancy to formulate GPU-efficient Ghost (G-Ghost) stage structure. The features in a stage are split into two parts where the first part is processed using the original block with fewer output channels for generating intrinsic features, and the other are generated using cheap operations by exploiting stage-wise redundancy. Experiments conducted on benchmarks demonstrate the effectiveness of the proposed C-Ghost module and the G-Ghost stage. C-GhostNet and G-GhostNet can achieve the optimal trade-off of accuracy and latency for CPU and GPU, respectively. Code is available at https://github.com/huawei-noah/CV-Backbones.

1 Introduction

Efficient CNN design targets mobile deployment by reducing computation while preserving accuracy. This paper exploits feature-map redundancy to develop CPU- and GPU-oriented Ghost architectures.

  • 25.6M parameters and 4.1B FLOPs make ResNet50 costly for 224×224 image processing, motivating portable architectures for mobile devices.
  • Feature maps in well-trained CNNs often contain similar pairs, motivating cost-efficient generation of redundant features rather than eliminating them.
  • C-Ghost splits convolution into controlled ordinary convolutions and cheap linear operations that generate additional feature maps from intrinsic features.
  • C-Ghost reduces parameters and computational complexity without changing output size, while preserving similar recognition performance.
  • GPU-efficient design must account for hardware behavior because low-FLOP operators such as depth-wise convolution may not reduce GPU latency as expected.
  • The extended work introduces G-Ghost stages and G-GhostNet to exploit stage-wise redundancy and provide lightweight CNNs with state-of-the-art GPU performance.

2 Related Work

Related work addresses efficient CNNs through model compression and compact architecture design. Prior GPU-focused approaches reduce complexity or guide architecture using hardware-relevant characteristics.

  • Model compression reduces computation, energy, and storage costs through techniques such as pruning connections and channels.
  • Compact model design develops lightweight architectures for embedded deployment, including depth-wise-convolution-based Xception and MobileNets.
  • GPU-oriented methods include SqueezeNext’s decomposed convolutions and RegNet’s empirical guidance based on activations and GPU speed.

3 CPU-Efficient GhostNet

C-Ghost reduces convolutional cost by generating a small set of intrinsic feature maps and expanding them with cheap operations. These modules form C-Ghost bottlenecks and C-GhostNet, with configurable width for latency, size, and accuracy needs.

  • 3.1 Ghost Module for More Features: C-Ghost targets redundant feature maps by producing intrinsic maps with ordinary convolutions and deriving additional ghost maps through cheap operations.This replaces generating every output feature map independently with large convolutions.
  • 3.1 Ghost Module for More Features: The module preserves intrinsic maps through identity mapping while applying per-channel linear transformations to generate additional ghost features.The output contains n = m · s feature maps from m intrinsic maps and s generated variants per intrinsic map.
  • 3.1 Ghost Module for More Features: C-Ghost can replace ordinary convolutional layers while producing the same number of feature maps with lower computational cost.Its cheap operations use substantially less computation than ordinary convolution, and the reported compression ratio equals the theoretical speed-up ratio.
  • 3.2 Building Lightweight C-GhostNet: C-Ghost bottlenecks stack two C-Ghost modules, using the first as an expansion layer and adapting the shortcut path for stride=2.For stride=2, the shortcut downsamples and a depth-wise convolution with stride=2 is inserted between the modules.
  • 3.2 Building Lightweight C-GhostNet: C-GhostNet replaces MobileNetV3 bottlenecks with C-Ghost bottlenecks arranged in stages, while omitting hard-swish because of its large latency.The architecture uses standard convolution initially, stage-wise stride changes, global average pooling, and a final 1280-dimensional feature vector; some residual layers include SE modules.
  • 3.2 Building Lightweight C-GhostNet: A width multiplier α uniformly scales channels, approximately controlling model size and computational cost by α^2.This allows C-GhostNet to be customized for smaller, faster models or higher accuracy on specific tasks.

4 GPU-Efficient GhostNet

G-Ghost targets GPU-efficient CNN design by exploiting redundancy across blocks within a stage, reducing expensive computation while preserving representation through intrinsic feature processing and cheap operations.

  • Motivation: G-Ghost addresses GPU inefficiency in lightweight CNNs, where low-FLOPs operators such as depth-wise convolution may have low arithmetic intensity and poor GPU utilization.The design targets a better trade-off between accuracy and GPU latency by simplifying the computationally heavy network body.
  • G-Ghost Stage: G-Ghost exploits similarity between early and later block features, allowing parts of a stage’s output to be approximated from shallow features rather than produced by all blocks.The proposed structure can replace a vanilla CNN stage while retaining the original output feature dimension.
  • G-Ghost Stage: The stage separates features into complicated and ghost parts, processing the complicated features with blocks and generating ghost features from shallow features using cheap operations.The ghost-feature ratio λ controls the proportion of features generated through the cheaper path.
  • Intrinsic Feature Aggregation: Intermediate features from the complicated path are transformed and mixed into ghost features to supplement information that cheap operations may lack.The mix operation aggregates intermediate representations using a simple transformation based on global average pooling and fully connected layers.
  • Complexity Analysis: G-Ghost reduces stage computation and memory cost, with larger reductions for stages containing at least three blocks, although an excessive ghost ratio weakens representation capacity.The reduction depends on the original stage structure, ghost ratio λ, cheap operation, and mix operation; the method therefore requires a cost–performance trade-off.
  • G-GhostNet: G-Ghost stages can remould existing CNN architectures, and G-GhostNet applies them throughout a lightweight GPU-oriented network using mix operations.The stage is presented as a general structure applicable to different block types, including residual, inverted residual, and ghost blocks.

5 Experiments

The experiments evaluate the efficiency of the proposed C-Ghost module on CPUs and the G-Ghost stage on GPUs.

  • Experiments assess the efficiency of the proposed C-Ghost module on CPU and the G-Ghost stage on GPU.

5.1 Datasets and Experimental Settings

The study evaluates its methods on CIFAR10, ImageNet, and MS COCO using standard training and evaluation setups, with models implemented in PyTorch.

  • Datasets: CIFAR10, ImageNet ILSVRC2012, and MS COCO are used to evaluate classification and object-detection performance.CIFAR10 contains 60,000 32×32 images; ImageNet contains 1,000 categories and 1.2 million images; COCO evaluation uses the 2014 minival split.
  • Experimental Settings: Models are implemented in PyTorch and trained on NVIDIA V100 GPUs with baseline-matched or established training settings.ImageNet models use the baseline networks’ training schemes for fair comparison, while object-detection experiments use MMDetection with a common 1× scheduler.

5.2 CPU-Efficient C-GhostNet

C-GhostNet replaces ordinary convolutions with intrinsic-feature extraction plus cheap linear operations, exploiting feature-map redundancy to reduce computation while preserving recognition performance. Experiments across CIFAR-10, ImageNet, and detection benchmarks show favorable accuracy–efficiency trade-offs.

  • Toy experiments: Extremely small MSE values indicate strong correlations between deep-network feature maps, supporting generation of redundant maps from fewer intrinsic maps.The toy experiment evaluates reconstruction error across feature-map pairs.
  • C-Ghost module: C-Ghost modules generate additional feature maps from intrinsic maps using cheap linear operations, reducing the computational cost of ordinary convolutional layers.The module can be used to replace convolutional layers in existing architectures.
  • Hyper-parameters: d = 3 performs better than smaller or larger kernels, while 1×1 kernels lack spatial information and 5×5 or 7×7 kernels add computation and overfitting risk.The experiments fix s = 2 while varying d on CIFAR-10.
  • Hyper-parameters: Increasing s reduces FLOPs substantially but gradually lowers accuracy, reflecting a direct trade-off between compression and recognition performance.The study varies s from 2 to 5 with d = 3.
  • CIFAR-10 experiments: 93.7% accuracy is achieved by C-Ghost-VGG16 (s = 2) with significantly fewer FLOPs than competing compression methods.On ResNet56, the method achieves comparable baseline accuracy with 2× speed-up.
  • ImageNet experiments: About 2× acceleration and compression are obtained on ResNet50 while maintaining original accuracy; at s = 4, the accuracy drop is 1.2% with about 4× speed-up.The comparisons report stronger performance than methods with similar weights or FLOPs.
  • C-GhostNet: C-GhostNet consistently outperforms competing lightweight networks across approximately 50, 150, and 200–300 MFLOPs computational levels.The reported comparisons include MobileNet, ShuffleNet, ProxylessNAS, FBNet, and MnasNet.
  • C-GhostNet: C-GhostNet reaches 75.2% accuracy with 40 ms latency, compared with about 46 ms for MobileNetV3 at similar accuracy.The comparison concerns processing one image on the evaluated mobile device.

5.3 GPU-Efficient G-Ghost Structure

G-Ghost uses stage-wise feature redundancy to reduce GPU cost without relying heavily on GPU-inefficient operations. Across CIFAR-10 and ImageNet evaluations, it improves or preserves accuracy while reducing FLOPs or GPU latency.

  • G-Ghost design: G-Ghost stages split features into intrinsic and cheap-operation paths, enabling GPU-efficient generation of redundant stage features.The method is evaluated on image recognition and object detection networks.
  • CIFAR10 experiments: G-Ghost-ResNet56 with mixing reduces FLOPs by 2.32× with only a 0.21% accuracy drop versus vanilla ResNet56.Mixing intrinsic features improves performance by 0.78% over the baseline comparison.
  • Cheap operations: Using 1×1 convolution as the cheap operation performs better than identity mapping or removing the cheap operation.The ablation tests whether cheap operations contribute beyond the surrounding architecture.
  • Cheap operations: The 3×3 convolution gives the best accuracy among tested cheap operations, while 1×1 convolution offers good performance with fewer FLOPs and parameters.The study adopts 1×1 convolution for subsequent experiments.
  • Ghost ratio: Increasing ghost ratio λ substantially decreases FLOPs while gradually reducing accuracy; λ = 0.5 is used as the usual trade-off.The ratio controls the proportion of cheap features and the resulting compression.
  • ImageNet experiments: G-Ghost-ResNet34 with mixing achieves comparable accuracy to original ResNet34 while reducing GPU latency by about 16%.The stage structure is applied by replacing all stages in ResNet34.
  • Network pruning: G-Ghost-ResNet34 outperforms the compared pruning methods with higher accuracy and fewer FLOPs and parameters.The comparison includes SFP, PFEC, Taylor-FO-BN, and FPGM.
  • Feature visualization: G-Ghost-ResNet34 feature patterns resemble original ResNet34 features, while the reduced complexity does not harm network capacity.The comparison uses feature maps from the third stage of both models.

6 Conclusion

GhostNet accelerates CNN inference on heterogeneous CPU and GPU devices by generating feature maps through cheap operations. C-GhostNet and G-Ghost stages preserve recognition performance while improving efficiency across their target hardware.

  • CPU-efficient design: C-Ghost modules split convolutional layers, using fewer filters for intrinsic maps and cheap operations for additional ghost maps.The module is designed as a plug-and-play component for converting conventional models into compact ones.
  • GPU-efficient design: G-GhostNet is evaluated against representative models using accuracy and GPU latency on an NVIDIA Tesla V100 with batch size 64.Advanced auto-augment is applied in the reported comparison.
  • CPU-efficient design: C-GhostNet outperforms state-of-the-art portable neural architectures in efficiency and accuracy.
  • GPU-efficient design: G-Ghost stages split stage outputs into intrinsic and ghost features, generating the latter with GPU-efficient cheap operations from reduced-channel original stages.Intermediate intrinsic features are additively aggregated into ghost features to enhance representation capacity.
Loading 2201.03297v1…