PaperlayerView exact revision on arXiv

Source-linked AI summary

Shift-based Primitives for Efficient Convolutional Neural Networks

Huasong Zhong, Xianggen Liu, Yihui He, Yuchun Ma

arXiv:1809.08458v2cs.CVcs.LG

TL;DR

CNNs can be accurate yet costly in computation and memory, and compact-model operations may still slow GPU inference. The paper introduces pointer-shift primitives and builds AddressNet and Enhanced AddressNet; across CIFAR100 and ImageNet, the models are reported faster with comparable or better accuracy.

  • Problem

    Compact CNNs can have operations that remain inefficient on GPUs, so fewer parameters or FLOPs do not necessarily mean faster inference.

    Source: §1 · p. 1

  • Method

    The paper combines channel, address, and shortcut shifts with point-wise group convolution to build AddressNet and Enhanced AddressNet.

    Source: §1 · p. 2, §5 · p. 7

  • Results

    The paper reports that its shift-based models are faster and achieve comparable or better accuracy on CIFAR100 and ImageNet.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The authors conclude that shift operations provide a scalable and flexible basis for designing compact architectures.

    Source: §4.4 · p. 7

    Limited cuDNN optimization for group convolution favors models not based on group convolution, constraining the reported results to comparable rather than best performance.

    Source: §4.4 · p. 7

Abstract

from arXiv · show

We propose a collection of three shift-based primitives for building efficient compact CNN-based networks. These three primitives (channel shift, address shift, shortcut shift) can reduce the inference time on GPU while maintains the prediction accuracy. These shift-based primitives only moves the pointer but avoids memory copy, thus very fast. For example, the channel shift operation is 12.7x faster compared to channel shuffle in ShuffleNet but achieves the same accuracy. The address shift and channel shift can be merged into the point-wise group convolution and invokes only a single kernel call, taking little time to perform spatial convolution and channel shift. Shortcut shift requires no time to realize residual connection through allocating space in advance. We blend these shift-based primitives with point-wise group convolution and built two inference-efficient CNN architectures named AddressNet and Enhanced AddressNet. Experiments on CIFAR100 and ImageNet datasets show that our models are faster and achieve comparable or better accuracy.

1. Introduction

CNNs offer high accuracy but can demand substantial computation and memory, motivating fast, energy-efficient models for latency-sensitive applications. The paper proposes three shift primitives and builds two compact architectures that are faster with comparable or better accuracy on CIFAR100 and ImageNet.

1. Introduction

  • CNN accuracy comes with increased computation time and memory use, while some applications require low latency, small storage, and appropriate accuracy.The introduction cites self-driving cars and speech recognition as examples.Source: §1 · p. 1
  • Reducing parameters or FLOPs does not ensure faster inference, because core operations in compact architectures can remain inefficient on GPUs.The introduction notes that channel shuffle and shortcut connections in ShuffleNet account for 30% of inference time despite requiring no FLOPs or parameters.Source: §1 · p. 1
  • AddressNet and Enhanced AddressNet combine channel, address, and shortcut shifts with point-wise group convolution, achieving faster inference with comparable or better accuracy on CIFAR100 and ImageNet.The primitives move pointers to minimize memory copying and avoid floating-point operations.Source: §1 · p. 2

2. Related Work

Related work distinguishes compressing pretrained models from designing small models directly, then reviews both strategies and their trade-offs. The paper places its approach among methods for compact layers and networks, where theoretical efficiency may not translate into efficient inference.

2. Related Work

  • Prior efficient-network approaches are grouped into compressing pretrained models or training small models directly.The paper takes the latter approach, designing compact networks directly.Source: §2 · p. 2

2.1. Compressing Neural Networks

  • Compression methods include pruning [22] [13] [7], low-rank factorization [32], quantization and binarization [5], and knowledge distillation [27].The paper distinguishes its direct compact-network design from compression of pretrained networks.Source: §2.1 · p. 2

2.2. Designing compact layers and networks

  • Compact architectures use structures such as ResNet bottlenecks, ResNeXt multi-branch layers, SqueezeNet fire modules [16], and GoogLeNet to reduce computation or parameters.The approaches described include reducing channel counts or replacing some 3 × 3 convolutions with 1×1 convolutions.Source: §2.2 · p. 2
  • Depthwise separable convolution can reduce computation in theory, but fragmented memory footprints make it expensive for inference despite low FLOPs.The passage notes this drawback is also mentioned in [37].Source: §2.2 · p. 2
  • ShiftNet’s feature map shift is parameter-free and FLOP-free but is not efficiently implemented as depthwise separable convolution; address shift instead uses pointer shifting.The paper reserves “feature map shift” for the ShiftNet method.Source: §2.2 · p. 2
  • ShuffleNet uses channel shuffle to restore information flow across pointwise group convolutions, but the shuffle moves channels into another memory space.Residual connections include identity mapping and channel concatenation, used in ResNet and ShuffleNet; DenseNet concatenates previous layer outputs.Source: §2.2 · p. 2

3. Approach

The approach combines three pointer-based shift primitives with pointwise group convolution to build efficient CNN modules. It introduces channel and spatial shifts, then uses these operations in Address-based and Address-enhanced modules.

3.1. Channel Shift

  • Channel shift blends information across adjacent channel groups by shifting addresses, avoiding data movement and using 8× less data movement than channel shuffle in the illustrated example.Stacked group convolutions can otherwise restrict information flow between groups.Source: §3.1 · p. 3, Figure 2 · p. 3

3.2. Address Shift

  • Address shift implements spatial movement by offsetting feature-map pointers, with right, left, up, and down offsets defined relative to the memory stride.Unlike ShiftNet's zero-padded feature-map shift, address shift has nonzero boundaries, which the authors report did not noticeably affect accuracy; four basic directions can represent other shifts.Source: §3.2 · pp. 3–4

3.3. Shortcut Shift

  • Shortcut shift preallocates continuous storage for consecutive layer outputs, allowing channel concatenation without copying data or spending computation time.The authors identify DenseNet, which relies heavily on channel concatenation, as a setting where this optimization can be useful.Source: §3.3 · p. 4

3.4. Address-based module

  • The Address-based module applies pointwise group convolution, channel shift, and four-direction address shift, while Address-enhanced embeds both shifts in its second group convolution.The enhanced design uses four groups corresponding to shift directions and arranges forward and backward offsets to avoid memory overflow or extra memory overhead.Source: §3.4 · p. 4, Figure 5 · p. 5

4. Experiments

The experiments evaluate the proposed architectures and modified MobileNet on CIFAR100 and ImageNet. AddressNet and Enhanced AddressNet are built in multiple depths using stacked modules.

4. Experiments

  • The evaluation plan compares the proposed architectures with primary operations and ShiftResNet on CIFAR100, then evaluates modified MobileNet and the proposed models on ImageNet.The section states the evaluation scope but supplies no benchmark outcomes in these passages.Source: §4 · p. 5
  • AddressNet and Enhanced AddressNet use three blocks at feature-map sizes 32, 16, and 8, with block output channels of {48, 60, 96} and {48, 96, 192}, respectively.Each block has 3, 5, or 7 modules in the 20-, 32-, or 44-layer variants.Source: §4 · p. 5
  • The networks end with global average pooling and a 100-way classifier, and use average pooling with doubled output channels when feature-map sizes change.Subsampling occurs in the second pointwise group convolution with stride 2.Source: §4 · p. 5

4.1. Implementation details

Implementation details specify training and evaluation procedures for CIFAR100 and ImageNet. Inference time is measured on a single GeForce GTX 1080 GPU with batch size 1.

4.1. Implementation details

  • CIFAR100 training uses 32×32 inputs, SGD for 300 epochs, batch size 128 across four GPUs, and expansion rate 3 for most experiments.The stated weight decay is 0.0005 and momentum is 0.9.Source: §4.1 · p. 5
  • ImageNet training uses random 224×224 crops from 256×256 images, while evaluation uses a single 224×224 center crop and expansion rates listed in Table 7.The learning rate starts at 0.1 and is divided by 10 every 30 epochs.Source: §4.1 · p. 5
  • Inference time is averaged over two epochs on one GeForce GTX 1080 GPU with batch size 1, after batch-normalization layers are removed for testing.The authors state that batch normalization can be fused into a convolution layer in advance.Source: §4.1 · p. 5

4.2. Shift-based Primitives vs. baselines

The experiments compare the shift primitives with channel shuffle, feature-map shift, identity-map shortcuts, and depthwise separable convolution. Across these comparisons, the proposed shifts preserve or improve accuracy while reducing operation or inference time in the reported settings.

4.2.1 Channel shift vs. Channel shuffle

  • 12.7 times faster operation time and 1.4 times faster total time, with the same accuracy, were achieved by channel shift versus channel shuffle.The comparison uses AddressNet-32 models with the same parameter and FLOP counts; operation time is the average time consumed by the operation.Source: §4.2.1 · p. 5, Table 2 · p. 6

4.2.2 Address Shift vs. Feature map Shift

  • Four basic shift directions achieved nearly the same performance as nine directions, while address shift achieved nontrivial inference speedup over feature-map shift.The experiments motivate using address shift with four directions in subsequent models.Source: §4.2.2 · pp. 5–6

4.2.3 Shortcut shift vs. Identity map

  • Shortcut shift achieved almost equivalent performance to identity-map layers with similar parameters and FLOPs, without spending inference time.The comparison reduces current-layer output channels to match the previous residual function and uses expansion rate four.Source: §4.2.3 · p. 6

4.2.4 Address shift vs. Depthwise separable convolution

  • Enhanced AddressNet-32 behaved better than MobileNet-32 with similar parameter and FLOP scales, supporting address shift's comparable or better expressive ability.MobileNet-32 was designed with the same number of layers for a fair comparison.Source: §4.2.4 · p. 6

4.3. Performance on CIFAR100

On CIFAR100, AddressNet and Enhanced AddressNet are compared with ShiftResNet across model scales. The proposed models achieve better accuracy and significantly reduce inference time in the reported comparisons.

4.3. Performance on CIFAR100

  • AddressNet-44 achieved better accuracy than the best-accuracy ShiftResNet with 3x fewer FLOPs and 6x fewer parameters.Across different parameter and FLOP levels, both proposed architectures consistently achieved better accuracy and significantly reduced inference time; Enhanced AddressNet was always better than AddressNet.Source: §4.3 · p. 6
  • AddressNet and Enhanced AddressNet consistently achieved better accuracy than ShiftResNet across different parameter and FLOP levels.The ShiftResNet results were quoted from [34].Source: §4.3 · p. 6
  • Enhanced AddressNet was always better than AddressNet in the reported CIFAR100 comparisons.The passage attributes this to its larger groups and shift-operation mechanism.Source: §4.3 · p. 6

4.4. ImageNet

The ImageNet experiments adapt MobileNet and Enhanced AddressNet using address shift. Address-shift MobileNet performs especially well in the low-accuracy scenario, while the paper reports comparable performance in modest-accuracy scenarios and notes a cuDNN group-convolution constraint.

4.4. ImageNet

  • Address-MobileNet replaces depthwise separable convolutions with address shift and is scaled to input sizes 192 and 160; Enhanced AddressNet-A and -B adapt depth and expansion rate for ImageNet.Enhanced AddressNet-B is described as shallower than A.Source: §4.4 · pp. 6–7
  • Address-shift MobileNet excelled in both accuracy and inference time in the low-accuracy scenario, while the models were comparable to other compact state-of-the-art models in modest-accuracy scenarios.The authors describe the low-accuracy result as validating address shift's scalability to a large dataset.Source: §4.4 · p. 7
  • Limited cuDNN optimization for group convolution favors models that do not use group convolution, constraining the reported results to comparable rather than best performance.The authors also conclude that the experiments demonstrate the scalability and flexibility of the three shift operations for compact architectures.Source: §4.4 · p. 7

5. Conclusions

The paper presents three shift-based operations for designing fast, energy-efficient CNNs and proposes AddressNet and Enhanced AddressNet. It reports improved accuracy on CIFAR and ImageNet and describes evaluations across different accuracy levels.

5. Conclusions

  • Shift-based operations use address offsets for spatial convolutions and are described as requiring no parameters, FLOPs, or inference time.The operations include channel shift, shortcut shift, and address shift.Source: §5 · p. 7
  • Table 6 compares ShiftResNet and AddressNet on CIFAR100, with ShiftResNet results quoted from [34].Source: Table 6 · p. 8
  • Figure 7 compares models across three accuracy levels; models nearer the upper left have better accuracy and lower inference time.Source: Figure 7 · p. 8
  • AddressNet and Enhanced AddressNet outperform ShiftNet significantly, and shift-based networks achieve better accuracy on CIFAR and ImageNet.The paper also presents variants at different accuracy levels, including a shallower Enhanced AddressNet-B for low accuracy.Source: §5 · p. 8
  • Enhanced AddressNet-B is a little shallower to fit the low-accuracy level.Source: Table 8 · p. 8

Related papers

Loading 1809.08458v2…