PaperlayerView exact revision on arXiv

Source-linked AI summary

PerforatedCNNs: Acceleration through Elimination of Redundant Convolutions

Michael Figurnov, Aijan Ibraimova, Dmitry Vetrov, Pushmeet Kohli

arXiv:1504.08362v4cs.CV

TL;DR

High CNN evaluation costs hinder deployment on low-power devices. PerforatedCNNs skip selected convolutional output positions and interpolate the rest; the paper reports 2×–4× acceleration and a combined method that outperforms both components.

  • Problem

    CNN evaluation is computationally costly, hindering deployment on low-power devices such as mobile phones.

    Source: §1 · p. 1

  • Method

    PerforatedCNNs calculate convolutional outputs at selected spatial positions and interpolate the remaining values.

    Source: §3.1 · p. 3

  • Results

    Perforation accelerates modern CNNs by 2×–4× with a small accuracy decrease, and the combined method outperforms both its components.

    Source: §1 · p. 1, §4.4 · p. 8

  • Takeaways & Limitations

    Perforation preserves CNN architecture and can be added through fine-tuning rather than complete retraining.

    Source: §5 · p. 8

Abstract

from arXiv · show

We propose a novel approach to reduce the computational cost of evaluation of convolutional neural networks, a factor that has hindered their deployment in low-power devices such as mobile phones. Inspired by the loop perforation technique from source code optimization, we speed up the bottleneck convolutional layers by skipping their evaluation in some of the spatial positions. We propose and analyze several strategies of choosing these positions. We demonstrate that perforation can accelerate modern convolutional networks such as AlexNet and VGG-16 by a factor of 2x - 4x. Additionally, we show that perforation is complementary to the recently proposed acceleration method of Zhang et al.

1 Introduction

CNN evaluation is costly, limiting deployment on low-power devices. PerforatedCNNs skip selected convolutional output positions and interpolate them, achieving 2×–4× faster evaluation with a small accuracy decrease.

1 Introduction

  • Convolutional layers usually take over 80% of CNN evaluation time, and VGG-16 requires 1.5 · 10^10 floating point multiplications per image.These computational requirements hinder deployment on systems without GPUs and in power-constrained scenarios such as mobile devices.Source: §1 · p. 1
  • PerforatedCNNs evaluate only selected spatial output positions and interpolate the others, avoiding the architecture changes caused by convolutional strides.The perforation mask determines which positions are calculated exactly; the authors propose strategies for selecting masks and combining them across layers.Source: §1 · p. 1
  • Perforation reduces evaluation time of modern CNN architectures by 2×–4× with a small decrease in accuracy.The method is inspired by loop perforation, which reduces computation by skipping selected iterations.Source: §1 · p. 1

2 Related Work

Prior CNN acceleration work includes optimized implementations and approximate methods. The paper distinguishes its spatial output sampling from methods that reduce convolutional kernels or exploit sparse inputs.

2 Related Work

  • CNN acceleration research includes highly optimized implementations and approximate methods for reducing evaluation cost.The paper cites GPU implementations, low-precision arithmetic, low-rank tensor decompositions, and methods for spatially sparse inputs.Source: §2 · pp. 1–2
  • Unlike sparse-input methods, PerforatedCNNs sparsely sample convolutional outputs and interpolate the remaining values.The approach does not rely on spatially sparse inputs.Source: §2 · p. 2
  • Lebedev and Lempitsky reduce convolutional kernel support, whereas PerforatedCNNs reduce the spatial positions evaluated; the approaches are complementary.Perforation can also be combined with compression methods for fully connected layers.Source: §2 · p. 2

3 PerforatedCNNs

PerforatedCNNs reduces convolution work by computing only selected spatial positions and using nearest-neighbor interpolation elsewhere. It describes several ways to choose those positions and a greedy strategy for configuring perforation across layers.

3 PerforatedCNNs

  • Input patches become rows of a data matrix multiplied by reshaped kernels, providing the matrix-multiplication implementation used for convolution.The resulting matrix is reshaped into the output tensor.Source: §3 · p. 3

3.1 Perforated convolutional layer

  • The layer computes selected positions exactly and nearest-neighbor interpolates the rest, storing only exact outputs and requiring masks shared across channels for matrix-multiplication lowering.This reduces activation storage by a factor of 1/(1-r); subsequent nonlinearities and 1 × 1 convolutions also operate on fewer elements.Source: §3.1 · p. 3

3.2 Perforation masks

  • Four input-independent mask strategies select exact spatial positions: uniform random, grid, pooling-structure, and loss-impact based.Uniform samples can cluster when few positions are retained; pooling-structure masks prioritize positions used most often by pooling operators.Source: §3.2 · pp. 3–4
  • Impact masks estimate each position’s loss change with a first-order Taylor approximation, then select the top-N positions by channel-aggregated impacts averaged over training data.The paper iterates between increasing a layer’s perforation rate and recalculating impacts, reporting improved results through co-adaptation of layer masks.Source: §3.2 · pp. 4–5

3.3 Choosing the perforation configurations

  • A greedy strategy tests increasing perforation rates across layers and selects the next layer according to the stated cost-function value.The procedure defines evaluation time and an objective for the accelerated network relative to the unaccelerated network.Source: §3.3 · p. 5

4 Experiments

The experiments compare perforation masks on AlexNet layers and whole networks, benchmark whole-network strategies, and test perforation combined with another acceleration method.

4.1 Single layer results

  • On AlexNet, impact masks work best for conv2 while grid masks perform very well for conv3; reducing conv2 kernels to 3 × 3 plus impact perforation gives 9.1× acceleration for 1% top-5 error increase.Layers followed by max-pooling are easier to accelerate than layers followed by another convolution; the kernel-size reduction complements.Source: §4.1 · p. 6, Table 2 · p. 6

4.2 Baselines

  • Against resize, stride, and fractional-stride baselines, impact perforation is best and grid perforation is slightly worse.Strides are a good strategy when changing activation sizes and training from scratch are options; fractional strides perform poorly relative to strides, likely because nonuniform downsampling makes outputs harder for the next convolution to process.Source: §4.2 · pp. 6–7, Figure 5 · p. 7

4.3 Whole network results

  • All considered networks achieve 2× acceleration on CPU and GPU with under 2.6% increase of error after tuning.Theoretical speedups are usually close to empirical ones, and storing only non-perforated activations significantly reduces activation memory.Source: §4.3 · p. 7, Table 3 · p. 8

4.4 Combining acceleration methods

  • At 4× theoretical speedup on VGG-16, the combined method outperforms both perforation and Zhang et al.’s linear asymmetric method.The decomposition transfers the perforation mask to its first layer, and the combined method is fine-tuned again at a lower learning rate.Source: §4.4 · pp. 7–8, Table 4 · p. 8

5 Conclusion

PerforatedCNNs exploit redundancy in CNN intermediate representations to reduce evaluation time and memory consumption. They retain the CNN architecture, require fine-tuning rather than complete retraining, and can combine with methods exploiting other redundancy.

5 Conclusion

  • PerforatedCNNs exploit redundancy in intermediate representations to reduce evaluation time and memory consumption.The approach requires only a minor convolution-layer modification and achieves speedups close to theoretical ones on CPU and GPU.Source: §5 · p. 8
  • Unlike the baselines discussed, PerforatedCNNs do not change the CNN architecture or the size of intermediate representations.Retaining the architecture allows integration into existing computer-vision pipelines with fine-tuning rather than complete retraining.Source: §5 · p. 8
  • Perforation can be combined with acceleration methods that exploit other types of network redundancy for further speedups.Source: §5 · p. 8
  • Future work proposes input-dependent masks that focus on salient image parts, including non-rectangular or disjoint regions.The paper contrasts this possibility with visual-attention works that use rectangular image crops.Source: §5 · p. 8

A Interpolation strategy

Nearest-neighbor interpolation is the paper’s default for filling perforated values, balancing interpolation complexity against error. Zero replacement is inadequate for successful conv3 acceleration, while barycentric interpolation helps only in some cases.

A Interpolation strategy

  • Nearest-neighbor interpolation offers a good trade-off between memory accesses per interpolated value and achieved error.It is the paper’s chosen strategy for filling missing values.Source: A Interpolation strategy · p. 10
  • Zero replacement is not sufficient for successful acceleration of the AlexNet conv3 layer.Source: A Interpolation strategy · p. 10
  • Barycentric interpolation slightly improves results for pooling masks in conv2 and grid masks in conv3, but performs similarly or worse in other cases.Interior perforated points use a weighted sum of triangle vertices; exterior points use the nearest neighbor.Source: A Interpolation strategy · p. 10

B Single layer results for NIN network

The paper extends single-layer acceleration experiments to the three non-1 × 1 convolutional layers of NIN. Grid and impact masks perform best, as in AlexNet conv2 and conv3.

B Single layer results for NIN network

C Empirical and theoretical speedups

Empirical acceleration generally approaches theoretical speedup, though additional memory accesses leave measured results below theory in almost all cases. CPU and GPU results differ, making target-device perforation rates important.

C Empirical and theoretical speedups

D Implementation details

The implementation evaluates convolution through matrix multiplication, with perforation reducing the data matrix rows. Stacking mini-batch images restores input size for comparisons, at the cost of a comparatively fast transpose.

D Implementation details

  • Increasing perforation rate reduces matrix-multiplication efficiency, especially on GPU, whose performance is best with large inputs.Source: D Implementation details · p. 11
  • The implementation stacks ⌊1/(1−r)⌋ mini-batch images to match the original data-matrix size, then transposes the result; the transpose is comparatively fast.The same stacking approach is applied to baseline methods.Source: D Implementation details · pp. 11–12
  • Figure 8 measures multiplications per second for data-matrix-by-kernel-matrix multiplication across perforation rates on AlexNet conv2.Source: D Implementation details · p. 11, Figure 8 · p. 12
Loading 1504.08362v4…