PaperlayerView exact revision on arXiv

Source-linked AI summary

A CNN Accelerator on FPGA Using Depthwise Separable Convolution

Lin Bai, Yiming Zhao, Xinming Huang

arXiv:1809.01536v2eess.SPcs.AR

TL;DR

CNNs are computationally intensive and difficult to deploy on portable devices because GPU platforms consume substantial power. This paper proposes a scalable FPGA accelerator optimized for depthwise separable convolution, using resource-performance balancing and hierarchical buffering. On Arria 10, MobileNetV2 achieved 266.6 frames per second, or a 20x speedup compared with CPU execution.

  • Problem

    CNN deployment on portable devices is difficult because convolution requires substantial computing power and GPUs have high power consumption.

  • Method

    The paper proposes a scalable FPGA CNN accelerator optimized for depthwise separable convolution, using matrix multiplication engines and hierarchical memory with ping-pong buffering.

  • Results

    266.6 frames per second and 170.6 GOPS were achieved for MobileNetV2 on an Arria 10 SoC FPGA at 133MHz, representing a 20x CPU speedup.

  • Takeaways & Limitations

    The accelerator can be configured for different FPGA sizes by balancing on-chip resources and processing performance, enabling CNN execution on portable devices.

Abstract

from arXiv · show

Convolutional neural networks (CNNs) have been widely deployed in the fields of computer vision and pattern recognition because of their high accuracy. However, large convolution operations are computing-intensive that often requires a powerful computing platform such as Graphics Processing Unit (GPU). This makes it difficult to apply CNNs to portable devices. The state-of-the-art CNNs, such as MobileNetV2 and Xception, adopt depthwise separable convolution to replace the standard convolution for embedded platforms. That significantly reduces operations and parameters with only limited loss in accuracy. This highly structured model is very suitable for Field-Programmable Gate Array (FPGA) implementation. In this paper, a scalable high performance depthwise separable convolution optimized CNN accelerator is proposed. The accelerator can be fit into an FPGA of different sizes, provided the balancing between hardware resources and processing speed. As an example, MobileNetV2 is implemented on Arria 10 SoC FPGA, and the results show this accelerator can classify each picture from ImageNet in 3.75ms, which is about 266.6 frames per second. This achieves 20x speedup if compared to CPU.

I. INTRODUCTION

The paper targets the computing and memory demands that make CNN deployment difficult on portable devices by proposing a scalable FPGA accelerator. It combines depthwise separable convolution with hardware and memory designs intended to balance performance and FPGA resources.

  • Motivation: CNNs consume substantial computing power during training and deployment, while GPUs’ high power consumption limits embedded use.FPGA-based accelerators offer a tradeoff between power consumption and reconfigurability.
  • Motivation: Depthwise separable convolution reduces the operations and parameters required by standard convolution, supporting its use in MobileNetV1 and MobileNetV2.The paper describes this as a way to lighten CNN computing burden while achieving comparable results.
  • Contributions: The accelerator processes all CNN layers in a matrix multiplication engine and uses hierarchical memory with ping-pong buffering to reduce off-chip bandwidth limitations.The framework stores parameters and intermediate data through coordinated on-chip and off-chip memory structures.
  • Contributions: The proposed scalable-design methodology fits the framework to different FPGAs by balancing on-chip resources and processing performance.The number of processing resources and available on-chip memory can be varied for different FPGA sizes.
  • Results: 266.6 frames per second and 170.6 GOPS were achieved for MobileNetV2 on an Arria 10 SoC FPGA at 133MHz.The reported result represents a 20x speedup compared with CPU execution.

II. DEPTHWISE SEPARABLE CONVOLUTION

Depthwise separable convolution factorizes standard convolution into depthwise and pointwise stages, reducing mathematical operations and parameters. MobileNetV2 applies this structure with bottleneck operations and further reduces weights in selected layers.

  • Depthwise separable convolution: Depthwise separable convolution factorizes standard convolution into a depthwise convolution followed by a 1×1 pointwise convolution.The depthwise stage convolves each input channel individually, while the pointwise stage combines channels.
  • Depthwise separable convolution: Depthwise separable convolution considerably reduces both mathematical operations and parameters compared with standard convolution.For stride 1, the paper gives WDSC = K × K × N + N × P and ODSC = M × M × K × K × N + M × M × N × P.
  • MobileNetV2: MobileNetV2 further decreases weights by shrinking output channels in some layers and improves performance by adding a pointwise convolution before depthwise separable convolution.This added operation is called a bottleneck.
  • MobileNetV2: Table I describes MobileNetV2 as sequences of one or more identical layers, except for stride, with all depthwise convolutions using 3×3 kernels.The table provides the network structure referenced by the paper.

A. Architecture Overview

The accelerator uses an MME array to perform CNN operations, with on-chip buffering and off-chip memory transfers organized to reduce memory-access latency. A finite-state machine controls the accelerator.

  • A. Architecture Overview: The MME array performs convolution, normalization, ReLU, and pooling for the CNN.A feature map buffer stores intermediate results, while biases are loaded into MME registers.
  • A. Architecture Overview: A ping-pong weight buffer is placed between the MME array and off-chip memory to maximize bandwidth.
  • A. Architecture Overview: The accelerator stores parameters and input images in off-chip memory while buffering intermediate feature maps on chip.

B. Matrix Multiplication Engine

Each MME combines buffering, multiplication, accumulation, normalization, activation, and pooling components. Its line-buffer length is configurable for different input sizes.

  • B. Matrix Multiplication Engine: Each MME contains 32 line-buffer slices, a 32-slice 3 × 3 multiplier array, an adder tree, normalization, ReLU, and pooling blocks.
  • B. Matrix Multiplication Engine: During convolution, the MME loads feature maps and weights into line buffers, multiplies them, and sums products according to convolution type.
  • B. Matrix Multiplication Engine: The line-buffer working length is selected by the control FSM to fit different input sizes.The implementation length is (K −1) × M + K.

2) Adder Tree:

The configurable adder tree supports both depthwise and pointwise convolution, while the first layer can use standard convolution for three-channel inputs.

  • 2) Adder Tree:: The adder tree can perform summation for either depthwise or pointwise convolution.
  • 2) Adder Tree:: Black blocks are shared, blue blocks support depthwise convolution, and red blocks support pointwise convolution.
  • 2) Adder Tree:: The accelerator supports standard convolution in the first layer when the input feature-map channel number is 3.
  • 2) Adder Tree:: Depthwise convolution sums products from each multiplier-array slice in parallel, producing 32 output channels per MME.

5) Pointwise Convolution:

Pointwise convolution is implemented as 1 × 1 standard convolution using tiled feature-map submatrices and line buffers. Supporting operations include batch-normalization arithmetic and pooling stages.

  • 5) Pointwise Convolution:: Pointwise convolution uses a kernel size of 1 × 1 and divides the input feature map into M × M × 32 submatrices.
  • 5) Pointwise Convolution:: The submatrices are shifted into line buffers sequentially so the MME multipliers can be fully utilized.
  • 5) Pointwise Convolution:: The tiling approach follows divide and conquer: small matrix products are computed separately and their results are summed.
  • 5) Pointwise Convolution:: After training, batch normalization becomes multiplication and addition, while average and max pooling use multiplication-accumulation and comparison stages, respectively.

8) ReLU:

The accelerator uses selectable ReLU processing and hierarchical buffering to balance limited on-chip memory against external-memory bandwidth. Its design also considers instantiated compute resources and feature-map size.

  • 8) ReLU:: Three activation options—no ReLU, standard ReLU, and ReLU6—are selectable after normalization.
  • 8) ReLU:: Hierarchical memory balances FPGA on-chip memory’s high bandwidth against external memory’s larger capacity and limited bandwidth.
  • 8) ReLU:: The design varies the number of matrix multiplication engines and feature-map buffer size to balance resources and processing requirements.

IV. RESULTS

MobileNetV2 was implemented on an Arria 10 SoC FPGA using a resource-balanced accelerator configuration and a memory architecture supporting automated DDR4 transfers. The implementation operated at 133MHz, with results reported in Table II.

  • IV. RESULTS: MobileNetV2 was implemented on an Arria 10 SoC Development Kit containing 251680 ALMs, 2131 M20K blocks, and 1687 DSP blocks.
  • IV. RESULTS: 16-bit quantization was selected because previous works widely used this strategy for CNNs.
  • IV. RESULTS: A 4-MME array, 36Kb ping-pong weight buffer, and 24.5Mb intermediate feature-map buffer were selected after balancing resource usage and processing time.
  • IV. RESULTS: 133MHz was achieved on the FPGA accelerator, with the adder tree limiting the operating frequency.
  • IV. RESULTS: The external memory interface and mSG-DMA bridge accelerator buffers with DDR4 memory, whose maximum bandwidth is 8.5GB/s.

V. CONCLUSION

The paper presents a high-performance, scalable accelerator optimized for depthwise separable convolution. By varying matrix multiplication engines and on-chip memory, it targets FPGAs of different sizes and achieves 266.6 fps and 170.6 GOPS on Arria 10 with MobileNetV2.

  • V. CONCLUSION: The accelerator is optimized for depthwise separable convolution, which requires fewer operations and parameters.
  • V. CONCLUSION: Different numbers of matrix multiplication engines and variable on-chip memories allow the accelerator to fit large or small FPGAs.
  • V. CONCLUSION: 266.6 fps and 170.6 GOPS were achieved when MobileNetV2 was implemented on an Arria 10 SoC FPGA.
Loading 1809.01536v2…