PaperlayerView exact revision on arXiv

Source-linked AI summary

An OpenCL(TM) Deep Learning Accelerator on Arria 10

Utku Aydonat, Shane O'Connell, Davor Capalija, Andrew C. Ling, Gordon R. Chiu

arXiv:1701.03534v1cs.DCcs.ARcs.CV

TL;DR

Prior FPGA CNN implementations were limited by external memory bandwidth and had lagged GPU performance. This paper introduces an OpenCL Deep Learning Accelerator that caches feature maps on-chip and uses Winograd transforms, achieving 10x faster AlexNet performance than the state-of-the-art FPGA implementation while reaching 23 img/s/W, competitive with TitanX results.

  • Problem

    FPGA CNN implementations were limited by external memory bandwidth, incomplete layer support, and performance below leading GPU results.

  • Method

    The OpenCL Deep Learning Accelerator caches intermediate feature maps on-chip, explores device-specific configurations analytically, and uses Winograd transforms to reduce convolution arithmetic.

  • Results

    10x faster than the state-of-the-art FPGA implementation of AlexNet, with 23 img/s/W competitive against publicly known TitanX GPU results.

  • Takeaways & Limitations

    The DLA demonstrates that reduced memory bandwidth and Winograd-based computation can support competitive FPGA CNN throughput and energy efficiency for AlexNet.

  • Takeaways & Limitations

    TitanX comparisons may be overly optimistic because the referenced results remove communication overhead and use random data instead of the ILSVRC database.

Abstract

from arXiv · show

Convolutional neural nets (CNNs) have become a practical means to perform vision tasks, particularly in the area of image classification. FPGAs are well known to be able to perform convolutions efficiently, however, most recent efforts to run CNNs on FPGAs have shown limited advantages over other devices such as GPUs. Previous approaches on FPGAs have often been memory bound due to the limited external memory bandwidth on the FPGA device. We show a novel architecture written in OpenCL(TM), which we refer to as a Deep Learning Accelerator (DLA), that maximizes data reuse and minimizes external memory bandwidth. Furthermore, we show how we can use the Winograd transform to significantly boost the performance of the FPGA. As a result, when running our DLA on Intel's Arria 10 device we can achieve a performance of 1020 img/s, or 23 img/s/W when running the AlexNet CNN benchmark. This comes to 1382 GFLOPs and is 10x faster with 8.4x more GFLOPS and 5.8x better efficiency than the state-of-the-art on FPGAs. Additionally, 23 img/s/W is competitive against the best publicly known implementation of AlexNet on nVidia's TitanX GPU.

1. INTRODUCTION

The paper targets FPGA CNN accelerators that are constrained by external memory bandwidth and underuse device operations. It introduces an OpenCL DLA combining on-chip caching, analytical design-space exploration, and Winograd transformations.

  • FPGA CNN implementations often underperform GPUs because limited external memory bandwidth constrains data movement and prior designs underuse available operations.
  • The DLA caches intermediate feature maps in on-chip stream buffers and batches fully connected layers, reducing external bandwidth requirements by an order of magnitude.
  • The design-space methodology uses analytical resource and throughput models to select the highest-throughput architecture configuration for a specific FPGA and CNN.
  • Winograd transformations reduce the multiply-accumulate operations required by convolution layers.

2. BACKGROUND

The background introduces CNN layer structure, FPGA execution challenges, related accelerator approaches, and Intel’s OpenCL programming model. It positions the DLA as an FPGA architecture designed to reduce memory traffic and improve compute utilization.

  • CNN Background: CNNs are directed graphs of computational layers that consume and produce multidimensional feature maps.
  • AlexNet: AlexNet contains five convolution layers, three fully connected layers, normalization and pooling layers, and a final 1000-class softmax output.
  • Related Work: Earlier FPGA CNN designs implemented all layers but could remain severely external-memory-bound or achieve low device utilization.
  • Related Work: Caffeine improved FPGA CNN performance by 3x and energy efficiency over K40, but remained 5.8x worse in power efficiency than TitanX.
  • Paper Positioning: The proposed approach caches feature maps on-chip, uses Winograd transforms, and targets compute-bound execution with efficient DSP utilization.
  • OpenCL FPGA Platform: Intel’s FPGA SDK for OpenCL provides a vendor-agnostic programming model in which host code controls transfers and kernel execution.
  • OpenCL FPGA Platform: The SDK supplies a pre-generated FPGA platform with reserved, pre-placed components, reducing the hardware-design burden for OpenCL programmers.

3. DLA ARCHITECTURE

The DLA implements AlexNet on an FPGA using OpenCL and is optimized for throughput, flexibility, parallel convolution, and on-chip data reuse.

  • The DLA implements all AlexNet layers on an FPGA using the Intel FPGA SDK for OpenCL.
  • Convolution hardware is optimized for throughput because convolutions account for 92% of AlexNet’s total floating-point operations.
  • The architecture supports other CNN topologies by changing vectorization factors rather than vectorizing different convolution loops.
  • Parallelism is extracted across output columns, output feature maps, input feature maps, and input columns using Qvec, Kvec, Cvec, and Wvec.
  • PEs receive shared input-feature sticks and produce output-feature vectors for tiled, time-multiplexed convolution execution.
  • On-chip double buffers and PE filter caches reuse feature maps and weights, reducing unnecessary external memory accesses.

3.3 Arithmetic Optimizations

The accelerator uses Winograd transformations to reduce convolution arithmetic while preserving parallel output generation through on-chip data transformations.

  • Winograd transformations reduce the arithmetic complexity of stride-1 convolutions, enabling faster execution.
  • The four standard dot-products use three filter values and three input values per output pixel.
  • Four output pixels are computed with six multiplications and additions instead of the 12 required by standard convolution.
  • The DLA transforms three filter coefficients and six feature inputs into six Winograd filters and six transformed inputs, then transforms six outputs back into four features.

3.4 PEs

Processing elements combine vectorized dot-product units, accumulators, caches, and buffering to compute transformed convolution results continuously.

  • Each PE contains dot-product units, accumulators, and caches for convolution processing.
  • Each PE contains Wvec dot-product units, and each cycle convolves a 1 × Wvec × Cvec input sub-region.
  • Accumulators use shift-registers whose locations hold partial sums for specific output features.
  • PE filter caches use Wvec×Cvec memory banks so one transformed filter weight can be read from each cache every cycle.
  • Double-buffering overlaps convolution execution with PE cache updates and prefetches weights for the next layer.
  • Each PE sends Wvec outputs per cycle to the ReLU unit for the Winograd output transform.

3.5 Stream Buffers

Stream buffers store feature data in on-chip RAM, feed PEs during convolution, and accept outputs while processing continues.

  • Stream buffers use on-chip RAM to store feature data and stream it to the PEs.
  • Double-buffered stream buffers overlap input streaming with storage of convolution outputs during layer execution.
  • The architecture uses Wvec×Cvec stream buffers, with feature-map width divided across Wvec buffers.
  • A crossbar converts each cycle’s 1×Qvec×Kvec output region into the 1 × Wvec × Cvec buffer layout.
  • Shared-exponent FP16 multiplication reduces PE resource overhead by enabling fixed-point multiplication and exploiting fractured Arria 10 DSP multipliers.

3.7 Fully Connected Layers

The DLA reconfigures its processing elements for fully-connected layers, where filter-weight reuse is lower than in convolutional layers. It batches images and streams weights while caching features to reduce bandwidth demands.

  • 3.7 Fully Connected Layers: Fully-connected layers reuse filter weights less than convolutional layers during single-image classification, motivating a different PE configuration.The architecture therefore processes fully-connected layers in image batches.
  • 3.7 Fully Connected Layers: Image batches are formed after convolution, then processed together during fully-connected layers to alleviate limited filter-weight reuse.Features are written to external memory before batching and later pre-loaded into PE caches.
  • 3.7 Fully Connected Layers: Fully-connected execution uses pre-loaded feature caches, streamed filter weights, and dot-product units assigned to individual images.Each cycle streams F unique filter weights to all PEs, which multiply them with different cached image features.
  • 3.7 Fully Connected Layers: No Winograd transformation is applied because fully-connected computations produce only a single output.The same processing elements support both convolutional and fully-connected layers, keeping dot-product units busy across both layer types.
  • 3.7 Fully Connected Layers: Wvec/N partial sums in each PE are summed to produce N outputs from each PE.This summarizes the final reduction step in the fully-connected PE configuration.

3.8 Overall Architecture

The overall DLA supports every AlexNet layer through concurrently executing OpenCL kernels connected by FIFO channels. Stream buffers, processing elements, normalization, pooling, ReLU, and sequencing coordinate data movement and topology-specific execution.

  • 3.8 Overall Architecture: The DLA adds hardware for normalization, max-pooling, and ReLU so the complete AlexNet topology can execute on the FPGA.Earlier processing elements handle convolutional and fully-connected layers.
  • 3.8 Overall Architecture: StreamBuffer applies Winograd transformations, manages feature-map streams, fetches filters from DDR, and sends transformed features to the processing elements.Features and outputs move through daisy-chained PE connections.
  • 3.8 Overall Architecture: Normalization outputs feed pooling, whose results return to stream buffers for later convolutions or go to external memory after the final convolution.Pooling operates independently on each feature map, avoiding buffering between feature-map tiles.
  • 3.8 Overall Architecture: A sequencer generates addresses, control signals, and bypass decisions from the CNN topology and layer dimensions.Normalization and pooling can be bypassed when the executed topology does not require them.
  • 3.8 Overall Architecture: All units are independent, concurrent OpenCL kernels connected by FIFOs implemented with Intel’s channel API.This establishes the architecture’s streaming execution model.

4. DESIGN SPACE EXPLORATION AND ANALYTICAL MODELS

The paper analytically models DLA resource usage and throughput to select device-specific configurations. The models account for vectorization, Winograd transforms, memory traffic, batching, and layer execution cycles.

  • 4. DESIGN SPACE EXPLORATION AND ANALYTICAL MODELS: Analytical resource models use Cvec, Kvec, Wvec, and Qvec to estimate processing-element, stream-buffer, and filter-cache usage.These estimates support architecture selection for a specific FPGA device and CNN.
  • 4. DESIGN SPACE EXPLORATION AND ANALYTICAL MODELS: M20K models estimate memory required for the largest input and output feature maps and for filter caches using 16-bit floating-point storage assumptions.The model assumes each M20K stores 1024 such values in a 2-word-wide by 512-deep memory.
  • 4. DESIGN SPACE EXPLORATION AND ANALYTICAL MODELS: Throughput modeling combines vector dimensions, feature-map and filter sizes, DSP efficiency, cycle counts, and DDR bandwidth utilization.DSP efficiency captures quantization from vectorization and spatial interleaving.
  • 4. DESIGN SPACE EXPLORATION AND ANALYTICAL MODELS: Fully-connected cycle models operate on image batches, count streamed filter-weight bytes, and omit quantization effects because measured DSP efficiency is close to 100%.The batch size appears explicitly in the fully-connected throughput calculation.
  • 4. DESIGN SPACE EXPLORATION AND ANALYTICAL MODELS: The final throughput divides clock frequency by total layer cycles, normalizing fully-connected execution by Sbatch while ignoring concurrent negligible overhead from normalization and ReLU.Searching Cvec and Kvec with these models identifies the optimal configuration when other parameters are fixed.

5. EXPERIMENTAL EVALUATION

The evaluation implements AlexNet on an Arria 10 development kit and compares the DLA with prior FPGA and TitanX GPU results. The setup uses separate batching choices for convolutional and fully-connected layers and a single DDR4 bank.

  • 5. EXPERIMENTAL EVALUATION: The evaluation maps AlexNet onto an Intel Arria 10 A10-1150 device and compares results with prior FPGA implementations and TitanX GPU results.The A10-1150 is a 20nm device.
  • 5. EXPERIMENTAL EVALUATION: Convolution layers use batch size 1, while fully-connected layers use batch size 96 as specified by the architecture.The evaluation uses one DDR4x64 bank at 1200MHz with 17GB/s bandwidth to reduce FPGA power.
  • 5. EXPERIMENTAL EVALUATION: Table 2 reports convolutional and fully-connected GFLOPS together with DSP efficiency, distinguishing effective GFLOPS from actual GFLOPS under Winograd.The table uses an 8 × 48 PE configuration.
  • 5. EXPERIMENTAL EVALUATION: Host-to-FPGA DDR transfers are pipelined with DLA execution, while compared studies use differing fixed- and floating-point precisions.Prior work reports limited impact from 16-bit fixed point relative to 16-bit floating point, but the studies are not precision-identical.

6. RESULTS

The DLA achieves high DSP utilization and closely matches its analytical throughput model. Comparisons show substantially higher FPGA throughput and competitive energy efficiency against TitanX, though some baselines use optimistic assumptions.

  • 6. RESULTS: Cvec = 8 and Kvec = 48 is identified as a peak-throughput configuration for the Arria 10 1150 under the stated exploration assumptions.The analysis assumes fmax = 300MHz, Qvec = 4, Wvec = 6, and even-multiple Kvec values.
  • 6. RESULTS: 16% scaling for transfer and host-device overhead makes the analytical throughput predictions closely match measured performance.The 16% adjustment equals the measured average difference between system-level and FPGA-device throughput.
  • 6.2 FPGA Comparisons to the state-of-the-art: 8.4x more GFLOPS than the latest Ultrascale result and 19x more than the latest Stratix V result are reported for AlexNet.The Ultrascale comparison uses batch size 32 for fully-connected layers.
  • 6.2 FPGA Comparisons to the state-of-the-art: 23 img/s/W is reported as 5.8x better than the prior FPGA result and competitive with TitanX and M4 after power normalization.TitanX exceeds the DLA in raw performance, while the DLA is competitive on normalized energy efficiency.
  • 6.2.1 Discussion on performance comparisons: The reported comparisons may overstate baseline performance because some references omit transfer overhead or non-linear-layer execution time.The authors therefore suspect the DLA’s relative system-level benefit is larger than Table 6 indicates.

7. CONCLUSIONS

The paper concludes that its OpenCL DLA reduces memory-bandwidth requirements and improves FPGA CNN performance through on-chip buffering, vectorization, and Winograd transforms. It reports 10x higher AlexNet performance than prior FPGA work and 23 img/s/W competitive with TitanX, while identifying broader CNN mapping and runtime reconfigurability as future work.

  • 7. CONCLUSIONS: An on-chip stream buffer reduces required memory bandwidth by an order of magnitude by storing input and output feature maps.The architecture targets CNN computation on FPGAs and keeps feature maps on-chip.
  • 7. CONCLUSIONS: Vectorization achieves over 60% DSP efficiency, while Winograd transforms reduce the DSPs required for convolution layers.These mechanisms contribute to the reported system-level performance improvements.
  • 7. CONCLUSIONS: 10x faster AlexNet performance than state-of-the-art FPGA work and 23 img/s/W competitive with TitanX are reported.The conclusion frames the result at overall system level and in energy efficiency.
  • 7. CONCLUSIONS: Future work includes mapping GoogLeNet and VGG and exploring how runtime reconfigurability affects DLA performance.These directions extend the architecture beyond the demonstrated setting.
Loading 1701.03534v1…