Source-linked AI summary
DLAU: A Scalable Deep Learning Accelerator Unit on FPGA
Chao Wang, Qi Yu, Lei Gong, Xi Li, Yuan Xie, Xuehai Zhou
TL;DR
Large-scale deep learning networks create challenges for high-performance, low-power hardware implementation. DLAU addresses this with a scalable FPGA accelerator using tiled computation and three pipelined processing units, achieving up to 36.1x speedup with low power utilization.
Problem
Large-scale deep learning networks challenge hardware implementations that must provide high performance while limiting power consumption and accommodating constrained accelerator resources.
Method
DLAU uses configurable tile techniques, FIFO buffers, and three pipelined processing units to reduce memory transfers, reuse computing resources, and support varied network topologies.
Results
36.1x speedup was achieved at 256×256 network size on an FPGA prototype, with reasonable hardware cost and low power utilization.
Takeaways & Limitations
DLAU provides a configurable and scalable FPGA architecture that trades tile size against hardware cost to support large neural networks.
Abstract
from arXiv · showhide
As the emerging field of machine learning, deep learning shows excellent ability in solving complex learning problems. However, the size of the networks becomes increasingly large scale due to the demands of the practical applications, which poses significant challenge to construct a high performance implementations of deep learning neural networks. In order to improve the performance as well to maintain the low power cost, in this paper we design DLAU, which is a scalable accelerator architecture for large-scale deep learning networks using FPGA as the hardware prototype. The DLAU accelerator employs three pipelined processing units to improve the throughput and utilizes tile techniques to explore locality for deep learning applications. Experimental results on the state-of-the-art Xilinx FPGA board demonstrate that the DLAU accelerator is able to achieve up to 36.1x speedup comparing to the Intel Core2 processors, with the power consumption at 234mW.
I. INTRODUCTION
DLAU addresses the challenge of implementing increasingly large deep-learning networks with high performance and low power on limited hardware resources. It uses scalable tiling and pipelined FPGA modules to balance speedup, hardware cost, and application flexibility.
- Motivation: Large deep-learning networks increase implementation difficulty because they require high performance and low power despite limited accelerator resources.FPGA and ASIC accelerators have limited computing resources, memory, and I/O bandwidths.
- Approach: DLAU is a scalable accelerator unit designed to speed up kernel computations in deep-learning algorithms.
- Approach: Tile techniques partition large input data and support configurable tile sizes that trade speedup against hardware cost.This configuration makes the FPGA accelerator more adaptable to different machine-learning applications.
- Architecture: Three fully pipelined processing units—TMMU, PSAU, and AFAU—can compose CNN, DNN, and emerging neural-network topologies.
II. TILE TECHNIQUES AND HOT SPOT PROFILING
The paper targets matrix multiplication as a major deep-learning computation and uses tiled data processing to fit large networks onto limited FPGA resources. Tiling partitions inputs and weights, accumulates partial sums, and enables accelerator reuse.
- Hot-spot profiling: Deep neural networks contain iterative computations with few conditional branches, making them suitable for parallel hardware optimization.
- Hot-spot profiling: Matrix multiplication is identified as a significant contributor to execution time for feedforward, RBM, and back-propagation operations.
- Tile techniques: Tile techniques partition massive input data into subsets that hardware accelerators buffer and process locally.The architecture is reused to support large-scale networks despite FPGA resource constraints.
- Tile techniques: Input and weight matrices are divided into corresponding tiles, whose partial products are accumulated to produce the result.The accelerator's hardware cost depends on tile size, enabling large networks to be implemented with fewer resources.
III. DLAU ARCHITECTURE AND EXECUTION MODEL
DLAU is a standalone, configurable FPGA accelerator that streams tiled data through three pipelined processing units. FIFO buffers and on-chip tiling support adaptable execution across neural-network sizes.
- System architecture: DLAU integrates an embedded processor, DDR3 memory controller, DMA module, and accelerator as a configurable standalone unit.The processor transfers inputs and weights to internal BRAM, activates DLAU, and returns results.
- Pipeline execution: Three pipeline stages—TMMU, PSAU, and AFAU—compute tiled data in sequence before writing results back to memory.
- Pipeline execution: FIFO input and output buffers absorb inconsistent throughput between processing units and help prevent data loss.
- Scalability: Tiling divides large data volumes into on-chip-cacheable tiles, allowing the accelerator to accommodate different neural-network sizes.
- Pipeline execution: A stream-like data path lets TMMU compute matrix products, PSAU accumulate partial sums, and AFAU apply activation functions by piecewise linear interpolation.
A. TMMU architecture
TMMU exploits tiled weight and node data using BRAMs, alternating registers, and a pipelined adder tree. Its design produces partial sums continuously for downstream accumulation.
- TMMU function: TMMU performs multiplication and accumulation operations while exploiting weight locality to calculate partial sums.Input and output FIFO buffers connect TMMU with DMA and PSAU.
- Data buffering: With tile size=32, TMMU distributes weight rows across BRAMs and alternates registers for computation and node-data loading.
- Pipelined computation: A pipelined binary adder tree and time-sharing of coarse-grained accelerators enable TMMU to produce one partial-sum result every clock cycle.
B. PSAU architecture
PSAU accumulates partial sums from TMMU and forwards completed results to the output buffer and AFAU while sustaining one accumulation per clock cycle.
- PSAU accumulates the partial sums produced by TMMU.
- Final partial sums are written to the output buffer and sent to AFAU in a pipeline manner.
- One-partial-sum-per-cycle accumulation matches TMMU’s partial-sum generation throughput.
C. AFAU architecture
AFAU implements sigmoid activation through piecewise linear interpolation, using buffered and pipelined computation to preserve throughput across DLAU’s processing units.
- AFAU implements activation functions with piecewise linear interpolation, y=ai*x+bi, over intervals [xi,xi+1).
- The sigmoid function uses four segments, with saturated outputs for x>8 and x≤-8 and configured functions between those bounds.
- Separate input and output buffers plus pipelined computation let AFAU operate one sigmoid value per clock cycle.
IV. EXPERIMENTS AND DATA ANALYSIS
The evaluation prototypes DLAU on a Xilinx Zynq Zedboard and tests tiled deep neural networks against an Intel Core2 baseline across three network sizes.
- DLAU is implemented on a Xilinx Zynq Zedboard with 667MHz ARM Cortex-A9 processors and programmable fabrics.
- Mnist benchmarks train 784×M×N×10 deep neural networks in Matlab, supplying DLAU with M×N weights and node values.
- The experiments use tile size 32, a 200MHz DLAU clock, and network sizes 64×64, 128×128, and 256×256.
A. Speedup Analysis
DLAU’s speedup increases with network size and tile size, while its configurable tiling supports performance–hardware-cost trade-offs.
- 36.1x speedup is achieved at the 256×256 network size compared with the Intel Core2 baseline.
- Speedup rises from 19.2x at 64×64 to 36.1x at 256×256 network size.
- At 128×128, increasing tile size from 8 to 32 raises speedup from 9.2x to 30.5x.
- DLAU’s configurable tile sizes allow speedup to be balanced against hardware cost.
B. Resource utilization and Power
DLAU uses FPGA resources modestly through tiled computation, while its reported power consumption is lower than DianNao. The floorplan shows TMMU as the dominant logic consumer.
- 167 DSP blocks are utilized for floating-point addition and multiplication operations.
- DLAU consumes 35/257=13.6% of the BRAMs used by Ly&Chow’s work while occupying similar FFs and LUTs.
- Tile techniques divide large-scale neural networks into small tiles, improving the architecture’s scalability and flexibility while reducing BRAM utilization.
- 234mW is the reported total power of DLAU, compared with 485mW for DianNao.
- TMMU occupies most of the FPGA programming-logic area because it uses substantial LUTs and FFs.
V. CONCLUSION AND FUTURE WORK
The paper presents DLAU as a scalable FPGA accelerator for large-scale neural networks and reports 36.1x speedup with reasonable hardware cost and low power utilization. Future work includes optimizing the weight matrix and memory access and analyzing FPGA–GPU trade-offs.
- DLAU is a scalable and flexible FPGA accelerator with three pipelined processing units reused for large-scale neural networks.
- 36.1x speedup is reported for DLAU with reasonable hardware cost and low power utilization.
- Future work includes optimizing the weight matrix and memory access and analyzing trade-offs between FPGA and GPU accelerators.