PaperlayerView exact revision on arXiv

Source-linked AI summary

Recurrent Neural Networks Hardware Implementation on FPGA

Andre Xian Ming Chang, Berin Martini, Eugenio Culurciello

arXiv:1511.05552v4cs.NE

TL;DR

RNNs are challenging to parallelize on conventional hardware, limiting their fit for embedded systems. This paper implements an LSTM on a Zynq 7020 FPGA and evaluates it with a character-level model. The hardware generates Shakespeare-like text and is reported as significantly faster than other mobile platforms, while further work is needed to parallelize gate computation.

  • Problem

    RNN recurrence makes full parallelization difficult on conventional hardware, while CPUs offer limited parallelism and small models may not benefit fully from GPUs.

    Source: §I · p. 1

  • Method

    The paper implements an LSTM hardware module on a Zynq 7020 FPGA and tests it with a character-level model of 2 layers and 128 hidden units.

    Source: §I · p. 1

  • Results

    The hardware was significantly faster than other mobile platforms, generated Shakespeare-like text, and had average percentage errors of 3.9% for ct and 2.8% for ht versus Torch7 output.

    Source: A. Accuracy · p. 5, §VII · p. 6

  • Takeaways & Limitations

    The implementation could potentially evolve into an RNN coprocessor for future devices, with parallel gate computation identified as future work.

    Source: §VII · p. 6

    Further work is needed to optimize the design for parallel gate computation, including a parallel MAC unit configuration for matrix-vector multiplication.

    Source: §VII · p. 6

Abstract

from arXiv · show

Recurrent Neural Networks (RNNs) have the ability to retain memory and learn data sequences. Due to the recurrent nature of RNNs, it is sometimes hard to parallelize all its computations on conventional hardware. CPUs do not currently offer large parallelism, while GPUs offer limited parallelism due to sequential components of RNN models. In this paper we present a hardware implementation of Long-Short Term Memory (LSTM) recurrent network on the programmable logic Zynq 7020 FPGA from Xilinx. We implemented a RNN with $2$ layers and $128$ hidden units in hardware and it has been tested using a character level language model. The implementation is more than $21\times$ faster than the ARM CPU embedded on the Zynq 7020 FPGA. This work can potentially evolve to a RNN co-processor for future mobile devices.

I. INTRODUCTION

RNNs are difficult to parallelize on conventional hardware, motivating optimized embedded architectures. The paper presents an FPGA LSTM implementation and reviews the memory-control mechanisms and vanilla architecture it uses.

I. INTRODUCTION

  • RNN recurrence complicates parallel execution, while CPUs offer limited parallelism and small models may not benefit fully from GPUs.The paper motivates optimized RNN hardware for embedded systems.Source: §I · p. 1
  • The work implements an LSTM module on a Zynq 7020 FPGA and tests a character-level language model with 2 layers and 128 hidden units.LSTM is described as a specific RNN architecture with a learned memory controller for avoiding vanishing or exploding gradients.Source: §I · p. 1

II. LSTM BACKGROUND

  • LSTM adds memory controllers that decide when to remember, forget, and output, supporting learning of long-term dependencies where standard RNNs fail.The passage says this makes training more stable.Source: §II · p. 1
  • LSTM variations include peephole gates influenced by cell memory and merged input-forget gates; the cited work reports similar performance across variations.Other variations are also noted in the passage.Source: §II · pp. 1–2
  • The implementation uses vanilla LSTM without peepholes, with gates formed from matrix-vector operations on the input and previous output followed by a nonlinear function.The input, forget, and output gates govern remembering, forgetting, and responding; deep networks cascade modules across layers.Source: §II · p. 2

III. RELATED WORK

Prior FPGA work includes accelerators for CNNs and vision, as well as RNN approaches with different computational arrangements. This work focuses on LSTM and uses a single module to consume the input and previous output simultaneously.

III. RELATED WORK

  • Prior FPGA vision and CNN accelerators include neuFlow, with up to 100× end-to-end speedups, and nn-X, with up to 240 G-op/s.The passage also describes a real-time megapixel-image recognition system.Source: §III · p. 2
  • A prior FPGA LSTM study focused on making the learning phase hardware-friendly using Simultaneous Perturbation Stochastic Approximation.Source: §III · p. 2
  • Unlike, this FPGA LSTM approach uses a single module that consumes the input and previous output simultaneously.The cited standard-RNN design unfolds a fixed number of timesteps, serially processes hidden states, and computes outputs in parallel.Source: §III · p. 2
  • Figure 3 shows the main hardware module that implements the LSTM gates, with a nonlinear module configurable as either tanh or logistic sigmoid.Source: Figure 3 · p. 3

A. Hardware

The hardware implements LSTM gate calculations with fixed-point matrix-vector operations and segmented nonlinear functions. Its staged module streams gate results through an element-wise stage to produce the cell and hidden states.

A. Hardware

  • Matrix-vector multiplication uses Q8.8 fixed point and a MAC unit; the bias is incorporated through an extra weight-matrix column and unity input element.The adder output feeds a linearly mapped element-wise nonlinear function.Source: A. Hardware · p. 3
  • The nonlinear function uses 13 pipelined line segments, each combining a MAC unit with a comparator to apply a configured linear mapping over an input range.The coefficients and ranges are stored in configuration registers.Source: A. Hardware · p. 3
  • Independent DMA streams require synchronization buffering so vector and matrix-row elements arrive aligned at the MAC units.The buffer waits until all ports are streaming before outputting synchronized data.Source: A. Hardware · p. 3
  • The LSTM computation runs in three sequential stages, with parallel gate modules generating vectors for a FIFO before the element-wise stage outputs c_t and h_t.An extra matrix-vector multiplication generates the final output after the last LSTM layer.Source: A. Hardware · p. 4
  • The Zedboard FPGA module runs at 142 MHz with 1.942 W on-chip power; Table I reports its Zynq ZC7020 resource utilization.MAC units multiply 16-bit values and accumulate at 32-bit precision.Source: A. Hardware · pp. 3–4, Table I · p. 4

B. Driving Software

The C control software loads weights and input vectors into memory and configures the hardware. It also manages recurrent state and the transfer of outputs between LSTM layers.

B. Driving Software

  • C control software populates main memory with weights and input vectors, then controls the hardware through configuration registers.Different layers use different weight locations set through the registers.Source: B. Driving Software · p. 4
  • The weight matrix stores bias as an extra final element in each row, while the input vector adds a unity element for bias addition.Zero padding matches matrix-row and vector sizes to ease stream synchronization.Source: B. Driving Software · p. 4
  • Cell and hidden-state outputs are overwritten in the memory locations used for the prior timestep’s states; multi-layer operation copies one layer’s hidden output to the next layer’s input location.That inter-layer copy was removed during profiling, and the software preserved hidden outputs between layers for error measurements.Source: B. Driving Software · p. 4

V. EXPERIMENTS

The experiments evaluate a Torch7 character-level language model on Shakespeare text using a two-layer LSTM. Runtime profiling compares the FPGA implementation with embedded CPU and GPU platforms.

V. EXPERIMENTS

  • The test model is a two-layer LSTM with 128 hidden units, trained on a subset of Shakespeare’s work to predict characters from prior characters.It uses batch size 50, sequence length 50, and learning rate 0.002; the predicted character is fed back as the next input.Source: §V · p. 4
  • The model uses 65-element one-hot character input and output vectors, and generates a text character by character from the preceding character.The Torch7 training script’s parameters were transferred to the control software.Source: §V · p. 4
  • Runtime profiling includes the Zedboard’s 667 MHz dual ARM Cortex-A9 CPU and 142 MHz FPGA, plus the Tegra K1 and Odroid XU4 platforms.The Tegra K1 includes a 192-core Kepler GPU; the Odroid XU4 has Cortex-A15 and Cortex-A7 cores.Source: §V · p. 4

A. Accuracy

The accuracy evaluation compares fixed-point FPGA outputs with Torch7 outputs and examines generated text. It reports average state errors and finds that the two implementations produce different text with similar behavior.

A. Accuracy

  • Even the small model used almost 530 KB of weights, motivating weight compression as a throughput-versus-accuracy trade-off.The Q8.8 fixed-point format introduces rounding errors.Source: A. Accuracy · pp. 4–5
  • 3.9% average c_t error and 2.8% average h_t error were measured against Torch7; across time steps, the best error was 1.3% and the worst was 7.1%.The passage reports that recurrent operation did not accumulate the errors and that they stabilized at a low average percentage.Source: A. Accuracy · p. 5
  • The FPGA and CPU each generated 1,000 characters of Shakespeare-like dialogue; their texts differed, but their behavior was the same.Figure 6 places the FPGA output on the left and CPU output on the right.Source: A. Accuracy · p. 5, Figure 6 · p. 5

B. Memory Bandwidth

The Zedboard platform provides four AXI DMA ports for FPGA–DDR3 transfers, while an LSTM module’s peak bandwidth demand constrains parallel replication.

B. Memory Bandwidth

  • Four 142 MHz AXI DMA ports, each sending 32-bit packages, provide up to 3.8 GB/s full-duplex bandwidth between the FPGA and external DDR3.Source: B. Memory Bandwidth · p. 5
  • At 142 MHz, one LSTM module computes 388.8 M-ops/s and requires 1.236 GB/s of memory bandwidth at peak usage.Source: B. Memory Bandwidth · p. 5
  • Memory bandwidth limits how many LSTM modules can run in parallel; replication requires more bandwidth or internal memory to reduce external DDR3 use.Source: B. Memory Bandwidth · p. 5

C. Performance

The hardware LSTM is reported significantly faster than other platforms despite its lower clock frequency, and two parallel cells can reach a 16× speedup over Exynos5422. The paper also reports that the small model ran slower on a MacBook GPU than its CPU, and describes its performance-per-power calculation.

C. Performance

  • 16× faster than Exynos5422 is possible with two LSTM cells in parallel; the implemented hardware was significantly faster than other platforms at 142 MHz versus the Zynq CPU’s 667 MHz.Source: C. Performance · p. 5
  • Execution time divided by the experiment’s 264.4 M-ops total gives performance in ops/s; power was measured to calculate performance per unit power.Source: C. Performance · p. 5
  • 0.304 s on the MacBook PRO 2016 CPU versus 0.569 s on its GPU for the same Torch7 test; the model was too small to benefit from the GPU and required memory copies.Source: C. Performance · pp. 5–6

VII. CONCLUSION

The work presents an LSTM hardware implementation that generated Shakespeare-like text and was faster than other mobile platforms. The authors identify parallel gate computation as future work toward a possible RNN co-processor.

VII. CONCLUSION

  • RNNs using LSTM have gained popularity in speech recognition, machine translation, scene analysis, and image caption generation.Source: §VII · p. 6
  • The implemented hardware produced Shakespeare-like text with a character-level model and was significantly faster than other mobile platforms.Source: §VII · p. 6
  • A future RNN co-processor for devices is a possibility, but further work is needed to optimize parallel gate computation using a parallel MAC configuration for matrix-vector multiplication.Source: §VII · p. 6
Loading 1511.05552v4…