PaperlayerView exact revision on arXiv

Source-linked AI summary

LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applications

Yaman Umuroglu, Yash Akhauri, Nicholas J. Fraser, Michaela Blott

arXiv:2004.03021v1eess.SPcs.ARcs.LG

TL;DR

Extreme-throughput applications require DNN implementations that meet stringent performance and latency constraints. LogicNets co-designs sparse, quantized network topologies that convert to FPGA truth-table netlists. Across two tasks, the approach demonstrated competitive accuracy, low latency, and throughput in the hundreds of millions of inferences per second.

  • Problem

    Extreme-throughput applications need DNN implementations that meet very high sample-rate and low-latency requirements.

    Source: §I · p. 1

  • Method

    LogicNets co-designs sparse, quantized DNN topologies that can be directly converted into equivalent FPGA truth-table netlists.

    Source: §I · p. 1

  • Results

    Hundreds of millions of inferences per second, competitive accuracy, and low latency were demonstrated on two extreme-throughput tasks.

    Source: §I · p. 1, §VI · p. 6

  • Takeaways & Limitations

    The evaluation demonstrates efficient FPGA implementations for sparse, quantized topologies on two extreme-throughput tasks.

    Source: §VI · p. 6

    The approach cannot map dense, high-fan-in networks and requires custom topologies; finding suitable designs also requires exploring a large design space.

    Source: D. Limitations · p. 6

Abstract

from arXiv · show

Deployment of deep neural networks for applications that require very high throughput or extremely low latency is a severe computational challenge, further exacerbated by inefficiencies in mapping the computation to hardware. We present a novel method for designing neural network topologies that directly map to a highly efficient FPGA implementation. By exploiting the equivalence of artificial neurons with quantized inputs/outputs and truth tables, we can train quantized neural networks that can be directly converted to a netlist of truth tables, and subsequently deployed as a highly pipelinable, massively parallel FPGA circuit. However, the neural network topology requires careful consideration since the hardware cost of truth tables grows exponentially with neuron fan-in. To obtain smaller networks where the whole netlist can be placed-and-routed onto a single FPGA, we derive a fan-in driven hardware cost model to guide topology design, and combine high sparsity with low-bit activation quantization to limit the neuron fan-in. We evaluate our approach on two tasks with very high intrinsic throughput requirements in high-energy physics and network intrusion detection. We show that the combination of sparsity and low-bit activation quantization results in high-speed circuits with small logic depth and low LUT cost, demonstrating competitive accuracy with less than 15 ns of inference latency and throughput in the hundreds of millions of inferences per second.

I. INTRODUCTION

LogicNets co-designs neural-network topologies for efficient FPGA inference in applications with extreme throughput and latency requirements. It converts quantized networks into truth-table netlists, using sparsity and quantization to reduce fan-in and guide hardware cost.

I. INTRODUCTION

  • LogicNets converts trained quantized networks directly into an equivalent hardware netlist of truth tables.The methodology co-designs network topologies to map directly to efficient FPGA implementations.Source: §I · p. 1
  • Sparsity and quantization reduce neuron fan-in, while an analytical cost model estimates FPGA resources to guide topology design.Source: §I · p. 1
  • A PyTorch library trains sparse, quantized topologies and converts them to Verilog netlists.Source: §I · p. 1
  • The evaluation covers two extreme-throughput tasks and reports competitive accuracy with FPGA throughput in the hundreds of millions of inferences per second.The approach is presented as an alternative to flexible hardware/software mappings that can incur performance overheads.Source: §I · p. 1

II. BACKGROUND

The background reviews sparse and quantized networks and FPGA inference architectures, contrasting designs with explicit computation datapaths against approaches that encode computation in LUTs.

A. Sparse and Quantized Neural Networks

  • Sparse networks connect each neuron to only a few activations, and low-bit quantization commonly uses training techniques such as STE to preserve accuracy.This work uses a priori fixed sparsity for simplicity and applies quantization to weights, activations, or both.Source: A. Sparse and Quantized Neural Networks · p. 2

B. Prior Work on FPGA DNN Inference

  • LUT-based designs such as LUTNet and binarized-network implementations can unroll computation, but retain explicit accumulation or activation datapaths.hls4ml supports folding or full unrolling and operates at 8- or 16-bit widths, mapping substantial computation to DSP blocks.Source: B. Prior Work on FPGA DNN Inference · p. 2
  • LogicNets packs operations and weights into truth tables without explicit MAC or activation datapaths, unlike the architectures reviewed above.Compared with NullaNet [14], LogicNets uses sparse topologies to avoid high fan-in, lossless truth-table conversion, and several low-precision activation variants.Source: B. Prior Work on FPGA DNN Inference · p. 2

III. BUILDING DNNS FROM SMALL TRUTH TABLES

LogicNets is motivated by FPGA hardware built from small truth tables and uses the equivalence between quantized neurons and truth tables to construct implementable networks.

A. Equivalence of Neurons and Truth Tables

  • A quantized neuron with X input bits and Y output bits can be implemented as an X:Y truth table by enumerating its possible inputs and recording the outputs.The truth table contains entries for 2^X possible inputs.Source: A. Equivalence of Neurons and Truth… · pp. 2–3
  • LogicNets calls an X:Y truth-table Verilog implementation a Hardware Building Block (HBB), and a trained neuron convertible to one a Neuron Equivalent (NEQ).The neuron’s internal complexity does not matter when its input and output bit counts are fixed.Source: A. Equivalence of Neurons and Truth… · p. 3
  • A 6:1 NEQ with floating-point weights and batch-normalization parameters uses 320 bits of storage, while its equivalent HBB needs 64 bits, five times less.Source: A. Equivalence of Neurons and Truth… · p. 3

B. The FPGA LUT Cost of Large Truth Tables

Implementing larger truth tables from FPGA LUTs requires controlling input fan-in because LUT cost grows exponentially with the number of inputs.

B. The FPGA LUT Cost of Large Truth Tables

  • Combinations of smaller LUTs can implement larger X:Y HBBs; for example, three 6:1 LUTs can implement a 7:1 LUT.The cost model estimates the required number of 6:1 FPGA LUTs for an HBB.Source: B. The FPGA LUT Cost of Large Truth… · p. 3
  • LUT cost grows exponentially with fan-in: a single 32:1 NEQ would require close to a hundred million 6:1 LUTs.The passage concludes that each NEQ must have sufficiently small fan-in to keep its LUT cost reasonable.Source: B. The FPGA LUT Cost of Large Truth… · p. 3

C. Designing Topologies with Restricted Fan-In

LogicNets restricts neuron fan-in to make DNN topologies practical for FPGA truth-table building blocks, using sparsity, activation quantization, and a hardware cost model to guide design.

C. Designing Topologies with Restricted Fan-In

  • Truth-table LUT cost grows exponentially with fan-in, motivating topology design that restricts the fan-in of each artificial neuron.The cost model guides exploration of neuron and layer counts while avoiding that exponential growth.Source: C. Designing Topologies with… · p. 3
  • Sparse connections limit each neuron to γ inputs from the previous layer, avoiding the large fan-in of dense connections.The design also quantizes activations to ≤4 bits to further reduce fan-in.Source: C. Designing Topologies with… · p. 3
  • The cost model estimates FPGA resources from each NEQ’s cost; a 12-input, 2-output NEQ is estimated at 170 6:1 LUTs.For 128 such NEQs, the example network’s estimated cost is 21,760 6:1 LUTs.Source: C. Designing Topologies with… · p. 3

IV. THE LOGICNETS DESIGN FLOW AND IMPLEMENTATION

  • The design flow defines HBBs and NEQs, trains and converts a network of NEQs into an HBB netlist, then postprocesses and synthesizes it into a bitfile.A prototype PyTorch library supports design-space exploration.Source: §IV · p. 3

A. Define HBBs and NEQs

The first design-flow step identifies X:Y values with reasonable LUT cost and defines corresponding trainable NEQs under sparsity and activation-quantization restrictions.

A. Define HBBs and NEQs

  • The step identifies X:Y values that produce HBBs with reasonable LUT cost.The identified range is used to define corresponding NEQs.Source: A. Define HBBs and NEQs · p. 3
  • NEQs are defined in PyTorch so they can be trained with sparsity and activation-quantization restrictions.Source: A. Define HBBs and NEQs · p. 3
  • The design-flow labels name the building blocks Hardware Building Blocks (HBBs) and Neuron Equivalents (NEQs).Source: A. Define HBBs and NEQs · p. 4

X:Y LUT

LogicNets defines FPGA-oriented HBBs and corresponding PyTorch NEQs, then trains a sparse, quantized network and converts it into an HBB truth-table netlist for synthesis.

X:Y NEQ

  • Batch normalization followed by uniform quantization with learned scale factors is used for accuracy and easier training.The quantization uses Brevitas.Source: X:Y NEQ · p. 4
  • The HBB range is identified once per FPGA device family, while the corresponding NEQs are defined for training with sparsity and activation quantization.Source: X:Y NEQ · p. 4

B. Train and convert a network of NEQs

  • For each new network, NEQs are connected using fixed random sparsity, and the analytical cost model estimates FPGA resources before training.Training uses standard DNN optimizers and backpropagation; knowledge distillation and ensembling can also be applied.Source: B. Train and convert a network of NEQs · p. 4
  • After training, each NEQ is enumerated over input combinations to create an HBB truth table, represented as a ROM for synthesis to map to FPGA LUTs.Source: B. Train and convert a network of NEQs · p. 4

C. Postprocessing, Synthesis and Deployment

The paper focuses on single-FPGA deployment, inserting registers between layers before synthesis and place-and-route; its evaluation setup covers jet classification and network intrusion detection.

C. Postprocessing, Synthesis and Deployment

  • The postprocessing considered is register insertion between layers for single-FPGA implementations, followed by synthesis and place-and-route to produce a bitfile.Other possible optimizations include logic minimization and splitting the netlist for dynamic partial reconfiguration, but they are not the focus here.Source: C. Postprocessing, Synthesis and… · p. 4

V. EVALUATION

  • Jet Substructure Classification is evaluated as a 16-input, 5-output task for FPGA triggers targeting 40 MHz data and sub-microsecond latency.Source: §V · p. 4
  • Network intrusion detection uses UNSWNB15 packet examples with 49 generated input features and bad or normal labels.Preprocessing and feature conversion follow Murovic et al..Source: §V · p. 4
  • Networks are trained for 1,000 epochs and evaluated on an xcvu9p-flgb2104-2-i FPGA using Vivado 2018.3, with registers between layers and a 1 ns clock constraint.Post-synthesis simulation checks that the generated hardware returns the same results as the PyTorch network.Source: §V · p. 5

A. Overview of Key Results

Across the explored LogicNets designs, FPGA resource use trades off against accuracy, while constrained fan-in supports shallow, fast circuits. Activation bitwidth also materially affects accuracy.

A. Overview of Key Results

  • Table I reports network topology, quantization, test accuracy, and analytical-model and post-synthesis FPGA resources for selected JSC and NID networks.The selected networks illustrate points from a partial design-space exploration.Source: A. Overview of Key Results · p. 5
  • 68% accuracy with 214 LUTs for JSC-S contrasts with 4 percentage points better accuracy for JSC-L at 37.9k LUTs.LogicNets' topology knobs expose resource savings at some accuracy cost.Source: A. Overview of Key Results · p. 5
  • β · γ ≤6 yields as little as one LUT level between registers; JSC-S has a reported Fmax of 1.5 GHz, while larger NID-L designs exceed 400 MHz.The reported 1.5 GHz cannot be achieved in practice because of global FPGA clock-network limitations; larger designs require more logic levels and resources and are harder to place and route.Source: A. Overview of Key Results · p. 5
  • Over 70% JSC accuracy and over 90% NID accuracy are reported as competitive with related work.The authors leave exploring better sparsity methods than fixed random sparsity for future work.Source: A. Overview of Key Results · p. 5

B. Impact of Activation Bitwidth

  • 88.7% accuracy at 2120 LUTs for the lowest-cost β=2 datapoint contrasts with 85.3% at 7392 LUTs for the highest-accuracy binary result.For JSC, higher activation bitwidth can yield higher accuracy at similar LUT cost.Source: B. Impact of Activation Bitwidth · p. 5, Figure 5 · p. 5
  • With the same number of neurons, greater activation bitwidth yields greater accuracy; adding neurons helps slightly at β=1 but brings little to no benefit for β>1.The experiments indicate that β is vital to achieving good LogicNets accuracy.Source: B. Impact of Activation Bitwidth · pp. 5–6, Figure 6 · p. 6

C. Lossless Pruning via Synthesis Optimizations

Synthesis pruning makes post-synthesis LUT counts substantially smaller than the analytical model predicts. LogicNets also requires custom fan-in-restricted topologies and a broad design-space search.

C. Lossless Pruning via Synthesis Optimizations

  • Post-synthesis LUT counts average 39× below analytical predictions; pruning removes unconnected neurons and registers, while logic minimization cuts individual HBB costs by 2.6–68×.JSC models show smaller post-synthesis savings of 1.5–8×.Source: C. Lossless Pruning via Synthesis… · p. 6

D. Limitations

  • LogicNets cannot map dense networks with commonly used high-fan-in neurons, instead requiring custom topologies that restrict fan-in through sparsity and activation quantization.Finding sufficiently accurate, low-resource networks also requires exploring combinations of layer and neuron counts, activation bitwidths, and fan-ins.Source: D. Limitations · p. 6

E. Comparison to Related Work

Against prior extreme-throughput FPGA implementations, LogicNets offers different accuracy, latency, throughput, and resource trade-offs across the two tasks. The conclusion reports competitive accuracy and very high throughput, while identifying training and synthesis tooling as future-work directions.

E. Comparison to Related Work

  • JSC-L achieved 1.9× higher throughput and 3.8× lower latency, with 3.2% points lower accuracy than Duarte et al..The latency comparison excludes the 5-cycle softmax.Source: E. Comparison to Related Work · p. 6
  • NID-M exceeded Murovic et al. on all reported metrics: 1.2% points higher accuracy, 9.2× higher throughput, 1.9× lower latency, and 3.2× fewer LUTs.Source: E. Comparison to Related Work · p. 6

VI. CONCLUSION AND FUTURE WORK

  • Across two extreme-throughput tasks, LogicNets demonstrated competitive accuracy, low latency, and throughput in the hundreds of millions of inferences per second.Source: §VI · p. 6
  • Future work targets accuracy in sparse, quantized topologies and improved analytical cost models and synthesis of larger LogicNets models.Proposed directions include sparse convolutions, mixing LogicNets-style and conventional layers, and studying heuristic logic minimization.Source: §VI · pp. 6–7

Related papers

Loading 2004.03021v1…