Compressing deep neural networks on FPGAs to binary and ternary precision with HLS4ML

1 Introduction

• At the LHC, the L1 trigger has O(1) µs to decide whether to retain an event, and the severe latency constraint limits complex rule-based algorithms.DNNs are being investigated as fast, parallelisable approximations of rule-based algorithms.Source: §1 · p. 2
• hls4ml automatically converts trained DNNs into FPGA firmware through high-level synthesis and targets fully on-chip logic for latency within typical O(1) µs values.The library aims to support popular DNN ingredients and training-library interfaces.Source: §1 · p. 2
• Limited FPGA computational resources are the main challenge in deploying DNNs, while binary and ternary weights use one and two bits, respectively.Binary weights are +1 or −1; ternary weights are +1, 0, or −1.Source: §1 · p. 2
• The paper investigates binary and ternary networks for FPGA inference; binary parameters (+1 or −1) use one bit and ternary parameters (+1, 0, or −1) use two, yielding much smaller resource consumption [15] [16] [17].Source: §1 · p. 2
• The study evaluates binary and ternary networks on MNIST digit classification and LHC jet tagging, a task particularly relevant to LHC applications.MNIST enables comparison with previous literature; jet tagging is used as a benchmark in prior work.Source: §1 · p. 2
• The LHC jet data set simplifies realistic conditions by excluding the time and resources needed to compute input features.The authors point to future hls4ml support for more complex architectures and raw-data inputs.Source: §1 · p. 3

2 Benchmark models and data sets

• MNIST images are flattened from 28 × 28 pixels and classified by an MLP with 784 inputs, three 128-node hidden layers, and 10 outputs.The data set provides 60,000 training-and-validation images and 10,000 test images; 75% of the former are used for training and 25% for validation.Source: §2 · p. 3
• The LHC jet classifier maps 16 high-level features to five jet classes using hidden layers of 64, 32, and 32 nodes.Its approximately 1 million simulated examples are split into 60% training, 20% validation, and 20% test data.Source: §2 · p. 3
• Both baseline architectures are Keras models trained in floating point precision, with performance shown as ROC curves and normalized confusion matrices.Figure 1 depicts the baseline architectures; Figure 2 presents test-set performance, and Table 1 reports AUC and per-class accuracy.Source: §2 · p. 3, Figure 1 · p. 3, Figure 2 · p. 4
• 98% and 75% are the total test accuracies for the baseline MNIST and LHC jet classifiers, respectively.The values are computed across all categories by applying Arg Max to the network scores and comparing predictions with targets.Source: §2 · pp. 3–4
• The authors describe a jet-classifier accuracy of roughly 60–70% as useful for LHC experiments, with the baseline potentially selecting over 80% of W, Z, and t jets while reducing bandwidth by about a factor of 10.The rationale given is that most LHC jets come from quarks and gluons.Source: §2 · p. 4
• The baselines are lightweight examples chosen to keep resource use manageable, not architectures optimized for the best performance in each use case.The study focuses on hardware implementation and relative performance drop rather than absolute performance.Source: §2 · p. 4

• Custom binary and ternary implementations speed up the repeated dense-layer architecture and are integrated into hls4ml starting with version 0.1.6.That version also supports large dense layers with hundreds of nodes.Source: §3 · p. 5
• For ternary weights or activations wider than 1 bit, the implementation uses FPGA logic rather than typically limited DSP blocks.Source: §3 · p. 6
• Binary weights and activations use 1-bit encoding, allowing multiplication to be implemented as an XNOR operation.The truth table maps equal encoded values to 1 and different values to 0.Source: Table 2 · p. 6, §3 · p. 6
• Batch normalization parameters yield compile-time thresholds, so binary and ternary tanh nodes can use comparisons instead of runtime scaling.The implementation also selects an integer bit width for each layer based on the largest expected input value.Source: §3 · p. 6
• Binary and ternary layers are integrated and compatible with hls4ml, which also supports mixing them with higher-precision layers.Mixed-precision models are supported but were not explored in this study.Source: §3 · p. 6

• The strategy spans direct precision reduction for resource savings, larger architectures to approach baseline accuracy at higher cost, and intermediate compromises.Source: §4 · pp. 6–7
• Seven model variants include a baseline, architecture-preserving BNN and TNN, optimized-width best BNN and TNN, and ReLU-based hybrid models.Source: §4 · pp. 6–7
• The BNN and TNN replace weights with ±1 or −1/0/+1, use binary or ternary tanh in hidden layers, insert batch normalization, and remove output softmax.Source: §4 · pp. 6–7
• Best BNN and TNN models use Bayesian optimization to choose larger or tuned layer widths; they are shown only for LHC jets, where direct quantization loses more performance.Source: §4 · p. 7
• Hybrid models retain ReLU or clipped ReLU activations with quantized weights, aiming for baseline-like performance without increasing node count and operations.Source: §4 · p. 7
• The paper discusses bit precision and quantization even for binary or ternary models whose intermediate outputs are not all represented at those precisions.Source: §4 · p. 7
• Training uses binary or ternary precision in forward propagation and full precision in backpropagation; binary/ternary models use hinge loss, while baselines use categorical cross entropy.Source: §4 · p. 7

5 Experiments

• Experiments use Vivado HLS 2018.2 on a Xilinx Virtex Ultrascale 9+ at 200 MHz; synthesized-firmware simulations are bit-identical to device execution, which the study omits for practical reasons.Logic synthesis further refines resource estimates and reduces resource consumption without changing accuracy.Source: §5 · p. 8

5.1 Handwritten digits classification

• MNIST baseline quantization to ⟨18, 8⟩ lowers accuracy from 98% to 95%, largely because of limited precision in the softmax exponential-function lookup table.Using ⟨22, 10⟩ precision for the softmax lookup table recovers 97% accuracy without affecting resources; alternatively, removing softmax is suitable when only the top-scoring class is needed.Source: Table 3 · p. 8, §5.1 · p. 9
• At O(100) ns, MNIST BNN/TNN models retain competitive performance while saving most FPGA resources; hybrid models recover about half the accuracy loss at a small DSP cost.The hybrid DSP cost comes from explicitly allocating batch-normalization layers before ReLU or clipped ReLU activations.Source: Table 3 · p. 8, §5.1 · pp. 9–10, Table 4 · p. 10

5.2 LHC jet identification

• Jet-model binarization or ternarization causes substantial accuracy loss, partly mitigated by ReLU activations; Bayesian optimization finds larger architectures that nearly match performance with moderate loss.The optimized BNN and TNN have about 40 and 4 times as many parameters as the baseline, respectively.Source: §5.2 · p. 11, Table 5 · p. 12
• The best TNN matches the best BNN's accuracy, baseline latency, and drastically reduces DSP utilization; ternary models generally offer a better resource–accuracy balance.The largest best BNN requires a higher initiation interval to fit FPGA resource limits, while the best TNN remains within the targeted latency boundary.Source: §5.2 · pp. 11–12, Table 5 · p. 12, Table 6 · p. 12

6 Summary and Outlook

• Binary and ternary networks preserve competitive accuracy while drastically reducing FPGA resource use and keeping inference latency at O(100) ns.The reported benchmarks are handwritten digit recognition on MNIST and jet identification at the LHC.Source: §6 · p. 12
• Compared with binary models, ternary models reach accuracy closer to the original baselines, typically with smaller resource cost and comparable latency.Source: §6 · p. 12
• The hls4ml implementation converts neural network models into FPGA firmware and supports binary and ternary model strategies.Source: §6 · p. 12
• Binarization and ternarization are presented as alternatives to compression approaches such as pruning, with minimal DSP utilization and potential for complex architectures in resource-constrained environments.Source: §6 · p. 12