Source-linked AI summary
DiffLUT-Net: Differentiable Training of FPGA LUT Networks with Learnable Connectivity
Jiaqi Ye, Xinrui Gong, Jingcun Wang, Olga Kondrateva, Bing Li, Grace Li Zhang
TL;DR
FPGA neural-network deployment commonly relies on arithmetic acceleration or post-training LUT conversion, leaving a gap between learned models and FPGA logic. DiffLUT-Net trains FPGA-native LUT6 networks by jointly learning truth-table functions and sparse hardware-valid connectivity, then hardens and exports them as Verilog. Across five benchmarks, it reports strong accuracy–hardware trade-offs, including resource reductions relative to comparison methods.
Problem
Existing FPGA flows either retain arithmetic structures or convert pretrained quantized models into LUTs, while prior LUT methods do not fully validate jointly learned connectivity through end-to-end hardware deployment.
Method
DiffLUT-Net jointly learns LUT6 truth-table entries and one hardware-valid source for each input port, then binarizes, fixes, prunes, and exports the network as synthesizable Verilog.
Results
DiffLUT-Net achieves strong accuracy–hardware trade-offs across five benchmarks, with compact configurations improving resource efficiency and high-accuracy configurations avoiding excessive hardware growth.
Takeaways & Limitations
Jointly learning LUT functions and sparse connectivity provides a direct training-to-deployment framework for compact FPGA-native inference.
Takeaways & Limitations
Prior truth-table approaches remain constrained by exponential growth in table size with increasing fan-in and input bit width.
Abstract
from arXiv · showhide
Field-programmable gate arrays (FPGAs) enable efficient neural-network inference, but most deployment flows either accelerate multiply-accumulate operations or convert pretrained quantized models into lookup tables (LUTs). We present DiffLUT-Net, an FPGA-native network connected by six-input LUTs that are trained from scratch. We jointly learn the 64 truth-table entries of a LUT and the source to each of its six input ports using a differentiable LUT function relaxation and hardware source selection. After training, the truth tables and connections are discretized, unused logic can be pruned, and the network is exported directly as synthesizable Verilog. Across five benchmarks, DiffLUT-Net achieves favorable accuracy-resource trade-offs. These results demonstrate the effectiveness of jointly learning LUT functions and sparse connectivity for compact FPGA-native inference. The code is available at https://github.com/TUDa-HWAI/DiffLUT-Network.
1 Introduction
FPGA inference efficiency depends on aligning trainable representations with FPGA logic, yet conventional flows separate model training from hardware realization. DiffLUT-Net addresses this gap by jointly learning LUT6 functions and hardware-valid connectivity, then exporting the trained network directly to synthesizable hardware.
- Conventional FPGA flows train networks with MAC operations before quantization, compilation, or mapping, separating learned models from inference logic.
- Truth-table methods convert trained quantized neurons or subnetworks into FPGA LUTs, while direct logic networks train hardware-oriented functions from scratch.
- DiffLUT-Net jointly learns all 64 entries of each LUT6 truth table and one source signal for each of its six input ports.
- After training, truth tables are binarized, connections fixed, unused logic pruned, and the deterministic network exported as synthesizable Verilog.
- 41% fewer LUTs are used by a compact MNIST configuration than NeuraLUT-Assemble at comparable accuracy, while LUT utilization falls 52% on Fashion-MNIST and 16% on CIFAR-10 versus DWN.
2 Related Work
Prior FPGA neural-network approaches either accelerate arithmetic, convert quantized functions into truth tables, or train logic networks directly. Their limitations include retained arithmetic structures, exponentially growing truth tables, and incomplete validation of jointly learned connectivity and end-to-end hardware deployment.
- Arithmetic FPGA accelerators preserve weighted sums and activations, mapping multipliers, adders, accumulators, buffers, and pipelines onto hardware resources.
- These methods can reduce arithmetic cost but may still require substantial resources for adder trees, nonlinear activations, batch normalization, and intermediate-value storage.
- Quantized-neuron methods enumerate low-bit input-output combinations into truth tables, eliminating explicit MAC operations for functions such as weighted sums, normalization, and activation.
- Truth-table size grows exponentially with fan-in and input bit width, motivating low precision, sparse connectivity, restricted fan-in, decomposition, or hierarchical assembly.
- Hardware-native methods train Boolean gates or LUTs from scratch, with later work improving parameterization, sparse connections, architectures, and output grouping.
- Existing LUT-level methods do not jointly establish the benefits of their training and connectivity choices through a complete training-to-Vivado workflow.
3 Methodology of DiffLUT-Net
DiffLUT-Net trains LUT6 functions and their sparse inter-layer connectivity jointly, then discretizes the learned network for pruning and direct Verilog export. Its methodology combines distribution-aware binary input encoding, differentiable truth-table parameters, learned source selection, and grouped output accumulation.
- Output and hardware export: The network assigns final-layer LUT outputs to class groups and predicts the class with the largest accumulated score.The complete learned network is translated directly into RTL, exported as Verilog, and implemented with Vivado.
- Differentiable LUT function: A LUT function is relaxed by representing each stored truth-table value as a sigmoid of a trainable parameter, then binarized after training.A LUT6 has 64 trainable parameters corresponding to its truth-table entries.
- Sparse connectivity: Each current-layer LUT input learns its source from previous-layer outputs through a trainable connection matrix whose maximum-scoring entry is selected during forward propagation.The matrix has one row per current-layer input port, and all candidates receive heuristic updates during backpropagation.
- Sparse connectivity: After training, the largest matrix entry fixes one source per input port, while unselected previous-layer LUTs become inactive and can be pruned.This produces the sparse mapping used for the final hardware network.
- Training procedure: Training uses symmetric signed initialization for LUT parameters and adds a penalty that pushes those parameters toward zero to reduce overfitting.The reported initialization values are +10 and -10.
- Input encoding: DiffLUT-Net converts real-valued inputs into binary signals with distribution-aware thermometer encoding before LUT processing.Thresholds are computed from training-data quantiles and fixed for inference.
4 Experimental Results
Across five benchmarks, DiffLUT-Net provides accuracy–resource trade-offs using post-implementation FPGA measurements, with compact configurations emphasizing matched-accuracy efficiency and high-accuracy configurations testing larger resource budgets. Width, depth, temperature, and second-layer size experiments show that scaling and architecture affect accuracy and hardware cost differently.
- Experimental setup: Five benchmarks evaluate compact matched-accuracy configurations and higher-resource high-accuracy configurations using FPGA hardware metrics.The reported setup includes post-placement-and-routing measurements where available.
- JSC benchmarks: 49.8× fewer LUTs and approximately 124× lower A×L are reported for compact JSC CERNBox DiffLUT-Net than NeuraLUT at similar accuracy.Ours (50) reaches 72.5% accuracy with 94 LUTs and 1.21 ns latency, compared with NeuraLUT at 72.0%.
- JSC benchmarks: At high accuracy on JSC CERNBox, DiffLUT-Net matches NeuraLUT-Assemble with 31% fewer post-implementation LUTs and achieves the lowest A×L among the listed high-accuracy configurations.The configuration reaches 75.0% accuracy and reduces A×L by 35% relative to NeuraLUT-Assemble.
- JSC benchmarks: On JSC OpenML, compact DiffLUT-Net improves accuracy while reducing LUT count, latency, and A×L relative to DWN at both reported model scales.At the larger compact configuration, accuracy is 74.1% versus 74.0%, with LUTs reduced from 110 to 94.
- Fashion-MNIST and CIFAR-10: On Fashion-MNIST and CIFAR-10, DiffLUT-Net reports 52% and 16% LUT reductions relative to DWN while closely matching or improving accuracy.The Fashion-MNIST compact configuration reaches 89.00% versus DWN’s 89.01%; CIFAR-10 reaches 59.50% and improves accuracy by 2.08% with 16% fewer LUTs.
- Scaling and architecture: Increasing width improves accuracy with diminishing gains, while two layers outperform one, deeper networks are non-monotonic, and intermediate temperature performs best.Width from 50 to 4000 raises accuracy from 74.1% to 76.3%; two layers reach 76.5%, and the best observed temperature is τ=40.
- Scaling and architecture: A 500–1000-node second layer provides the best observed accuracy–hardware trade-off by preserving accuracy while reducing LUT utilization.A 500-node second layer retains 76.0% accuracy with 1,724 LUTs, a 63% reduction from the single-layer reference.
5 Conclusion
DiffLUT-Net provides a complete train-to-deployment framework for FPGA-native LUT6 networks by jointly learning functions and sparse hardware-valid connections. Across five benchmarks, it achieves strong accuracy–hardware trade-offs, while convolutional and transformer extensions remain future work.
- DiffLUT-Net jointly learns LUT6 functions and sparse hardware-valid connections, then hardens and exports them as synthesizable Verilog.Distribution-aware thermometer encoding and GroupSum complete the train-to-deployment framework.
- Across five benchmarks, DiffLUT-Net achieves strong accuracy–hardware trade-offs across different model scales.Compact configurations emphasize resource efficiency, while high-accuracy configurations improve accuracy without excessive hardware growth.
- Ablations show that model width provides a controllable path for scaling accuracy and learned sparse connectivity can reduce logic across multiple network stages.
- Future work includes extending differentiable LUT networks to convolutional and transformer-based architectures with cross-layer parameter sharing.
A Training Configurations
The appendix summarizes training configurations shared by the benchmark and ablation experiments, with dataset-specific settings reported separately.
- The appendix summarizes training configurations used in the benchmark and ablation experiments.
- Table 5 lists the training settings shared by the benchmark and ablation experiments.
A.1 Configurations for the JSC Benchmarks
The JSC benchmark configurations are documented in the appendix and correspond to the DiffLUT-Net configurations reported in Table 2.
- All JSC models are trained without data augmentation.
- Table 6 lists the DiffLUT-Net configurations reported in Table 2.
A.2 Configurations for MNIST
The MNIST appendix documents the training configurations used for the reported results, including the data-augmentation distinction between model variants.
- The two-layer MNIST model is trained without data augmentation.
- Data augmentation is enabled for the two single-layer configurations marked with an asterisk in the main benchmark table.
- Table 7 lists the training configurations reported for the MNIST results in Table 3.
A.3 Configurations for Fashion-MNIST and CIFAR-10
Table 8 reports the training configurations used for the Fashion-MNIST and CIFAR-10 results in Table 4. Fashion-MNIST uses no data augmentation, whereas CIFAR-10 uses augmentation and is trained for 400,000 steps.
- Table 8 lists the DiffLUT-Net configurations reported in Table 4.
- Fashion-MNIST configurations are trained without data augmentation.
- CIFAR-10 uses data augmentation and is trained for 400,000 steps.
A.4 Configurations for the Ablation Studies
The ablation studies are conducted on JSC OpenML using shared training settings unless an experiment explicitly varies them. Four ablations are reported separately.
- All ablation studies are conducted on JSC OpenML.
- Ablations use shared training settings unless an experiment explicitly varies a setting.
- The four ablation studies are reported separately.
A.4.1 Single-Layer Width
The ablation configurations vary layer width, network depth, GroupSum temperature, and second-layer width under specified fixed settings. The temperature study contains 80 completed runs and reaches its highest observed accuracy at τ = 40.
- Single-Layer Width: The single-layer-width ablation varies the number N of trainable LUT6 nodes and adjusts temperature with model width according to Table 9.
- Network Depth: The network-depth ablation fixes every LUT6 layer at width N = 2,000 and varies the number of layers from one to four.
- GroupSum Temperature: The temperature ablation fixes the Ours (1000, 500) architecture and varies only the GroupSum training temperature.
- GroupSum Temperature: 80 completed runs were obtained in the temperature ablation, with the highest observed accuracy at τ = 40.
- Second-Layer Width: The second-layer-width ablation fixes the first-layer width at N_1 = 2,000 and varies the second-layer width N_2.
- Second-Layer Width: The None configuration contains only the 2,000-node first layer, which connects directly to GroupSum.