Source-linked AI summary
FPGA/DNN Co-Design: An Efficient Design Methodology for IoT Intelligence on the Edge
Cong Hao, Xiaofan Zhang, Yuhong Li, Sitao Huang, Jinjun Xiong, Kyle Rupnow, Wen-mei Hwu, Deming Chen
TL;DR
Embedded FPGAs provide efficient, low-latency DNN acceleration for edge devices, but scarce resources make deployment difficult. The paper proposes simultaneous FPGA/DNN co-design with automatic hardware-oriented model search and accelerator generation, achieving stronger FPGA results and similar GPU accuracy with lower energy use.
Problem
Scarce resources on edge-scale FPGAs make it difficult to deploy DNNs while meeting accuracy, latency, power, and resource requirements.
Method
The methodology combines bottom-up hardware-oriented DNN exploration with top-down DNN-driven accelerator design through Auto-DNN and Auto-HLS.
Results
6.2% higher IoU, 40% lower power, and 2.5× better energy efficiency than the 1st-place FPGA design were achieved on PYNQ-Z1 object detection.
Takeaways & Limitations
The co-design implementation outperformed the 1st-place FPGA design across reported factors and delivered similar accuracy to GPU designs with better energy efficiency.
Abstract
from arXiv · showhide
While embedded FPGAs are attractive platforms for DNN acceleration on edge-devices due to their low latency and high energy efficiency, the scarcity of resources of edge-scale FPGA devices also makes it challenging for DNN deployment. In this paper, we propose a simultaneous FPGA/DNN co-design methodology with both bottom-up and top-down approaches: a bottom-up hardware-oriented DNN model search for high accuracy, and a top-down FPGA accelerator design considering DNN-specific characteristics. We also build an automatic co-design flow, including an Auto-DNN engine to perform hardware-oriented DNN model search, as well as an Auto-HLS engine to generate synthesizable C code of the FPGA accelerator for explored DNNs. We demonstrate our co-design approach on an object detection task using PYNQ-Z1 FPGA. Results show that our proposed DNN model and accelerator outperform the state-of-the-art FPGA designs in all aspects including Intersection-over-Union (IoU) (6.2% higher), frames per second (FPS) (2.48X higher), power consumption (40% lower), and energy efficiency (2.5X higher). Compared to GPU-based solutions, our designs deliver similar accuracy but consume far less energy.
1 INTRODUCTION
Embedded FPGAs offer low-latency, energy-efficient DNN acceleration for IoT, but limited resources complicate deployment. The paper addresses this through simultaneous FPGA/DNN co-design and an automatic search-and-generation flow.
- IoT applications require sophisticated DNN models alongside strict latency, power, and resource constraints.
- Embedded FPGAs are attractive for IoT intelligence because of their high energy efficiency and low cost, despite scarce resources.
- The proposed methodology develops DNNs and FPGA accelerators simultaneously, making DNN designs FPGA-architecture driven and accelerators DNN-aware.
- The combined FPGA/DNN co-design space is extremely large, making candidate training and evaluation time-consuming even with NAS and HLS.
- The automatic flow combines hardware-oriented DNN search, a fine-grained tile-based accelerator, Auto-DNN, and Auto-HLS for synthesizable accelerator generation.
2 RELATED WORK
Prior work advances DNN design and FPGA acceleration largely as separate activities. Existing platform-aware searches target CPUs and GPUs, while FPGA accelerators often remain constrained by the chosen DNN models.
- DNN design and FPGA accelerator design are often conducted independently.
- General DNN search can achieve high QoR but may produce complex structures unsuitable for FPGA deployment.
- Platform-aware DNN searches described here consider inference latency on CPUs and GPUs rather than FPGAs.
- FPGA accelerator methods use quantization, compression, latency-directed resource allocation, and fine-grained pipelines, but may be limited by the DNN models.
3 FPGA/DNN CO-DESIGN
The co-design space jointly represents DNN and FPGA accelerator choices, while the flow searches both sides iteratively. Bundle-Arch and Tile-Arch provide structured templates for hardware-aware models and low-latency implementations.
- 3 FPGA/DNN CO-DESIGN: The joint co-design space is exponentially larger than either the DNN or accelerator design space alone.
- 3.1 Co-Design Space: Table 1 variables specify DNN layers, FPGA IP instances, parallelism, quantization, channel expansion, and down-sampling configurations.
- 3 FPGA/DNN CO-DESIGN: Auto-DNN explores DNN candidates under hardware constraints, while Auto-HLS maps them to FPGA accelerators and the two execute iteratively.
- 3 FPGA/DNN CO-DESIGN: The flow accepts a target task, FPGA resource constraints, performance targets, and configurable IP templates, then outputs hardware-oriented DNNs and accelerators.
- 3 FPGA/DNN CO-DESIGN: Auto-DNN evaluates building blocks for latency, resource utilization, and accuracy, selecting Pareto-curve blocks for further exploration.
- 3 FPGA/DNN CO-DESIGN: The search uses stochastic coordinate descent under resource and latency constraints, with Auto-HLS providing precise results for feedback and updates.
4 DNN AND ACCELERATOR TEMPLATE
The methodology combines hardware-aware DNN construction with FPGA-specific accelerator templates and analytical performance/resource modeling. Bundle-Arch and Tile-Arch support exploration and implementation under edge-FPGA constraints.
- Bundle-Arch: Hardware-Aware DNN Template: DNN templates narrow the design space and incorporate hardware knowledge to guide hardware-oriented model exploration.The paper uses templates to speed search and direct designs toward hardware-oriented configurations.
- Co-Design Flow: Auto-DNN explores DNN models while Auto-HLS generates corresponding FPGA implementations through the four-component co-design flow.Bundle-Arch and Tile-Arch provide the DNN and accelerator templates, respectively.
- Bundle-Arch: Hardware-Aware DNN Template: Bundle-Arch builds DNNs from sequential layer Bundles that are replicated, shaped, and configured bottom-up, with down-sampling spots between Bundles.The corresponding FPGA Bundle combines the IP instances used for DNN layer computation.
- Bundle Generation: Bundle candidates use convolution, depth-wise convolution, pooling, normalization, and activation IPs, with at most two computational IPs per Bundle for resource-scarce IoT devices.Eighteen Bundle candidates are generated offline, then promising candidates are selected using accuracy contributions and hardware characteristics.
- Tile-Arch: Low Latency Accelerator Template: Tile-Arch maps DNNs onto embedded FPGAs with a fine-grained tile-based pipeline designed for low latency and resource saving.Its folded structure reuses IPs across layers and tiles, while BRAM supports intra-Bundle communication and DRAM supports inter-Bundle communication.
- Bundle and DNN Performance Modeling: Latency modeling estimates each Bundle from computation and data-transfer costs, including IP computation, processed data, memory bandwidth, pipelining overlap, and transfer overlap.Auto-HLS sampling determines the overlap and control-related parameters used by the model.
- DNN Performance Modeling: DNN latency and resource models aggregate repeated Bundle costs, inter-Bundle data movement, Bundle resources, and additional control-logic overhead.The model uses N for Bundle repetitions, while Auto-HLS sampling determines inter-Bundle movement and control-overhead parameters.
- DNN Performance Modeling: Coarse- and fine-grained Bundle evaluations compare latency, resource usage, and accuracy to select promising candidates for hardware-aware DNN exploration.The evaluations construct DNNs from Bundle replications and examine characteristics across Bundle choices and configurations.
5 DNN EXPLORATION AND UPDATE
Auto-DNN and Auto-HLS jointly explore hardware-oriented DNNs and FPGA accelerators under latency and resource constraints. Bundle evaluation, stochastic coordinate descent, training, and accelerator synthesis form an iterative co-design flow.
- Bundle Evaluation: The flow first evaluates Bundle candidates on latency, resource usage, and accuracy, then selects promising candidates for hardware-aware DNN exploration.Coarse and fine-grained evaluation supports scalable Bundle selection before model search.
- Bundle Evaluation: Each Bundle is tested through compact DNN constructions, including insertion into a fixed template or repeated replication, with proxyless task training for accuracy.The experiments train each candidate for 20 epochs to accelerate evaluation.
- Bundle Evaluation: Bundle 1 and 3 favor higher-accuracy DNNs with greater resource use and latency, whereas Bundle 13 favors real-time responses with lower resource use.Fine-grained evaluation exposes distinct latency, accuracy, and resource characteristics across Bundles.
- Hardware-Aware Search: Auto-DNN incrementally updates DNNs using Bundle replication, down-sampling, and channel expansion until latency targets are met within resource constraints.The SCD unit randomly selects coordinates and updates the DNN while enforcing the resource limit.
- Hardware-Aware Search: The SCD unit receives an initial DNN, a latency target, a latency tolerance, and a resource constraint, then generates K candidate DNNs for training and FPGA implementation.Auto-HLS synthesizes corresponding implementations to obtain resource usage and latency.
- Accelerator Generation: Auto-HLS generates synthesizable C accelerator code from the Tile-Arch template using IP calls, weight loading, and data buffering based on DNN dimensions and feature maps.The generated code can be directly synthesized by HLS tools, with later manual optimization mentioned in the implementation flow.
6 EXPERIMENTAL RESULTS
The experiments evaluate latency–accuracy trade-offs and board-level object-detection implementations on PYNQ-Z1. The proposed designs achieve strong IoU, throughput, power, and energy-efficiency results against FPGA and GPU baselines.
- Experimental Setup: The evaluation sets latency targets of 10, 15, and 20 FPS at 100MHz to provide accuracy–latency trade-off options.Explored DNNs must meet target latency within resource constraints.
- DNN Exploration: 68 DNN models are built from 5 Bundles, and the most accurate model for each FPS target is selected for final evaluation.The selected models are DNN1, DNN2, and DNN3.
- DNN Exploration: 68.6% IoU is achieved by DNN1 at 12.5 FPS@100MHz and 17.4 FPS@150MHz.DNN2 reaches 61.2% IoU at 16.0 FPS@100MHz and 22.7 FPS@150MHz, while DNN3 reaches 29.7 FPS@150MHz with 59.3% IoU.
- Comparison Method: The comparison uses board-level implementations, 50K official-dataset images, and runtime-inclusive FPS measurements covering loading, preprocessing, and inference.Power and energy are measured with a POWER-Z KT001 USB Power Monitor.
- Comparison Results: 6.2% higher IoU, 40% lower power, and 2.5× better energy efficiency are achieved than the first-place FPGA design.Against the first-place GPU design, energy efficiency is 3.6× better, with 40% longer latency and nearly 6× lower clock frequency.
7 CONCLUSION
The proposed FPGA/DNN co-design methodology combines bottom-up DNN exploration with top-down accelerator design for embedded-FPGA IoT intelligence. On object detection, it outperformed the DAC-SDC first-place design across accuracy, power, and energy efficiency, while approaching GPU accuracy with better energy efficiency.
- The methodology combines bottom-up DNN model exploration with top-down FPGA accelerator design for embedded FPGAs.
- 6.2% higher IoU, 40% lower power, and 2.5× better energy efficiency were achieved than the DAC-SDC first-place design.
- Similar accuracy to GPU designs was achieved, ranging from 0.1% better than third place to 1.2% worse than first place.
- 3.1× to 3.8× better energy efficiency was achieved than the compared GPU designs.