Improving Performance Estimation for FPGA-based Accelerators for Convolutional Neural Networks

• Estimating performance across hardware specifications supports architectural exploration to identify an optimal FPGA accelerator configuration.Source: §1 · p. 2
• Latency estimation without scheduling information is challenging because execution varies with communication, synchronization, control, I/O interruptions, and CNN architecture, while representative feature selection is difficult.Source: §1 · p. 2
• The proposed method achieved the top result among the compared latency estimators: linear regression, zero-mean GP, ANN-mean GP, gradient tree boosting, and ANN.Source: §1 · p. 2
• The method couples a Gaussian process with a standard analytic method and statistical data to estimate performance for FPGA-based CNN accelerators.Source: §1 · p. 2

• Software-level performance estimates make design-space exploration more practical than repeated synthesis and hardware runs for different configurations; the paper focuses on latency.Source: §2 · p. 2
• Standard 2D convolution layers account for over 90% of CNN processing time, and design-space exploration determines filter, channel, spatial, and kernel parallelism.Source: §2 · p. 3
• Prior approaches include analytic accelerator-count estimation, synthesis-report-based gradient tree boosting, and a general heuristic used as the standard method for CNN accelerators.Source: §2 · p. 3
• The standard heuristic estimates each convolution layer by the maximum of its loading, computation, and storing times, then sums layer times for the CNN.Source: §2 · pp. 3–4
• The heuristic uses convolution input and output sizes, operation counts, memory bandwidth, clock cycle time, and hardware parallelism, but can miss communication, synchronization, and control delays.Source: §2 · p. 4

• The GP uses a mean function and covariance kernel: the mean represents the supposed average, while the kernel captures correlations between inputs.Source: §3 · p. 4
• A GP prediction is a multivariate Gaussian distribution with a predictive mean and variance for test targets given training data and test features.Source: §3 · p. 5
• Hyperparameters associated with the mean function and kernel are selected by maximizing marginal likelihood or by likelihood observation or cross-validation.Source: §3 · p. 5
• The analytic latency model for each convolution layer serves as the GP mean function, while past measurements train the model to learn nonlinearities that cannot be modeled analytically.Source: §3 · p. 5
• The proposed method reuses standard-estimation features and combines the analytic model with collected data, avoiding complete reliance on data and allowing predictions beyond observed samples without collapsing.Source: §3 · p. 5

• Per-layer latency is characterized by input loading, computation, and result storing times.Source: §4.1 · p. 6
• Only the first layer loads its input into on-chip memory, intermediate-layer outputs are buffered on-chip, and the output is stored off-chip once.Source: §4.1 · p. 6
• The accelerator section introduces notation and convolution operation and data-size quantities before presenting per-layer latency equations.Source: §4.1 · p. 6

1. Loading time i.e., the time to load the input into the on-chip memory

• Per-layer latency combines loading, computation, and storing, with loading plus computation for the first layer, overlapped weight and computation times for intermediate layers, and storing added for the last layer.Loading includes weights and input data; storing returns the output to off-chip memory.Source: §1 · p. 6

4.2 Dataset

• The dataset contains 156 convolution-layer samples from SSD, YOLO, and ResNet-50, with 24, 75, and 57 convolutions respectively.SSD and YOLO are described as irregular, while ResNet-50 is known for residual blocks; the networks were quantized to 8-bit integers after 32-bit floating-point training.Source: §4.2 · p. 7
• Each network was run on the implemented accelerator on an Intel Arria GX 1150 FPGA.Table 3 is identified as the evaluation dataset, but its caption alone does not specify table contents.Source: §4.2 · p. 7, Table 3 · p. 7

5 Evaluation

• The comparison included the standard method, zero-mean and ANN-mean GPs, LR, GTB, and ANN, evaluated using LOOCV MAE.Hyperparameters were selected using LOOCV MAE; the proposed GP’s grid search considered learning rates and linear, Gaussian, and Matérn kernels.Source: §5 · pp. 7–8
• 30.7% improvement in LOOCV MAE: the analytic-mean GP reached 0.312 ms, versus 0.450 ms for LR and the standard method.The authors report the analytic-mean GP as the best method.Source: §5 · p. 8
• The method reuses a standard analytic approximation with recorded measurements, and the authors say more measurements and kernel or mean-function tuning could improve it.This is presented as an implementation-simplicity advantage.Source: §5 · p. 8
• Kernel computation scales as O(P^3), so inference time can be prolonged when training samples are numerous.The authors suggest k-Means selection of important points as one possible remedy and state inference remains much faster than synthesis and hardware execution.Source: §5 · p. 8

6 Conclusion and Future Work

• The proposed Gaussian process pairs domain-specific knowledge with collected data and was compared with the standard method, GP variations, linear regression, gradient tree boosting, and an artificial neural network.Source: §6 · p. 8
• 30.7% accuracy improvement over the standard method or linear regression was demonstrated for the proposed approach.The evaluation table covers latency estimation for different methods.Source: Table 5 · p. 9, §6 · p. 10
• Users select software and hardware features, use an analytic approximation as the GP mean function, provide profiling features and per-layer latency targets, and choose a kernel for training.Source: §6 · p. 10
• Future work will validate the method on more configurations and hardware boards and extend analytic approximations to objectives such as resource usage and power consumption.Source: §6 · p. 10