PaperlayerView exact revision on arXiv

Source-linked AI summary

Comparing Energy Efficiency of CPU, GPU and FPGA Implementations for Vision Kernels

Murad Qasaimeh, Kristof Denolf, Jack Lo, Kees Vissers, Joseph Zambreno, Phillip H. Jones

arXiv:1906.11879v1cs.CVeess.IV

TL;DR

Embedded vision developers need evidence for selecting among heterogeneous accelerators under runtime and energy constraints. The paper benchmarks vision kernels and complete pipelines across CPU, GPU, and FPGA platforms using their vendor-optimized libraries, finding that GPUs favor simpler kernels while FPGAs outperform for more complicated pipelines and improve as pipeline complexity grows.

  • Problem

    Selecting among heterogeneous embedded-vision accelerators and their vendor-optimized libraries is difficult when runtime performance and energy constraints must both be considered.

  • Method

    The study benchmarks representative vision kernels and complete pipelines on ARM57 CPU, Jetson TX2 GPU, and ZCU102 FPGA platforms using OpenCV, VisionWorks, and xfOpenCV.

  • Results

    GPU implementations achieve 1.1–3.2× energy/frame reduction for simple kernels, while FPGAs achieve 1.2–22.3× reduction for more complete vision pipelines and improve as pipeline complexity grows.

  • Takeaways & Limitations

    Accelerator suitability depends on vision-kernel and pipeline complexity: GPUs perform well for simple parallel kernels, whereas FPGAs increasingly outperform for complete pipelines.

Abstract

from arXiv · show

Developing high performance embedded vision applications requires balancing run-time performance with energy constraints. Given the mix of hardware accelerators that exist for embedded computer vision (e.g. multi-core CPUs, GPUs, and FPGAs), and their associated vendor optimized vision libraries, it becomes a challenge for developers to navigate this fragmented solution space. To aid with determining which embedded platform is most suitable for their application, we conduct a comprehensive benchmark of the run-time performance and energy efficiency of a wide range of vision kernels. We discuss rationales for why a given underlying hardware architecture innately performs well or poorly based on the characteristics of a range of vision kernel categories. Specifically, our study is performed for three commonly used HW accelerators for embedded vision applications: ARM57 CPU, Jetson TX2 GPU and ZCU102 FPGA, using their vendor optimized vision libraries: OpenCV, VisionWorks and xfOpenCV. Our results show that the GPU achieves an energy/frame reduction ratio of 1.1-3.2x compared to the others for simple kernels. While for more complicated kernels and complete vision pipelines, the FPGA outperforms the others with energy/frame reduction ratios of 1.2-22.3x. It is also observed that the FPGA performs increasingly better as a vision application's pipeline complexity grows.

I. INTRODUCTION

Embedded vision increasingly requires energy-efficient image processing, while developers must choose among heterogeneous accelerators and partition pipelines across them. This study benchmarks representative kernels and complete pipelines on CPUs, GPUs, and FPGAs to compare performance, energy, and EDP.

  • Energy-efficient image processing is especially important for real-time embedded systems with constrained communication power budgets or capabilities.
  • CPU, GPU, and FPGA accelerators use different approaches to accelerating embedded vision applications.The passage identifies multicore CPUs, GPUs, and FPGAs as three common accelerator types.
  • Choosing an accelerator requires considering runtime performance, energy efficiency, programmability, and efficient pipeline partitioning.
  • The study benchmarks representative vision kernels and complete pipelines on ARM57 CPU, Jetson TX2 GPU, and Xilinx UltraScale FPGA platforms.It also analyzes the reasons behind runtime, power, and energy differences and compares platforms using EDP.
  • Unlike prior work focused mainly on single kernels or subsets of vision workloads, this study evaluates a wide range of standard vision kernels and explains platform-specific behavior.

III. BACKGROUND

The study contrasts CPU, GPU, and FPGA architectures as alternative ways to accelerate embedded vision. Their parallelism, memory organization, and programmability shape how efficiently they execute vision workloads.

  • CPUs use SIMD instructions and multiple ALUs to apply repeated operations across image data.Examples include ARM NEON and Intel SSE instruction sets.
  • GPUs specialize SIMD-style processing with simpler cores, enabling many more cores per chip than general-purpose CPUs.The passage notes simpler control logic, typically no branch prediction or prefetch, and small per-core memory.
  • FPGAs comprise configurable logic, DSPs, on-chip memories, I/O pads, and routing channels rather than a fixed processor-like design.
  • FPGA custom datapaths can stream pixels directly between computing units and keep data local in distributed on-chip memory.This organization can avoid external-memory transfers and exploit locality in vision kernels.

B. Computer Vision Libraries

The paper uses vendor- and platform-oriented vision libraries and organizes computer-vision kernels into six categories. The categories progress from simple input operations to composite workloads with increasing complexity.

  • Computer Vision Libraries: OpenCV is a standard C/C++ library for image and vision processing used in desktop and embedded applications.The passage also notes bindings for Python and Java and identifies version 4.0 as current at writing.
  • Computer Vision Libraries: VisionWorks is an NVIDIA toolkit implementing and extending OpenVX for CUDA-capable GPUs.It provides immediate-mode, graph-mode, and CUDA API programming models.
  • Computer Vision Libraries: xfOpenCV provides OpenCV functions optimized for Zynq and Zynq UltraScale devices and implemented with HLS.It offers a software interface for building FPGA vision pipelines in the SDx environment.
  • Vision Kernel Categories: The six kernel categories are input processing, image arithmetic, image filters, image analysis, geometric transformation, and composite kernels.Kernel complexity grows through the first five categories, while composite kernels combine kernels from earlier categories.
  • Vision Kernel Categories: Input-processing kernels perform simple format or channel changes, including channel operations, color conversion, and bit-depth conversion.
  • Vision Kernel Categories: Image arithmetic applies localized pixel operations that can benefit from highly parallel GPUs and FPGAs.The passage emphasizes limited data dependencies across processing units.
  • Vision Kernel Categories: Image filters compute correlations between images and fixed-size kernels, while nonlinear filters have less regular behavior.Local memory can support parallel distribution when it accommodates the kernel size.
  • Vision Kernel Categories: Image-analysis kernels reduce images to decision variables but include branching and complex memory accesses that negatively affect CPU and GPU performance.

IV. EXPERIMENTAL METHODOLOGY

This section introduces the performance metrics, measurement techniques, and benchmarking approach used in the study.

  • The section describes the performance metrics used to evaluate the hardware platforms.
  • The section presents the measurement techniques used in the experiments.
  • The section introduces the study’s benchmarking approach.

A. Performance Metrics

The study evaluates run-time, energy/frame, and EDP to compare accelerator efficiency while accounting for both speed and energy consumption.

  • Run-time is measured as elapsed kernel execution time using a high-resolution timer.
  • Energy/frame measures electrical energy dissipated to perform one frame’s kernel operations.It is calculated from power consumed during the frame-processing delay.
  • Dynamic power is the power consumed above the static power level while the system computes.Static power is consumed when no active computation takes place.
  • EDP multiplies energy/frame by delay time, enabling comparison of speed–energy trade-offs; lower EDP is better.Run-time or energy/frame alone may not capture practical efficiency.

B. Measurement Techniques and Platforms:

Power measurements compare CPU processing with accelerated FPGA or GPU processing across standardized 1080p grayscale workloads and specified hardware/software environments.

  • Each benchmark processes 1000 frames on the platform CPU followed by 1000 frames on the hardware-accelerated component.Average frame rate is computed from the time between the corresponding measurement boundaries.
  • The evaluated hardware includes an ARM-A57 CPU, Jetson TX2 GPU, and ZCU102 FPGA with different clock rates and memory configurations.The FPGA runs at 300 MHz, ARM-A57 at 1.7 GHz, and GPU at 998.4 MHz.
  • The study uses OpenCV, VisionWorks, and xfOpenCV as publicly available vision libraries.OpenCV-compatible wrappers allow the same OpenCV code to compile for GPU and FPGA.
  • Figure 1 separates power samples for CPU cores in the first 1000 frames from FPGA or GPU samples in the second 1000 frames.

C. Benchmarking Approach

The benchmarking approach favors publicly available, out-of-the-box library kernels and measures accelerator energy while separately examining data-movement costs and GPU implementations.

  • The study evaluates unmodified publicly available kernels to compare accelerator efficiency without platform-specific code around kernel calls.Single kernels are run through OpenCV and VisionWorks on CPU and GPU, and xfOpenCV in FPGA fabric.
  • Single-kernel efficiency is compared using energy consumption per frame and dynamic power, excluding static platform power.This is intended to better reflect the deployed workload, especially for small kernels.
  • Data movers are measured with passthrough kernels that copy image pixels without arithmetic or logical operations.The measurements estimate energy consumed for data movement rather than computation.
  • Figure 2 compares VisionWorks and the OpenCV CUDA module by frame rate and energy/frame, with stereoBM unavailable in VisionWorks publicly.

V. EXPERIMENTAL RESULTS

The experimental results section presents single-kernel benchmarks across six vision categories followed by evaluations of representative complete vision pipelines.

  • The results cover single kernels from six categories and then evaluate representative vision pipelines.

A. Single Kernel Performance:

Single-kernel results vary with algorithm complexity and data-access structure: GPUs lead on simple, highly parallel operations, while FPGAs increasingly excel on filters and irregular kernels. VisionWorks also substantially accelerates CPU OpenCV across kernel categories.

  • Input processing: 1.79× and 1.41× average energy/frame reductions made GPU and FPGA implementations more efficient than the CPU for input processing.The GPU additionally achieved a 2.4× reduction versus the FPGA for bit-depth conversion.
  • Image arithmetic: 4.6× and 7.2× energy/frame reductions made the GPU more efficient than the CPU and FPGA, respectively, for image arithmetic.SIMT suits operations that can be divided into many pieces performing the same computation.
  • Image filters: 1.8× and 7.4× average energy/frame reductions made the FPGA more efficient than the GPU and CPU for image filters.Irregular median and morphological filters are less straightforward for GPUs because they involve sorting, comparisons, and branching.
  • Image analysis: The FPGA reduced image-analysis energy/frame by 1.2× for lookup and histogram kernels and 3.5× for kernels with more branching and complex memory access.The latter group includes integral image, mean/std, and min/max locations.
  • Geometric transformation: The FPGA reduced energy/frame by 1.6× for resize/remap and 2× for affine/perspective warp relative to the GPU.Warp operations require address generation from 2×3 or 3×3 matrices before mapping.
  • Complexity trend: As kernel complexity increases, FPGA performance and energy efficiency improve relative to GPU and CPU, while GPUs scale less well for irregular division, poor locality, conditions, or complex memory access.More complex algorithms can occupy more programmable-logic resources.

B. Complete Vision Pipeline Performance:

The study evaluates four complete embedded-vision pipelines spanning preprocessing, feature extraction, and post-processing. FPGA implementations consume less energy/frame and achieve lower EDP than CPU and GPU implementations, with advantages increasing for more complex pipelines.

  • Evaluated pipelines: Four pipelines were evaluated: background subtraction, color segmentation, stereo block matching, and Harris corner tracking.The pipelines follow common preprocessing, feature-extraction, and post-processing stages.
  • Background subtraction: Background subtraction combines subtraction, Gaussian filtering, thresholding, erosion, and dilation to detect foreground changes in image sequences.Its components are shown in Figure 10.
  • Color segmentation: Color segmentation converts RGB to HSV, applies range thresholding to three channels, and then performs erosion and dilation.Its components are shown in Figure 11.
  • Harris corner tracking: Harris corner tracking takes successive frames, computes Harris corners from the current frame, and outputs tracked corners for the next frame.The pipeline uses five kernels.
  • Stereo block matching: Stereo block matching generates a disparity map from stereo-camera inputs and parameters using stereo rectification, remapping, and local block matching.The pipeline supports creation of a three-dimensional environmental map.
  • Results: The FPGA consumed less energy/frame than the CPU and GPU for all four pipelines and was more efficient in EDP, where lower EDP is better.FPGA energy/frame and EDP reduction relative to the GPU increased with pipeline complexity.
  • Complexity trend: Increasing pipeline complexity improves FPGA relative efficiency because more programmable logic is used and inter-module data remains on-chip in the streaming implementation.Keeping communicated pixels on-chip reduces external-memory traffic.

VI. CONCLUSION

The study benchmarks vision algorithms across OpenVX categories and finds that accelerator energy efficiency depends on kernel and pipeline complexity. GPUs are effective for simple kernels, while FPGAs increasingly outperform GPUs and CPUs on complete, more complex pipelines.

  • The benchmark covers algorithms from all computer vision categories defined by the OpenVX open standard on GPU- and FPGA-accelerated embedded platforms.
  • 1.1–3.2× energy/frame reduction is achieved by GPUs for many simple and easily parallelized kernels.
  • 1.2–22.3× energy/frame reduction is achieved by FPGAs over GPUs and CPUs for more complete vision pipelines.
  • FPGA energy efficiency improves relative to the other platforms as vision-pipeline complexity grows.The comparison uses energy-delay product, which incorporates both energy/frame and algorithm throughput.
Loading 1906.11879v1…