PaperlayerView exact revision on arXiv

Source-linked AI summary

Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications

Mostafa Rahimi Azghadi, Corey Lammie, Jason K. Eshraghian, Melika Payvand, Elisa Donati, Bernabe Linares-Barranco, Giacomo Indiveri

arXiv:2007.05657v2cs.ARcs.LGeess.SP

TL;DR

Healthcare edge systems need efficient hardware because deep-learning workloads are computationally and memory intensive, while devices must remain compact, low-power, rapid, and affordable. The paper tutorials and compares CMOS, FPGA, memristive, DNN, and neuromorphic approaches, including a multimodal EMG–vision benchmark. It reports that spiking multimodal fusion improved accuracy by approximately 4%, while accelerator specialization trades generality for optimized processing.

  • Problem

    Healthcare edge AI requires efficient hardware for computationally intensive DNN workloads under compactness, power, throughput, cost, and deployment constraints.

  • Method

    The paper reviews hardware accelerators and neuromorphic processors, provides implementation tutorials, and benchmarks platforms on multimodal EMG and visual hand-gesture recognition.

  • Results

    Approximately 4% accuracy improvement occurred when EMG and DVS spiking signals were fused, whereas mismatched non-spiking fusion produced marginal or destructive improvements.

  • Takeaways & Limitations

    Neuromorphic processors may complement DL inference engines for edge biomedical processing, while more specialized hardware improves selected-task processing at the expense of generalizability.

Abstract

from arXiv · show

The advent of dedicated Deep Learning (DL) accelerators and neuromorphic processors has brought on new opportunities for applying both Deep and Spiking Neural Network (SNN) algorithms to healthcare and biomedical applications at the edge. This can facilitate the advancement of medical Internet of Things (IoT) systems and Point of Care (PoC) devices. In this paper, we provide a tutorial describing how various technologies including emerging memristive devices, Field Programmable Gate Arrays (FPGAs), and Complementary Metal Oxide Semiconductor (CMOS) can be used to develop efficient DL accelerators to solve a wide variety of diagnostic, pattern recognition, and signal processing problems in healthcare. Furthermore, we explore how spiking neuromorphic processors can complement their DL counterparts for processing biomedical signals. The tutorial is augmented with case studies of the vast literature on neural network and neuromorphic hardware as applied to the healthcare domain. We benchmark various hardware platforms by performing a sensor fusion signal processing task combining electromyography (EMG) signals with computer vision. Comparisons are made between dedicated neuromorphic processors and embedded AI accelerators in terms of inference latency and energy. Finally, we provide our analysis of the field and share a perspective on the advantages, disadvantages, challenges, and opportunities that various accelerators and neuromorphic processors introduce to healthcare and biomedical domains.

I. INTRODUCTION

Healthcare increasingly uses DL for monitoring, prediction, diagnosis, treatment, and prognosis, but edge deployment requires compact, low-power, rapid, and affordable hardware. This tutorial reviews CMOS, memristive, and FPGA accelerators and considers neuromorphic processors as complements for biomedical edge processing.

  • Healthcare motivation: DL systems process heterogeneous patient data for monitoring, prediction, diagnosis, treatment, and prognosis.Inputs may include biosamples, medical images, temperature, and movement.
  • Hardware gap: DL compute loads have doubled every 3.4 months since 2012, outpacing Moore’s Law’s 18–24-month doubling target.The paper identifies this mismatch as a challenge for dedicated hardware development.
  • Edge requirements: Healthcare edge processing must combine compactness, low power, rapid throughput, and low cost for viable smart monitoring.Edge processing also moves sensitive medical-data processing away from the cloud and supports offline operation.
  • Edge requirements: Embedded accelerators show promise but remain relatively power hungry and costly, while some algorithms exceed resource-constrained devices’ memory bandwidth.The paper argues that innovative systems are still needed for affordable large-scale healthcare edge deployment.
  • Paper scope: The tutorial reviews CMOS, memristors, and FPGAs for biomedical DL acceleration and examines how event-based neuromorphic processors can complement DL accelerators.It includes implementation guidance, literature review, benchmarking, and a perspective on future biomedical processing.

II. DEEP ARTIFICIAL AND SPIKING NEURAL NETWORKS

The paper distinguishes deep artificial neural networks from spiking neural networks and summarizes common architectures for different biomedical data structures. MLPs, CNNs, RNNs, and LSTMs provide different mechanisms for extracting features from cross-sectional, spatial, and sequential inputs.

  • Terminology: In this paper, “deep” refers to Deep Artificial Neural Networks, whereas Deep Spiking Neural Networks are called SNNs.DNN denotes networks with more than one hidden layer regardless of architecture, while the paper reserves DL terminology for deep ANNs.
  • ANN architectures: MLPs use layered perceptron cells for hierarchical feature extraction, while CNNs add spatial filtering and pooling for spatial invariance and downsampling.These architectures are used in biomedical tasks including seizure detection and medical imaging.
  • ANN architectures: RNNs use feedback loops to retain information from previous steps, making them suitable for sequential signals such as ECG and medical text.LSTMs are a well-known RNN type designed to identify patterns in data sequences.

1) Automatic hierarchical feature extraction:

DNNs automatically learn hierarchical features by optimizing network parameters through loss minimization and backpropagation. This process is computationally expensive because repeated vector–matrix multiplications and memory operations dominate training and inference costs, motivating transfer learning and accelerator design.

  • Automatic hierarchical feature extraction: DNNs combine hierarchies of simpler learned features to extract intricate representations without manually specified human knowledge.Feature learning is performed across multiple computational layers and abstraction levels.
  • Backpropagation: Training minimizes a loss function, commonly cross-entropy, by optimizing weights and biases through backpropagation.Optimization proceeds backward through the network using methods such as stochastic gradient descent and adaptive-learning-rate algorithms.
  • Backpropagation in DNNs is computationally expensive: O(N^2) time and power complexity arises when a length-N vector is multiplied by an N × N matrix on a conventional von Neumann architecture.DNN training repeats computationally intensive VMM operations across thousands of epochs and huge datasets.
  • Transfer learning: Transfer learning adapts existing trained parameters when training and test distributions differ, reducing the expense of training from scratch.In medical imaging, pretrained ImageNet weights are used to speed convergence and improve accuracy, including for patient-specific tuning.

C. DL Accelerators

DL accelerators target the computational and memory demands of neural networks, while SNNs process temporal events efficiently and can interface with event-based sensors. The paper benchmarks FPGA, memristive, DNN, and neuromorphic approaches on multimodal hand-gesture recognition using EMG and visual data.

  • DL Accelerators: Accelerator design seeks massive parallelism and reduced memory access to enable fast, low-power healthcare IoT and PoC devices.These goals address the dominant MAC and memory-read costs of DNNs.
  • D. Spiking Neural Networks: The paper presents SNNs and neuromorphic processors as complementary options for efficient biomedical signal processing alongside conventional DL accelerators.Its perspective emphasizes event-based processing and edge-oriented hardware choices.
  • D. Spiking Neural Networks: SNNs process all-or-none spikes over time, encode sequential inputs temporally, and process event-based sensor outputs as events are generated.Dynamic Vision Sensors produce ON/OFF spikes from changes in visual input.
  • Benchmark task: The benchmark uses a five-gesture dataset combining forearm EMG from a Myo armband with DVS events and conventional camera video.The task supports comparisons among FPGA, memristive, DNN-accelerator, and neuromorphic implementations.
  • Benchmark task: Inference comparisons report accuracy across 3-fold session-based cross-validation together with energy, inference time, and Energy-Delay Product.EDP is defined as average energy consumption multiplied by average inference time.

III. DNN ACCELERATORS TOWARDS HEALTHCARE AND BIOMEDICAL APPLICATIONS

The paper surveys CMOS, FPGA, and memristor accelerators for biomedical edge applications, including DNN processing of multimodal signals and healthcare data. It also contrasts synchronous DNN pipelines with asynchronous SNN neuromorphic processing.

  • Edge-AI accelerator platforms: CMOS edge-AI chips target portable healthcare applications involving image-based data and sequential signals such as EEG and ECG.Examples include medical imaging, monitoring, diagnosis, and biomedical signal analysis.
  • DNN and SNN operation models: DNNs process digitally sampled inputs in serially propagated batches, whereas SNN processors asynchronously process temporally encoded signals independently.EMG and photographic inputs can be represented either as digital frames or as spike trains for the corresponding processor type.
  • Edge-AI accelerator platforms: Most listed CMOS accelerators consume hundreds of mW per chip, while a few consume around 10 Watts.Their computational capability is reported in GOP/s and relates to the size of networks they can support.
  • Biomedical applications: Some CMOS chips accelerate both CNNs and RNNs, broadening their suitability for sequential biomedical applications.Examples include ECG analysis, MRI construction, medical consultation, respiratory-sound classification, and personalized prediction.
  • Biomedical applications: General-purpose accelerators support applications including skin cancer detection, fall detection, chest X-ray analysis, bowel-sound monitoring, and ECG arrhythmia detection.These examples use CNN, LSTM, or related neural architectures on portable or edge platforms.

2) Common approaches to CMOS-driven DL acceleration:

CMOS acceleration primarily improves throughput and efficiency through architectural dataflow choices, reduced precision, parallel processing, and reduced data movement. These principles appear across specialized processors, tile-based systems, tensor cores, and systolic arrays, while FPGA implementations add reconfigurability and high-level synthesis.

  • Edge constraints: Edge accelerators must operate within restrictive power budgets, with thermal limits such as 5 W imposed by portable battery constraints.Data-center and edge platforms differ in scale and clock speed but optimize related computational tasks.
  • CMOS optimization strategies: Reduced-precision arithmetic is commonly combined with in- or near-memory computing and heightened parallelism to improve throughput.These strategies target lower data movement and more efficient neural-network execution.
  • Parallel architectures: Specialized processors commonly use multiple sub-processing elements to increase overall throughput.Examples include neuFlow processing tiles and NPUs that compute neuron responses through multiplication, accumulation, and activation.
  • Parallel architectures: Tensor cores combine GPU parallelism with application-specific mixed-precision computing.The NVIDIA Tesla V100 is described as combining 672 tensor cores in one unit.
  • Parallel architectures: Systolic arrays are a prominent technique for achieving MAC parallelism in specialized processors.The passage identifies their use by Groq, Google, and other chip developers.
  • Reconfigurable hardware: FPGAs provide low-cost reconfigurability and parallel computation, while HLS tools synthesize C-like descriptions into RTL.This supports tasks requiring simultaneous processing, such as multiple ECG channels.

1) Accelerating DNNs on FPGAs:

The FPGA case study demonstrates DNN acceleration for biomedical signal processing using fixed-point representations and OpenCL on an OpenVINO FPGA platform. FPGA-based designs can be energy-efficient, but their development requires specialized engineering expertise.

  • FPGA implementation: The benchmark deploys a DNN using fixed-point parameter representations on an OpenVINO FPGA platform with OpenCL.OpenCL can pipeline and execute work items in parallel or sequentially depending on available computation resources.
  • FPGA implementation: The compilation flow converts a trained PyTorch model through Caffe, fixed-point weights, PipeCNN-generated RTL libraries, and an FPGA bitstream.Weights and biases are exported as a binary .dat file before integration and compilation.
  • Biomedical applications: Biomedical FPGA accelerators have targeted ECG anomaly detection, cancer detection from mass spectrometry, and brain-computer interfaces.Application-specific quantization and network design are reported to improve inference speed relative to CPU and GPU baselines.
  • Efficiency and limitations: FPGAs are reported as projected to provide >10 times energy-delay efficiency compared with state-of-the-art GPUs for DL acceleration.The projection depends on algorithmic design and hardware-software co-design techniques.
  • Efficiency and limitations: FPGA design is less straightforward than high-level DL accelerator design and requires skilled engineers and stronger tools.This creates a practical development constraint despite the projected efficiency advantage.

2) Mapping memristive crossbars to DNN layers:

Memristive crossbars map neural-network weights to conductances so that analog MAC and VMM operations occur in parallel and near memory. The paper demonstrates pretrained-DNN conversion and variability simulation, while emphasizing that device nonidealities can degrade accuracy and add system complexity.

  • Mapping DNN layers: Convolutional operations are transformed into MAC operations through unrolling so they can be implemented as VMMs on memristive crossbars.This mapping is required because fully connected layers map more directly to crossbar weights and input voltages.
  • Device nonidealities: Experimental memristor imperfections can significantly degrade performance when crossbars scale to real-world healthcare DNNs.Mitigation circuitry and system-level techniques reduce these effects but add computation time and system complexity.
  • Mapping DNN layers: Pretrained PyTorch linear and convolutional layers are converted into memristive equivalents using 1T1R crossbars and double-column positive-negative weight representation.Converted models are tuned using linear regression.
  • Variability modeling: The benchmark models device variability with a VTEAM-based RRAM model and sampled RON and ROFF distributions.The example uses mean ROFF = 2,500Ω and mean RON = 100Ω with specified variability.
  • Device nonidealities: Increasing device-to-device variability increases performance degradation across the converted memristive MLP and CNN networks.The simulations vary σ from 0 to 500 while comparing mean test-set accuracy across three folds.

6) Memristive DNNs towards biomedical applications:

Memristive DNNs offer potential energy and delay advantages, but biomedical implementations remain sparse and largely simulation-based. Existing hardware demonstrations have generally been limited to simple tasks rather than practical large-scale biomedical processing.

  • Biomedical MDNN research includes small-scale simulations for arrhythmia classification and a physical programmable memristive array for breast cancer diagnosis.
  • Practical biomedical processing remains unrealized at large scale, even at the simulation level.
  • Hardware MDNN implementations have shown significant energy savings versus state-of-the-art GPUs, but have performed only simple MNIST and CIFAR classification tasks.
  • Quantized and binarized MDNN simulations show potential energy savings over full-precision MDNNs in biomedical ECG processing, but full hardware design is needed for verification.
  • The paper reviews memristive, FPGA, and CMOS technologies and analyzes their roles in healthcare and biomedical edge inference.

A. CMOS Technology Has Been the Main Player for DL Inference in the Biomedical Domain

CMOS remains the dominant basis for DL inference engines, while neuromorphic processors provide complementary possibilities for real-time, low-power biomedical signal processing. The section also highlights unresolved challenges in programmability, online learning, and deployment safety.

  • A. CMOS Technology Has Been the Main Player for DL Inference in the Biomedical Domain: Current non-GPU DL inference engines are implemented in CMOS, but deployment for biomedical applications faces power, generalization, bias, liability, and workflow obstacles.
  • A. CMOS Technology Has Been the Main Player for DL Inference in the Biomedical Domain: Hardware-software co-design is required to make accelerators readily programmable for general and complex biomedical tasks.
  • B. Towards Edge Processing for Biomedical Applications With Neuromorphic Processors: Always-on medical monitoring requires orders-of-magnitude better power efficiency, motivating SNN-based neuromorphic processors.
  • B. Towards Edge Processing for Biomedical Applications With Neuromorphic Processors: Spiking RNNs temporally integrate ECG and EMG patterns, with reported anomaly-detection accuracies of 91% and 95%.
  • B. Towards Edge Processing for Biomedical Applications With Neuromorphic Processors: For hand-gesture recognition, SVM on spiking hidden-layer outputs reached 84%, while a spiking learning method reached 74% at about 0.05 mW.
  • B. Towards Edge Processing for Biomedical Applications With Neuromorphic Processors: Efficient on-chip online learning remains unresolved because weight updates require local information and suitable weight storage.

C. Why Is the Use of MDNNs Very Limited in the Biomedical Domain?

Biomedical MDNN use is limited by sparse memristive implementations and hardware constraints across FPGA and analog-array platforms. FPGAs offer practical power and development advantages, but memory and bandwidth limitations restrict complex architectures.

  • General-programmable memristive-CMOS biomedical implementations are very few, with most other memristive designs limited to simulations of crossbars.
  • FPGAs support simple MLPs and some LSTMs and CNNs, but limited on-chip memory and bandwidth constrain complex DL architectures.
  • Customized FPGA blocks for EEG processing produced almost an order of magnitude greater power efficiency than GPUs.
  • A customized FPGA design can be synthesized into ASIC CMOS, with one reported implementation achieving near 100 times the energy efficiency of its FPGA counterpart.
  • FPGAs are appealing for biomedical edge devices because low power and affordable cost are especially important for wearables, monitoring systems, and PoC devices.
  • FPAAs have biomedical applications, but an FPAA DNN accelerator for biomedical and general use remains unexplored.

E. Benchmarking EMG Processing Across Multiple DNN and SNN Hardware Platforms

The benchmark compares conventional DNN and spiking neuromorphic implementations for single-sensor and fused EMG–vision hand-gesture recognition. It shows efficiency advantages for spiking hardware, while sensor choice and architecture specificity affect accuracy, generality, area, and power.

  • Benchmark setup: The benchmark evaluates Loihi, ODIN+MorphIC, embedded GPU, FPGA, and memristive implementations on EMG-only and EMG–vision hand-gesture recognition.The task uses five sign-language gestures, with spiking vision and EMG streams fused at the last layer.
  • Hardware trade-offs: Moving from GPU to FPGA to memristive networks improves MLP and CNN processing by orders of magnitude, but reduces task generality and exposes endurance, ADC-power, and area constraints.Memristor calculations are network-specific; duplicated weights increase parallelism but can impose prohibitive ADC power for deep computer-vision networks.
  • Efficiency results: Approximately two orders of magnitude improvement in EDP is reported for spike-based hardware compared with GPU and FPGA counterparts.The result supports prospective use of spiking architectures for always-on monitoring.
  • Accuracy results: Approximately 10% lower classification accuracy occurs for EMG-only Loihi and ODIN+MorphIC implementations than for GPU and FPGA counterparts.The paper suggests immature spike-based training or discarded subthreshold information as possible explanations.
  • Sensor fusion: Approximately 4% accuracy improvement occurs when EMG and DVS data are fused as spiking signals, whereas mismatched non-spiking representations yield marginal or destructive improvements.The paper links this pattern to the potential benefit of consistent data representations across sensory mechanisms.

F. Deep Network Accelerators and Patient-specific Model Tuning

Patient-specific variability motivates individualized model tuning, including online adaptation on edge accelerators. The paper presents accelerators as potential complements to DNNs for hardware efficiency, while stressing that hardware progress alone is insufficient for medical-AI integration.

  • Patient-specific modeling: Patient-specific variability makes it difficult to train and deploy one model across individuals with unique signatures.Patient-specific modeling develops computational models of pathophysiology from individualized patient data.
  • Patient-specific modeling: Transfer learning and tuning can retrain parts of a broadly trained ANN or neuromorphic model using patient-specific data.The initial model learns domain-specific features from data across various patients before individual tuning.
  • Patient-specific modeling: Online tuning on edge hardware can individualize an initial generic model while addressing concerns about sensitive patient data.The accelerator gathers sufficient patient data over time and adapts its operation to patient-specific needs.
  • Patient-specific modeling: Offline off-chip transfer learning may require prior measurements and remote storage or processing of private patient data.These requirements distinguish offline tuning from online adaptation.
  • Conclusion: SNNs may complement DNNs when hardware efficiency limits widespread integration, but hardware progress is necessary rather than sufficient for successful medical-AI integration.The paper frames adoption as requiring coordinated attention from healthcare, engineering, data, policy, neuroscience, and materials disciplines.
Loading 2007.05657v2…