Source-linked AI summary
Digital Electronics and Analog Photonics for Convolutional Neural Networks (DEAP-CNNs)
Viraj Bangari, Bicky A. Marquez, Heidi B. Miller, Alexander N. Tait, Mitchell A. Nahmias, Thomas Ferreira de Lima, Hsuan-Tung Peng, Paul R. Prucnal, Bhavin J. Shastri
TL;DR
CNN convolutions are computationally intensive, motivating hardware that can process them more efficiently. The paper proposes a DEAP CNN architecture combining digital electronics and analog photonics, and estimates faster convolution performance with lower energy use than conventional GPUs while demonstrating MNIST recognition.
Problem
CNN convolutions consume substantial computation, motivating alternatives to GPU-based processing for efficient neural-network inference.
Method
The paper combines digital electronics and analog photonics in a DEAP CNN architecture, mapping a trained two-layer MNIST network onto photonic convolution hardware.
Results
DEAP is estimated to perform convolutions 2.8 to 14 times faster than a GPU while using roughly 0.75 times its energy consumption, and the CNN reaches 98% MNIST accuracy.
Takeaways & Limitations
The results support DEAP as a potentially scalable photonic approach to CNN convolution, with speed increasing linearly alongside energy consumption.
Takeaways & Limitations
Physical implementation still faces challenging high-I/O silicon-photonic/electronic packaging and control-circuitry requirements.
Abstract
from arXiv · showhide
Convolutional Neural Networks (CNNs) are powerful and highly ubiquitous tools for extracting features from large datasets for applications such as computer vision and natural language processing. However, a convolution is a computationally expensive operation in digital electronics. In contrast, neuromorphic photonic systems, which have experienced a recent surge of interest over the last few years, propose higher bandwidth and energy efficiencies for neural network training and inference. Neuromorphic photonics exploits the advantages of optical electronics, including the ease of analog processing, and busing multiple signals on a single waveguide at the speed of light. Here, we propose a Digital Electronic and Analog Photonic (DEAP) CNN hardware architecture that has potential to be 2.8 to 14 times faster while maintaining the same power usage of current state-of-the-art GPUs.
I. INTRODUCTION
CNN convolutions impose substantial computational cost, motivating faster hardware. The paper proposes a DEAP architecture that combines digital electronics with analog photonics and maps CNN processing onto it.
- Over 80% of total processing time is spent on convolution, making forward propagation a primary CNN bottleneck.
- The proposed DEAP architecture performs CNN image recognition using digital electronic and analog photonic components.
- The study trains a standard two-layer CNN offline, uploads its parameters to DEAP, and evaluates forward propagation on MNIST.
- The design extends prior photonic-CNN architecture by addressing input representation, multi-channel summation, multi-dimensional kernels, weight constraints, and kernel or input depth.
- The paper describes photonic devices, develops a hardware-inspired convolution algorithm, builds a two-layer MNIST DEAP CNN, and benchmarks energy and speed against DeepBench.
II.1. Convolutions Background
CNN convolution applies kernels across image channels and can be reformulated as matrix multiplication. This formulation explains both the operation's computational burden and its suitability for parallel hardware.
- With stride one and equal image dimensions, convolution produces an output with spatial dimensionality (H − R + 1) × (H − R + 1).
- Convolution applies an R × R × D kernel across an H × W × D input, combining channel-wise results into output features.
- The operation uses K different kernels, with output dimensionality determined by the kernel count and the convolution parameters.
- 86% to 94% of CNN execution time can be consumed by convolutions, which are computationally intensive operations.
- The convolution can be generalized as a matrix-matrix multiplication by vectorizing the kernel and arranging image patches as matrix columns.
II.2. Silicon Photonics Background
Silicon photonics offers a route to faster convolution processing through optical parallelism and tunable microring resonators. The described devices encode and manipulate analog values using resonance, balanced detection, and amplification.
- Silicon photonics motivates optical alternatives to GPUs, and broadcast-and-weight architectures have demonstrated multiply-accumulate frequencies up to five times faster than conventional electronics.
- An all-pass microring resonator uses a single coupled waveguide, with its transfer behavior determined by interference and phase-dependent resonance.
- Microring resonators tune optical transfer functions by heating the ring with current, shifting its resonance peak and modulating intensity.
- An add-drop configuration uses through and drop ports with balanced photodetection to produce a differential transfer function for optical processing.
- Balanced photodetection represents positive and negative kernel values, while a transimpedance amplifier can amplify the resulting signal by a factor of two.
II.3. Dot Products with Photonics
Photonics computes dot products by wavelength-multiplexing intensity-coded inputs, filtering them with tunable MRRs, and subtracting balanced photodiode outputs. Electronic amplification and bias handling extend the representable weight and input ranges.
- Signal representation: Wavelength-multiplexed electromagnetic waves encode input amplitudes through nonnegative power envelopes, with µ_i restricted to 0 ≤µ_i ≤ 1 when amplification is prevented.The multiplexed signal uses distinct angular frequencies ω_i and slowly varying envelopes.
- Optical weighting: Tunable filters H+(ω) and H−(ω) create weighted optical signals that are sent to a balanced photodiode for photocurrent computation.The balanced photodiode has spectral response R(ω).
- Dot-product formation: The effective filter value is F∗_i = |H+(ω_i)|^2 − |H−(ω_i)|^2, while the input factor is set as A_i = E0R0µ_i.These definitions yield a photocurrent proportional to the weighted sum.
- Hardware implementation: MRR through and drop ports implement the complementary filters, and their balanced-PD outputs perform the matrix multiplication before TIA amplification.The electro-optic architecture is termed a photonic weight bank (PWB).
- Range extension: A TIA gain gTIA accommodates weight vectors whose components exceed the filter range −1 to 1.The gain is applied to the photocurrent and can be manufactured in standard CMOS.
- Signed inputs: Supporting negative inputs introduces a predictable bias current requiring conceptual subtraction and reduces precision because a larger input range must be represented.The additional subtraction circuitry is an optical or electronic design disadvantage.
III. PERFORMING CONVOLUTIONS USING PHOTONICS
DEAP is a photonic architecture designed to perform CNN convolutions. Its supported convolutional parameter range is bounded by maximum input channels and kernel edge length.
- Architecture: DEAP is introduced as a photonic architecture capable of performing convolutions for CNNs.The architecture is intended specifically for CNN convolution operations.
- Supported parameters: The implementation support range is parameterized by maximum input channels Dm and maximum kernel edge length Rm.Parameters without complementary bounding parameters can be supported at arbitrary values.
III.1. Producing a Single Convolved Pixel
A DEAP convolutional unit produces one convolved pixel by encoding a receptive field across wavelengths and channels, performing parallel weighted multiplications, and electrically summing the results.
- Input encoding: A single convolved pixel is represented as the dot product of two 1×R^2 vectors, requiring Rm^2 lasers with unique wavelengths and DmR_m^2 modulators.Each modulator keeps its carrier intensity proportional to a normalized input pixel value.
- Channel mapping: Wavelength division multiplexing combines the laser carriers and splits them into Dm lines, each carrying Rm^2 MRRs for a channel-specific subset of pixels.The architecture uses DmR_m^2 MRRs in total.
- Photonic multiplication: Each PWB produces an electrical signal proportional to the dot product between the kernel weights and the corresponding channel inputs.The MRRs in a PWB contain weights for one kernel channel.
- Output formation: A passive voltage adder sums the weight-bank outputs, producing the value of one convolved pixel.Figure 5 depicts this complete single-pixel architecture.
- Parameter flexibility: Smaller kernels or lower-dimensional kernels are supported by setting unused kernel weights or channel modulators to zero.This allows bounded hardware dimensions to represent smaller convolution configurations.
Performing a Full Convolution
Full convolutions are performed by striding the input image through a DEAP unit while retaining kernel weights, and can be parallelized across multiple convolutional units.
- Convolution cycling: An arbitrary-size convolution cycles through the input image, readjusting the modulation array while leaving the weight banks unchanged until a new kernel is applied.Inputs are zero-padded before processing and strided according to S.
- Hardware-inspired algorithm: Algorithm 1 loads each kernel, iterates over image positions using stride S, performs the convolution, and stores the results in memory block O.The loops span kernel channels and valid spatial positions.
- Parallelization: Creating nconv convolutional-unit instances produces nconv output pixels per cycle by passing the next input set to each unit.Output-pixel computation is distributed across the units.
IV. PHOTONIC CONVOLUTIONAL NEURAL NETWORKS
The DEAP CNN maps convolutional processing onto photonic hardware while retaining digital storage, activation, pooling, and classification stages. A simulated two-layer MNIST implementation achieved 98% accuracy versus 98.6% for a standard CNN.
- CNNs combine convolution, activation, pooling, and fully connected layers, with convolution providing parallel kernel-based feature extraction.Pooling reduces feature dimensions, while fully connected layers perform classification.
- The DEAP simulator uses photonic weight banks and voltage summation to perform channel-wise convolution from trained kernels.The simulator models microring-resonator transfer functions and assumes 7-bit weight-bank control.
- The DEAP architecture stores images, weights, and convolved features digitally, transfers them through DACs, performs convolution on-chip, and returns outputs through ADCs.Activation commands and memory remain associated with the computer-side digital system.
- A 28 × 28 input filtered by two 5 × 5 × 8 kernel groups produces 24 × 24 × 8 and then 20 × 20 × 8 convolved features before pooling.The second-stage output undergoes ReLU and average pooling with a 2×2 kernel whose elements are 1/4.
- 98% accuracy was obtained on 500 MNIST test images, compared with 98.6% for a standard two-layer CNN with max pooling.The authors conclude that the simulator remains sufficiently robust despite 7-bit precision.
V.1. Energy Estimation
The DEAP architecture is evaluated through estimated energy, throughput, and runtime comparisons with GPU convolution benchmarks. Its projected performance depends on photonic and electronic component limits, with precision and physical integration remaining important constraints.
- Energy estimation: 100 W is the estimated energy usage of a single convolution unit when 1024 modulators represent inputs.The two evaluated configurations require 112 W for a large kernel and 95 W for a smaller kernel.
- Throughput: 21 ps is the estimated propagation time for 100 MRRs with a 10 m radius, corresponding to 50 GS/s before detector and TIA bottlenecks.Balanced photodiodes and TIAs are limited to 25 GS/s and 10 GS/s, respectively, while individual MRR modulation reaches 128 GS/s.
- Throughput: 5 GS/s limits DEAP system speed because both DACs and ADCs operate at 5 GS/s and support 7-bit precision.This corresponds to producing one convolved pixel in 200 ps.
- GPU comparison: DEAP is estimated to perform convolutions 2.8 to 14 times faster than conventional GPUs while using roughly 0.75 times their energy consumption.These estimates use DeepBench GPU runtimes and compare configurations with one and two convolutional units.
- Constraints: Higher-precision photonic DACs and ADCs could increase the speedup, while physical implementation still requires high-I/O photonic-electronic packaging and control circuitry.The architecture also faces stochastic photonic noise, ADC quantization noise, and RF distortion.
- Constraints: 7-bit MRR precision is below the 16-bit precision supported by half-precision floating-point representations, making precision a central implementation challenge.The conclusion identifies increasing photonic-component precision as a major challenge for matching classical floating-point representations.