Source-linked AI summary
Photonic tensor cores for machine learning
Mario Miscuglio, Volker J. Sorger
TL;DR
Large neural-network matrix operations motivate alternatives to serial, power-intensive electronic processing. This work introduces a photonic tensor core that combines wavelength-parallel optical MACs with nonvolatile phase-change memory for stored weights. The architecture performs fixed-kernel inference with O(1) runtime complexity and near-zero static power in the reported design.
Problem
Large neural networks require high-bandwidth, low-latency matrix multiplication, while electronic accelerators can remain power-hungry and incur memory-access overhead.
Method
The paper implements a photonic tensor core using WDM-parallel dot-product engines, phase-change weight memories, microring filtering, and photodetection.
Results
The photonic tensor core performs fixed-kernel inference with O(1) runtime complexity and near-zero static power consumption.
Takeaways & Limitations
The design provides a pathway toward passive photonic tensor processors for optical-domain inference at network edges without repeated external-memory access.
Abstract
from arXiv · showhide
With an ongoing trend in computing hardware towards increased heterogeneity, domain-specific co-processors are emerging as alternatives to centralized paradigms. The tensor core unit (TPU) has shown to outperform graphic process units by almost 3-orders of magnitude enabled by higher signal throughout and energy efficiency. In this context, photons bear a number of synergistic physical properties while phase-change materials allow for local nonvolatile mnemonic functionality in these emerging distributed non van-Neumann architectures. While several photonic neural network designs have been explored, a photonic TPU to perform matrix vector multiplication and summation is yet outstanding. Here we introduced an integrated photonics-based TPU by strategically utilizing a) photonic parallelism via wavelength division multiplexing, b) high 2 Peta-operations-per second throughputs enabled by 10s of picosecond-short delays from optoelectronics and compact photonic integrated circuitry, and c) zero power-consuming novel photonic multi-state memories based on phase-change materials featuring vanishing losses in the amorphous state. Combining these physical synergies of material, function, and system, we show that the performance of this 8-bit photonic TPU can be 2-3 orders higher compared to an electrical TPU whilst featuring similar chip areas. This work shows that photonic specialized processors have the potential to augment electronic systems and may perform exceptionally well in network-edge devices in the looming 5G networks and beyond.
I. Introduction
Deep neural networks depend on large, low-latency matrix operations, while conventional processors face serial execution, memory-access overhead, and substantial power demands. The paper proposes a photonic tensor core using parallel optical computation and on-chip nonvolatile weights.
- Deep neural networks rely heavily on large matrix multiplications that require high bandwidth and low latency.
- GPUs and TPUs improve matrix processing but remain power-hungry and can incur substantial inference latency, including for smaller tasks.
- Photonic computing is explored as a massively parallel, distributed alternative that can exploit optical signals and reduce power consumption.
- The proposed photonic tensor core performs 4x4 matrix multiplication and accumulation in one shot using trained weights stored in on-chip 4-bit multilevel photonic memory.
- The work presents a first step toward photonic tensor processors intended to scale MAC operations while suppressing power consumption and latency relative to existing accelerators.
II.1 Matrix multiplication algorithms
Matrix multiplication retains high operation complexity even with optimized algorithms, but tensor-core parallelism reduces execution-time scaling by performing many operations concurrently. The paper maps this principle onto photonic hardware with photon-limited latency.
- Naïve matrix multiplication requires O(n3) operations, while Strassen and Winograd reduce this to O(n2.373) without eliminating scaling with matrix size.
- Tensor cores retain O(n3) total operations but can achieve O(n2) operational time through parallel and systolic execution.
- A conventional tensor core computes D = A×B + C on 4x4 matrices, with FP16 multiplication inputs and FP16 or FP32 accumulation matrices.
- The proposed silicon-photonic tensor core targets latency determined mainly by optical time of flight and detection, below tens of picoseconds in modern photoreceivers.
- Integrated photonics can perform fixed-kernel linear operations noniteratively with O(1) time complexity when sufficient parallel wavelengths and hardware are available.
II.2 Photonic Tensor Core architecture
The photonic tensor core combines 16 wavelength-parallel dot-product engines with passive phase-change weighting and photodetection. This architecture performs matrix multiplication and accumulation through optical filtering, modulation, and summation.
- The photonic tensor core contains 16 independent dot-product engines that perform row-by-column multiplication and accumulation.
- Wavelength-division multiplexing parallelizes element-wise multiplication across the input vector and stored kernel weights.
- Microring resonators spectrally select channels, while phase-change memories encode kernel weights through quantized optical attenuation.
- A photodetector incoherently sums the weighted optical signals, producing each matrix-accumulation output D_ij.
- Passive microring filtering and nonvolatile multistate memories reduce active tuning, retain weights without static power, and avoid considerable additional loss.
II.3 Photonic Memories
The proposed photonic memory uses Ge2Sb2Se5 phase-change wires to encode multilevel weights through reversible electrothermal switching. Simulations indicate compact 4-bit storage with low insertion loss and quantized optical modulation.
- Ge2Sb2Se5 provides a transparent amorphous-state platform for nonvolatile multistate photonic memories at telecommunication wavelengths.
- Programming uses different thermal regimes: crystallization occurs near 523 K, whereas amorphization requires temperatures above 900 K.
- Electrothermal Joule heating reversibly switches each memory element between amorphous and crystalline phases using tungsten electrodes.
- A 4-bit memory uses 16 states encoded by 30 nm-thin, 250 nm-wide PCM wires arranged along the waveguide.
- ~1 dB insertion loss is obtained for the 4-bit multilevel memory in the all-amorphous state.
- A 4-bit memory provides a 0.2 dB/bit quantization step and about 3.5 dB maximum extinction ratio.
II.4 Performances
The photonic tensor core performs 4-bit matrix multiplication passively and achieves substantial throughput advantages over electronic tensor cores within a single-die area.
- 4-bit precision enables passive matrix multiplication once weights are stored in the photonic network.Inference avoids logic architecture and off-chip memory transduction.
- ~65 ps latency processes a 4x4 multiplication containing 64 MACs, yielding 0.5-2 POPS/s across approximately 250 units.The estimate assumes a maximum die area of 800 mm2.
- 2-8x throughput improvement is reported over NVIDIA’s T4 and A100 for electronic data-fed operation, rising to ~60x for optical data.The comparison is limited to a single die of approximately 800 mm2.
III. CONCLUSION
The proposed photonic tensor core combines wavelength-multiplexed signals with phase-change photonic memories for passive, constant-time tensor processing. Once the kernel is set, execution is limited by photon flight time and detector latency, supporting faster edge-network processing without external-memory access or EO conversions.
- WDM signals are weighted through engineered multistate Ge2Sb2Se5 photonic memories patterned on waveguides.Memory states are reprogrammed through electrothermal switching between amorphous and crystalline phases.
- Pre-set kernels allow passive operation with no dynamic or static power dissipation and runtime complexity O(1).The architecture avoids additional optical losses and external memory access during operation.
- Once the kernel is set, execution time is limited by on-chip photon flight time and photodetector latency O(<10-1ns).The limit depends on ring size, wavelength selectivity, and detector latency.
- Advancing phase-change materials and photonic-memory integration could enable full-precision floating-point matrix multiplication and accumulation.The authors connect this prospective capability to all-optical tensor units for edge-network tasks.