Superconducting optoelectronic circuits for neuromorphic computing

Jeffrey M. Shainline; Sonia M. Buckley; Richard P. Mirin; Sae Woo Nam

Source-linked AI summary

Superconducting optoelectronic circuits for neuromorphic computing

Jeffrey M. Shainline, Sonia M. Buckley, Richard P. Mirin, Sae Woo Nam

arXiv:1610.00053v2Sep 30, 2016cs.NEcond-mat.supr-conphysics.optics

TL;DR

The paper addresses the computational cost of implementing large, highly connected neural networks in software and the need for scalable hardware. It proposes a hybrid semiconductor-superconductor platform using photonic spiking neurons, optical waveguides, and variable synaptic connections. The platform is predicted to achieve 20 aJ/synapse event, while its information analysis is device-level rather than system-level.

Problem
Implementing complex neural networks in software is computationally expensive, motivating hardware with many neurons and connections for large-scale information processing.
Method
The platform combines semiconductor light emitters, superconducting nanowire detectors, superconducting electronics, optical waveguides, and reconfigurable coupling mechanisms to implement spiking neural networks.
Results
20 aJ/synapse event is predicted for the optoelectronic platform, compared with approximately 20 pJ/synapse event for many CMOS systems.
Takeaways & Limitations
Photonic interconnects and superconducting circuits provide a proposed route toward neuromorphic systems with massive interconnectivity and high information capacity.
Takeaways & Limitations
The information measure described by Equation G1 is at the device level rather than the system level.

Abstract

from arXiv · show

Neural networks have proven effective for solving many difficult computational problems. Implementing complex neural networks in software is very computationally expensive. To explore the limits of information processing, it will be necessary to implement new hardware platforms with large numbers of neurons, each with a large number of connections to other neurons. Here we propose a hybrid semiconductor-superconductor hardware platform for the implementation of neural networks and large-scale neuromorphic computing. The platform combines semiconducting few-photon light-emitting diodes with superconducting-nanowire single-photon detectors to behave as spiking neurons. These processing units are connected via a network of optical waveguides, and variable weights of connection can be implemented using several approaches. The use of light as a signaling mechanism overcomes fanout and parasitic constraints on electrical signals while simultaneously introducing physical degrees of freedom which can be employed for computation. The use of supercurrents achieves the low power density necessary to scale to systems with enormous entropy. The proposed processing units can operate at speeds of at least $20$ MHz with fully asynchronous activity, light-speed-limited latency, and power densities on the order of 1 mW/cm$^2$ for neurons with 700 connections operating at full speed at 2 K. The processing units achieve an energy efficiency of $\approx 20$ aJ per synapse event. By leveraging multilayer photonics with deposited waveguides and superconductors with feature sizes $>$ 100 nm, this approach could scale to systems with massive interconnectivity and complexity for advanced computing as well as explorations of information processing capacity in systems with an enormous number of information-bearing microstates.

I. INTRODUCTION

The paper motivates a neuromorphic hardware platform that combines photonic communication with superconducting circuits to scale neural networks in width, connectivity, and information capacity. It proposes semiconductor light emitters, superconducting detectors, optical waveguides, and reconfigurable synapses as the core components.

Neural-network expressivity scales as k^mn, motivating increases in both network width and depth.Here, m is input dimension, n is the number of hidden layers, and k is the number of nodes per layer.
Spike-based hardware should maximize processing units, connections, and temporal utilization to support high information-processing capacity.Pulse communication is described as advantageous for noise resilience and temporally encoded information.
Photonic fanout is proposed to overcome electronic fanout limits, while superconducting circuits are proposed to reduce power density.Together, these choices target highly scaled systems with many processing units and high total complexity.
The platform uses waveguide-integrated semiconductor emitters, superconducting detectors, electronics, and reconfigurable nanophotonic waveguides for weighted directed networks.Light also provides access to frequency, polarization, mode, intensity, statistics, and coherence as physical degrees of freedom.
Synaptic memory can use branching waveguides, electro-mechanically actuated waveguide couplers, magnetic Josephson junctions, or other magnetic and flux-storage components.The discussed suspended-waveguide approach is reconfigurable on a 1 µs timescale and draws no steady-state power.
The proposed optoelectronic platform is predicted to achieve 20 aJ/synapse event, compared with approximately 20 pJ/synapse event for many CMOS systems.The architecture focuses on faint-light sources and superconducting nanowire single-photon detectors, whose cryogenic cost is weighed against lower operating energy.

B. Integrate-and-fire circuit

The integrate-and-fire circuit uses photon absorption and superconducting-to-normal transitions to accumulate inputs until a threshold redirects current into an LED, producing a spike. Its threshold and photon-number resolution can be tuned through bias current and nanowire-array size, while the spiderweb geometry supports many connections.

PND operation: Photon absorption or excess current drives nanowires normal, redirecting current to a parallel LED and producing a firing event.The circuit resets when the PND returns to the superconducting state and LED current ceases.
PND operation: A single SNSPD can provide near-unity single-photon firing, while the PND extends integration-and-fire behavior to higher photon thresholds.The threshold is set by the number of nanowires and the array bias current.
Threshold simulations: Monte Carlo simulations estimate spike probability versus incident photon number across bias currents for PND arrays with 10, 20, and 40 nanowires.The simulations average 1,000 trials and model stochastic photon absorption.
Threshold simulations: Increasing nanowire count improves photon-number resolution: 10-wire arrays produce discrete bands, whereas 40-wire arrays yield approximately continuous threshold tuning.Bias current changes the firing-function shape, but integer nanowire counts discretize threshold control.
Scalable geometry: The PND state space scales as 2^Nnw, and the spiderweb neuron combines inputs on one waveguide passing repeatedly through many SNSPDs.The detector portion can be as small as 10µm × 10µm, while a 1,000-photon threshold gives an approximately 35µm × 35µm device.

C. Differentiable response circuit

The differentiable-response circuit uses a series nanowire detector whose resistance grows with absorbed photons, driving a nonlinear LED response rather than the PND’s step response. Its response can be tuned through nanowire design, bias, and amplification for hundreds or thousands of connections.

SND operation: The SND uses a continuous nanowire with photon-induced hotspots that redistribute current between a detector branch and a parallel LED.This produces a differentiable nonlinear response rather than a discrete PND firing step.
SND response: SND resistance increases with absorbed photons, then saturates when the absorbing region is fully driven normal.The LED output shows exponential turn-on followed by flattening, with response depending on refractory period.
SND response: Critical currents of 4 µA and 8 µA demonstrate hardware tuning of the photon input-output response.The LED efficiencies associated with these designs are η = 1% and η = 0.1%, respectively.
SND response: In the SND, detector resistance and applied bias determine LED voltage and photon output, unlike the PND’s bias-controlled step response.The LED design and circuit resistance jointly determine the generated photon number.
Scaling and amplification: The SND supports hundreds or thousands of connections, while an nTron amplifier decouples receiver bias from emitted photon number and expands the encoding state space.The amplifier can provide gain to the light emitter and allow I2 to be less than I1.

E. Other neuromorphic circuits

The paper presents circuit variants that add stopping, inhibitory, self-feedback, and upstream-feedback behaviors to superconducting optoelectronic neurons, then estimates their energy consumption. The modeled energy reaches 20 aJ per photon for a 1% efficient LED and 100 aJ per synapse event for an SND design.

Circuit variants: Integrate-and-stop-firing circuits quench LED-driven activity after a firing event, enabling stimulation until a specified activity level is reached.
Circuit variants: Inhibitory configurations let photons or an upper neuron suppress downstream firing, while a series connection moderates group activity.
Circuit variants: Self-feedback uses an LED power tap to direct emitted light back to the receiver and quench the receiver current when activity rises.
Energy consumption: 20 aJ/photon is achieved with a 1% efficient LED at larger photon numbers, while an SND circuit reaches 100 aJ/synapse event under specified operating conditions.The SND estimate assumes 0.6Ic drive, 10^3 received photons, a 50 ns hotspot recovery time, and a 1% efficient LED.
Energy consumption: For a 10% efficient LED, inductance and photon production contribute nearly equally near 100 photons, whereas capacitor charging dominates at low photon numbers.Capacitive energy remains nearly constant and becomes negligible at larger photon numbers.

G. Electrically-injected light source

The paper evaluates electrically injected light sources for scalable neuromorphic photonics, including III-V, hybrid, and silicon emissive-center implementations. A silicon approach uses lithographically localized emitters in a ridge-waveguide p-i-n LED, but integrating very large numbers of emitters remains challenging.

Source platforms: The platform targets photonic-electronic integration and operation efficiencies around 10%, with III-V semiconductors such as GaAs and InP offering attainable efficiency.
Source platforms: III-V sources can be electrically injected and coupled to high-index waveguides, or connected to deposited low-loss a-Si or SiN waveguides.
Source platforms: Hybrid III-V/silicon integration includes direct mounting, wafer bonding, and III-V growth on silicon, but the proposed architecture requires a separate source for each neuron.
Scope boundary: Hetero-epitaxy may be limited by the significant cost and difficulty of growing the required materials.
Silicon emissive centers: Silicon emissive centers can be fabricated in CMOS-compatible processes, but the principal challenge is integrating billions of emitters in one system.
Silicon emissive centers: A monolithically integrated silicon LED places emitters in the intrinsic region of a ridge-waveguide p-i-n junction, requiring lithographic localization to avoid intolerable waveguide loss.

H. Summary

The platform connects superconducting optoelectronic neuron circuits through optical waveguides that collect and distribute signals across many connections. Proposed dendritic designs and multilayer photonics address the challenge of scaling interconnectivity.

Connectivity: Optical waveguides connect SPON processing units while allowing neurons to integrate signals from many sources without time-multiplexing.Each neuron can transmit through branching axon waveguides and receive through integrating dendritic waveguides.
Dendritic arbor: The dendritic arbor combines optical signals from many upstream neurons onto photon detectors with low loss.Two proposed approaches explore this functionality for optoelectronic neurons.
Dendritic arbor: ≈40 upstream neurons can be connected compactly using the spiderweb dendritic arbor and receiver design.The design uses multimode waveguides to combine many single-mode inputs.
Dendritic arbor: A stingray SPON directly combines input waveguides on a detector-array landing pad and is better suited to larger input counts.The design uses an array of sine bends after the input ports.
Scaling: Multilayer photonics could route dendritic arbors across several or tens of photonic and superconducting layers for future massive interconnectivity.The authors identify networks supporting tens to hundreds of connections as a current technical challenge.

B. The axon and its arborization

The platform distributes neuron outputs through compact optical power splitters and proposes reconfigurable couplers for variable synaptic weights. These designs support fixed connectivity, learning-related reconfiguration, and broader network construction.

Axon arborization: Small-footprint, low-loss power splitters can generalize to three dimensions and enable thousands of synapses at 10 µm^3/synapse.The splitters distribute one neuron’s output across its connections and can be extended with multilayer photonics.
Synaptic weights: Fixed connection weights can be implemented by branching one neuron’s output waveguide to downstream target inputs.This provides a preliminary approach to weighted connectivity.
Synaptic weights: Electro-mechanically actuated waveguide couplers provide variable coupling from 0% to 100% by controlling waveguide separation.The couplers can be arranged vertically or laterally and are proposed for cryogenic operation.
Learning and plasticity: The proposed platform supports supervised systems with externally updated weights and unsupervised systems in which circuit activity reconfigures synapses.The two application classes differ in how voltages and activity update the weight matrix.
Learning and plasticity: Alternative electronic or magnetic memory elements could avoid mechanically mobile components, while tunable Mach–Zehnder interferometers are poorly suited to highly scaled systems.These alternatives are identified as future work or as limited by device size.

A. Multi-layer perceptron

The proposed SOEN platform can implement multilayer perceptrons using stacked routing-waveguide planes and die, with scaling determined by connectivity, neuron count, and layer structure. Modeling indicates promising densities for moderate connectivity but severe constraints at brain-like connectivity.

Architecture: An MLP can be implemented with processing layers connected through one or more vertically stacked planes of routing waveguides and stacked die.The processing layers run horizontally while routing-waveguide planes stack vertically.
Performance: 500 to 3,000 photons span an input dynamic range of approximately 11 bits for the 0.7I_c response.The response turns on near 500 photons and roughly levels out by 3,000 photons.
Limitations: The proposed synaptic bit depth is unlikely to reach the 32 bits used in modern GPU software implementations.The authors note that this constraint may be minor for applications prioritizing speed, complexity, and connectivity.
Spatial scaling: 400,000 neurons per cm^2 are achievable when each neuron has 10 connections, under the model’s assumptions.This density is obtained for N_conn = 10 and is more compact with N_wg = 1.
Spatial scaling: For 100 to 1,000 connections per neuron, the model estimates over 10,000 to 300 neurons per cm^2, respectively.Using N_wg = 10 becomes advantageous across this connectivity range.
System scaling: A proposed 1 m^3 tiled system would contain 7 × 10^10 neurons, roughly 10% of the number in the human brain.The construction uses 10^7 die arranged in stacked sheets.
Power scaling: Each 700-connection device consumes 2×10^-17 J/synapse event, while 20 MHz is achievable based on device limitations.The scaling discussion evaluates sparse event rates and includes cryogenic power considerations.

V. DISCUSSION AND OUTLOOK

The SOEN platform is presented as a few-photon, scalable neuromorphic hardware platform with potential applications in experimental models of visual processing. Integrated SNSPD arrays could provide a built-in retina for multilayer networks.

Discussion: Optical signaling can route non-interacting signals in three dimensions without wiring parasitics, supporting the interconnectivity associated with brain performance.The discussion connects optical routing with the importance of extensive neuronal connections.
Discussion: The SOEN platform operates in the few-photon regime with compact, energy-efficient components intended to support scalability.The authors distinguish this operating regime from other optical neuromorphic approaches.
Visual cortex: An integrated SNSPD pixel array could function as a built-in retina for monolithic image acquisition and analysis.The proposed visual-system architecture combines the array with a multilayer neural network.
Visual cortex: The envisioned SOEN visual system separates retina, thalamus, granular, and supragranular processing across interconnected layers.The design includes feedforward, inhibitory, excitatory, feedback, and recurrent connections as described for the biological system.
Visual cortex: An initial testbed would use one die for retina and thalamus, plus two chips of 700 neurons for cortical layers.The system is proposed for experiments including object recognition, edge detection, motion, and spatial-frequency perception.

C. High-performance application spaces

Neuromorphic systems are positioned for analyzing large, complex, interacting data, while superconducting electronics could provide fast, energy-efficient hardware. The proposed SOEN platform adds optical signaling and may connect superconducting processors to external systems.

Neuromorphic systems can extract features from large, noisy datasets and learn from temporal data evolution.The paper identifies markets and other complex interacting-unit systems as application areas suited to neuromorphic computing.
Superconducting electronics could address supercomputer bottlenecks through high speed and energy efficiency.Josephson-junction processors are described as roughly 100 times faster than CMOS while retaining high energy efficiency.
The SOEN platform may transduce single-flux-quantum pulses into optical signals for chip-to-chip and cryostat I/O links.
SOEN combines superconducting detectors and electronics with faint-photon semiconductor sources to form massively interconnected processors with access to multiple photonic degrees of freedom.The proposed signaling degrees of freedom include frequency, polarization, mode index, intensity, and coherence.
The paper proposes semiconductor LEDs, superconducting nanowire detectors, and reconfigurable optical waveguides as a route to advanced computing systems.It links massive interconnectivity and multiple physical degrees of freedom to systems with enormous entropy.
The platform is motivated partly by using alternative physical systems to investigate information processing in systems with brain-like complexity.

Appendix A: Threshold condition for the PND array

The PND threshold analysis combines detector-current conditions, optical absorption modeling, and Monte Carlo statistics to select absorption designs that distribute photons across nanowires. Ten-pass configurations improve low-absorption performance, while one-pass operation favors approximately 1% absorption.

The PND threshold condition is derived by relating absorbed-photon count to current redistribution among the nanowires.The current decreases as more nanowires are driven normal, and firing occurs when the current reaches the critical current.
Waveguide thickness, spacer height, and nanowire orientation provide tunable parameters for SNSPD absorption.The analysis considers parallel and perpendicular propagation and absorption probabilities for 100 nm and 200 nm waveguides.
The PND simulations use 1,000 Monte Carlo trials for different incident photon numbers with an array of 40 SNSPDs.
For single-pass operation, 1% absorption is close to ideal because both mean absorption and standard deviation are near one.At 0.1% absorption, the standard deviation is lower but the mean absorbed-photon count is only approximately 0.2.
Ten passes improve the performance of the 0.1% absorption case, although not all incident photons are absorbed.
For 40 incident photons, 0.1% absorption gives a more desirable spread than 1% or 10% in the ten-pass design.At 10% absorption, photons are absorbed on the first pass, producing a larger standard deviation and uneven absorption across nanowires.

Appendix C: Integration time and refractory period

The platform’s integration time and refractory period are controlled by superconducting relaxation and circuit parameters. Hotspot dynamics set a lower integration-time scale, while resistive and inductive elements can extend integration or delay refiring.

SPON integration time is the interval from photon absorption until the receiver no longer retains memory of that event.In the basic case, it is determined by the superconductor’s hotspot relaxation time and quasiparticle dynamics.
Parallel shunt resistors can set the PND’s L/R integration time above the hotspot relaxation-time lower bound.Adjusting the L/R value can extend integration to times much longer than other system timescales.
Cylindrically symmetric nanowire arrays are proposed to improve current uniformity after firing and address flux trapping in PNDs.The geometry avoids placing nanowires at an edge, promoting even supercurrent distribution.
Parallel resistors and nanowire inductance can engineer the SND integration time without a steady-state power penalty.
10–100 kHz event rates correspond to using a 1 ns quasiparticle lifetime as the integration time.The paper describes this operating range as straightforward to achieve.
The refractory period is governed by the SNSPD L/R time constant, with the LED’s several-kilohm impedance shortening it relative to a standard SNSPD.Additional series inductance can provide a longer delay when required.

light-emitting diode

The LED and waveguide analysis models low-temperature emitter behavior approximately and selects compact photonic geometries for controlled mode propagation. The design uses 200 nm waveguides, a 600 nm inter-waveguide gap, and supports multimode dendritic structures.

light-emitting diode: The LED model uses an analytical p–n junction current–voltage relationship to estimate emitter behavior.Electron and hole diffusion coefficients, mobilities, lifetimes, and carrier concentrations enter the model.
light-emitting diode: The photonic current is calculated from the electronic junction current using the modeled emitter relationship.
light-emitting diode: The p–n junction model is only approximate because the intended device uses a p–i–n junction and the model breaks down at low temperature.More thorough numerical and experimental investigation is deferred to future work.
waveguide design: For waveguides thinner than 200 nm, only the first vertical-order TE and TM modes are present in the analysis.The design therefore assumes a 200 nm waveguide height.
waveguide design: A compact waveguide can support tens of modes while maintaining a compact bend radius.
waveguide design: A 600 nm inter-waveguide gap is selected because symmetric and antisymmetric supermodes converge toward the uncoupled value there.At a 100 nm gap, the supermode splitting is substantially larger.

Appendix F: Scaling

The appendix estimates multilayer perceptron scaling and discusses how bandwidth and bit depth affect device-level information capacity. The information measure is explicitly limited to the device level rather than the full system.

Scaling: MLP layer length is modeled from tap, gap, crossing, neuron, routing-plane, and interlayer-coupler dimensions.The model uses 10 µm taps, 5 µm gaps, 3 µm crossings, and 10 µm interlayer couplers, with neuron and routing counts determining overall length.
Scaling: Each neuron is assumed to connect by synapse to every neuron in the next layer, with layer width set by the interwaveguide gap multiplied by neuron count.The model takes the interwaveguide gap as 600 nm × Nn and assumes equal neuron length and width.
Information capacity: Increasing stimulus and response bandwidths expands the mutual-information integral, while the proposed signals can provide roughly 11 bits of discretization.The intrinsic speed of SPONs is stated to exceed biological systems by a factor of 10^4, affecting both bandwidths.
Information capacity: Including SNSPD current alongside incident photons can further increase discernible-stimulus bit depth, but higher bit depth trades against size and efficiency.The information analysis considers photonic inputs and outputs, while current through the SNSPD provides an additional stimulus dimension.
Information capacity: The mutual-information calculation measures information content at the device level, not the system level.A full system-level information analysis is therefore outside the scope of this measure.