PaperlayerView exact revision on arXiv

Source-linked AI summary

Single Neuromorphic Memristor closely Emulates Multiple Synaptic Mechanisms for Energy Efficient Neural Networks

Christoph Weilenmann, Alexandros Ziogas, Till Zellweger, Kevin Portner, Marko Mladenović, Manasa Kaniselvan, Timoleon Moraitis, Mathieu Luisier, Alexandros Emboras

arXiv:2402.16628v1cs.NEcond-mat.mes-hall

TL;DR

Conventional artificial synapses omit short-term dynamics and meta-plasticity, while GPU implementations of multi-functional synapses face computational overhead and memory bottlenecks. The paper demonstrates a two-terminal SrTiO3 memristor with these synaptic functions and evaluates device-informed modified STPN units on Atari Pong. The network’s estimated energy efficiency gain over GPU implementations is 96×–970×, depending on the GPU implementation.

  • Problem

    Conventional ANN synapses omit short-term memory, short-term plasticity, and meta-plasticity, while GPU implementations of multi-functional synapses face computational overhead and a von Neumann bottleneck.

    Source: §1 · pp. 2–3

  • Method

    The authors propose a two-terminal SrTiO3 memristor with six synaptic operations and evaluate device-informed modified STPN units on Atari Pong.

    Source: §1 · pp. 3–4

  • Results

    The memristive-synapse network achieved an estimated 96×–970× energy-efficiency gain over GPU implementations, depending on the GPU implementation.

    Source: §1 · p. 4

  • Takeaways & Limitations

    The device controls short-term decay without an additional contact or complex control signals, supporting its use in neuromorphic circuit implementations.

    Source: §4 · p. 16

    Further hardware realization requires vertical device structures for crossbar arrays and improved long-term retention while preserving short-term plasticity.

    Source: §4 · p. 16

Abstract

from arXiv · show

Biological neural networks do not only include long-term memory and weight multiplication capabilities, as commonly assumed in artificial neural networks, but also more complex functions such as short-term memory, short-term plasticity, and meta-plasticity - all collocated within each synapse. Here, we demonstrate memristive nano-devices based on SrTiO3 that inherently emulate all these synaptic functions. These memristors operate in a non-filamentary, low conductance regime, which enables stable and energy efficient operation. They can act as multi-functional hardware synapses in a class of bio-inspired deep neural networks (DNN) that make use of both long- and short-term synaptic dynamics and are capable of meta-learning or "learning-to-learn". The resulting bio-inspired DNN is then trained to play the video game Atari Pong, a complex reinforcement learning task in a dynamic environment. Our analysis shows that the energy consumption of the DNN with multi-functional memristive synapses decreases by about two orders of magnitude as compared to a pure GPU implementation. Based on this finding, we infer that memristive devices with a better emulation of the synaptic functionalities do not only broaden the applicability of neuromorphic computing, but could also improve the performance and energy costs of certain artificial intelligence applications.

1 Introduction

Conventional ANNs omit short-term synaptic dynamics and meta-plasticity, while GPU implementations of richer synapses incur computational and memory-transfer overhead. This work introduces an SrTiO3 memristor and modified STPN network, reporting faster, more stable Pong training and an estimated 96×–970× energy-efficiency gain over GPU implementations.

1 Introduction

  • Biological synapses exhibit Hebbian plasticity, short-term plasticity, and meta-plasticity, whereas conventional ANN synapses capture only a subset of these dynamics.Meta-plasticity includes activation-dependent control of short-term decay timescales, from milliseconds to minutes.Source: §1 · p. 2, Figure 1 · p. 5
  • ST-Hebb synapses add decaying short-term memory, short-term weight updates, and decay-time control to conventional synaptic functions, enabling STPN meta-learning.The STPN combines conventional neurons with ST-Hebb synapses and can be integrated into multilayer networks.Source: §1 · p. 2
  • GPU implementations of multi-functional synapses face extra computational overhead and memory-bound operations across networks with 10^6 to 10^14 synapses.The physically separated memory and processing units impose a von Neumann bottleneck.Source: §1 · pp. 2–3
  • Earlier memristor demonstrations used spike-timing rules that could not be integrated into DNNs, while a three-terminal approach required externally applied decay signals.The latter approach raised control-circuit and wiring complexity concerns for large-scale implementation.Source: §1 · p. 3
  • The SrTiO3 device encodes long- and short-term weights in conductance and supports plasticity updates, spontaneous short-term decay, decay-time control, and in-memory multiplication.It operates at conductances in the tens of nS and has short-term timescales from 10 milliseconds to hundreds of seconds.Source: §1 · pp. 3–4
  • The memristive-synapse network achieved an estimated 96×–970× energy-efficiency gain over GPU implementations, depending on the GPU implementation.The estimate compares GPU and memristive synapse energy in a Pong-playing network.Source: §1 · p. 4
  • A modified STPN unit incorporating measured device energy was evaluated in Atari Pong, with faster and more stable training than the original version.The reported training difference is attributed mainly to the device-imposed constraint on short-term decay time; long short-term-weight timescales were required for robust, fast training.Source: §1 · p. 4

2 Multi-functional Synaptic Behavior in Single Memristor

A single SrTiO3 memristor encodes long- and short-term synaptic weights in its conductance and intrinsically supports their plasticity and decay-time control. Pulse parameters tune short-term updates, while bias voltage controls their decay.

2 Multi-functional Synaptic Behavior in Single Memristor

  • The device uses Cr/Pt and Ti electrodes on an SrTiO3 single-crystal substrate, with an approximately 40 nm electrode gap and a 15 nm SiN cover layer.Annealing at 300°C for 20 minutes in flowing Ar forms a thermal oxide at the Ti–STO interface.Source: §2 · pp. 5–6
  • In the proposed switching picture, oxygen-vacancy motion modulates the Pt–STO Schottky barrier, while vacancies diffuse back at zero bias and negative bias accelerates decay.The authors note that a role for interfacial trap states cannot be excluded.Source: §2 · p. 6
  • 100 SET pulses at 4 V for 500 µs switch conductance from low to high, and subsequent 2–3 V pulses induce short-term increases followed by decay.Long-term conductance values from 12 to 23 nS can be set; 2, 2.5, and 3 V pulses produce short-term increases of 3, 5, and 10 nS, respectively.Source: §2 · p. 7, Figure 2 · p. 8
  • The memristor intrinsically implements all six ST-Hebb synaptic operations, including long- and short-term weight storage, plasticity, meta-plasticity, and input-weight multiplication.Long-term and short-term components are both encoded in conductance; short-term decay occurs spontaneously.Source: §2 · pp. 7, 9
  • Short-term updates span 0.7–38.6 nS across pulse amplitudes and widths, while bias voltage controls the decay time constant Λ from 0.08 to 0.92.With the main pulse fixed at 3.5 V and 500 µs, decay times range from hundreds of milliseconds at Vbias = −0.6 V to tens of seconds at 0.6 V.Source: §2 · p. 9, Figure 3 · p. 10

3 DNN with Multi-functional Memristive Synapses

The authors model Atari Pong learning with a modified STPN network incorporating measured memristor characteristics, then estimate energy use against GPU implementations. Their analysis reports substantially lower energy for the memristive network, with device constraints shaping training and operation.

3.1 Modified short-term plasticity neuron

  • The simulated actor-critic network uses 64 m-STPN units connected to 2,592 inputs and 64 recurrent outputs, totaling 169,984 synapses.Frames pass through two convolutional layers before the m-STPN layer; fully connected layers compute the next action and state value.Source: §3.1 · p. 11
  • Restricting the learned decay constant Λ to the device-achievable range [0.08, 0.92] affects training, while larger upper limits produce faster reward increases.With Λ = 0, agents still learn, but more slowly and less robustly than with Λ = [0.08, 0.92].Source: §3.1 · pp. 11–12

3.2 Energy consumption of deep STPN network

  • Short-term weight updates are sparse during a Pong game, so only a small number of energy-consuming update pulses are needed.The modeled memristor energy includes pulse energy for ΔF updates and bias power for controlling the decay time constant.Source: §3.1 · p. 12, §3.2 · p. 14
  • At a 0.6 V bias, a synapse with simulated weight G = 0 still draws 4.3 nW because the physical device has a finite minimum conductance.The passage attributes the low power to the memristor’s low conductance values, enabled by non-filamentary switching.Source: §3.2 · p. 14
  • The energy estimate assumes a 0.6 V bias for every synapse when accounting for decay control and input-weight multiplication.The network-wide energy distribution also reflects that most synapses have no short-term weight updates during the game.Source: §3.2 · pp. 15–16
  • A factor of 96 lower energy consumption was obtained for the memristor network than the GPU (optimal) fp16 implementation.The comparison covers all network synapses and the synaptic operations considered; the memristor estimate excludes required peripheral circuitry.Source: Figure 4 · p. 13, §3.2 · p. 15
  • The authors argue that the energy gain and simple two-terminal device structure indicate that scaled-up memristive hardware is feasible.This is presented as an indication of feasibility, not a demonstrated scaled-up implementation.Source: §3.2 · p. 16

4 Conclusion

The study presents an SrTiO3 memristor and modified STPN units for collocating long- and short-term synaptic dynamics, and reports higher simulated inference energy efficiency than a GPU implementation. Further hardware development and investigation of decay models remain necessary.

4 Conclusion

  • At least 96-fold higher inference energy efficiency was obtained for the simulated memristor STPN network than for the GPU implementation.The comparison was conducted in simulation.Source: §4 · p. 16
  • The two-terminal STO memristor stores and computes long- and short-term synaptic weight updates, with short-term decay controlled without an extra contact or complex control signals.The device’s decay-time control implements a form of intrinsic meta-plasticity.Source: §4 · p. 16
  • Full hardware realization requires vertical device structures for crossbar arrays and improved long-term retention while preserving short-term plasticity.The authors suggest an oxide layer between the Pt electrode and STO as a possible retention improvement.Source: §4 · p. 16
  • Because decay time constants significantly affect training performance, the authors propose investigating different decay models for both long- and short-term components.Source: §4 · p. 17

5 Methods

The methods cover fabrication and measurement of the STO devices, a modified STPN model, Pong training, and GPU energy benchmarking. The GPU comparison measures operations that can also be performed by the memristor and includes a register-only compute benchmark that excludes memory traffic.

5.1 Device Fabrication

  • The Cr-Pt and Ti-Pt electrode stacks were patterned by e-beam lithography, deposited by e-beam evaporation, annealed at 300 degrees for 20 minutes in Ar, and encapsulated with 30 nm SiN.The authors report that annealing forms thermal oxide at the Ti-STO interface, leaving oxygen vacancies; it likely also diffuses chromium into STO.Source: §5.1 · p. 17

5.2 Experimental Setup

  • Quasi-static I-V characteristics were measured with a Keysight M9601A, and voltage pulses were generated with a Keysight 33500 waveform generator.Current was amplified with a Femto DHPCA-100 and read with a Rohde&Schwarz RTE 1104 oscilloscope.Source: §5.2 · p. 17

5.3 Modified STPN Model

  • The modified STPN layer learns long-term weight W, Hebbian association strength Γ, and short-term decay parameter Λ by back propagation through time.Removing plastic-weight normalization while retaining input normalization avoids a time-varying decay constant that can destabilize training and makes memristor implementation feasible.Source: §5.3 · pp. 17–18

5.4 Network Training

  • PongNoFrameskip-v4 agents were trained from experience collected by four parallel agents, following the stated rollout, gradient-clipping, discount, and learning-rate schedule.Frame stacking was omitted; the learning rate decayed from 0.0001 to 10^-11 over 200 million iterations.Source: §5.4 · p. 18

5.5 GPU Energy measurement

  • GPU energy was measured for W+F, decay, ΔF, and weight multiplication, with repeated kernel executions, median reporting, and bootstrapped 99% confidence intervals.The GPU (optimal) benchmark scales matrices to improve energy efficiency, while GPU (compute) uses registers without global-memory traffic; that compute-only result is not a usable implementation because its outputs are inaccessible to a GPU program.Source: §5.5 · pp. 18–19

Related papers

Loading 2402.16628v1…