Source-linked AI summary
Training and Operation of an Integrated Neuromorphic Network Based on Metal-Oxide Memristors
Mirko Prezioso, Farnood Merrikh-Bayat, Brian Hoskins, Gina Adam, Konstantin K. Likharev, Dmitri B. Strukov
TL;DR
The paper demonstrates an integrated neural network using memristors in a dense, transistor-free crossbar circuit. The network performed analog vector-by-matrix multiplication and reached classification after an average of 15 training epochs, supporting further development of larger neuromorphic networks.
Problem
Implementing neuromorphic networks with many devices remains challenging, motivating integrated crossbar-based approaches.
Method
The authors built a dense, transistor-free memristor crossbar that performs analog vector-by-matrix multiplication using input signals, bias, and memristor conductances.
Results
15 training epochs on average were required to reach classification.
Takeaways & Limitations
The demonstration is presented as an important step toward effective analog-hardware implementation of more complex neuromorphic networks.
Takeaways & Limitations
The actual training rate ΔG depends on the initial conductance G, and some effects are less critical for network performance.
Abstract
from arXiv · showhide
Despite all the progress of semiconductor integrated circuit technology, the extreme complexity of the human cerebral cortex makes the hardware implementation of neuromorphic networks with a comparable number of devices exceptionally challenging. One of the most prospective candidates to provide comparable complexity, while operating much faster and with manageable power dissipation, are so-called CrossNets based on hybrid CMOS/memristor circuits. In these circuits, the usual CMOS stack is augmented with one or several crossbar layers, with adjustable two-terminal memristors at each crosspoint. Recently, there was a significant progress in improvement of technology of fabrication of such memristive crossbars and their integration with CMOS circuits, including first demonstrations of their vertical integration. Separately, there have been several demonstrations of discrete memristors as artificial synapses for neuromorphic networks. Very recently such experiments were extended to crossbar arrays of phase-change memristive devices. The adjustment of such devices, however, requires an additional transistor at each crosspoint, and hence the prospects of their scaling are less impressive than those of metal-oxide memristors, whose nonlinear I-V curves enable transistor-free operation. Here we report the first experimental implementation of a transistor-free metal-oxide memristor crossbar with device variability lowered sufficiently to demonstrate a successful operation of a simple integrated neural network, a single layer-perceptron. The network could be taught in situ using a coarse-grain variety of the delta-rule algorithm to perform the perfect classification of 3x3-pixel black/white images into 3 classes. We believe that this demonstration is an important step towards the implementation of much larger and more complex memristive neuromorphic networks.
Pattern Classification Experiment
The authors implemented and trained a transistor-free memristor-crossbar perceptron to classify 3×3 black-and-white images into three classes. Perfect classification was achieved after average training of 15 epochs, while key computations were performed physically in the crossbar.
- Network and inputs: The network was a single-layer perceptron with 10 inputs and 3 outputs, fully connected through 30 synaptic weights.Nine inputs represented image pixels and one input served as bias.
- Network and inputs: The network classified 3×3-pixel black-and-white images into three classes using 9 pixel inputs.The test set contained 30 patterns, including stylized “z”, “v”, and “n” letters plus noisy variants; because of its limited size, it served for both training and testing.
- Network and inputs: Inputs were encoded as +0.1 V for black pixels and -0.1 V for white pixels, with a -0.1 V bias input.Two memristors implemented each synapse to sustain balanced output coding, producing 60 memristors in the crossbar.
- Training procedure: Training occurred in situ using the Manhattan Update rule, a coarse-grain batch-mode variation of the supervised delta rule.The method applied training patterns sequentially, accumulated output-error-dependent increments, and modified weights with fixed-amplitude set and reset pulses.
- Results: At conductance initialization around 35 μS, perfect classification was reached on average after 15 training epochs.The device-specific switching dynamics made performance depend on initial conductance; for example, set and reset changes differed substantially at 20 μS and 65 μS.
- Results: The transistor-free crossbar physically performed the analog vector-by-matrix multiplication, while activation and training operations were handled by external electronics.The authors identify this demonstration as a step toward larger CrossNets, including multilayer-perceptron classifiers with deep learning.
Methods Summary
The devices were fabricated as dense metal-oxide memristive crossbars on silicon and packaged for electrical characterization. Device states were adjusted and read using external instrumentation, voltage pulses, and a V/2 biasing scheme.
- Fabrication: The switching stack used a 4-nm Al2O3 barrier and a 30-nm TiO2 switching layer.The bilayer was deposited and etched while preserved in the future crossbar area.
- Fabrication: After fabrication, dies were annealed for 30 minutes at 300°C in a reducing atmosphere and wire-bonded to a DIP40 package.The reducing atmosphere contained 10% H2 and 90% N2.
- Characterization: Electrical characterization used an Agilent B1500A parameter analyzer and B5250A switching matrix controlled by custom C code through GPIB.The equipment was used for testing packaged devices and wafers.
- Operation: All write and read pulses were 500 μs long, and memristor adjustment used the V/2 scheme with selected row and column wires biased at ±V/2.For readout, the selected column was biased, the selected row connected to virtual ground, and other lines grounded.
Supplementary Information
The supplementary information characterizes individual memristors and integrated crossbars, including device retention, endurance, conductance uniformity, forming, switching dynamics, neural-network outputs, and training convergence.
- Figure S1 presents a typical Pt/Al2O3/TiO2-x/Pt memristor together with its state retention and switching endurance.
- Figure S2 shows microphotographs of the bonded 12×12 crossbar and a zoomed view of its integrated memristive devices.
- Figure S3 characterizes virgin devices using representative I-V curves, a conductance map, and a histogram of effective conductances measured at 0.1 V.
- Figure S4 examines forming through switching curves, a forming-voltage map, and a histogram while progressively increasing the formed array from 2×2.
- Figure S6 measures effective-conductance evolution under repeated fixed-amplitude 500-μS pulse trains for reset and set polarities.
- Figures S7 and S8 assess output-current and weight distributions and simulate training convergence through epochs and the percentage of experiments achieving perfect convergence.