Source-linked AI summary
Implementation of Multilayer Perceptron Network with Highly Uniform Passive Memristive Crossbar Circuits
F. Merrikh Bayat, M. Prezioso, B. Chakrabarti, I. Kataeva, D. Strukov
TL;DR
Scaling neuromorphic network complexity is linked to monolithic memristor–CMOS integration. The work demonstrates board-level integration of metal-oxide memristive synapses with silicon neurons, enabled by improved memristor uniformity and reliability.
Problem
Further scaling of neuromorphic network complexity may require monolithic integration of memristors with CMOS circuits.
Method
The work demonstrates board-level hardware integration of metal-oxide memristive synapses with silicon neurons.
Results
Improved memristor uniformity and reliability facilitated the successful board-level demonstration.
Takeaways & Limitations
Monolithic memristor–CMOS integration is presented as a path toward higher neuromorphic network complexity.
Takeaways & Limitations
Drops may lead to lower effective precision in vector-by-matrix computation, and some patterns are difficult even for humans to classify.
Abstract
from arXiv · showhide
The progress in the field of neural computation hinges on the use of hardware more efficient than the conventional microprocessors. Recent works have shown that mixed-signal integrated memristive circuits, especially their passive ('0T1R') variety, may increase the neuromorphic network performance dramatically, leaving far behind their digital counterparts. The major obstacle, however, is relatively immature memristor technology so that only limited functionality has been demonstrated to date. Here we experimentally demonstrate operation of one-hidden layer perceptron classifier entirely in the mixed-signal integrated hardware, comprised of two passive 20x20 metal-oxide memristive crossbar arrays, board-integrated with discrete CMOS components. The demonstrated multilayer perceptron network, whose complexity is almost 10x higher as compared to previously reported functional neuromorphic classifiers based on passive memristive circuits, achieves classification fidelity within 3 percent of that obtained in simulations, when using ex-situ training approach. The successful demonstration was facilitated by improvements in fabrication technology of memristors, specifically by lowering variations in their I-V characteristics.
2 Research Laboratories, DENSO CORP., 500-1 Minamiyama, Komenoki-cho, Nisshin, Japan 470-0111
The paper demonstrates a board-integrated mixed-signal neuromorphic classifier using passive memristive crossbars, enabled by improved device uniformity. Its hardware classification closely approaches software performance while illustrating scaling prospects and remaining constraints.
- Motivation: Mixed-signal circuits implement vector-by-matrix multiplication physically through Ohm’s and Kirchhoff’s laws, using adjustable-conductance memristors as synapses.This approach targets the efficiency gap between conventional digital neuromorphic hardware and biological prototypes.
- System demonstration: The demonstrated system is a fully functional, board-integrated mixed-signal network based on metal-oxide memristive devices and passive “0T1R” crossbars.Inference runs directly in hardware rather than through external-computer post-processing.
- System demonstration: The network contains almost an order of magnitude more devices than previously reported passive-memristive neuromorphic classifiers.Its focus on passive crossbars also supports possible three-dimensional integration and local storage of synaptic weights.
- Device technology: Uniform, gradual analog switching and narrow set/reset-voltage distributions enabled precise conductance tuning across the 20×20 arrays.The authors describe this as the first report of such precise adjustment at this integration scale.
- Pattern Classification: 95% training and 79.06% test fidelity were measured for hardware-oblivious operation, versus 100% and 82.34% in software; hardware-aware operation reached 100% and 81.4%.The classifier processed about 300,000 patterns per second, although some benchmark patterns were difficult even for humans.
- Discussion and Summary: Projected mixed-signal implementations could reduce the inference energy-delay product by six orders of magnitude versus advanced digital circuits, or more than eight orders with three-dimensional 10-nm memristors.These projections depend on scaling crossbar dimensions and achieving substantially higher synaptic density.
1. Additional Details for the Experimental Setup
The supplementary setup automates memristor forming with configurable current-sweep parameters, reset handling, and retry logic, while the experimental boards implement the neurons, switching matrix, and measurement control.
- Memristor forming: The forming procedure accepts device lists, attempt counts, current-sweep limits and steps, reset voltage, resistance thresholds, and a minimum current ratio.These parameters are specified before devices are formed sequentially.
- Memristor forming: Devices below the pristine-resistance threshold are treated as effectively pre-formed by annealing and switched to a low-conducting state before further forming.This reduces leakage currents during forming of the remaining crossbar devices.
- Memristor forming: Failed forming triggers repeated current sweeps with increased maximum current, followed by a second attempt after resetting all devices to low-conductive states.The second attempt can address leakage caused by previously formed devices accidentally turning on.
- Forming limitations: The maximum sweep current and stopping-current adjustment remained manual in some cases, including approximately 10% of automatic-forming failures.The authors note that this adjustment could be automated.
- Forming outcomes: 1% to 2.5% of crossbar devices could not be formed with the chosen parameters, but failed devices had negligible effect on tuning accuracy.Larger stress might have formed some devices, but it was not attempted to avoid permanent crossbar damage.
- Experimental hardware: The implementation uses discrete AD8034 operational amplifiers for CMOS neurons and ADG1438 multiplexers for the onboard switch matrix.The setup includes wire-bonded crossbar chips, separate printed circuit boards, and computer-controlled measurements.
2. Additional Details and Results for Pattern Classification Experiment
The experiment evaluates a custom four-class benchmark and compares ex-situ and in-situ training for a 10-hidden-neuron multilayer perceptron. Ex-situ approaches outperform fixed-amplitude in-situ training because read feedback enables device-specific pulse adjustment.
- The benchmark contains 40 training and 640 test 4×4-pixel black-and-white patterns representing “A”, “T”, “V”, and “X”.
- The patterns are not linearly separable, and multi-bit analog weights significantly improve performance for the implemented training algorithm.
- In-situ training used hardware inference but relied on an external computer for weight-update computation and storage.
- The in-situ setup was limited to fixed-amplitude training pulses, while variable-amplitude implementation was identified as a route for improvement.
- 70% experimentally measured fidelity was obtained for three-pattern in-situ classification, versus 100% training-set performance for both ex-situ approaches.
- Ex-situ training performs better because read feedback allows unique write-pulse amplitudes to compensate for switching-threshold variations.
3. Crossbar Circuit Scaling
The scaling analysis examines voltage drops in passive crossbars across inference, read, and write operations. It estimates feasible larger arrays, while identifying inference precision and line resistance as important scaling constraints.
- Increasing crossbar dimensions raises line currents and can make crosspoint voltages differ from the voltages applied at the periphery.
- Write operation produces the largest voltage drops because switching requires the largest applied voltages and currents.
- 30% and 25% estimated voltage drops for set and reset switching indicate possible 70×70 arrays with demonstrated technology and up to 400×400 arrays with improved electrode resistance.
- The V/2 biasing scheme used here permits only about 7% worst-case voltage drop under the corresponding scaling estimate.
- Read-operation drops are smaller and can be incorporated when determining the actual voltage across a specific memristor.
- Conductance compensation can address inference voltage drops, but exact adjustment is prevented by input-dependent drops on virtually grounded lines.
- Inference is more sensitive to voltage variation, so even small drops can reduce vector-by-matrix precision and limit scaling.
4. Temperature Sensitivity
The paper analyzes temperature sensitivity in practical neuromorphic hardware and describes circuit-level compensation strategies. Experimental sensitivity of functional performance was not measured.
- Practical neuromorphic hardware should operate correctly across wide temperature ranges, motivating temperature-sensitivity analysis.
- Differential synapses partially compensate temperature-dependent conductance changes by implementing each weight as W ≡ G+ - G-.
- Temperature dependence is weakest at higher conductive states and can be further reduced by choosing larger G_BIAS values.
- A feedback memristor in the second op-amp stage can additionally compensate temperature drift when it has similar temperature dependence.