Source-linked AI summary
Physics for Neuromorphic Computing
Danijela Markovic, Alice Mizrahi, Damien Querlioz, Julie Grollier
TL;DR
Neuromorphic computing faces challenges in developing hardware that can keep pace with conventional CMOS chips and AI algorithms. This article synthesizes physics- and materials-based approaches inspired by AI and neuroscience, highlighting functional integrated systems and reported 94.4% accuracy.
Problem
Neuromorphic physics must address the challenge of keeping pace with advances in conventional CMOS chips and AI algorithms.
Method
The article synthesizes physics- and materials-based neuromorphic approaches inspired by conventional AI algorithms and neuroscience.
Results
Fully integrated neuromorphic systems are functional with reasonable raw performance, including one inference system reporting 94.4% accuracy.
Takeaways & Limitations
The paper supports developing neuromorphic algorithms and hardware hand in hand while identifying challenges beyond current CMOS and AI algorithms.
Takeaways & Limitations
The discussion identifies limitations associated with chip area and with the STDP rule itself.
Abstract
from arXiv · showhide
Neuromorphic computing takes inspiration from the brain to create energy efficient hardware for information processing, capable of highly sophisticated tasks. In this article, we make the case that building this new hardware necessitates reinventing electronics. We show that research in physics and material science will be key to create artificial nano-neurons and synapses, to connect them together in huge numbers, to organize them in complex systems, and to compute with them efficiently. We describe how some researchers choose to take inspiration from artificial intelligence to move forward in this direction, whereas others prefer taking inspiration from neuroscience, and we highlight recent striking results obtained with these two approaches. Finally, we discuss the challenges and perspectives in neuromorphic physics, which include developing the algorithms and the hardware hand in hand, making significant advances with small toy systems, as well as building large scale networks.
Why should we take inspiration from the brain?
Brains inspire neuromorphic computing because they perform complex cognitive tasks with remarkably low energy, unlike conventional digital computers. Their learnable algorithms and physical implementation use dynamical, stochastic, plastic, neuron–synapse-based computation rather than high-precision synchronous circuits.
- Biological brains perform complicated tasks, including analyzing data, making decisions, and moving toward goals.
- The human brain categorizes, predicts, and creates with power consumption of only about 20 W.
- 1000 kW.h trains a state-of-the-art natural language processing model, matching the energy consumed by a human brain over six years.
- “Algorithms” of the brain: Brain algorithms are dynamical, reconfigurable, and able to learn from experience, while exhibiting phenomena such as criticality, synchronization, chaos, and stochastic resonance.
- Physical implementation: Brains compute through neurons and synapses, stochasticity, binary and analog coding, asynchronous communication, collective behavior, and high plasticity instead of precision digital circuits.
Why are physics and material science essential to neuromorphic computing?
Neuromorphic computing requires physics and material science because conventional electronics separates memory from computation, creating data-movement and energy costs while limiting scalable neural hardware. New materials and physical phenomena must enable nanoscale, energy-efficient devices with dense, three-dimensional interconnectivity and integrated memory.
- Why are physics and material science essential to neuromorphic computing?: Conventional electronics separates memory and computation, creating a von Neumann bottleneck that slows processing and increases energy consumption.Artificial-intelligence algorithms repeatedly read, process, and rewrite substantial data.
- Why are physics and material science essential to neuromorphic computing?: CMOS alone requires dozens of transistors per neuron, external synapse memories, and micrometer-scale devices, limiting integration by chip area.Neural-network performance increases with neuron and synapse counts, while assembling chips can produce bulky systems and interconnect energy losses.
- Why are physics and material science essential to neuromorphic computing?: New materials and physical phenomena must provide nanoscale neurons and synapses that combine low energy consumption, memory, learning, nonlinearity, and high endurance.These devices should also be tunable, dynamical, reconfigurable, multifunctional, and manufacturable at low cost.
- Why are physics and material science essential to neuromorphic computing?: CMOS technology also struggles to reproduce the brain’s dense connectivity because it is mainly 2D, has limited fan-out, and distributes energy inefficiently.The brain averages 10,000 synapses per neuron and uses three-dimensional structure with high fan-in/fan-out.
- Why are physics and material science essential to neuromorphic computing?: Future devices must support large fan-in/fan-out, extensive interconnectivity, self-assembly, 3D interconnects, mass manufacture, and low cost.They must also be easy to address in large networks while providing signal gain and memory.
How can we imitate something that we do not understand yet?
Because the brain lacks a precise working model, neuromorphic computing follows two paths: mapping current AI algorithms onto physical systems or adding neuroscience-inspired features and dynamics. These approaches use memristive and photonic hardware, as well as device physics, to pursue efficient and more complex computation while facing scalability and learning challenges.
- Two approaches: Researchers pursue two approaches: physically implementing conventional AI algorithms for efficiency or adding neuroscience-inspired features and dynamics for more complex computing.The distinction arises because there is not yet a precise model of how the brain works.
- AI algorithms on physical systems: Hybrid CMOS/memristive and photonic systems are the two proposed scalable physics-based technologies for mapping deep networks onto chips.Memristive crossbars connect neuron layers, while photonics can convey information in parallel through wavelength multiplexing and passive neural systems.
- AI algorithms on physical systems: Memristors provide nanoscale resistors with non-volatile analog conductance states tunable by applied voltage, enabling crossbars to perform weighted-sum operations between neuron layers.Their conductance can arise from several material and physical effects, including conductive filaments and phase transitions.
- AI algorithms on physical systems: 100-fold gain in energy consumption and speed compared to graphical processing cards has been estimated for memristive neural-network training.Learning with memristors nevertheless raises challenges, and purely resistive arrays are limited by current sneak paths and array size.
- Neuroscience-inspired hardware: Neuroscience-inspired devices reproduce biological-like dynamics including periodic spiking, chaotic spiking, bursting, synchronization, and stochastic nano-neuron behavior.Device physics is also used to craft synapses and neurons from the same materials and implement learning algorithms.
Developing physical neuromorphic systems in a lab
Developing physical neuromorphic systems requires co-designing algorithms, materials, device physics, and bio-inspired models while addressing imperfect-device learning and system scaling. Small physical “toy” systems can test hypotheses and reveal system-level insights beyond single-device or theoretical studies.
- Co-designing algorithms and hardware: Neuromorphic systems require algorithms, materials, nanodevice physics, and bio-inspired computing models to be developed together.This applies to both artificial-intelligence and neuroscience-inspired approaches.
- Learning with imperfect devices: Device-level learning requires precise weight updates and weight-independent variations, which noisy, nonlinear nanodevices make difficult.Backpropagation may require updates much smaller than 0.1% of the weight value, while device nonlinearities can prevent convergence.
- Learning with imperfect devices: Algorithm adaptation can accommodate imperfect substrates, including binary-weight deep networks and methods more tolerant of memristor nonlinearity.Binary weights simplify hardware implementation, although online learning still requires real-valued weights.
- Learning with imperfect devices: STDP offers spatially local, biologically inspired learning that memristive devices can implement naturally, but extending it to complex multilayer systems remains challenging.Pattern recognition has been demonstrated with small memristor ensembles and larger phase-change-memory ensembles.
- Small-scale “toy” systems: Small physical “toy” neuromorphic systems let academic laboratories test hypotheses and obtain system-level insights not identified by theory or single-device studies.Such systems provide a route for contributions even when large-scale systems require CMOS foundries and industrial partners.
- Small-scale “toy” systems: Spintronic nano-oscillator experiments showed that nanoscale noise and drift can hinder reservoir computing, yet an exploitable stability range achieved state-of-the-art pattern-recognition performance.The approach also classified spoken vowels, while tunability and cyclability supported the observed performance.
Conclusion and perspectives
Neuromorphic computing remains an open-ended, fast-evolving field in which physics and materials enable emerging hardware approaches, including quantum and large-scale physical neural networks. Future progress depends on co-developing algorithms and hardware while addressing challenges that conventional CMOS and current AI methods cannot solve.
- Conclusion and perspectives: The materials and physical principles for future neural-network hardware remain open-ended because neuromorphic approaches have distinct advantages and disadvantages.The review emphasizes physics as relevant to this still-developing field.
- Conclusion and perspectives: Quantum neurons and synapses could exploit superposition and entanglement for parallel, high-dimensional processing while neuromorphic methods address noise and device variations.The review identifies cross-fertilization between quantum and neuromorphic computing as a source of potentially remarkable results.
- Conclusion and perspectives: More than a million memristors have been integrated into functional systems with reasonable raw accuracy, but performance remains significantly below software AI standards.Long fabrication and testing delays mean these systems do not incorporate the field’s latest progress.
- Conclusion and perspectives: 94.4% accuracy was achieved by an optimized fully integrated inference system on MNIST, whereas large functional learning systems remain challenging.Most learning demonstrations remain hybrid, while a fully hardware system implemented STDP learning with 1.4 Million synapses.
- Conclusion and perspectives: Conventional CMOS processors are also advancing rapidly, with recent GPUs reducing data motion and bit precision to achieve extremely low power for specific neural-network applications.Physics-based neuromorphic computing must keep pace with both CMOS hardware and rapidly advancing AI algorithms.
- Conclusion and perspectives: Interdisciplinary teams should develop algorithms and hardware together, while novel physics and materials should target problems beyond the long-term reach of CMOS and AI algorithms.The review draws inspiration from both neuroscience and artificial intelligence to guide these efforts.