PaperlayerView exact revision on arXiv

Source-linked AI summary

Neuromorphic electronic circuits for building autonomous cognitive systems

Elisabetta Chicca, Fabio Stefanini, Chiara Bartolozzi, Giacomo Indiveri

arXiv:1403.6428v1cs.ETq-bio.NC

TL;DR

The paper addresses the open challenge of building compact, low-power physical systems that interact autonomously and exhibit cognitive abilities, beyond accelerating neural-network simulations. It reviews and develops neuromorphic circuits and networks for neural dynamics, plasticity, recurrent processing, working memory, and decision making. Experimental systems demonstrate cognitive functions, while the paper also discusses hardware constraints and system-level strategies for scaling the approach.

  • Problem

    Compact, low-power physical artifacts that behave intelligently in real-world autonomous interactions remain difficult to build, despite hardware advances for fast neural-network simulation.

  • Method

    The paper reviews and develops neuromorphic analog and mixed-signal circuits and recurrent networks for real-time neural dynamics, plasticity, working memory, and decision making.

  • Results

    A neuromorphic multi-chip system performed context-dependent task selection, demonstrating cognitive abilities with variable and imprecise circuits.

  • Takeaways & Limitations

    The presented circuits and networks form components for building autonomous neuromorphic cognitive systems, with progress requiring biological inspiration and system-level strategies.

  • Takeaways & Limitations

    Hardware plasticity mechanisms must operate with bounded, low-precision synaptic weights and account for their effects on network memory capacity.

Abstract

from arXiv · show

Several analog and digital brain-inspired electronic systems have been recently proposed as dedicated solutions for fast simulations of spiking neural networks. While these architectures are useful for exploring the computational properties of large-scale models of the nervous system, the challenge of building low-power compact physical artifacts that can behave intelligently in the real-world and exhibit cognitive abilities still remains open. In this paper we propose a set of neuromorphic engineering solutions to address this challenge. In particular, we review neuromorphic circuits for emulating neural and synaptic dynamics in real-time and discuss the role of biophysically realistic temporal dynamics in hardware neural processing architectures; we review the challenges of realizing spike-based plasticity mechanisms in real physical systems and present examples of analog electronic circuits that implement them; we describe the computational properties of recurrent neural networks and show how neuromorphic Winner-Take-All circuits can implement working-memory and decision-making mechanisms. We validate the neuromorphic approach proposed with experimental results obtained from our own circuits and systems, and argue how the circuits and networks presented in this work represent a useful set of components for efficiently and elegantly implementing neuromorphic cognition.

I. INTRODUCTION

The paper motivates neuromorphic circuits as compact, low-power, real-time hardware for autonomous cognitive systems, rather than only fast neural-network simulation. It develops circuit building blocks for biologically plausible dynamics, plasticity, recurrent processing, and cognitive functions.

  • Motivation: Conventional computers remain disadvantaged relative to biological nervous systems for real-time interaction with noisy, uncontrolled sensory inputs.The paper attributes this gap partly to the nervous system’s different computational and organizing principles.
  • Motivation: Existing custom hardware can accelerate large-scale neuroscience simulations, but the paper targets compact, real-time, energy-efficient systems that operate in the physical world.The proposed focus differs from architectures primarily intended as alternative substrates for neural simulation.
  • Neural dynamics: Subthreshold log-domain low-pass circuits provide compact, low-power building blocks for reproducing biologically plausible first-order neural and synaptic temporal dynamics.The reviewed circuits include the Differential Pair Integrator, Low-Pass Filter, and Tau-Cell.
  • Neural dynamics: The Differential Pair Integrator, Low-Pass Filter, and Tau-Cell are analyzed through translinear principles and their characteristic equations and general solutions.The analysis uses circuit translinear loops and derives equations for the three log-domain filters.
  • Circuit trade-offs: The three filter designs trade component count, layout area, current polarity, complementary CMOS compatibility, and control flexibility.The LPF has fewest components but requires an isolated p-FET well; the Tau-Cell requires precise current-source matching, while the DPI supports positive and negative currents and adds an Ith control parameter.

III. SILICON NEURONS

The paper presents compact generalized integrate-and-fire silicon neurons that combine biologically realistic dynamics with low-power operation and configurable spiking behavior. Their circuit blocks implement membrane filtering, spike generation, refractory reset, and spike-frequency adaptation.

  • III. SILICON NEURONS: Conductance-based neuron circuits are complex and area-intensive, whereas simplified integrate-and-fire models use fewer transistors but may omit richer behaviors.Generalized integrate-and-fire models offer a compromise between these extremes.
  • III. SILICON NEURONS: The proposed generalized integrate-and-fire circuit is compact, low-power, and supports biologically realistic time constants, refractory periods, and spike-frequency adaptation.These features support resonances and oscillatory behaviors associated with more complex neuron models.
  • III. SILICON NEURONS: The circuit combines an input DPI low-pass filter, positive-feedback spike generator, refractory reset block, and adaptation DPI filter.The blocks model leak conductance, sodium activation and inactivation, refractory behavior, and adaptation current, respectively.
  • III. SILICON NEURONS: The neuron equation uses Imem for the membrane-potential variable, Iahp for adaptation, u(t) for spike periods, and f(Imem) for membrane-dependent feedback.The function f(Imem) depends on membrane current and the positive-feedback current Ia.
  • III. SILICON NEURONS: Experimental measurements showed that f(Imem) can be fitted with an exponential function of Imem.This fit supplies the nonlinear relationship used in the generalized neuron description.
  • III. SILICON NEURONS: By changing bias controls for time constants, refractory period, and adaptation, the circuit produces behaviors ranging from regular spiking to bursting.The same circuit also supports time constants of hundreds of milliseconds and compatibility with fast asynchronous digital pulses.

IV. SILICON SYNAPSES

The silicon synapse circuit is designed to reproduce biologically realistic excitatory postsynaptic currents while incorporating temporal dynamics, short-term plasticity, NMDA voltage gating, and conductance dependence.

  • IV. SILICON SYNAPSES: Dedicated neuromorphic hardware can faithfully model synaptic dynamics, with throughput constrained mainly by the number of received and transmitted spikes.The text contrasts this bottleneck with the computational cost of software simulation.
  • IV. SILICON SYNAPSES: The DPI-based excitatory synapse produces exponentially rising and decaying EPSCs with a time constant set by Vτ.The input spike is applied to both the basic synaptic and short-term-depression blocks.

A. Short term depression and short-term facilitation

The synapse implements short-term plasticity through spike-dependent response changes that transition from nonlinear accumulation to linear low-pass filtering as synaptic current increases.

  • A. Short term depression and short-term facilitation: Short-term depression and facilitation provide circuit mechanisms for processing temporal signals and decoding temporal information.The paper identifies short-term depression and short-term facilitation as the two subclasses of these dynamics.
  • A. Short term depression and short-term facilitation: When Isyn ≪ Ith, each spike increases the circuit response by an amount greater than one.This spike-dependent nonlinear regime persists while the stated current inequality holds.
  • A. Short term depression and short-term facilitation: As Isyn grows beyond the low-current regime, the circuit approaches Isyn ≫ Ith and behaves as a first-order low-pass filter.The transition marks the onset of the circuit’s linear regime.

B. NMDA voltage gating and conductance behavior

The synapse extends DPI dynamics with voltage-gated NMDA channels and conductance-based voltage dependence. These functions are assigned to distinct differential-pair blocks in the circuit.

  • B. NMDA voltage gating and conductance behavior: The MN1−2 output differential pair implements NMDA voltage-gated channels, while MG1−2 implements conductance-based behavior.The response properties of these blocks were characterized previously.
  • B. NMDA voltage gating and conductance behavior: The complete synapse schematic combines short-term depression, basic DPI dynamics, NMDA voltage gating, and conductance-based voltage dependence.The corresponding blocks are MS1−3, MD1−6, MN1−2, and MG1−2, respectively.

C. Homeostatic plasticity: synaptic scaling

Synaptic scaling is a homeostatic mechanism that globally adjusts afferent synaptic weights to keep neuronal firing within a functional range while preserving relative synaptic differences.

  • C. Homeostatic plasticity: synaptic scaling: Synaptic scaling globally scales all synapses afferent to a neuron to maintain its firing rate within a functional range.It compensates for chronic activity changes while preserving relative differences between individual synapses.
  • C. Homeostatic plasticity: synaptic scaling: In VLSI, synaptic scaling can compensate for temperature drifts and sudden changes in system input activity.
  • C. Homeostatic plasticity: synaptic scaling: Independent controls over Vw and Vthr allow DPI synapses to combine local learning with neuron-wide homeostatic scaling.Learning changes each synapse’s Vw, whereas adaptive circuits adjust Vthr across synapses connected to one neuron.

V. SYNAPTIC PLASTICITY: SPIKE-BASED LEARNING

The paper presents hardware mechanisms for spike-based plasticity, including locally updated synapses, postsynaptic control signals, stochastic bistable weights, and calcium- and voltage-dependent learning rules.

  • V. SYNAPTIC PLASTICITY: SPIKE-BASED LEARNING: Long-term plasticity circuits automatically set synaptic weights on neuromorphic VLSI chips without dedicated external access to each synapse.
  • V. SYNAPTIC PLASTICITY: SPIKE-BASED LEARNING: Hardware synapses require plasticity mechanisms that operate with bounded, low-precision weights and account for physical constraints affecting memory capacity.Continuous operation also raises the blackout-effect problem in networks approaching storage capacity.
  • V. SYNAPTIC PLASTICITY: SPIKE-BASED LEARNING: Two-state plastic synapses use input-spike variability to control transitions and switch between slow and fast learning without changing internal parameters.Slow learning targets uncorrelated mean-rate patterns, whereas fast learning targets highly correlated patterns.
  • V. SYNAPTIC PLASTICITY: SPIKE-BASED LEARNING: The Fig. 4a circuit updates weights from presynaptic spikes using postsynaptic VUP and VDN signals, then refreshes the analog weight toward one of two rails.The stored voltage Vw sets the amplitude of the synaptic excitatory postsynaptic current.
  • V. SYNAPTIC PLASTICITY: SPIKE-BASED LEARNING: A calcium- and membrane-potential-dependent rule supports supervised and unsupervised classification, including perceptron-like binary outputs.Postsynaptic activity is represented by VCa, while threshold comparisons generate potentiation or depression signals.
  • V. SYNAPTIC PLASTICITY: SPIKE-BASED LEARNING: The VCa conditions implement stop-learning, improving generalization by preventing over-fitting after an input pattern has already been learned.Highly correlated learned and incoming patterns raise VCa above Vthk3, halting further synaptic modification.

VI. FROM CIRCUITS TO NETWORKS

The paper combines silicon synapses, integrate-and-fire neurons, plasticity, and recurrent winner-take-all networks into configurable spiking systems supporting memory, decision-making, and pattern processing.

  • VI. FROM CIRCUITS TO NETWORKS: Spiking neural network chips combine multiple synapse, silicon-neuron, and plasticity elements using on-chip or off-chip connectivity.These elements can be integrated onto single chips and connected through hardwired or reconfigurable infrastructures.
  • A. Recurrent neural networks: Recurrent neural networks store internal representations through connectivity-dependent dynamics and support associative memory, working memory, and context-dependent decisions.
  • VI. FROM CIRCUITS TO NETWORKS: A silicon neuron block integrates input spikes in synapses, sums resulting currents at an integrate-and-fire soma, and emits output spikes.Its synapses can implement local plasticity and global homeostatic scaling.
  • A. Recurrent neural networks: sWTA networks use recurrent excitation and inhibition so strongly responding neurons suppress competitors while similarly tuned neurons cooperate.
  • A. Recurrent neural networks: sWTA competition supports feature extraction, signal restoration, pattern classification, and unsupervised learning of common input features.Competition enhances tuned feature responses and suppresses background noise.
  • A. Recurrent neural networks: The sWTA topology uses excitatory neurons, a global inhibitory neuron, and hardwired synaptic connections represented in a corresponding chip architecture.

B. Distributed multi-chip networks

Distributed neuromorphic systems use modular organization and asynchronous spike-based communication to scale networks across chips, while bandwidth and memory constrain speed and size.

  • B. Distributed multi-chip networks: Cortical modularity motivates scalable systems whose properties can be qualitatively reproduced across scales.Such scalability depends on principles for arranging modules and enabling coherent long-distance communication.
  • B. Distributed multi-chip networks: Multi-chip pulse-based neural networks provide greater computational power and flexibility than single-chip systems for exploring neuromorphic scalability.
  • B. Distributed multi-chip networks: Address Event Representation communicates analog information through temporal relationships among asynchronous digital spike events across chips.
  • B. Distributed multi-chip networks: Asynchronous arbitration queues colliding events, transmitting one address at a time and delaying additional events by nanoseconds or fractions of microseconds.
  • B. Distributed multi-chip networks: With millisecond-scale time constants, communication delays are negligible; assuming 1 ms jitter tolerance, up to 4 K coincident events can be processed in 350 nm CMOS.
  • B. Distributed multi-chip networks: AER system performance is bounded by communication memory and bandwidth, forcing a trade-off between neural-processing speed and implementable network size.

C. A SW/HW echo-system

The paper combines hardware experiments with software infrastructure for configuring, monitoring, and validating real-time neuromorphic circuits and systems. Measurements demonstrate synaptic depression, adaptive bursting, and flexible control of neural dynamics.

  • C. A SW/HW echo-system: The software infrastructure is intended to define, configure, and control spiking neural networks and monitor their activity in real time.Scaling reconfigurable platforms requires tools that translate neural models into hardware parameters and event routing.
  • C. A SW/HW echo-system: The pyNCS framework bridges high-level neural-network programming libraries and hardware processing through reconfigurable APIs and scripting.It interprets hardware specification files and constructs abstract representations of neuromorphic devices.
  • C. A SW/HW echo-system: Experimental data collected from multiple neuromorphic VLSI devices demonstrates the behavior of the described circuits.The experiments validate circuit functionality across several chips and systems.
  • C. A SW/HW echo-system: 50 Hz presynaptic stimulation reveals membrane-potential responses under different short-term depression adaptation settings.Short-term depression reduces EPSC amplitude after repeated spikes and recovers slowly, on the order of 100 ms.
  • C. A SW/HW echo-system: Tuned spike-frequency adaptation circuits produce bursting behavior in response to a constant input current.DPI flexibility provides an additional control parameter without requiring extra circuits or dedicated resources.

B. Spike-based learning

The paper demonstrates spike-based learning circuits and recurrent winner-take-all networks as hardware mechanisms for learning, selective amplification, persistent activity, and state-dependent computation. Experiments show stochastic synaptic transitions, pattern learning, and finite-state behavior in VLSI systems.

  • B. Spike-based learning: Poisson-driven pre- and postsynaptic activity regulates synaptic transition probabilities, enabling learning-rate modulation or teacher signals.The same stochastic inputs support unsupervised and supervised learning conditions.
  • B. Spike-based learning: Poisson variability enables stochastic weight updates whose direction depends on input-spike timing and postsynaptic state.The demonstrated regime permits both potentiation and depression, with depression more likely.
  • B. Spike-based learning: Repeated presentations of the “INI” pattern train 3,472 plastic synapses while a stochastic teacher signal drives the postsynaptic neuron.As the pattern is stored, increased postsynaptic firing reduces the number of synapses undergoing LTP.
  • B. Spike-based learning: The learning circuits support perceptron-like supervised learning without explicit error control at every synapse and attractor-network learning without random generators at each synapse.Feed-forward and recurrent configurations use the same neuron and plasticity circuits.
  • C. sWTA networks of I&F neurons: A 128-neuron sWTA network selectively amplifies the strongest localized input through recurrent excitation and suppresses competitors through global inhibition.The response remains invariant when the relative input amplitudes are swapped.
  • C. sWTA networks of I&F neurons: Recurrent sWTA populations maintain activity after transient stimulation, allowing subpopulations to represent states in finite-state machines.After a second stimulus, the first population becomes silent while the second remains self-sustained.

VIII. DISCUSSION

The discussion presents low-power hybrid analog/digital circuits as building blocks for adaptive, fully parallel, real-time neuromorphic architectures. It addresses noise and mismatch by emphasizing energy efficiency, design strategies, and plasticity-based compensation.

  • VIII. DISCUSSION: The proposed circuits are intended to construct efficient, biophysically realistic, real-time neural architectures and autonomous behaving systems.The approach uses sub-threshold analog operation and exploits the physics of VLSI.
  • VIII. DISCUSSION: Sub-threshold current-mode circuits offer lower noise energy and better bandwidth-over-power efficiency than above-threshold circuits.Noise remains a recognized criticism of the design approach, but the discussion compares energy-related properties favorably.
  • VIII. DISCUSSION: Device mismatch is more prominent in sub-threshold circuits, yet aggressive mismatch reduction can enlarge designs and reduce neuron and synapse density.The authors argue that appropriate analog design techniques can minimize mismatch without requiring large circuits.
  • VIII. DISCUSSION: Plasticity and homeostatic mechanisms can reduce the effects of mismatch and compensate for large changes in neuron-driving signals.These mechanisms provide system-level feedback rather than relying only on precise transistor matching.

B. Exploiting variability and imprecision

The paper treats variability and imprecision as compatible with neuromorphic cognition when embedded in compact, distributed, massively parallel systems. Its circuits and networks support adaptive behavior, learning, working memory, decision making, and context-dependent task selection, while neuroscience understanding remains a major field-level constraint.

  • B. Exploiting variability and imprecision: The proposed strategy integrates many compact, inaccurate, and inhomogeneous circuits into dense distributed arrays rather than relying on a few precise elements.Intrinsic variability and diverse activation patterns are presented as potentially fundamental to neural computation.
  • C. Towards autonomous cognitive systems: The circuits provide building blocks for compact, low-power, scalable systems that interact with environments, learn input signals, and exhibit adaptive abilities.The paper links sWTA selective amplification and reverberating activity to working-memory and decision-making models.
  • C. Towards autonomous cognitive systems: A neuromorphic multi-chip system performed context-dependent task selection using variable and imprecise circuits.The authors describe this as a concrete example of cognitive abilities comparable to those used in primate cognition studies.
  • D. Challenges and progress in Neuromorphic Engineering: Neuromorphic Engineering progress is limited partly by a small research community and a lack of automated design, verification, and configuration infrastructure.The discussion identifies limited understanding of brain function and neural computation as the major limiting factor.
  • IX. CONCLUSIONS: The paper reviews neuromorphic circuits, spike-based learning solutions, recurrent-network mechanisms, and system-level strategies for autonomous cognitive systems.Its conclusion combines circuit demonstrations with an assessment of the field and proposed strategies for accelerating progress.
Loading 1403.6428v1…