PaperlayerView exact revision on arXiv

Source-linked AI summary

Control of criticality and computation in spiking neuromorphic networks with plasticity

Benjamin Cramer, David Stöckel, Markus Kreft, Michael Wibral, Johannes Schemmel, Karlheinz Meier, Viola Priesemann

arXiv:1909.08418v2cs.ETstat.CO

TL;DR

Criticality is often assumed to optimize computation, but whether its benefits depend on task requirements remains unclear. The authors test tasks of varying complexity in a plastic spiking neuromorphic network, adjusting its state through input strength. Criticality maximizes information-theoretic capacity, yet only complex tasks benefit from it; simple tasks suffer.

  • Problem

    How collective network state relates to task requirements remains unclear, and task-dependent performance calls for ways to characterize processing beyond a specific task.

    Source: §1 · p. 1

  • Method

    The authors evaluate tasks of varying complexity in a plastic spiking neuromorphic network and tune its state by changing input strength.

    Source: §1 · p. 2

  • Results

    Criticality maximizes information-theoretic measures, but only complex tasks benefit from it; simple tasks suffer.

    Source: §3 · pp. 7–8

  • Takeaways & Limitations

    The network’s working point can be adapted to task requirements by changing mean input strength.

    Source: §3 · p. 7

    A stated scope boundary is that 512-neuron circuits with adaptive-exponential LIF dynamics and intercompartmental conductances are planned for future chip versions.

    Source: §2 · p. 2

Abstract

from arXiv · show

The critical state is assumed to be optimal for any computation in recurrent neural networks, because criticality maximizes a number of abstract computational properties. We challenge this assumption by evaluating the performance of a spiking recurrent neural network on a set of tasks of varying complexity at - and away from critical network dynamics. To that end, we developed a spiking network with synaptic plasticity on a neuromorphic chip. We show that the distance to criticality can be easily adapted by changing the input strength, and then demonstrate a clear relation between criticality, task-performance and information-theoretic fingerprint. Whereas the information-theoretic measures all show that network capacity is maximal at criticality, this is not the case for performance on specific tasks: Only the complex, memory-intensive tasks profit from criticality, whereas the simple tasks suffer from it. Thereby, we challenge the general assumption that criticality would be beneficial for any task, and provide instead an understanding of how the collective network state should be tuned to task requirement to achieve optimal performance.

1 Introduction

Criticality is widely treated as an optimal design target because it maximizes basic processing properties, but its value may depend on task requirements. This study uses a plastic spiking neuromorphic network to compare task performance and information-theoretic measures across network states.

1 Introduction

  • Criticality is a guiding design principle because it maximizes sensitivity, dynamic range, correlation length, information transfer, and susceptibility.Precisely tuning a system to criticality can be challenging; local homeostatic learning may instead tune dynamics by adjusting input strength.Source: §1 · p. 1
  • Local plasticity can shape connectivity according to pre- and postsynaptic spike timing, supporting sequence learning and potentially speeding convergence.The authors relate this pattern-learning role to shaping biological networks during development through spontaneous activity.Source: §1 · p. 1
  • The relation between collective network state and specific task requirements remains unknown, despite the widespread belief that criticality optimizes task performance.The authors ask whether long correlation times may help memory-intensive tasks but hinder tasks requiring little memory.Source: §1 · p. 1
  • Task performance depends on the chosen task, motivating task-independent measures of how local circuits process information.Information theory measures information transfer, past-input information, storage, and—through partial information decomposition—unique, redundant, and synergistic contributions.Source: §1 · pp. 1–2
  • The BrainScaleS 2 neuromorphic chip combines physical neuron and synapse models with a processor that implements flexible plasticity for efficient long-term learning experiments.Its analog elements provide speedup and energy efficiency, while parallel operation makes computation speed largely independent of system size.Source: §1 · p. 2
  • The study finds that criticality benefits complex tasks but not simple ones, while information-theoretic measures indicate maximum network capacity near criticality.The network’s collective state can be adjusted by changing input strength.Source: §1 · p. 2

2 Results

Low external input brings the network close to critical dynamics, but the best operating point depends on task complexity: memory-intensive tasks benefit from criticality, while simple tasks can suffer. Information-theoretic measures increase toward criticality even when that state is not optimal for task performance.

Critical dynamics

  • At low Kext, avalanche distributions and dynamic measures indicate critical-like behavior, including branching ratio tending toward unity, longer autocorrelation, and stronger perturbation responses.Avalanche sizes follow power laws over two orders of magnitude, with exponents near 1.5; for low Kext, the maximum-likelihood comparison favors power-law fits in most cases.Source: §2 · pp. 3–4

Scaling and implementation

  • Software simulations show avalanche cutoffs scaling with system size at exponent 1.6 ± 0.2, while the N = 32 neuromorphic chip runs about 100 times faster than Brian 2.A 600 s biological-time experiment takes 570 s in Brian 2 and 6 s on the chip.Source: Figure 3 · p. 3, §2 · p. 3

Task performance

  • For n = 5 sum, performance is best away from criticality, whereas n = 25 sum and parity tasks benefit from criticality, especially as Nread decreases.With Nread reduced from 8 to 4, only the nonlinear parity task increasingly benefits; sum information appears globally available even under subcritical dynamics.Source: Figure 5 · p. 4, §2 · pp. 4–5

Dynamic adaptation

  • Switching Kext adapts network dynamics and task performance; adapting from subcritical to critical takes about 500 synaptic updates, versus about 50 in the reverse direction.The transitions take about 0.5 s in wall-clock time on the neuromorphic chip, and the resulting working points match those reached from zero initial weights.Source: §2 · p. 5, Figure 6 · p. 5

Memory

  • Low Kext extends input memory to longer delays and increases memory stored across recurrent neuron pairs, while single-neuron memory capacity stays fairly constant.The reservoir's internal memory capacity increases over almost two orders of magnitude toward criticality and accompanies better performance on complex parity tasks.Source: Figure 7 · p. 5, §2 · p. 6

Information-theoretic measures

  • Information transfer, active information storage, mutual information, and all PID components increase toward criticality, although shared and synergistic contributions exceed unique contributions.The authors interpret the increased transfer and storage as greater network computational capacity, while noting that maximal capacity can be unfavorable for simple tasks.Source: §2 · pp. 6–7, Figure 9 · p. 6

3 Discussion

Criticality maximizes abstract information-theoretic properties but benefits only complex tasks; simple tasks can perform worse. The network state can be tuned through input strength, while plasticity and task-independent measures help characterize and adapt its computational properties.

3 Discussion

  • Network state can be tuned by changing input strength, reaching criticality without fine-tuning network parameters.Changing input rate has effects similar to changing Kext, and the lowest rates can cross the critical point.Source: §3 · p. 7
  • Input strength and inhibition provide multiple control parameters, allowing flexible adjustment when only one can be freely set without perturbing input coding.The discussion identifies input degree, input rate, and inhibition as control parameters.Source: §3 · p. 7
  • Adding the causal STDP arm destabilized the network despite homeostasis; the reduced anticausal implementation remains stable and resembles full STDP.The authors suggest this reduced implementation may be useful in future studies.Source: §3 · p. 7
  • Task-dependent performance and task-independent measures together characterize criticality's computational effects; lagged stimulus–neuron information estimates task memory timescales.The paper reports that capacity peaks at criticality without necessarily yielding maximal task performance.Source: §3 · p. 7
  • Plasticity equalizes parameter noise, reducing the need to know parameter variations or perform specific calibration; analog-device parameter and thermal noise can potentially destabilize networks.The discussion presents the framework as particularly useful for analog neuromorphic devices.Source: §3 · pp. 7–8
  • Despite having only 32 neurons, the network showed criticality signatures and complex computational capabilities; the authors expect larger systems to offer richer possibilities.They expect scaling without fine-tuning because local learning rules self-tune the network.Source: §3 · p. 8
  • Only complex tasks profit from criticality, while it is detrimental for simple tasks despite maximizing abstract computational properties and information-theoretic measures.The discussion attributes harm on a simple task to maintenance of task-irrelevant information when network capacity is high.Source: §3 · pp. 7–8

4 Methods

The methods describe a plastic spiking network implemented on neuromorphic hardware and evaluation procedures spanning network dynamics, information measures, and reservoir tasks. The model and readout setup are used to compare activity and task performance across conditions.

4.1 Model

  • The network model approximates current-based LIF neurons with synaptic currents from external and recurrent inputs.The hardware uses 32 presynaptic partners per neuron, and each synapse is assigned to external input or recurrent transmission; Brian 2 simulations complement hardware emulations without parameter or temporal noise.Source: §4.1 · pp. 8–9
  • The network uses analog LIF neurons and plastic synapses, with weights developed during burn-in and then frozen for experiments unless otherwise stated.The hardware comprises 32 neurons; the plasticity rule combines drift, a correlation-sensitive STDP term, and biased noise, with only the negative STDP arm implemented.Source: §4.1 · pp. 8–9

4.2 Evaluation

  • The model uses temporal binning to represent activity and extract neural avalanches from population spikes.Avalanche size is the spike count across consecutive non-empty bins, with at least one empty bin separating successive avalanches.Source: §4.2 · p. 9
  • The evaluation estimates branching and autocorrelation from binned population activity, alongside avalanche statistics and responses to perturbations.Avalanches are separated by empty bins, and their size distributions are compared with power-law and exponential fits; perturbation sensitivity and trial-to-trial distance are also measured.Source: §4.2 · pp. 9–10
  • Information-theoretic analyses quantify activity storage, inter-neuron transfer, lagged input information, and information modification.The measures include AIS, transfer entropy, lagged mutual information and memory capacity, plus PID components; activity is binned at the refractory time and measures are typically normalized by entropy.Source: §4.2 · pp. 10–11
  • Reservoir performance is tested with n-bit parity and sum tasks using linear readouts trained on selected neuron activity.Parity performance is assessed on test data using mutual information with the predicted label, offset-corrected with shuffled labels; the sum task uses multiple readout classes and a winner-take-all decision.Source: §4.2 · p. 11

Related papers

Loading 1909.08418v2…