PaperlayerView exact revision on arXiv

Source-linked AI summary

Supervised Learning in Spiking Neural Networks with FORCE Training

Wilten Nicola, Claudia Clopath

arXiv:1609.02545v3q-bio.NC

TL;DR

Spiking-network models need to learn diverse behaviors, but existing approaches can depend on particular neuron implementations or closed-form task equations. This paper applies FORCE training directly to spiking networks, using supervised error signals to train dynamical systems, classifiers, sequences, songbird singing, and movie replay. The trained networks reproduce complex behaviors and support perturbations that reveal effects such as altered replay and spike timing.

  • Problem

    Existing spiking-network approaches can constrain the underlying neuron organization and require tasks to be specified as closed-form differential equations, while FORCE training had mainly been implemented in rate-equation networks.

  • Method

    The paper applies FORCE training to spiking networks, using high-dimensional dynamical reservoirs and supervisor-provided error signals to learn network connections and behaviors.

  • Results

    FORCE-trained spiking networks mimic dynamical systems, classify inputs, store sequences, reproduce zebra-finch singing, and encode and replay a movie scene.

  • Takeaways & Limitations

    The networks reproduce complex spatio-temporal behaviors and permit perturbations yielding behavioral responses, replay changes, and spike timing statistics.

  • Takeaways & Limitations

    The hippocampal models could be extended with biological constraints such as Dale’s law and explicit representations of hippocampal circuit components.

Abstract

from arXiv · show

Populations of neurons display an extraordinary diversity in the behaviors they affect and display. Machine learning techniques have recently emerged that allow us to create networks of model neurons that display behaviours of similar complexity. Here, we demonstrate the direct applicability of one such technique, the FORCE method, to spiking neural networks. We train these networks to mimic dynamical systems, classify inputs, and store discrete sequences that correspond to the notes of a song. Finally, we use FORCE training to create two biologically motivated model circuits. One is inspired by the zebra-finch and successfully reproduces songbird singing. The second network is motivated by the hippocampus and is trained to store and replay a movie scene. FORCE trained networks reproduce behaviors comparable in complexity to their inspired circuits and yield information not easily obtainable with other techniques such as behavioral responses to pharmacological manipulations and spike timing statistics.

Acceptance

The manuscript was accepted and published in Nature Communications on December 20, 2017, and should be cited as Nicola and Clopath (2017).

  • The manuscript was accepted and published on December 20, 2017.
  • The paper appeared in Nature Communications, volume 8, issue 1, as article 2208.
  • The recommended citation is Nicola, W., and Clopath, C. (2017), “Supervised learning in spiking neural networks with FORCE training.”

Introduction

The introduction motivates FORCE training as a flexible approach for spiking networks because it does not require a fixed neuron implementation or a closed-form task description. The paper applies it directly to spiking networks and tests biologically motivated circuits for singing and movie replay.

  • Motivation: Human learning spans diverse tasks, including complex motion sequences and replaying musical notes, motivating computational models of flexible behavior.
  • Existing approaches: Existing top-down methods determine recurrent spiking-network connection strengths from an intended task or behavior.
  • Existing approaches: NEF and spike-based coding approaches constrain networks by their underlying neuron organization and require tasks specified as closed-form differential equations.
  • FORCE training: FORCE training uses a high-dimensional dynamical system as a reservoir and requires a supervisor-provided error signal rather than a closed-form differential equation.
  • Paper approach: The paper applies FORCE training directly to spiking networks, testing robustness across implementations, neuron models, and supervisors, then modeling zebra-finch singing and hippocampal movie replay.

Results

FORCE training successfully taught spiking networks to reproduce dynamical systems, sequences, songbird singing, and high-dimensional movie signals. High-Dimensional Temporal Signals improved training and replay, while perturbation experiments revealed circuit-specific robustness and vulnerabilities.

  • FORCE method: FORCE training minimized squared error between network dynamics and target dynamics by learning feedback weights online with Recursive Least Squares.The weight matrix combined static chaos-inducing weights with learned feedback and a linear decoder.
  • Dynamical systems: Spiking networks learned oscillators across neuron models and supervisor types, including sinusoids, sawtooth signals, Van der Pol oscillators, and noisy inputs.Training was robust to different initial chaotic states, with convergence regions remaining contiguous across parameter sweeps.
  • Dynamical systems: Spiking networks converged at approximately N^-1/2 in L2 error, whereas rate networks converged at approximately N^-1.This comparison was obtained for a 5 Hz sinusoidal supervisor.
  • Dynamical systems: Spiking networks reproduced Lorenz-like chaotic dynamics after training, although the task required 5000 neurons and 45 seconds of training.Their attractor-density performance was comparable to that of rate networks, and both regenerated the stereotypical Lorenz tent map.
  • Discrete sequences: An Izhikevich network reproduced Ode to Joy with 82% testing accuracy, while replay errors were stereotyped around repeated notes and note sequences.The trained network had an average firing rate of 34 Hz and a stable peri-stimulus time histogram despite replay-to-replay variability.
  • Biologically motivated circuits: FORCE-trained Izhikevich networks reproduced zebra-finch song spectrograms and remained functional after excitatory synapses were reduced by 90%, but 20% upscaling destroyed singing.The model’s seizure-like activity after excitation upscaling resembled experimentally observed effects of bicuculline in RA.
  • High-Dimensional Temporal Signals: Internally generated High-Dimensional Temporal Signals made longer song learning faster, more accurate, and more robust, including error-free learning of a 16-second four-bar sequence.The network simultaneously learned a 64-dimensional temporal signal and the song.
  • High-Dimensional Temporal Signals: Movie replay achieved a time-averaged correlation of r = 0.98, while HDTS inputs were necessary for effective training and correctly ordered replay.Without HDTS, individual frames remained replayable, but scene order became chaotic.

Discussion

FORCE training enables spiking networks to reproduce complex dynamics and biologically motivated behaviors, while high-dimensional temporal signals improve learning and replay. The method is robust across models and settings, but its learning rules remain biologically implausible and require further development.

  • FORCE training is robust to neuron models, initial states, and synaptic connection types, with Izhikevich neurons most accurate for arbitrary tasks or dynamics.The Izhikevich model’s slower spike-frequency adaptation variables may increase reservoir capacity.
  • Songbird circuit: In the songbird model, FORCE training accurately reproduces area RA spike statistics and song spectrograms, while excitation–inhibition perturbations degrade singing behavior.Excess excitation changes the spectrogram nonlinearly, whereas excess inhibition reduces all frequency amplitudes.
  • Hippocampal circuit: In the episodic-memory model, manipulating the HDTS robustly speeds up or reverses movie replay, while reducing or blocking it disrupts learning and accurate replay.The HDTS also induces a slow population oscillation reminiscent of hippocampal theta.
  • Successful training requires balancing the chaos-inducing and feedback matrices, whereas the method’s reliance on non-local correlation information limits biological plausibility.The authors identify scaling between Q and G as important for preserving a functional chaotic reservoir.
  • High-dimensional temporal signals improve accuracy and capability for reproducing long signals by partitioning neurons into temporally organized assemblies.The signal discretizes time by segregating neurons into assemblies.
  • FORCE training reproduces complex spatio-temporal dynamics and extends spiking networks beyond low-dimensional tasks to classification, song sequences, songbird singing, and movie replay.

Methods

The networks use chaotic recurrent spiking dynamics with synaptic filtering, neuron-specific current and voltage dynamics, and FORCE learning through a decomposed weight matrix and online RLS updates.

  • Neuron models: The networks use coupled integrate-and-fire neurons implemented as LIF, theta, or Izhikevich models.Voltage variables reset when thresholds or voltage peaks are reached; the Izhikevich model additionally includes an adaptation current.
  • Synaptic filtering: Synaptic currents filter presynaptic spikes using simple exponential, double exponential, or alpha filters.The primary configuration uses a double-exponential filter with a 2 ms rise time and 20 ms decay time.
  • Learning target: The target is an m-dimensional teaching signal x(t), which the network approximates through its learned recurrent dynamics and decoder.The synaptic connection matrix controls postsynaptic currents arriving at each neuron.
  • Network architecture: The synaptic matrix combines static chaotic weights with a learned decoder and feedback term, ωij = Gω0ij + Qηi · φTj.The decoder φ is learned and also represents the network’s linear firing-rate readout.
  • FORCE optimization: RLS updates the decoder online using network-generated correlations represented by P(t), with λ providing regularization and controlling error reduction.The network starts with φ(0) = 0 and P(0) = INλ^-1; a denominator modification can slightly improve accuracy and stability without changing convergence order.

G and Q parameters

The FORCE parameters G and Q balance chaotic static input with learned feedback so that the learned and static contributions remain comparable in magnitude.

  • Parameter roles: G scales the static weight matrix and controls the network’s chaotic behavior, while Q scales the feedback term.The static matrix is sampled with zero mean and variance determined by network sparsity.
  • Parameter balance: The learned and static synaptic contributions are both O(1), motivating a Q scaling that balances their fluctuations.The derived scaling depends on G, the static-weight standard deviation σω, and mean squared firing rates.

High Dimensional Temporal Signals

High-Dimensional Temporal Signals stabilize learning by partitioning time into pulse-defined neuron assemblies that activate sequentially and propagate long signals.

  • Signal construction: HDTS discretizes an interval T into m subintervals, adding one supervisor component and a pulse or upward deflection for each interval.Gaussian or piecewise-defined pulses can construct the supervisor, with σ controlling pulse width.
  • Assembly organization: The signals divide the network into assemblies whose membership depends on static weights and encoding variables ηin.Each assembly is activated by preceding assemblies, propagating a long signal at a frequency set by pulse width.
  • Signal variants: Sinusoidal HDTS is used for the main figures, while Gaussian HDTS appears in Supplementary Fig. 18C.

The FORCE Method with Rate Networks (Figure 1)

A 1000-neuron rate network learned a 5 Hz sinusoidal teaching signal using an initial untrained period, online RLS training, and a post-training evaluation period.

  • Setup: A 1000-neuron rate network was trained to approximate a 5 Hz sinusoidal oscillation with F = 10, τs = 10 ms, G = 1, Q = 1.5, and p = 0.1.The network ran for 1 second before RLS, trained for 4 seconds, and continued for 5 seconds after learning.

Using Spiking Neural Networks to Mimic Dynamics Using FORCE training (Figure 2)

FORCE training enabled spiking networks to reproduce several target dynamical systems, including oscillators and Lorenz-like chaotic trajectories. The trained Lorenz network reproduced the target trajectories and attractor beyond its training interval.

  • Neuron-model robustness: 22.9 Hz and 36.7 Hz average firing rates were obtained for LIF and Izhikevich networks reproducing the 5 Hz sinusoidal oscillator.The simulations used 2000 neurons in each network with distinct connectivity and integration parameters.
  • Lorenz dynamics: 22.21 Hz average firing rate accompanied a 45-second-trained network’s reproduction of Lorenz-like trajectories and the attractor for 50 additional seconds.The Lorenz system used ρ = 28, σ = 10, and B = 8/3 with 5000 theta neurons.

Using Spiking Neural Networks for Pattern Storage and Replay with the FORCE Method (Figure 3)

FORCE training stored and replayed discrete note sequences by aligning a time-varying teaching signal with network output. Correct replays appeared as low-error local minima and were classified using a critical error threshold.

  • Pattern encoding: Teaching pulses encoded quarter notes at 2 Hz and half notes at 1 Hz, with each pulse marking a note’s presence.The pulses could also provide harmonic amplitudes or envelopes for generating an audio signal.
  • Replay classification: As replay time t varied, alignment between the teaching signal x(t′) and output x̂(t′) produced local minima of E(t) at correct replay times.A replay was automatically classified as correct when E(t*) fell below a critical value.
  • Training setup: The trained network averaged 34 Hz firing, with λ^-1 = 2 ms used for recursive-least-squares training.

Using Spiking Neural Networks for Pattern Storage and Replay with the FORCE Method: Song Bird Example (Figure 4)

A FORCE-trained spiking network reproduced zebra-finch singing from a spectrogram teaching signal. Post-training weight manipulation varied the balance between excitation and inhibition.

  • Songbird model: A 1000-neuron RA network was FORCE trained on a 5-second male zebra-finch song spectrogram and fired at an average rate of 52.93 Hz.The spectrogram was generated with a 22.7 ms window, and training lasted 50 seconds.
  • Songbird model: The teaching signal was a spectrogram of recorded male zebra-finch singing obtained from the CRCNS data repository.
  • Post-training manipulation: The parameter α controlled post-training excitation-inhibition balance, with α = 1 preserving the original weight matrix.Values α > 1 amplified excitatory connections, whereas α < 1 diminished them.

Using Spiking Neural Networks for Pattern Storage and Replay with the FORCE Method: Movie Replay (Figure 5)

The hippocampus-inspired replay network was trained on an 8-second movie clip represented as a smoothed, interpolated, downsampled supervisor. This representation allowed recursive-least-squares training at arbitrary time points.

  • Movie supervisor: An 8-second Predator movie clip supplied the teaching signal for the replay network.
  • Movie supervisor: Frames were smoothed and interpolated so recursive least squares could operate at arbitrary time points rather than only native frame times.
  • Movie representation: The movie was downsampled to 1920 pixels arranged as 30 × 64, forming the supervisor.

Figure Captions

FORCE-trained spiking networks mimic oscillatory dynamics, store and replay song sequences, reproduce birdsong, and replay movie scenes using high-dimensional temporal signals.

  • Figure 1: FORCE method: FORCE training adds learned decoder-weighted connections to static weights that induce chaotic reservoir dynamics, converting them into controlled outputs.The learned weights are determined with Recursive Least Squares (RLS), while the static weights scale to initialize network-level chaos.
  • Figure 2: Dynamics: FORCE-trained networks learn oscillators and other dynamical signals, including sinusoids, sawtooth oscillations, Van der Pol oscillators, and noisy teaching signals.Theta-neuron performance depends on parameters such as the synaptic decay time constant.
  • Figure 2: Dynamics: Spiking networks of theta, leaky integrate-and-fire, and Izhikevich neurons exhibit divergent spike trains after a single spike deletion, indicating chaotic behavior.The comparison covers 2000-neuron networks and displays voltage traces for five randomly selected neurons.
  • Figure 4: Songbird singing: A 1000-neuron Izhikevich network reproduces a zebra-finch song spectrogram from a pulse sequence modeled on HVC-to-RA activity.The pulse chain represents the firing output of HVC RA-projecting neurons that trigger singing behavior.
  • Figure 5: Movie replay: Movie replay uses either an internally trained or externally supplied HDTS, with the signal imposing an 8 Hz or 4 Hz population-activity oscillation, respectively.The external HDTS can be compressed in time to speed replay, and performance is measured by time-averaged correlation with the teaching signal.

Supplementary Note 1: The FORCE Method can Train Spiking Neural Networks to Classify Inputs

FORCE-trained spiking networks classify inputs, obey Dale’s law through synaptic boundaries, and learn temporal signals for song and movie replay. Classification generalizes to new inputs, while HDTS inputs support accurate movie-sequence replay.

  • Input classification: FORCE-trained Izhikevich networks classify inputs separated by linear or nonlinear boundaries and generalize to new inputs with test error below 7%.Targets are upward or downward pulses for the two classes, with a rest period between sequential inputs.
  • Input classification: A 2000-neuron Izhikevich network uses positive and negative components of a 2 Hz sinusoid as binary class outputs.Supplementary figures show repeated classifications and distinguish consistent from inconsistent responses near the class boundary.
  • Dale’s law: Dale’s law is enforced by initializing static weights with excitatory and inhibitory neuron groups and dynamically bounding the learned low-rank weights.The decoders and approximant remain determined through RLS, while synaptic interactions are constrained by neuron type.
  • Song sequence replay: A 5000-neuron Izhikevich network learns the 64-note Ode to Joy sequence together with a 64-dimensional HDTS that encodes temporal position.The HDTS components correspond to successive elements of the supervisor and provide time information during replay.
  • Movie replay: Movie replay is trained with HDTS inputs across frequencies and amplitudes, with supplementary analyses showing reduced replay as HDTS amplitude decreases.The parameter sweep explores HDTS frequency and amplitude over a 10 by 10 mesh.
  • Movie replay: Spike distributions during movie replay are strongly non-uniform across HDTS phase, with a unimodal peak offset from the center phase.The distribution is reported for three repetitions of replay in the externally generated HDTS condition.
Loading 1609.02545v3…