Recurrent Network Models Of Sequence Generation And Memory

Kanaka Rajan; Christopher D Harvey; David W Tank

Source-linked AI summary

Recurrent Network Models Of Sequence Generation And Memory

Kanaka Rajan, Christopher D Harvey, David W Tank

arXiv:1603.04687v1Mar 14, 2016q-bio.NCcond-mat.dis-nnphysics.bio-ph

TL;DR

The paper asks whether neural sequences and working memory require specialized, pre-wired connectivity. Using Partial In-Network Training to modify a small fraction of random recurrent connections, it shows that disordered networks can generate cue- and choice-specific sequences through recurrent interactions and external inputs.

Problem
Existing sequence and memory models emphasized specialized architectures with mechanisms pre-wired into connectivity, despite neural sequences appearing across brain regions.
Method
Partial In-Network Training modifies a desired fraction of connections in initially random recurrent networks using a synaptic change algorithm.
Results
The networks generated different sequences for different cues and choices, with propagation relying on recurrent interactions and external inputs rather than feedforward or asymmetric connections.
Takeaways & Limitations
Neural sequences may emerge through learning in largely unstructured network architectures.
Takeaways & Limitations
The discussed prior algorithm used a separate feedback network from the network generating the actual trajectories.

Abstract

from arXiv · show

Sequential activation of neurons is a common feature of network activity during a variety of behaviors, including working memory and decision making. Previous network models for sequences and memory emphasized specialized architectures in which a principled mechanism is pre-wired into their connectivity. Here we demonstrate that, starting from random connectivity and modifying a small fraction of connections, a largely disordered recur- rent network can produce sequences and implement working memory efficiently. We use this process, called Partial In-Network Training (PINning), to model and match cellular resolution imaging data from the posterior parietal cortex during a virtual memory- guided two-alternative forced-choice task. Analysis of the connectivity reveals that sequences propagate by the cooperation between recurrent synaptic interactions and external inputs, rather than through feedforward or asymmetric connections. Together our results suggest that neural sequences may emerge through learning from largely unstructured network architectures.

AFFILIATIONS … RESULTS

The paper introduces PINning, which modifies a small fraction of initially random connections to model flexible neural sequences and working memory. Applied to posterior parietal cortex data, the approach yields minimally structured networks whose sequences depend on recurrent interactions cooperating with external inputs.

SUMMARY: Different cues and choices activate different neural sequences, enabling the model to represent multiple task-dependent trajectories.The study also analyzes multiple cue-initiated sequences and their short-term-memory capacity.
RESULTS: Sequence propagation arises from cooperation between recurrent synaptic interactions and external inputs rather than feedforward or asymmetric connections.This mechanism contrasts with specialized sequence architectures such as synfire chains and ring attractors.
SUMMARY: The framework develops a generic modeling approach in which largely unstructured network architectures acquire sequence-generating dynamics through learning.The authors extend beyond reproducing recordings by constructing models that extrapolate beyond the experimental data.
INTRODUCTION: The work addresses the ubiquity and variability of transient neural sequences by proposing general circuit-level mechanisms learned from task experience.Sequences occur across multiple brain regions and span diverse time durations, motivating models beyond highly specialized connectivity.
INTRODUCTION: PINning modifies any desired fraction of initially random connections, spanning networks from completely random to fully structured architectures.This framework tests how much connectivity structure is required to support experimentally observed sequences.
INTRODUCTION: PPC-like sequences are most consistent with minimally structured circuitry: a small structured component supports sequential activity amid a much larger unstructured network.The models target sequences recorded from mice performing a virtual-reality 2AFC task.
INTRODUCTION: The modeled sequences support short-term memory by storing whether the task indicated a left or right turn during the delay period.This directly links sequential dynamics to memory demands in the 2AFC task.

1. Sequences from highly structured or random networks do not match PPC data

Highly structured connectivity does not explain PPC sequence variability: moving-bump stereotypy is limited, and completely random networks fail to reproduce the data even with external inputs. These results motivate training initially disordered networks to generate realistic sequences.

1. Sequences from highly structured or random networks do not match PPC data: 40% of variance in trial-averaged, cross-animal PPC data was explained by a translated invariant activity profile, falling to 10–15% in single-trial data and 15% in one-mouse averages.The low bVar, together with weak relationships between neuronal activity patterns and anatomical location, motivated disordered network architectures.
1. Sequences from highly structured or random networks do not match PPC data: Because realistic PPC sequences arise during task learning, the study next asked whether training could modify initially disordered networks to produce them.Balanced random networks can be chaotic, whereas external stimuli can channel their dynamics and reduce variability, motivating this training question.
1. Sequences from highly structured or random networks do not match PPC data: 5 + 2% bVar and 0.15 + 0.1% pVar characterized ordered chaotic spontaneous activity, while sparsification increased bVar only to 22%.The random network used N = 437 rate-based neurons, with firing rates normalized and sorted by center-of-mass time to match the experimental analysis.
1. Sequences from highly structured or random networks do not match PPC data: 10 + 2% bVar and 0.2 + 0.1% pVar resulted when time-varying visual-stimulus inputs drove the random network, still failing to match PPC data.The comparison metrics quantify sequence stereotypy through bVar and the percentage of PPC variance captured through pVar.
1. Sequences from highly structured or random networks do not match PPC data: External inputs and completely disordered connectivity produced sequences that did not match PPC data, which were more structured and temporally constrained.The authors conclude that sequences observed during timing and memory experiments are unlikely to be an inherent property of completely random networks [Harvey, Coen & Tank, 2012].

2. Temporally Constrained Neural Sequences Emerge With Synaptic Modification

PINning modifies only a small, randomly selected subset of synapses so recurrent networks can generate temporally constrained sequences matching PPC activity. With p = 12% modification, networks achieved high-fidelity outputs while remaining largely disordered and exhibited reduced effective dimensionality.

PINning method: PINning applies RLS or FORCE learning only to outgoing synapses from a randomly selected subset of neurons, leaving the remaining randomly initialized connections unmodified.Each neuron has a target function, but only a fraction p of the total synapses is plastic.
Architecture spectrum: PINning spans architectures from fully random networks (p = 0) through partially structured networks with p < 25% task-specific connections to fully trained networks (p = 100%).The examples in Figures 2, 5, and 6 use less than 25% modified connections while most connections remain disordered.
Synaptic modification: Sparsely PINned networks required less total connectivity change despite larger changes at individual modified synapses.This compares the aggregate synaptic change across the connectivity matrix for different values of p.

3. Circuit Mechanism For Sequential Activation Through PINning

PINning produces sequential activity through sparse, largely symmetric recurrent interactions whose structured fluctuations cooperate with changing external inputs to move an activity bump. This non-autonomous mechanism differs from fully PINned ring-attractor-like connectivity and requires only a small fraction of modified weights.

Connectivity structure: Sparse PINning produced symmetric localized excitation with inhibitory self-interaction and diffuse flanking inhibition, whereas full PINning produced localized asymmetric connectivity and a moving Gaussian bump.The full-PINning dynamics were qualitatively similar to moving ring-attractor dynamics [Yishai, Bar-Or & Sompolinsky, 1995; Zhang, 1996].
Connectivity structure: Neither the sparse network’s band-averaged connectivity nor its fluctuations alone generated moving sequences; combining both components instead produced stationary bumps.Synthetic networks were driven by the same external inputs as the original PINned networks.
Mechanism: Mean interactions in sparse PINned networks form a localized excitatory bump, while their fluctuations interacting with external inputs move it across the network.Population-aligned inputs revealed the movement-producing asymmetry, which is difficult to visualize at individual-neuron resolution because of fluctuations.
Synaptic weight changes: At p = 8%, sequence-facilitating changes were concentrated in a small fraction of strong, asymmetric, heavy-tailed synaptic weights, while 92% remained unchanged.Decreasing p increased the magnitudes spanned by synaptic weights; the partially structured matrix had mean −0.1, variance 2.2, skewness −2, and kurtosis 30.

4. Delayed Paired Association And Working Memory Can Be Implemented Through · Sequences In PINned Networks

PINning enabled a largely disordered recurrent network to perform a 2AFC task by using distinct neural sequences to maintain cue identity during identical sensory-input delays. With only 16% plastic synapses, the network generated outputs consistent with experimental data and reproduced choice-specific and non-choice-specific sequential activity.

Sequences In PINned Networks: The simulated task used separate time-varying inputs for cue and turn periods, while neuron-specific inputs converged to the same waveform during both delay conditions.Correct performance consequently required the network to generate more than one sequence despite identical delay-period inputs.
Sequences In PINned Networks: Distinct left and right sequences maintained visual-cue identity during a delay when sensory inputs were identical.The task therefore demonstrates that sequences can mediate an alternative form of short-term memory.
Sequences In PINned Networks: With p = 16% plastic synapses, the network produced outputs consistent with experimental data, achieving pVar = 85% and bVar = 40% (Figures 5D and E).The comparison also references Figure 2c in [Harvey, Coen & Tank, 2012].
Sequences In PINned Networks: The network comprised 211 left-selective, 226 right-selective, and 132 non-choice-specific neurons among N = 569 network neurons.The selected neurons were assigned to activate in left, right, or shared sequence orders, respectively.
Sequences In PINned Networks: During right trials, left-preferring neurons were silenced, whereas during left trials, right-preferring neurons were silenced, allowing active neurons to generate the appropriate sequence.This selective silencing preserved cue memory across the delay.
Sequences In PINned Networks: Non-choice-specific neurons activated sequentially in the same order for both trial types, matching experimentally observed no-preference PPC neurons.The corresponding outputs are shown in Figure 5E and Supplemental Figure 7b in [Harvey, Coen & Tank, 2012].

5. Comparison With Fixed Point Memory Networks

Sequential-memory networks matched fixed-point networks in selectivity as a function of the modified-synapse fraction, while idealized sequences achieved higher selectivity with fewer modified synapses than the PPC-like DPA network. Comparable synaptic changes and noise robustness support sequences as a viable memory mechanism, with model noise reproducing experimentally observed forgetting.

Comparison With Fixed Point Memory Networks: The comparison embedded different fractions of PINned synapses against targets representing stored variables, using idealized sequential, PPC-derived sequential, or constant fixed-point values.The PPC-derived targets used firing rates from PPC data [Harvey, Coen & Tank, 2012], while fixed-point targets were constant valued.
Comparison With Fixed Point Memory Networks: Both sequential-memory networks performed comparably with the fixed-point network in their selectivity-p relationships, and required comparable magnitudes of synaptic change.These results suggest sequences may be a viable alternative to fixed points for storing memories in neural circuits.

6. Capacity Of Sequential Memory Networks

PINned recurrent networks can store multiple non-interfering sequential memories, with capacity determined by plasticity, network size, and sequence sparseness. Non-choice-specific neurons stabilize sparse sequences, increase capacity, and improve noise tolerance, but cannot rescue inadequately PINned networks or noise beyond tolerance.

6. Capacity Of Sequential Memory Networks: A 500-neuron network with p = 25% plastic synapses generated Ns = 5 cue-specific sequences with NActive/N = 3% during the delay.Each sequence was active only for trials matching its cue preference and remained silent on other trial types.
6. Capacity Of Sequential Memory Networks: Networks failed to maintain five cue identities when sequences were sparser than NActive/N < 1.6%, but adding NNon-choice-specific/N = 4% neurons rescued performance.The added neurons fired in the same temporal order across all trial types, providing one shared sequence alongside the five choice-specific sequences.
6. Capacity Of Sequential Memory Networks: Non-choice-specific neurons may act as a working-memory “conveyor belt” by supplying recurrent synaptic current to sparse sequences that otherwise cannot sustain themselves.This proposed role is consistent with the presence of such neurons in posterior parietal cortex [Harvey, Coen & Tank, 2012].
6. Capacity Of Sequential Memory Networks: Memory capacity Ns/N scaled with p and N and inversely with NActive/N, while non-choice-specific neurons increased the capacity slope by enabling sparser sequences.Their presence increased the capacity of sequence-based memory networks without requiring more synaptic modification.
6. Capacity Of Sequential Memory Networks: Higher plastic-synapse fractions and non-choice-specific neurons made networks more stable to stochastic perturbations, until noise exceeded each network’s tolerance.Beyond that tolerance, non-choice-specific neurons no longer repaired memory capacity.

DISCUSSION

PINning extends reservoir-style sequence models by modifying only a small fraction of recurrent synapses in an initially random network, reproducing key PPC working-memory dynamics without pre-wired specialized circuitry. The resulting sequences arise through input-dependent interactions between recurrent connectivity and external inputs, supporting multi-purpose sequential memory.

PINning: PINning modifies only a fraction of recurrent synapses in an initially random, heterogeneously wired network, avoiding separate feedback circuitry and readout-only learning while retaining learning efficiency.The plastic fraction is tunable, allowing interpolation between structured and random architectures.
PPC application: PINned networks reproduced PPC data features including choice-specific neural sequences and cue-identity memory retention during the delay.The data came from mice performing a virtual-reality working-memory decision task [Harvey, Coen & Tank, 2012].
PPC application: PPC sequence stereotypy was much lower than expected from highly specialized intrinsic sequence-generating connectivity or sequential readout from a structured upstream region.This motivated starting from random recurrent networks and adding only a small amount of structure through PINning.
Connectivity: PINned connectivity developed a heavy-tailed synaptic-strength distribution containing a small percentage of strong interactions, consistent with a sparse structured component embedded in disorder.The model also showed sensitivity to structural noise that training did not completely remove.
Circuit mechanisms: Sequence propagation was non-autonomous, relying on complex interplay between recurrent connections and external inputs rather than a pre-wired moving-bump or ring mechanism.PINning produced localized excitation with center-surround-like connectivity, while propagation depended on input-dependent dynamics.
Implications: The authors propose that sequences are a general and effective dynamical form of working memory, predicting their presence across diverse working-memory tasks and enabling multi-purpose timing-based computations.Sequential-memory capacity is framed as the computational bandwidth of a general-purpose neural circuit, with small structured connections embedded in a larger disordered network.

EXPERIMENTAL PROCEDURES

The experiments use fully connected firing-rate networks with initially random recurrent weights, selectively modified synapses, and externally driven inputs to reproduce target activity. Network states, sequence performance, and memory selectivity are quantified using PCA-based analyses and task-specific metrics.

Network model: Networks contain N fully interconnected firing-rate neurons with recurrent weights drawn independently from a Gaussian distribution, while only a fraction p of synapses is plastic during PINning.The remaining synapses are held fixed, allowing activity-dependent modification of a small subset of initially disordered connections.
External inputs: External inputs model sensory and proprioceptive stimulation as filtered, spatially delocalized white noise that is frozen and repeated identically across simulated trials.Each model neuron receives its own input pattern, and separate experiments additionally inject independently varying Gaussian noise to assess memory resilience.
Target functions: PINning targets are derived from PPC calcium fluorescence by complementary calcium deconvolution methods whose estimated firing rates agree well for the dataset.One method uses an alpha-function approximation with a 200 ms time constant, while the other uses fast Bayesian deconvolution with 52 ms rise and 384 ms decay times.
Analysis metrics: PCA-based state-space analysis diagonalizes firing-rate cross-correlations, while Selectivity Index measures cue-specific delay-period memory from preferred-neuron activity during preferred versus opposite trials.The Selectivity Index is based on the ratio of the difference to the sum of the corresponding mean activities.

FIGURES AND CAPTIONS

The figures show that randomly connected networks can exhibit time-ordered activity, while PINning produces temporally constrained PPC-like sequences and supports cue-specific working-memory sequences with limited synaptic modification.

Figure 2: Partial In-Network Training (PINning) Matches PPC-Like Sequences: PINning modifies recurrent synapses from a randomly selected neuron subset toward target waveforms, with p controlling the fraction of plastic neurons.The scheme updates selected recurrent weights at each time step according to the difference between neuronal input and its target waveform.
Figure 2: Partial In-Network Training (PINning) Matches PPC-Like Sequences: PINning yields PPC-like sequences with little extra-sequential activity and explains 85% of the experimental variance.The caption reports bVar = 40% and pVar = 85% for the PINned network.
Figure 2: Partial In-Network Training (PINning) Matches PPC-Like Sequences: At p = 12%, 14 principal components explain over 95% of network variance, compared with 38 for the random untrained network.The caption denotes this effective dimensionality by Qeff.
Figure 2: Partial In-Network Training (PINning) Matches PPC-Like Sequences: In a 500-neuron network, five choice-specific sequences correctly maintain cue identity during the delay, whereas adding non-choice-specific neurons restores memory without increasing p.Choice-preferring neurons are active only for matching cue trials, while non-choice-specific neurons participate in both trial types.

Inputs

The inputs show that random recurrent networks can lose cue identity during delay periods, whereas sparse PINning produces idealized sequences with high variance capture. PINned connectivity also reshapes network modes and supports either sequential or fixed-point working memory depending on training strength.

Random-network inputs: In a 436-neuron random network, left- and right-preferring neurons are active during both delay conditions, and cue memory disappears as inputs become cue-invariant.The cue memory is lost between 4–7 s because the inputs coalesce during the delay.
Idealized sequence generation: At p = 8%, PINning produces an idealized sequence capturing ~92% of the target variance, with performance plateauing at higher PINning fractions.Performance rises from 0 in the unmodified network and reaches pVar = ~92% for and above p = 8%.
Idealized sequence generation: The 500-neuron sequence network is low-dimensional: 10 principal components capture over 95% of activity variance at p = 8%.This corresponds to Qeff = 10 out of 500 possible components.
Connectivity and network modes: PINning alters only a scattered subset of weights, but 8% PINning redistributes eigenvalues into several large positive and negative modes.The largest changes involve 40,000 weights, equal to 8% of the total, with a negative bias in their spread.

PINned Networks

PINned networks retain high target-function variance as the fraction of untrained neurons increases, although their full-network outputs become noisier and less stereotyped. This robustness holds for both idealized and PPC-like sequences, with slight improvements in targeted-neuron variance under sparse PINning.

PINned Networks: Overall, increasing the relative number of untrained neurons did not appreciably reduce pVar, and sparse PINning slightly improved it.The improvement occurred at p = 10% for the idealized sequence and p = 15% for the PPC-like sequence, becoming smaller as p increased.
PINned Networks: Untrained neurons model unobserved active cells that may influence experimental activity, but they can add irregularity and reduce full-network output stereotypy.This reduction was indicated by a decrease in bVar computed over targeted and non-targeted neurons.
PINned Networks: pVar remained high as untrained neurons increased, reaching 90%, 95%, and 98% at Nnt/N = 5%, 50%, and 75% in an idealized sequence.The sequences became noisier overall as randomly fluctuating untrained neurons were added, while target variance remained largely unaffected.
PINned Networks: For a 437-neuron PPC-like network, pVar likewise increased from 80% to 85% and 88% as Nnt/N rose from 25% to 50% and 75%.These values confirmed the same general trend observed in the idealized sequence.

Random Network Output

Random networks produce increasingly stereotyped sequences as output sparsity thresholds rise, but an additive structured-random hybrid produces poorly stereotyped outputs. PINned networks remain functional under input and structural noise, with noise-trained networks somewhat more robust.

Random network outputs: Output stereotypy rises from 0 to 12% at threshold = 0 and saturates at 22% for thresholds > 2.The threshold = 0 networks were used as initial configurations throughout the paper.
Structured-random hybrid: An additive hybrid combining moving-bump and random connectivity produces outputs with only 1% explained variance and less structured connectivity fluctuations than PINned networks.Its band averages are larger and more asymmetric, while remaining positive near the principal diagonal.
Input noise robustness: Input stochastic noise reduces explained variance, with tolerance defined as the noise amplitude at which pVar falls below 50%.This test assesses whether sequences learned through PINning remain stable under perturbations.
Structural noise robustness: Training with stochastic input noise increases structural-noise tolerance from 0.6 to 0.8, indicating slightly more robust PINned networks.The comparison uses the structural-noise amplitude at which pVar drops below 50%.

Supplemental Data S12: Cross-Validation Analysis

Cross-validation showed that PINned networks captured about 90% of variance in held-out data from the same synthetic sequence but about 45% from the other sequence. This performance approached the 49% cross-sequence variance ceiling and far exceeded the 0.2% captured by a random network.

Supplemental Data S12: Cross-Validation Analysis: The analysis split 436 neurons into two 218-neuron synthetic sequences and trained separate 218-neuron PINning networks on each sequence.Even-numbered cells formed Sequence A, odd-numbered cells formed Sequence B, and each network was evaluated on both sequences.
Supplemental Data S12: Cross-Validation Analysis: 91 + 2% pVarTest A, Train A and 90 + 2% pVarTest B, Train B showed that PINned networks captured nearly all variance within their trained sequences.The corresponding cross-sequence values were 45 + 2% and 46 + 2%, indicating limited generalization between the synthetic sequences.
Supplemental Data S12: Cross-Validation Analysis: 0.2% pVarTest Data, Random Network showed that a random network captured only a tiny fraction of the data variability.This comparison used the random network shown in Figure 1B and also referenced the right panel of Figure 1H.
Supplemental Data S12: Cross-Validation Analysis: 49% pVar marked the variance that one synthetic data set accounted for in the other, so the model approached the maximum possible cross-sequence performance.The authors therefore concluded that the model performed almost as well as possible on the cross-validation comparison.