PaperlayerView exact revision on arXiv

Source-linked AI summary

Recurrent Network Models Of Sequence Generation And Memory

Kanaka Rajan, Christopher D Harvey, David W Tank

arXiv:1603.04687v1q-bio.NCcond-mat.dis-nnphysics.bio-ph

TL;DR

Neural sequence and memory models often rely on specialized pre-wired connectivity, limiting their connection to experimental recordings. This paper uses Partial In-Network Training to modify a small fraction of random connections, finding that disordered recurrent networks reproduce cue- and choice-specific sequences and memory retention through recurrent interactions and external inputs.

  • Problem

    Existing sequence models emphasized specialized pre-wired architectures, while neural sequences are widespread and models need sufficient complexity to connect with experimental recordings.

    Source: INTRODUCTION · p. 2

  • Method

    The authors use Partial In-Network Training to modify selected connections in initially random recurrent networks.

    Source: INTRODUCTION · p. 3, DISCUSSION · p. 14

  • Results

    The networks reproduce cue- and choice-specific neural sequences and retain cue identity during the delay, with propagation relying on recurrent interactions and external inputs rather than feedforward or asymmetric connections.

    Source: SUMMARY · p. 1, DISCUSSION · p. 15

  • Takeaways & Limitations

    Neural sequences may emerge through learning in largely unstructured network architectures.

    Source: SUMMARY · p. 1

    The authors note that the earlier algorithm in [Sussillo & Abbott, 2009] used a separate feedback network from the network generating the trajectories.

    Source: DISCUSSION · p. 14

Abstract

from arXiv · show

Sequential activation of neurons is a common feature of network activity during a variety of behaviors, including working memory and decision making. Previous network models for sequences and memory emphasized specialized architectures in which a principled mechanism is pre-wired into their connectivity. Here we demonstrate that, starting from random connectivity and modifying a small fraction of connections, a largely disordered recur- rent network can produce sequences and implement working memory efficiently. We use this process, called Partial In-Network Training (PINning), to model and match cellular resolution imaging data from the posterior parietal cortex during a virtual memory- guided two-alternative forced-choice task. Analysis of the connectivity reveals that sequences propagate by the cooperation between recurrent synaptic interactions and external inputs, rather than through feedforward or asymmetric connections. Together our results suggest that neural sequences may emerge through learning from largely unstructured network architectures.

AFFILIATIONS … RESULTS

The paper introduces Partial In-Network Training (PINning), showing that largely random recurrent networks can generate experimentally realistic sequences and support working memory after modifying only a small fraction of connections. Applied to posterior parietal cortex data, the framework links sequence generation to cooperation between recurrent interactions and external inputs rather than pre-wired feedforward or asymmetric circuitry.

SUMMARY

  • Different cues and choices activated different sequences, demonstrating task-dependent sequential activity in the modeled network.This extends the framework beyond a single fixed sequence and connects the models to the task structure.Source: SUMMARY · p. 1, INTRODUCTION · p. 3
  • Sequence propagation depended on cooperation between recurrent synaptic interactions and external inputs rather than feedforward or asymmetric connections.The mechanism contrasts with architectures that impose sequence generation through specialized connectivity.Source: SUMMARY · p. 1, INTRODUCTION · p. 3
  • Together, the results establish a modeling framework in which neural sequences emerge through learning from largely unstructured network architectures.The framework combines generic recurrent networks, partial connectivity modification, experimental matching, and analysis of sequence-generation mechanisms.Source: SUMMARY · p. 1

INTRODUCTION

  • The work addresses whether relatively unstructured random networks can produce sequential dynamics resembling experimental recordings while remaining adaptable across tasks.The motivation arises from the ubiquity and variability of neural sequences and the limitations of highly specialized circuit models.Source: INTRODUCTION · pp. 2–3
  • PINning modifies any desired fraction of initially random connections, spanning networks from completely random to fully structured.The synaptic change algorithm enables systematic control over the amount of learned connectivity.Source: INTRODUCTION · p. 3
  • Models built with PINning best matched posterior parietal cortex sequences using minimally structured circuitry with mostly unstructured connections.The modeled data came from mice performing a virtual-reality two-alternative forced-choice task.Source: INTRODUCTION · p. 3
  • The modeled sequences supported short-term memory by retaining whether an early cue indicated a left or right turn during the delay period.The study also examined multiple cue-initiated sequences and computed the capacity of this form of memory.Source: INTRODUCTION · p. 3

1. Sequences from highly structured or random networks do not match PPC data

Highly structured and completely random network architectures failed to reproduce the structured, temporally constrained PPC sequences. This motivated testing whether initially disordered networks could be modified by training to generate realistic activity.

Structured networks

  • PPC sequence stereotypy was low, with bVar = 40% in pooled trial-averaged data and 10–15% in single-trial data, arguing against a moving activity bump.Trial-averaged data from a single mouse had bVar = 15%, and neuronal activity patterns had a weak relationship with anatomical location.Source: §1 · p. 4

Random networks

  • Random networks produced sequential activity, but spontaneous dynamics had excessive background activity, with bVar = 5 + 2% and pVar = 0.15 + 0.1%.Increasing the activation threshold raised bVar to 22%, still below the data value of 40%.Source: §1 · pp. 4–5
  • Adding time-varying external inputs to completely disordered networks still failed to match PPC data, yielding bVar = 10 + 2% and pVar = 0.2 + 0.1%.The resulting sequences were less structured and temporally constrained than the PPC sequences.Source: §1 · p. 5

Implications

  • External inputs and completely random connectivity were insufficient to evoke PPC-like sequences, making such sequences unlikely to be inherent to random networks.Because real sequences arise during learning, the study next asked whether initially disordered networks could be modified by training.Source: §1 · p. 5

2. Temporally Constrained Neural Sequences Emerge With Synaptic Modification

PINning modifies only a selected subset of synapses in a recurrent network, allowing initially disordered connectivity to generate temporally constrained sequences matching PPC activity. Small synaptic modifications reduce output dimensionality while preserving a largely random network architecture.

Experimental targets

  • Training used PPC templates from 437 trial-averaged neurons pooled across 6 mice during left- and right-correct-choice trials in a 2AFC task.The learning rule continued until network rates matched the target functions.Source: §2 · p. 5

PINning method and sequence generation

  • PINning spans architectures from fully random connectivity (p = 0) through partially structured networks with p < 25% task-specific connections to fully trained networks (p = 100%).Only a fraction p of synapses was modified, while the majority remained disordered in the partially structured examples.Source: §2 · p. 6
  • 12% synaptic modification made an initially disordered network’s sequences more temporally constrained (bVar = 40%) and matched PPC data with high fidelity (pVar = 85%).PINning compared neuron inputs with PPC-derived target functions and used RLS or FORCE learning on selected outgoing synapses, leaving the remaining connectivity randomly initialized.Source: §2 · pp. 5–6

Magnitude of synaptic modification

  • Sparsely PINned networks changed individual synapses more but required a smaller total change across the synaptic connectivity matrix.This distinguishes the fraction of modified synapses from the overall magnitude of synaptic change.Source: §2 · p. 6

Population dynamics

  • The PPC-matching network had effective dimensionality Qeff = 14, below the untrained random network’s Qeff = 38 and the data’s Qeff = 24.Qeff asymptoted around 12 dimensions at higher p values; the PINned circuit was higher-dimensional than a sinusoidal bump attractor but lower-dimensional than the data.Source: §2 · pp. 6–7

3. Circuit Mechanism For Sequential Activation Through PINning

This section examines the circuit mechanism underlying sequential activation through PINning, focusing on how localized recurrent interactions, structured fluctuations, and changing external inputs drive sequence propagation. It also considers the sensitivity and robustness of PINned networks to structural and stochastic noise.

  • PINning an idealized 10.5-second sequence achieved pVar = 92% at p = 8%, requiring less plasticity than the p = 12% PPC-like sequence.The idealized sequence lacked experimental irregularities in transient timing and background activity, which reduced the required plasticity.Source: §3 · p. 7
  • At p = 100%, band-averaged connectivity instead formed a localized asymmetric profile and generated a moving Gaussian bump resembling classic moving ring-attractor dynamics.The comparison involved the fully PINned matrix as a useful structured-network benchmark.Source: §3 · p. 8
  • The 8% PINned network’s mean connectivity is symmetric near sequence distance zero, with localized excitation, inhibitory self-interaction, and diffuse flanking inhibition, producing stationary bumps without fluctuations.Neither band averages nor fluctuations alone reproduced the moving sequence; the full sparse matrix was required for sequence-like output.Source: §3 · p. 8
  • Mean synaptic interactions form a localized excitatory bump, while their fluctuations interacting with external inputs create the asymmetry that moves it, enabling non-autonomous sequence propagation.This mechanism does not require asymmetric mean connectivity in sparsely PINned networks and is revealed most clearly through population-level alignment.Source: §3 · pp. 8–9
  • In JPINned, 8% of synaptic weights were modified while the other 92% remained unchanged, producing an asymmetric, heavy-tailed distribution with mean –0.1, variance 2.2, skewness –2, and kurtosis 30.Source: §3 · p. 9
  • PINned networks are sensitive to small amounts of structural noise, including perturbations of 8% in the PINned matrix.Source: §3 · p. 9
  • Training with stochastic noise produces slightly more robust networks.The stochastic-noise procedure is described in Experimental Procedures 2, with the result shown in Supplemental Figure 11.Source: §3 · p. 9

4. Delayed Paired Association And Working Memory Can Be Implemented Through · Sequences In PINned Networks

PINning enables a partially structured recurrent network to perform a delayed two-alternative forced-choice task, supporting the claim that sequences can mediate an alternative form of short-term memory. The network preserves cue identity during identical sensory delays by generating distinct left and right sequences, while matching experimental neural activity with only 16% plastic synapses.

Sequences In PINned Networks

  • The PINned network performed a 2AFC task, supporting sequences as an alternative form of short-term memory.The task was based on the experiment in [Harvey, Coen & Tank, 2012].Source: Sequences In PINned Networks · p. 10
  • The model represented the cue and turn periods with two different time-varying inputs, while the middle maze segment provided a visually identical delay.This setup followed the left/right cue and turn experiences in the 2AFC experiment [Harvey, Coen & Tank, 2012].Source: Sequences In PINned Networks · p. 10
  • During identical sensory delays, distinct left and right sequences maintained the identity of the visual cue required for the correct response.The simulation used shared delay-period inputs for left and right trials, so performance depended on generating more than one sequence.Source: Sequences In PINned Networks · p. 10
  • With p = 16% plastic synapses, the network generated outputs consistent with experimental data, achieving pVar = 85% and bVar = 40%.These results are shown in Figures 5D and 5E and compared with Figure 2c in [Harvey, Coen & Tank, 2012].Source: Sequences In PINned Networks · p. 10
  • The N = 569 network contained 211 left-selective neurons, 226 right-selective neurons, and 132 non-choice-specific neurons.The three groups were selected to activate in left trials, right trials, or both trial types, respectively.Source: Sequences In PINned Networks · p. 10
  • Choice-selective neurons were silenced during opposite-choice delays, whereas non-choice-specific neurons fired in the same order across both trial types.This organization preserved cue memory and reproduced the behavior of no-preference PPC neurons observed experimentally [Harvey, Coen & Tank, 2012].Source: Sequences In PINned Networks · pp. 10–11

5. Comparison With Fixed Point Memory Networks

Sequential-memory networks performed comparably to fixed-point networks for memory selectivity, while idealized sequences achieved higher selectivity with fewer modified synapses. Model noise also reproduced the experimentally inferred level of cue forgetting, near a regime of abrupt performance decline.

Comparison With Fixed Point Memory Networks

  • The three networks were compared by a selectivity index after PINning fractions of synapses against mechanism-specific memory targets.Targets represented stored-variable values: idealized sequential targets, PPC firing rates [Harvey, Coen & Tank, 2012], or constant values for fixed-point memory.Source: §5 · p. 11
  • Selectivity was 0.91 for idealized sequences with p = 10%, versus 0.85 for the PPC-like DPA network with p = 16%.The idealized sequential network used long-duration population dynamics, whereas the PPC-like DPA network matched data [Harvey, Coen & Tank, 2012].Source: §5 · p. 11
  • Both sequential-memory networks had selectivity–p relationships comparable to the fixed-point network, which reached asymptotic selectivity 0.81 at p = 23%.Required synaptic-change magnitudes were also comparable, supporting sequences as a viable alternative to fixed points for memory storage.Source: §5 · p. 11
  • Adding noise reproduced the inferred delay-period forgetting level, although the model’s performance decreased fairly abruptly in that noise regime.The inferred forgetting corresponds to cue identity being lost on 30–40% of trials, based on 83 ± 9% correct behavioral performance [Harvey, Coen & Tank, 2012].Source: §5 · pp. 11–12
  • The model reproduced this forgetting level with a noise amplitude shown in Supplemental Figure 10B.The relevant noise value lies where performance shows a fairly abrupt decrease.Source: §5 · p. 12

6. Capacity Of Sequential Memory Networks

PINned recurrent networks can maintain multiple non-interfering sequential memories, with capacity shaped by sequence sparsity, network size, plasticity, and non-choice-specific neurons. These neurons stabilize sparse sequences and improve both memory capacity and noise tolerance, but cannot rescue inadequately trained networks or excessive noise.

  • A 500-neuron network with p = 25% plastic synapses generated Ns = 5 cue-specific sequences with NActive/N = 3% temporal sparseness.During the 4–8s delay, each sequence activated only for trials matching its cue preference and remained silent on other trial types.Source: §6 · pp. 12–13
  • NActive/N < 1.6% caused the five-cue network to lose cue identity across the delay, but adding NNon-choice-specific/N = 4% neurons restored successful sparse-memory execution.The added neurons formed one temporal sequence shared across trial types, stabilizing five choice-specific sequences without increasing synaptic modification.Source: §6 · p. 13
  • Capacity scaled with p and N and inversely with NActive/N, while non-choice-specific neurons increased the capacity-to-network-size slope by enabling sparser sequences.The capacity metric was the maximum fraction Ns/N; non-choice-specific neurons increased capacity without requiring more synaptic modification.Source: §6 · p. 13
  • Networks with more plastic synapses or non-choice-specific neurons were more stable against stochastic perturbations.Non-choice-specific neurons may act as a recurrent “conveyor belt,” supplying current to sequences too sparse to sustain themselves independently.Source: §6 · p. 13
  • Non-choice-specific neurons failed to restore capacity after noise exceeded a network’s tolerance or when the multi-sequential scheme was inadequately PINned with small p.Thus, their stabilizing effect has limits under excessive stochastic perturbation or insufficient training.Source: §6 · p. 13

DISCUSSION

The Discussion examines how PINning enables largely disordered recurrent networks to reproduce cortical sequence and working-memory features while retaining heterogeneous connectivity and supporting multiple context-dependent tasks. It considers how recurrent interactions and external inputs generate sequences without pre-wired mechanisms.

  • PINning modifies only a fraction of recurrent synapses in an initially random network, preserving heterogeneous wiring while retaining learning efficiency.The plastic fraction is tunable, allowing interpolation between structured and random architectures.Source: DISCUSSION · p. 14
  • PINned networks reproduced choice-specific neural sequences and retained cue-identity memory during the delay period in posterior parietal cortex data.The model matched data from mice performing a virtual-reality, working-memory-based decision task [Harvey, Coen & Tank, 2012].Source: DISCUSSION · pp. 14–15
  • PPC sequence stereotypy was much lower than expected from highly specialized intrinsic connectivity or readout from a structured upstream region, motivating a random-network starting point.The stereotypy measure was bVar (Figure 1C–G).Source: DISCUSSION · p. 15
  • PINned connectivity had a heavy-tailed synaptic-strength distribution caused by a small percentage of strong interactions, consistent with reported cortical strong-synapse structure.The heavy-tailed distribution was shown in Figure 3F.Source: DISCUSSION · p. 15
  • Sequence bumps formed through center-surround-like mean synaptic interactions, whereas their propagation was non-autonomous and depended on recurrent currents interacting with external inputs.This propagation mechanism differs from standard moving bump/ring attractors [Yishai, Bar-Or & Sompolinsky, 1995; Zhang, 1996].Source: DISCUSSION · pp. 15–16
  • The authors propose sequences as a more general and effective dynamical form of working memory, predicting their observation across diverse working-memory tasks and storage of more than two memories.Source: DISCUSSION · p. 16
  • Sparse structured connections embedded in a larger disordered network suffice for sequential timing signals on the order of 10s, suggesting substantial computational bandwidth.The discussion frames this capacity as the bandwidth of a general-purpose neural circuit for different timing-based computations.Source: DISCUSSION · p. 16
  • PINning could sparsely train one network for multiple temporally structured tasks and enable context-dependent switching, supporting future biological predictions about multi-purpose circuits.Candidate tasks include accumulation of evidence and delayed paired association, with analyses applied to areas such as PPC.Source: DISCUSSION · pp. 16–17
  • Unlike pre-wired moving-bump or synfire-chain architectures, these models were constructed without bias or assumed task mechanisms.Here, “pre-wired” denotes incorporating a principled task mechanism into network circuitry before analysis.Source: DISCUSSION · p. 17
  • Starting from random connectivity and learning limited structure revealed an alternative input-dependent sequence-propagation mechanism that pre-wiring a different mechanism would not have uncovered.Although the models were trained for the task, their unbiased construction left open the discovery of mechanisms not specified a priori.Source: DISCUSSION · p. 17

EXPERIMENTAL PROCEDURES

This section describes the network, input, training, and evaluation procedures used to implement PINning and assess network activity, memory, sequence capacity, and connectivity. It also details analyses of synaptic structure across partially and fully PINned networks and related matrix configurations.

13.1. Network elements

  • Random fully connected firing-rate networks use recurrent weights drawn independently from a Gaussian distribution, with a plastic fraction p modified by learning.The control parameter g is set between 1.2 and 1.5, producing chaotic spontaneous activity before modification and regular target-matching outputs afterward.Source: §13.1 · p. 18

13.2. Design of External Inputs

  • External inputs model PPC sensory and proprioceptive stimuli as filtered, spatially delocalized white noise frozen across trials, with optional injected stochastic noise testing memory resilience.Each model neuron receives its own repeated input pattern; injected noise is independently redrawn at every time step.Source: §13.2 · p. 19

13.3. Extracting Target Functions From Calcium Imaging Data

  • PPC calcium-fluorescence traces from Harvey, Coen & Tank, 2012 are converted into firing-rate targets using complementary deconvolution methods that agree well for the dataset.One method assumes a calcium impulse response with 52 ms rise and 384 ms decay, while the other uses fast Bayesian deconvolution; the extracted rates are transformed into PINning targets.Source: §13.3 · pp. 19–20

13.4. Synaptic Modification Rule For PINning

  • PINning compares each neuron’s input with its target function and updates only the selected plastic recurrent weights in proportion to error, presynaptic activity, and inverse rate cross-correlation.The synaptic modification uses the RLS/FORCE procedure, with p denoting both the fraction of plastic neurons’ outgoing weights and the fraction of plastic synapses in the fully connected network.Source: §13.4 · p. 21
  • PINning updates selected synapses through a regularized matrix rule, initializes P as the identity matrix multiplied by a learning-rate factor, and applies learning at every timestep across 500 learning steps.Source: §13.4 · pp. 21–22
  • PINning convergence is assessed by comparing outputs with data or target functions and tracking squared error, while performance uses pVar and the memory Selectivity Index.These measures are reported during learning and at simulation end, with pVar quantifying captured variance and Selectivity quantifying memory-task performance.Source: §13.4 · p. 22

13.5. Dimensionality Of Network Activity (Qeff)

  • Effective dimensionality Qeff is the number of principal components required to capture 95% of variance in the network activity dynamics.Qeff is obtained from PCA of the equal-time cross-correlation matrix of network firing rates, whose normalized eigenvalues distribute variance across orthogonal activity directions.Source: §13.5 · p. 22

13.6. Stereotypy Of Sequence (bVar), %

  • bVar measures the variance explained by translating an invariant activity profile along a sequence, including the stereotypy of individual-neuron transients.The aggregate waveform is formed by averaging center-of-mass-realigned firing rates from trial-averaged PPC data during a 2AFC task [Harvey, Coen & Tank, 2012].Source: §13.6 · pp. 22–23

13.7. Percent Variance Of Data Explained By Model (pVar), %

  • pVar quantifies model fit as the percentage of data variance captured by network outputs, based on the Frobenius-norm discrepancy from trial-averaged data.The measure is one minus the ratio of the squared Frobenius norm of the data-output difference to the variance of the data.Source: §13.7 · p. 23

13.8. Magnitude Of Synaptic Change

  • The normalized mean synaptic change, combined with plastic-synapse fraction p, measures the structure imposed on an initially random network to generate temporally structured dynamics.This metric is applied to PPC-like and idealized sequences and to three memory tasks, comparing the PINned connectivity with the initial random matrix.Source: §13.8 · p. 23

13.9. Selectivity Index For Memory Task

  • The Selectivity Index evaluates cue-specific memory maintenance during delayed paired-association tasks by comparing preferred-neuron activity on preferred versus opposite trials at the delay endpoint.The delay endpoint is approximately 10 s in Figure 5 and about 7 s in Figure 6, following [Harvey, Coen & Tank, 2012].Source: §13.9 · p. 24

13.10. Temporal Sparseness Of Sequences (NActive/N)

  • Sparser sequences increase capacity for multiple non-interfering sequences without increasing plastic-synapse fraction or synaptic-change magnitude, but propagation fails at NActive/N = 1.6%.For Figure 2A data, NActive/N = 3% corresponds to 16 of 437 neurons; reducing sparseness narrows sequences, whereas below the critical value insufficient current prevents propagation.Source: §13.10 · p. 24

13.11. Analyzing the Structure of PINned Synaptic Connectivity Matrices

  • The procedure computes means and standard deviations for the principal diagonals of three connectivity matrices, including sparsely PINned, randomly initialized, and fully PINned matrices.Source: §13.11 · p. 25
  • It applies the same mean-and-standard-deviation analysis to successive off-diagonal stripes moving away from the principal diagonal.Source: §13.11 · p. 25
  • The resulting profiles are plotted for the sparsely PINned matrix relative to random initialization and for the fully PINned matrix, with additional comparisons shown in Supplemental Figure 9.Source: §13.11 · p. 25

FIGURES AND CAPTIONS

The figures and captions illustrate how PINning transforms randomly connected networks into sparse, temporally constrained sequence generators that match PPC-like working-memory dynamics. They examine the roles of plasticity, connectivity, external inputs, network size, and temporal sparsity in sequence propagation, task selectivity, capacity, and convergence.

FIGURES AND CAPTIONS

  • An untrained random network appears time ordered after sorting neurons by tCOM, but its sequence contains substantial extra-sequential or background activity.This contrasts with the stereotyped PPC activity and the cleaner sequence produced after PINning.Source: Figure 1 · p. 26, FIGURES AND CAPTIONS · p. 26

A Rates from correct trials of 2AFC task

  • With p = 12% plastic synapses, the PINned network matches PPC sequences with bVar = 40% and pVar = 85%.The activity is temporally constrained and contains relatively little extra-sequential activity.Source: Figure 2 · p. 28, A Rates from correct trials of 2AFC task · pp. 28–29
  • PINning modifies only recurrent synapses from a randomly selected fraction p of the 437 neurons, using activity-based updates toward staggered target waveforms spanning the task.The targets are extracted from PPC firing rates, and the resulting neuron activity forms staggered bumps across the task duration.Source: A Rates from correct trials of 2AFC task · p. 28
  • pVar rises from 0 at p = 0 to 50% at p = 8%, then asymptotes at 85% for p > 12%.Thus, a small fraction of plastic synapses is sufficient to reproduce the PPC-like sequence.Source: A Rates from correct trials of 2AFC task · p. 29
  • The normalized mean synaptic change grows from ~7 at p = 12% to ~9 at p = 100%, while total matrix-wide change is smaller for small-p networks.Individual synapses change more in sparsely PINned networks, but fewer synapses are modified overall.Source: A Rates from correct trials of 2AFC task · p. 29
  • At p = 12%, 14 principal components capture over 95% of variance in the 437-neuron network, reducing effective dimensionality below the 38 components required by the random untrained network.Source: A Rates from correct trials of 2AFC task · p. 29
  • Fluctuations around the mean interaction are much larger and more structured for JPINned, 8% than for JRand, while JPINned, 100% band-averages are bigger, more asymmetric, and negative near i − j = 0.Fluctuations around JPINned, 100% band-averages are smaller than those for JPINned, 8%.Source: A Rates from correct trials of 2AFC task · pp. 30–31
  • Mean JPINned, 8% band-averages form the activity bump, but their averaged connectivity alone yields a fixed point rather than a sequence.In contrast, fully PINned mean band-averages can evoke a moving Gaussian-like bump because their asymmetry drives movement.Source: A Rates from correct trials of 2AFC task · pp. 30–31
  • The sequence forms through mean JPINned, 8% interactions, while movement arises from fluctuations cooperating with external inputs and their asymmetric summed current.The aligned population average of fluctuation and external-input currents reveals the asymmetry responsible for moving the bump across the network.Source: A Rates from correct trials of 2AFC task · p. 32
  • After PINning, left-preferring neurons activate only on left trials, right-preferring neurons only on right trials, and non-choice-specific neurons on both trial types.The network uses plastic outgoing synapses from small subsets of randomly interconnected neurons, with targets taken from PPC calcium-imaging firing rates.Source: A Rates from correct trials of 2AFC task · pp. 33–34
  • A 569-neuron recurrent network with p = 16% plastic synapses matches the experimental PPC firing-rate data, with pVar = 85%.The simulated task inputs are choice-specific during cue and turn periods but coalesce into a shared time-varying pattern during the delay.Source: A Rates from correct trials of 2AFC task · pp. 34–35
  • For three PINned working-memory networks—an idealized sequential model, a PPC-like-dynamics model, and a fixed-point memory network—the idealized-sequence network had selectivity = 0.91 at p = 10%.Source: A Rates from correct trials of 2AFC task · p. 35
  • The memory task is correctly executed through five choice-specific sequences; during the delay, neurons fire only on trials matching their cue preference and remain silent otherwise.Sequential activation alone does not maintain cue identity during the delay, whereas non-choice-specific neurons restore cue-identity memory.Source: A Rates from correct trials of 2AFC task · p. 37
  • Maximum sequence-carrying capacity Ns is directly proportional to pN and inversely proportional to NActive/N.Capacity decreases as network size, plasticity fraction, or temporal sparseness changes in the directions described by the figure, while additional non-choice-specific neurons moderately reduce it.Source: A Rates from correct trials of 2AFC task · p. 37
  • Memory-task noise tolerance decreases as p is lowered and capacity increases, but falls more slowly when non-choice-specific neurons are included.The figure evaluates resilience using the critical stochastic noise level tolerated before task failure as a function of Ns/N.Source: A Rates from correct trials of 2AFC task · p. 37
  • Deconvolution converts PPC Ca2+ fluorescence into normalized firing-rate estimates and re-convolution reproduces inferred Ca2+ signals for comparison with the data.The Bayesian method supplies probabilistic spike trains, which are used as normalized rate estimates given the imaging frame rate.Source: A Rates from correct trials of 2AFC task · pp. 38–39
  • PINning error drops below 0.02 for both p = 8% and p = 100% plastic-synapse networks before the 500th learning step; a p = 8% network reached 0.018 at 500 steps.Source: A Rates from correct trials of 2AFC task · p. 40

Inputs

Filtered white-noise inputs drive a random recurrent network in a two-alternative forced-choice task, but cue identity disappears during the delay as inputs become cue-invariant. Partial in-network training produces idealized sequences with sparse modification, low-dimensional activity, and altered connectivity eigenmodes.

Inputs

  • During 4–7 s delays, both left- and right-preferring neurons remain active, but cue memory disappears because inputs coalesce and become cue-invariant.The network is a 436-neuron random recurrent system driven by filtered white-noise inputs for a two-alternative forced-choice task.Source: Inputs · p. 41
  • At p = 8%, PINning produces pVar = ~92%, while the total synaptic change rises from ~3-fold to 8–9-fold as p increases to 100%.Although individual synapses change more in sparsely PINned networks, the total connectivity change is smaller.Source: Inputs · p. 42
  • Only 10 of 500 principal components explain over 95% of activity variance in the p = 8%, pVar = 92% PINned sequence network.This corresponds to Qeff = 10, indicating strongly low-dimensional sequential activity.Source: Inputs · p. 43
  • PINning changes 8% of the weights, with the largest changes showing a negative bias, while the modified network develops several large positive and negative non-zero eigenmodes.The random network’s eigenvalues lie uniformly within a circle, whereas the PINned network’s eigenvalues are distributed non-uniformly.Source: Figure S5 · p. 44, Inputs · p. 44
  • A network with p = 23% plastic synapses has an asymptotic selectivity of 0.81.Source: Inputs · p. 45

PINned Networks

PINning remains effective when many neurons are left untrained: target variance is largely preserved, although full-network outputs become noisier and less stereotyped. This robustness holds for both idealized and PPC-like sequence-generating networks.

PINned Networks

  • Overall, pVar does not decrease appreciably with more untrained neurons and can improve slightly in sparsely PINned networks.The improvement is observed for p = 10% idealized and p = 15% PPC-like networks, and becomes smaller as p increases.Source: PINned Networks · pp. 46–47
  • Untrained neurons model unobserved active cells but can add irregularity that reduces stereotypy in full-network outputs.This reduction is indicated by decreased bVar computed across targeted and non-targeted neurons.Source: PINned Networks · p. 47
  • Target variance remains high as untrained-neuron fractions increase, reaching 90%, 95%, and 98% for Nnt/N = 5%, 50%, and 75% in an idealized sequence network.Sequences become noisier as randomly fluctuating untrained neurons are added, while variance captured by PINned neurons remains largely unaffected.Source: PINned Networks · p. 47
  • The same trend appears in a 437-neuron PPC-like network, with pVar = 80%, 85%, and 88% for Nnt/N = 25%, 50%, and 75%.These values confirm that increasing the untrained fraction does not appreciably reduce target variance.Source: PINned Networks · p. 47

Random Network Output

Random networks produce increasingly stereotyped sequences as output sparsity thresholds rise, but an additive structured-random hybrid performs poorly. PINned networks remain functionally useful under input and structural noise, with noise-aware training improving robustness.

Random network outputs

  • Output stereotypy bVar increases from 0 to 12% at threshold 0 and saturates at 22% for thresholds above 2.Example networks show bVar values of 12% at threshold 0 and 22% at threshold 5.Source: Random Network Output · p. 48

Structured-random hybrid

  • The additive hybrid JHybrid = JMoving bump + JRand achieves only 1% pVar, despite containing structured moving-bump and random connectivity components.Its band averages are larger and more asymmetric than those of JPINned, 12%, while its fluctuations are less structured and arise from JRand.Source: Random Network Output · pp. 49–50

Input-noise robustness

  • Increasing stochastic input noise reduces explained variance, while both idealized-sequence and PPC-like memory networks show finite noise tolerance.Tolerance is defined where pVar falls below 50%; the supplied passage reports distinct tolerances for the two memory networks but omits their numeric values.Source: Figure S10 · p. 51, Random Network Output · pp. 51–52

Working-memory task

  • The network achieves selectivity 0.91 in one condition, but the same network fails the task (selectivity = 0) under high stochastic input noise that quenches delay-period memory.Source: Random Network Output · p. 52

Structural-noise robustness

  • Structural-noise tolerance increases from 0.6 without input noise during training to 0.8 with noise-aware training, indicating slightly more robust PINned networks.Tolerance is the structural-noise amplitude at which pVar drops below 50%.Source: Figure S11 · p. 53, Random Network Output · p. 53

Supplemental Data S12: Cross-Validation Analysis

Cross-validation split 436 neurons into two 218-neuron synthetic sequences and trained PINning networks on each sequence. Each network captured its training sequence well and the held-out sequence nearly as well as the data-to-data variance ceiling, unlike a random network.

Cross-Validation Analysis

  • PINning captured 91 + 2% of Sequence A when trained on A and 90 + 2% of Sequence B when trained on B.The analysis used 218 model neurons and evaluated all fractions of plastic synapses.Source: Supplemental Data S12:… · p. 54
  • Cross-sequence generalization reached 45 + 2% for A tested after training on A and 46 + 2% for B tested after training on B.The complementary sequence tests quantify how well each trained network reproduces the other synthetic sequence.Source: Supplemental Data S12:… · p. 54
  • The PINning model nearly reached the 49% variance explained between the two data sets, whereas a random network captured only 0.2%.The random-network comparison refers to the network shown in Figure 1B, with the random-network result also reported for Figure 1H.Source: Supplemental Data S12:… · p. 54
Loading 1603.04687v1…