full-FORCE: A Target-Based Method for Training Recurrent Networks

Introduction

• The credit-assignment problem arises because training has a target for the output but not for hidden-unit activity; backpropagation through time can require thousands of effective layers during silent inputs.The authors seek alternatives that can also extend to spiking network models.Source: Introduction · pp. 2–3
• The network uses continuous-time firing-rate units, with tasks specified by mappings from input fin(t) to target output fout(t).The recurrent matrix J, output weights w, and input weights uin are trainable in general, but this paper leaves uin fixed.Source: Introduction · pp. 2–3
• Liquid- and echo-state networks avoid internal targets by training only output weights, while FORCE uses output feedback but restricts effective recurrent changes to low rank and ties them to output weights.These approaches simplify learning but have limited performance or constrained recurrent modifications.Source: Introduction · pp. 3–4
• Extended FORCE generates targets for internal units from the desired output, but its success relies on incomplete minimization that leaves the target as a partial component of unit responses.At zero cost, the resulting dynamics are generally incompatible with reproducing the target output.Source: Introduction · p. 4
• Full-FORCE uses a second randomly connected network driven strongly enough to suppress chaos, with target output fout(t) among its inputs, to generate targets for modifying the full recurrent matrix.Unlike extended FORCE, the approach aims to minimize its cost function as much as possible.Source: Introduction · p. 5
• Full-FORCE constructs networks that perform tasks with fewer units and greater noise resistance than FORCE, with performance degrading smoothly as units decrease or noise increases.Task hints extend the range of learnable tasks and allow dynamics ranging from complex random-network-like behavior to simpler hand-built solutions.Source: Introduction · p. 5

The full-FORCE Algorithm

• The target-generating network receives the desired output as an input, creating activity targets for the task-performing network while remaining unmodified by learning.The target-generating network uses random recurrent connectivity; the task-performing network is the final product.Source: The full-FORCE Algorithm · pp. 5–6
• full-FORCE adjusts the task-performing network’s recurrent matrix so its internal drive matches the target network’s recurrent and externally driven signals.Learning uses recursive least squares; the task-performing network does not receive the target output as an input.Source: The full-FORCE Algorithm · p. 6
• The method assumes the target output can be linearly read out from the target network’s activity, an assumption that can be checked by fitting a readout.This check tests whether the target network acts as an auto-encoder of the desired output.Source: The full-FORCE Algorithm · pp. 6–7
• Even when task-performing and target activities are close, full-FORCE need not approximate FORCE because small deviations can make the connectivity matrix differ substantially.In the principal-component basis, deviations along low-variance components can have large effects as the activity-correlation eigenvalues decrease exponentially.Source: Differences between full-FORCE and… · pp. 7–8
• The deviations separating full-FORCE from FORCE lie in low-variance activity spaces, where modifying recurrent connections helps stabilize fluctuations.Stability is a critical challenge in recurrent networks with non-trivial dynamics.Source: The full-FORCE Algorithm · p. 7, Differences between full-FORCE and… · p. 8

An Oscillation Task

• A 300-unit full-FORCE network performs the task, but a same-sized FORCE network does not; the FORCE units cannot generate the teacher-forced activity.The comparison uses the target-generating network’s connectivity as the FORCE network’s initial random connectivity.Source: An Oscillation Task · p. 8, Figure 2 · p. 9
• 200 units suffice for reliable oscillation-task performance with full-FORCE, whereas FORCE requires 400; full-FORCE also degrades more gradually as network size decreases.The authors attribute this difference to full-FORCE’s superior stability: its networks can remain stable even when inaccurate.Source: An Oscillation Task · p. 8, Figure 3 · p. 10
• Longer training does not appreciably improve unsuccessful networks, and the minimum network size for success is smaller with full-FORCE than with FORCE.Training was extended to 100 batches, each comprising 100 oscillation periods.Source: An Oscillation Task · p. 9, Figure 4 · p. 10

Noise robustness

• Full-FORCE is more noise-resistant, with performance degrading more continuously as network size falls or noise increases.Noise was added as independent Gaussian white noise to each unit during both learning and testing.Source: Noise robustness · pp. 9–10, Figure 5 · p. 11
• Under noise, FORCE performance can worsen as network size increases during partial learning; this effect disappears after successful learning and is not observed with full-FORCE.The authors suggest full-FORCE’s superior stability as the reason the effect does not occur there.Source: Noise robustness · p. 10, Figure 5 · p. 11

Eigenvalues of the recurrent connectivity after learning

• After successful training on the oscillation task, full-FORCE changes the recurrent connectivity eigenvalues more substantially than FORCE, shrinking most into a smaller region.The authors connect pulling eigenvalues with real parts above 1 toward values closer to 0 with stabilization.Source: Eigenvalues of the recurrent… · p. 11

Using Additional Input to the Target-Generating Networks as Hints

• Task hints add a selected fhint input to target-generator dynamics; the signal should fit the task, such as a ramp for timing, and is never supplied to the task-performing network.Hints are provided only to the target-generating network during training.Source: Using Additional Input to the… · pp. 12–13

Interval matching task

• On interval matching, adding a ramp hint raised correct test trials from 4% to 91%, with residual errors mainly modest timing errors at long delays.Without a hint, correct responses occurred only for interpulse intervals shorter than 300 ms.Source: Interval matching task · pp. 13–14, Figure 7 · p. 14

Delayed comparison task

• Hints raised delayed-comparison correct classifications to 95% from 71% in-domain, and 94% from 53% on delays outside the 0.1–1 s training range.Extrapolation was tested on interpulse delays from 20 ms to 2 s.Source: Figure 8 · p. 15, Delayed comparison task · pp. 15–16

• Full-FORCE modifies the full recurrent connectivity, yielding networks that perform complex tasks with few units and considerable noise robustness.Source: Discussion · p. 16
• Well-chosen hints can improve performance on challenging tasks previously considered beyond the range of these recurrent networks.The authors distinguish inability to meet task demands from failure to understand the task; in the latter case, a hint can help.Source: Discussion · p. 17
• For interval-timing and delayed-comparison tasks, silent inputs require memory traces; BPTT would need more than 2,000 unrollings, challenging given gradient instability.The authors do not claim that gradient-based learning cannot learn these tasks.Source: Discussion · p. 17
• Hints can place models along the designed-to-random spectrum and regulate dynamics, addressing cases where random-network complexity exceeds neural data.Source: Discussion · pp. 17–18
• The target-generating network need not match the task-performing network in size, time constant, or connectivity; alternative designs could enhance properties and extend task range.The examples used networks with matching size and time constants and a randomly connected target-generating network.Source: Discussion · p. 18