Source-linked AI summary
Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules
Wulfram Gerstner, Marco Lehmann, Vasiliki Liakoni, Dane Corneil, Johanni Brea
TL;DR
Elementary behaviors unfold over seconds whereas neuronal spikes occur over milliseconds, creating a need for learning rules that bridge these timescales. This review examines neoHebbian three-factor rules and recent experiments testing whether synaptic eligibility traces can link neuronal co-activation to later modulatory signals. The reviewed evidence supports eligibility traces operating on behavioral timescales, while some hippocampal findings may instead reflect longer-timescale synaptic consolidation.
Problem
Brain learning rules must bridge millisecond neuronal activity and behaviors or memories unfolding over seconds, but direct evidence for second-scale eligibility traces was previously limited.
Method
The paper reviews experimental studies testing three-factor plasticity rules in which pre- and postsynaptic activity creates an eligibility trace converted into a weight change by a third signal.
Results
The reviewed experiments support eligibility traces combined with neuromodulatory signals, including approximately 1-second traces in striatum and 5–10-second traces in cortex.
Takeaways & Limitations
Eligibility traces provide experimental support for neoHebbian three-factor learning rules as a biological mechanism linking synaptic activity with later special-event signals.
Takeaways & Limitations
Some hippocampal findings involve eligibility traces lasting minutes and may relate to synaptic consolidation rather than reinforcement learning.
Abstract
from arXiv · showhide
Most elementary behaviors such as moving the arm to grasp an object or walking into the next room to explore a museum evolve on the time scale of seconds; in contrast, neuronal action potentials occur on the time scale of a few milliseconds. Learning rules of the brain must therefore bridge the gap between these two different time scales. Modern theories of synaptic plasticity have postulated that the co-activation of pre- and postsynaptic neurons sets a flag at the synapse, called an eligibility trace, that leads to a weight change only if an additional factor is present while the flag is set. This third factor, signaling reward, punishment, surprise, or novelty, could be implemented by the phasic activity of neuromodulators or specific neuronal inputs signaling special events. While the theoretical framework has been developed over the last decades, experimental evidence in support of eligibility traces on the time scale of seconds has been collected only during the last few years. Here we review, in the context of three-factor rules of synaptic plasticity, four key experiments that support the role of synaptic eligibility traces in combination with a third factor as a biological implementation of neoHebbian three-factor learning rules.
1 Introduction
Learning behaviors and forming memories are linked to changes in synaptic connections, while traditional Hebbian protocols do not account for additional factors. This motivates updated experimental support for three-factor learning rules.
- Novel behaviors and memories are associated with changes in synaptic connections.Examples include button pressing, tennis, traffic responses, salient events, flower discrimination, and spatial mapping.
- Hebbian protocols can induce long-lasting synaptic changes by combining presynaptic activation with postsynaptic voltage or firing-state manipulation.These protocols underlie studies of LTP, LTD, and spike-timing-dependent plasticity.
- The review updates earlier theoretical and experimental coverage by examining recent physiological evidence for eligibility traces and three-factor rules.The reviewed experiments span striatum, cortex, and hippocampus.
2 Hebbian rules versus three-factor rules
Hebbian rules use presynaptic and postsynaptic activity to set synaptic flags, whereas three-factor rules require an additional modulatory or event-related signal to convert eligibility into weight change. This framework distinguishes transient synaptic eligibility from measurable synaptic strength and accommodates delayed, reward-related plasticity.
- Hebbian rules versus three-factor rules: Synaptic strength wij is distinguished from an internal eligibility variable eij, which acts as a transient flag for a candidate weight change.wij is measurable through spine volume or postsynaptic-potential amplitude, whereas eij is not directly visible in standard electrophysiological experiments.
- Hebbian rules versus three-factor rules: Conventional Hebbian protocols include presynaptic stimulation paired with postsynaptic voltage or firing, producing LTP or LTD in the stimulated pathway.In voltage-dependent models, postsynaptic spikes are not necessary; wiring together can occur without firing together.
- Hebbian rules versus three-factor rules: Hebbian learning changes the synaptic flag through the joint presence of presynaptic activity xj and postsynaptic state yi.The flag can function as a correlation detector, with a decay time constant τe in models that include decay.
- Three-factor learning rules: Three-factor rules leave an eligibility trace after Hebbian co-activation and produce an actual weight change only when a third factor is present at the same time or soon afterward.The third factor may be a phasic neuromodulatory signal or an additional input associated with reward, surprise, attention, or another special event.
- Three-factor learning rules: The third factor scales learning and can determine its direction: without it, the synaptic weight remains unchanged, while a negative third factor can reverse a positive eligibility signal.Because neuromodulatory signals can be shared broadly, the third-factor variable is modeled without neuron-specific indices.
- Examples and theoretical predictions: In reward-based learning, eligibility rises during joint activation of state and action neurons and then decays, while the third factor is defined as reward minus expected reward.The framework predicts that the eligibility-trace duration should roughly match the interval between action initiation and reward delivery.
3 Experimental evidence for eligibility traces
Recent experiments across striatum, cortex, and hippocampus provide physiological support for eligibility traces that bridge synaptic activity and delayed neuromodulatory or postsynaptic signals. The reviewed findings identify distinct time windows, molecular pathways, and unresolved alternatives for three-factor plasticity.
- Striatum: Striatal STDP-like induction left an eligibility trace that dopamine converted into LTP for delays up to 1 second, but not after 4 seconds.Spine enlargement was maximal when dopamine began during induction and corresponded to strengthened excitatory postsynaptic currents.
- Striatum: Striatal three-factor plasticity depended on NMDA, CaMKII, protein synthesis, and dopamine D1 receptors.Localized CaMKII activity matched the dopamine-sensitive window, whereas cell-wide PKA was associated with the dopamine-triggered factor.
- Cortex: Cortical pre-before-post pairing produced LTP through noradrenaline after delays of 5–10 seconds, whereas post-before-pre pairing produced LTD through serotonin after about 3 seconds.The results were consistent across pharmacological and optogenetic neuromodulator delivery, including physiologically minimal pairing protocols.
- Cortex: A cortical model with two eligibility traces stabilized and prolonged network activity to support event prediction.The associated molecular transformations involved beta adrenergic receptors and cyclic AMP for LTP, and 5-HT2c receptors for LTD.
- Hippocampus: In hippocampus, delayed dopamine converted LTD-associated induction into LTP within about 1 minute, while a calcium plateau potential converted prior synaptic activation into LTP after several seconds.The hippocampal dopamine effect required continued weak presynaptic stimulation; the plateau-potential findings were interpreted as an eligibility-trace mechanism involving presynaptic activation and postsynaptic depolarization.
- Hippocampus: The hippocampal interpretation remains experimentally distinguishable from a model in which presynaptic activation alone sets the eligibility trace.The three-factor account predicts that a single activated synapse or less than 2 mV depolarization above rest will not suffice for later LTP.
4 Discussion and Conclusion
The discussion situates eligibility traces within broader learning theory, explains how three-factor rules preserve synaptic specificity, and emphasizes that reviewed experiments support—but do not uniquely establish—this mechanism.
- Eligibility traces arise naturally in policy-gradient and continuous space-time TD-learning, where the third factor can be reward minus expected reward.
- Broad neuromodulatory signals can remain synaptically selective because eligibility flags are set only when presynaptic input coincides with elevated postsynaptic voltage.
- Three-factor rules do not require assigning each neuromodulator exclusively to reward, surprise, or novelty, and can accommodate region- and pathway-specific functions.
- Eligibility traces are not theoretically mandatory because recurrent-network activity patterns and short-term synaptic plasticity can also associate events separated by seconds.
- The reviewed experiments support synaptic eligibility traces as one biological solution on behavioral time scales, while other mechanisms may operate in parallel.
- The framework reflects a long theoretical development, from verbal and discrete-time models to spiking models with explicit eligibility-trace time scales.