Generative Replay for Equilibrium Propagation: Toward Edge-Friendly Continual Learning in Hopfield Networks

1 INTRODUCTION

• Continual learning studies acquiring knowledge from a changing data stream while retaining earlier information; standard gradient-descent networks can suffer catastrophic forgetting as new tasks degrade earlier-task performance.Source: §1 · p. 1
• EqProp's continual-learning potential remains largely unexplored despite its local learning rule and shared energy-minimization dynamics for inference and learning in Hopfield Networks.EqProp compares free and weakly nudged equilibria without requiring additional mechanisms.Source: §1 · p. 2
• EB-Gen uses a frozen copy of the currently trained network to generate pseudo-examples by relaxation and interleave them with current-task data, without stored data, auxiliary components, or topology changes.The method builds on learned tasks being represented as stable attractors.Source: §1 · p. 2
• EqProp does not naturally train Hopfield Networks to model the input distribution, motivating a modification to the algorithm that addresses this limitation.Source: §1 · p. 2
• The work evaluates EB-Gen on Split-MNIST and Split-CIFAR10 against FineTune, ER, SRC, and standard Generative Replay to assess stability–plasticity trade-offs.Source: §1 · p. 2

2 RELATED WORKS

• Replay methods interleave samples representing past tasks with current-task examples, using either stored examples or learned generative distributions.Experience Replay uses a buffer, while Generative Replay commonly uses a separate generator.Source: §2 · p. 2
• Prior continual-learning work on energy-based models includes contrastive-divergence-trained Neural EBMs that outperformed conventional softmax classifiers in alleviating forgetting (Li et al. (2022)).Source: §2 · p. 3
• Wake-sleep approaches include an EqProp-trained Hopfield method, but its sleep phase requires conversion to a spiking neural network (Kubo et al. (2025)).Other cited wake-sleep methods use backward gradient propagation and synchronous feed-forward EBM architectures.Source: §2 · p. 3
• Ans (2004) introduced pseudo-rehearsal with a model and frozen copy for solving tasks and generating replay, but tested it on small toy problems using non-EBM recurrent networks trained with CHL.EB-Gen instead uses EqProp for energy-based models.Source: §2 · p. 3

3.1 HOPFIELD NETWORK AND EQUILIBRIUM PROPAGATION

• Hopfield network states relax through energy minimization, and their attractors store information; the network is recurrent and uses symmetric connections.The discretized relaxation dynamics resemble conventional neural-network inference.Source: §3.1 · pp. 3–4
• EqProp trains Hopfield Networks using local information from two successive relaxation phases, avoiding explicit gradient storage and a separate backward pass.The free phase clamps inputs to a sample and relaxes the remaining units; the nudged phase adds a task-loss perturbation to outputs, and the resulting update approximates gradient descent on that loss.Source: §3.1 · p. 4

3.2 PROBLEM SETTING

• Split-MNIST and Split-CIFAR10 each contain five sequential tasks of two classes, evaluated class-incrementally without task identity at training or test time.The study's scope is sequential image classification with defined task boundaries and i.i.d. samples within each task.Source: §3.2 · p. 5
• Incremental performance is summarized using Learning Accuracy for plasticity and Average Accuracy for stability, alongside measurements of current-task performance and retention.Source: §3.2 · p. 5, §3.3 · p. 5

3.3 CONTINUAL LEARNING BASELINES

• The evaluation uses FineTune, ER, Generative Replay, SRC, and Joint Training; FineTune and Joint Training serve as lower and upper bounds for average accuracy, respectively.Replay methods use a fixed replay-to-current-sample ratio across tasks, and use latent features when experiments use extracted features.Source: §3.3 · p. 5

4 ENERGY-BASED GENERATIVE REPLAY (EB-GEN)

• EB-Gen uses stable attractors in an EqProp-trained Hopfield Network to generate replay samples through relaxation, without stored data or auxiliary generative models.The same network supports prediction and generation by varying which state variables are constrained during relaxation.Source: §4 · p. 5, §4.1 · p. 6

4.1 SAMPLE GENERATION FROM EQPROP-TRAINED ENERGY LANDSCAPES

• Releasing all state variables lets the network relax to attractors that ideally represent previously learned input–output associations.Deterministic relaxation yields limited recurrent attractors, so generation adds noise during relaxation or stops relaxation early to increase diversity.Source: §4.1 · p. 6

4.2 WHY INPUT RECONSTRUCTION MUST BE EXPLICITLY TRAINED

• Standard EqProp training can produce degenerate input patterns because its task loss shapes outputs for clamped inputs, not the input distribution itself.When inputs are released, the energy provides no incentive for them to converge toward data-like configurations, making explicit input reconstruction necessary for meaningful input generation.Source: §4.2 · p. 6

4.3 HYBRID TRAINING FOR INPUT RECONSTRUCTION

• The hybrid objective adds an input reconstruction loss during EqProp training, encouraging low-energy equilibria to encode consistent input–output configurations.The relative importance of input reconstruction is controlled by λ; the resulting energy landscape supports relaxation-based generation of inputs or CIFAR10 features and outputs.Source: §4.3 · p. 6

4.4 THE GENERATIVE REPLAY PROCESS

• For each new task, a frozen copy of the network trained on earlier tasks generates replay samples that are interleaved with new-task examples.The replayed-to-new batch-size ratio is a method hyperparameter, often chosen greater than one.Source: §4.4 · pp. 6–7

5 RESULTS

• The reported results cover hybrid EqProp training, pseudo-example generation, and their use in EB-Gen.The model, training, and continual-learning hyperparameters are summarized in annex C.Source: §5 · p. 7
• The results section draws on the methodology introduced in Sections 3 and 4 and includes evaluation of the proposed EB-Gen approach.The supplied results passages do not give accuracy values or comparative outcomes.Source: §5 · p. 7

5.1 OFFLINE HYBRID TRAINING

• Full offline hybrid training is evaluated on MNIST and CIFAR10 under three regimes.The supplied passage does not specify the regimes.Source: §5.1 · p. 7

1. Regular regime: input nodes are fully clamped to input data, no input cost,

• Partial input clamping degrades accuracy by roughly 5–10%, while hybrid training improves input reconstruction over regular training.The hybrid regime frees only 10% of input nodes; partial clamping produces an input-error explosion, and the authors state that input cost is necessary for input reconstruction.Source: §1 · p. 7, Table 1 · p. 7, Table 2 · p. 7
• Hybrid training improves partial input reconstruction, but its generated MNIST inputs rarely resemble digits or even numbers.The authors assess whether such samples can still support replay in the following section.Source: §1 · p. 7

5.2 CONTINUAL LEARNING WITH ENERGY-BASED GENERATIVE REPLAY

• ER is most stable on Split MNIST and Split CIFAR10, losing 5–6% average accuracy versus Joint; EB-Gen loses at least 10% versus ER.Joint-Hybrid is the more relevant upper bound for EB-Gen and loses 6–7% average accuracy versus regular Joint training.Source: Table 3 · p. 8, §5.2 · p. 8
• EB-Gen improves over SRC by 10% on Split-MNIST and 3% on Split-CIFAR10; on Split-ImageNet10, its gain over the lower bound is only 5%.The CIFAR10/ImageNet10 comparisons use feature extractors pretrained on those same datasets, unlike earlier CIFAR10 results using an ImageNet-1k-pretrained ResNet18.Source: Table 4 · p. 8, §5.2 · p. 8
• FineTune performed worst, achieving 19% and 21% average accuracy on Split MNIST and CIFAR10, respectively, corresponding to complete forgetting of four tasks out of five.Source: §5.2 · p. 8
• Replay methods show reduced plasticity after the first task, especially on Split CIFAR10; EB-Gen’s first-task bias is most pronounced.The authors suggest EB-Gen’s shared parameter space for input and output representations may contribute; Task 1 accuracy also drops when learning Task 5 on Split CIFAR10.Source: Figure 3 · p. 8, §5.2 · p. 8

6.1 FORGETTING IN EBMS

• Optimizing for a new task can raise the energy of prior-task minima through the EBM negative phase, eroding them and causing forgetting.EqProp’s free and nudged phases resemble the model and data phases of contrastive divergence, respectively.Source: §6.1 · p. 9

6.2 REPLAY-BASED CONTINUAL LEARNING, MEMORY FOOTPRINT AND ANALOG HARDWARE FRIENDLINESS

• Despite underperforming ER on Split-MNIST-CIFAR10 and conventional Gen Replay on Split-MNIST, EB-Gen’s Hybrid regime uses standard EqProp for input generation and the network’s natural dynamics for replay.This avoids auxiliary modules, which the analysis identifies as a hardware-efficiency advantage.Source: §6.2 · p. 9
• ER may require external digital memory or on-chip capacitive arrays, while Gen Replay requires a separate generative model and its training, inference, and weight-storage costs.The analysis contrasts these requirements with EB-Gen’s replay using the Hopfield network’s dynamics.Source: §6.2 · p. 9

7 CONCLUSION

• EB-Gen uses a Hopfield network as both classifier and generator in class-incremental learning, reducing the need for additional memory or generator modules.The conclusion reports that it significantly reduces catastrophic forgetting and may offer a privacy opportunity because past data are encoded in the weights and generated samples are non-human-readable.Source: §7 · p. 9

8 FUTURE WORK

• Future work would test EB-Gen on Domain-Incremental and task-boundary-less scenarios, broaden comparisons to regularization and isolation methods, and explore other weight topologies and sample-selection strategies.These are proposed extensions rather than evaluations reported here.Source: §8 · p. 9

A.1 HOPFIELD NETWORK

• The Hopfield energy includes an interaction term between units and a separate term for individual biases, with optional noise in the relaxation dynamics.The appendix says the interaction term uses activation function ρ and weights wij; it also describes Gaussian noise with variance dt.Source: §A.1 · p. 12

A.2 EQUILIBRIUM PROPAGATION

• The EqProp variant adds a negative nudging phase after positive nudging to better approximate the loss gradient, reversing β’s sign for that phase.For EB-Gen, the input-cost β remains positive during negative nudging to avoid associating inputs with incorrect outputs; trials found this more efficient.Source: §A.2 · p. 12

B.1 GENERATED INPUT EXAMPLES

• Without input reconstruction, relaxed inputs are diffuse and capture coarse MNIST properties but no meaningful digit structure.The described coarse properties include spatial extent and centering.Source: §B.1 · p. 13
• Partial input release with an explicit reconstruction cost produces more structured patterns, although they are not human-interpretable; their task relevance is assessed through online knowledge transfer.The compared training regimes include full data input, 1% input damage, and 10% input damage with reconstruction cost.Source: §B.1 · p. 13

B.2 GENERATIVE REPLAY ALGORITHM

• EB-Gen generates pseudo-examples by relaxing a frozen copy of the model, then interleaves them with current-task data for EqProp training.The algorithm’s replay uses network dynamics without storing past data or adding an independent generative model.Source: §B.2 · p. 14, §B.1 · p. 14

• General settings specify relaxation steps and tolerance, network hidden size, layer learning rates, feature extractor, optimizer, and loss.Relaxation stops when the state-update amplitude falls below the tolerance; free and nudge phases have separate maximum step counts.Source: Table 5 · p. 15
• ER settings define the maximum replay-buffer capacity and the replay-to-current-sample ratio per training step.Source: Table 6 · p. 15
• Gen-Replay settings specify task-wise replay ratios, latent-space size, and generator learning rate, with CIFAR10-specific layer learning rates.The remaining generator architecture and training settings follow Stoychev et al. (2023).Source: Table 7 · p. 15
• EB-Gen settings specify task-wise replay ratios, the maximum generation steps, and the number of generated samples retrieved per relaxation.Source: Table 8 · p. 16