PaperlayerView exact revision on arXiv

Source-linked AI summary

AFEC: Active Forgetting of Negative Transfer in Continual Learning

Liyuan Wang, Mingtian Zhang, Zhongfan Jia, Qian Li, Chenglong Bao, Kaisheng Ma, Jun Zhu, Yi Zhong

arXiv:2110.12187v2cs.LG

TL;DR

Negative forward transfer can make retaining old knowledge interfere with learning new tasks. AFEC uses Bayesian continual learning to expand parameters for new tasks and selectively merge them, and the authors report state-of-the-art results across several benchmark types.

  • Problem

    Without old training samples, the direction of knowledge transfer to a new task is difficult to determine, and negative transfer can make retaining old knowledge interfere with learning it.

    Source: §1 · p. 1

  • Method

    AFEC uses a forgetting factor within Bayesian continual learning, expanding parameters for a new task and then selectively merging them with the main network.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    AFEC achieves state-of-the-art performance across CIFAR-10 regression, visual classification, and Atari reinforcement tasks, and improves new-task learning without increasing catastrophic forgetting.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors present AFEC as a plug-and-play approach that boosts representative weight-regularization and memory-replay strategies.

    Source: §1 · p. 2

    The setting assumes each task's training data is available only while that task is being learned.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Continual learning aims to learn a sequence of tasks from dynamic data distributions. Without accessing to the old training samples, knowledge transfer from the old tasks to each new task is difficult to determine, which might be either positive or negative. If the old knowledge interferes with the learning of a new task, i.e., the forward knowledge transfer is negative, then precisely remembering the old tasks will further aggravate the interference, thus decreasing the performance of continual learning. By contrast, biological neural networks can actively forget the old knowledge that conflicts with the learning of a new experience, through regulating the learning-triggered synaptic expansion and synaptic convergence. Inspired by the biological active forgetting, we propose to actively forget the old knowledge that limits the learning of new tasks to benefit continual learning. Under the framework of Bayesian continual learning, we develop a novel approach named Active Forgetting with synaptic Expansion-Convergence (AFEC). Our method dynamically expands parameters to learn each new task and then selectively combines them, which is formally consistent with the underlying mechanism of biological active forgetting. We extensively evaluate AFEC on a variety of continual learning benchmarks, including CIFAR-10 regression tasks, visual classification tasks and Atari reinforcement tasks, where AFEC effectively improves the learning of new tasks and achieves the state-of-the-art performance in a plug-and-play way.

1 Introduction

Negative forward transfer can make retaining old knowledge interfere with learning new tasks. AFEC draws on biological active forgetting to selectively forget conflicting knowledge while preserving continual learning performance.

1 Introduction

  • When forward transfer is negative, old knowledge can interfere with learning a new task, and precisely remembering old tasks can worsen that interference.The passage defines negative transfer as learning a new task from old knowledge performing worse than learning it from a randomly initialized network.Source: §1 · p. 1
  • Biological active forgetting regulates learning-triggered synaptic expansion and convergence: connections expand for a new experience, then are pruned to their prior amount.Figure 1 identifies this expansion-convergence process as the mechanism of biological active forgetting.Source: §1 · p. 2, Figure 1 · p. 2
  • AFEC uses Bayesian continual learning to forget conflicting old knowledge, expand parameters for each new task, and selectively merge them into the main network.The forgetting factor regulates the selective merging penalty.Source: §1 · p. 2
  • AFEC achieves state-of-the-art performance across CIFAR-10 regression, visual classification, and Atari reinforcement tasks, while improving new-task learning without increasing catastrophic forgetting.The authors also report that AFEC boosts representative weight-regularization and memory-replay strategies in a plug-and-play way.Source: §1 · p. 2
  • The authors describe AFEC as the first model of biological active forgetting at synaptic structures and suggest it offers a potential theoretical explanation of that mechanism.The proposed explanation concerns how biological active forgetting may support forgetting the past while continually learning conflicting experiences.Source: §1 · p. 2

2 Related Work

Prior continual-learning work has emphasized mitigating catastrophic forgetting through regularization, parameter isolation, and replay. Work directly addressing negative transfer is limited, and this paper focuses on incremental learning without storing old training data.

2 Related Work

  • Existing approaches mainly mitigate catastrophic forgetting through weight regularization, parameter isolation [24], or memory replay.AGS-CL combines weight regularization and parameter isolation and is reported to achieve state-of-the-art performance on various continual-learning benchmarks.Source: §2 · p. 2
  • Studies specifically addressing negative transfer are limited; this paper focuses on learning tasks incrementally from scratch without storing old training data.The authors also extend their method to settings with pretraining or memory buffers, and to regression and reinforcement tasks.Source: §2 · p. 3

3 Method

AFEC builds on Bayesian continual learning, then uses a forgetting factor and a two-stage expansion-convergence procedure to learn new tasks while selectively merging parameters. Its design is presented as formally consistent with biological active forgetting.

3 Method

  • The method section frames AFEC as Bayesian continual learning with an active-forgetting strategy formally consistent with biological synaptic active forgetting.The section first introduces the Bayesian continual-learning framework.Source: §3 · p. 3

3.1 Basics of Bayesian Continual Learning

  • In Bayesian continual learning, the posterior after a task incorporates its knowledge and can serve as the prior for the next task.The training data for each task is available only while that task is being learned.Source: §3.1 · p. 3
  • Because the posterior is generally intractable, the method uses approximations such as Laplace's, representing it with a Gaussian centered at the posterior mode and covariance from the inverse Hessian.The Fisher Information matrix indicates each parameter's importance for the previous task, and λ controls the penalty that alleviates catastrophic forgetting.Source: §3.1 · pp. 3–4

3.2 Active Forgetting with Synaptic Expansion-Convergence

  • When retaining an earlier task interferes with learning a new one, AFEC introduces a forgetting factor β to replace the original posterior with an actively forgetting distribution.The factor determines how much information from the earlier task is forgotten; β is selected by grid search between 0 and 1.Source: §3.2 · p. 4
  • AFEC first learns expanded parameters for the new task, then selectively converges them with the main network using weight-merging regularizers.The forgetting factor controls the penalty promoting active forgetting; the resulting algorithm is formally consistent with biological synaptic expansion followed by convergence.Source: §3.2 · p. 4
  • AFEC is designed for plug-and-play use and does not add storage cost over regular weight-regularization approaches because expanded parameters are not stored and the main network architecture stays fixed.The method can be extended to more than two tasks.Source: §3.2 · p. 5
  • Unlike regular weight regularization that penalizes changes to old parameters, AFEC expands parameters for the new task to avoid potential negative transfer, then merges weights by their representational contributions.The conceptual comparison with EWC depicts expansion for learning and convergence for selective weight-merging.Source: §3.2 · p. 5, Figure 2 · p. 5

4 Experiment

AFEC is evaluated on CIFAR-10 regression, visual classification, and Atari tasks spanning different transfer conditions, with experiments averaged over five runs. Results assess new-task learning alongside retention of earlier tasks.

4.1 CIFAR-10 Regression Tasks

  • In CIFAR-10 regression experiments, AFEC learns new tasks while remembering old ones, and transfers better to rotated similar tasks than EWC.For the transfer experiment, the feature extractor is fixed and only a linear classifier is fine-tuned; EWC performs worse than sequential fine-tuning without weight regularization.Source: §4.1 · p. 6, Figure 4 · p. 6, Table 1 · p. 6

4.2 Visual Classification Tasks

  • AFEC achieves the best averaged accuracy across all visual-classification benchmarks, outperforming AGS-CL on small-scale images from scratch, CUB-200 with ImageNet pretraining, and large-scale images from scratch.Table 2 summarizes averaged accuracy across tasks learned so far.Source: Table 2 · p. 7, §4.2 · p. 7
  • AFEC achieves the highest forward transfer across the three CIFAR-100 benchmarks, while its backward transfer is comparable to EWC.AFEC also learns each new task better than the other baselines; AGS-CL's new-task learning drops more rapidly despite its best backward transfer.Source: §4.2 · pp. 8–9, Figure 5 · p. 8

4.3 Atari Reinforcement Tasks

  • On Atari tasks with severely negative forward transfer, AFEC improves new-task learning and boosts EWC and MAS, particularly as more tasks are added.AFEC also outperforms reproduced AGS-CL results; Table 3 reports averaged performance increases in learning each new task.Source: §4.3 · p. 9, Table 3 · p. 9, Figure 7 · p. 9

5 Conclusion

AFEC targets potential negative transfer through active forgetting, improving new-task learning and benchmark performance while boosting representative strategies in a plug-and-play way.

5 Conclusion

  • AFEC achieves state-of-the-art performance across continual-learning benchmarks by improving new-task learning and boosting representative strategies in a plug-and-play way.The paper presents its Bayesian active-forgetting algorithm as formally consistent with synaptic expansion and convergence and functionally consistent with biological memory flexibility.Source: §5 · p. 10
  • The proposed approach is inspired by biological active forgetting and aims to mitigate potential negative transfer in continual learning.The paper describes the biological connection as a potential theoretical explanation for forgetting past knowledge while learning conflicting experiences.Source: §5 · p. 10

A Neural Mechanism of Biological Active Forgetting

The section describes Rac1-dependent active forgetting as synaptic expansion followed by convergence, and adapts that process to continual learning.

A Neural Mechanism of Biological Active Forgetting

  • New learning triggers an increase and subsequent elimination of presynaptic active zones, a process regulated by Rac1 signaling.The reported AZ increase and elimination in the mushroom body γ lobe occur in parallel with forgetting that lasts only hours; blocking AZ addition-induced elimination prevents forgetting.Source: A Neural Mechanism of Biological… · p. 14
  • For task B, the proposed adaptation first expands synapses so expanded and old active zones can learn, then converges them to the prior amount.The passage describes this sequence as synaptic expansion followed by synaptic convergence.Source: A Neural Mechanism of Biological… · p. 14
  • The detailed biological evidence is not included in the published version because it is to be published elsewhere.This limits what biological evidence the section itself provides.Source: A Neural Mechanism of Biological… · p. 14

B.1 Laplace Approximation

  • A second-order Taylor expansion around θ*_A is used to approximate the function, with the higher-order term omitted.The section identifies R2(x) as the higher-order term.Source: §B.1 · p. 15

B.2 Weighted Product Distribution with Forgetting Factor

  • The weighted product uses p1(x)^(1−β) and p2(x)^β with a normalizer Z to keep the resulting distribution normalized.The passage identifies p1(x) and p2(x) as Gaussian distributions.Source: §B.2 · p. 16

B.3 New Log Posterior of AFEC

  • AFEC forms a new log posterior and applies a Laplace approximation to it when the relevant distributions are Gaussian.The appendix states that the distributions are both Gaussian before introducing the approximation.Source: §B.3 · pp. 16–17

B.4 Continual Learning of More than Two Tasks

For learning more than two tasks, AFEC uses forgetting when incorporating each new task and continues its expansion-convergence procedure across the sequence.

B.4 Continual Learning of More than Two Tasks

  • To mitigate potential negative transfer to task C, AFEC replaces the old posterior with a weighted product involving a prior and a forgetting factor.The forgetting factor β is used for task C, and the normalizer Z depends on β.Source: §B.4 · p. 17
  • For the three-task case, the loss for learning task C is derived after updating the Fisher Information Matrix for tasks A and B.The expanded parameters for task C are optimized with its loss, and their Fisher Information Matrix is calculated.Source: §B.4 · p. 17
  • The algorithm learns expanded parameters for each task, then learns the main parameters and updates the Fisher Information Matrix across tasks.The pseudocode labels the two learning phases synaptic expansion and synaptic convergence.Source: §B.4 · p. 17
  • The visual-classification hyperparameter table reports search ranges and selected values, with AFEC searching λe while keeping λ tied to the corresponding weight-regularization approach.The caption notes that β is integrated into λe.Source: Table 4 · p. 18

C.1 Implementation

The implementation uses different optimizers and training settings for small-scale and large-scale visual-classification datasets, and searches hyperparameters for fair comparison.

C.1 Implementation

  • Small-scale image datasets use Adam at learning rate 0.001, batch size 256, and 100 epochs, while large-scale datasets use SGD with momentum 0.9 at learning rate 0.005, batch size 64, and 40 epochs.The listed datasets distinguish CIFAR variants from CUB-200 and ImageNet-100.Source: §C.1 · p. 18
  • The implementation performs extensive hyperparameter searches for all methods and reports the best performance for fair comparison.Search ranges and selected hyperparameters are summarized in Table 4.Source: §C.1 · p. 18
  • Classification-result error bars report standard deviations.The standard deviations are included in Table 5.Source: §C.1 · p. 18

C.2 Longer Task Sequence

AFEC retains an accuracy advantage over EWC on a 50-task Omniglot sequence. The section defines transfer metrics and describes adapting AFEC’s regularization to existing weight-regularization methods.

C.2 Longer Task Sequence

  • On 50-split Omniglot, averaged accuracy was 83.00% for AFEC versus 76.53% for EWC across all 50 tasks.For the first 25 tasks, the averages were 84.08% for AFEC and 66.45% for EWC.Source: §C.2 · p. 18

D ACC, FWT and BWT

  • AAC summarizes performance across tasks seen so far, while FWT and BWT describe influence across future and previously learned tasks.FWT can be positive or negative; BWT reflects how learning a new task affects old-task performance.Source: D ACC, FWT and BWT · pp. 18–19

E Adapt AFEC to Representative Weight Regularization Approaches

F Adapt AFEC to Representative Memory Replay Approaches

AFEC is applied to representative memory-replay methods under a small-buffer CIFAR-100 and ImageNet-100 setup. The reported results show substantial performance boosts for those methods.

F Adapt AFEC to Representative Memory Replay Approaches

G.1 Implementation

The Atari evaluation averages unstable rewards over multiple runs and task orders. The reproduced AGS-CL results do not match its reported advantage, and the reproduced method trails both AFEC variants.

G.1 Implementation

  • The implementation uses Adam with an initial learning rate of 0.0003 and evaluates normalized accumulated reward during training on each task.The paper also searches hyperparameters for AGS-CL, EWC, and MAS.Source: §G.1 · p. 20
  • Because normalized Atari rewards varied substantially across runs, the evaluation averages five random-seed runs and tests three task orders.The sequences use different orderings of the same eight Atari tasks.Source: §G.1 · p. 20

G.2 Reproduced Results of AGS-CL

  • The released code did not reproduce AGS-CL’s reported performance; reproduced AGS-CL only slightly exceeded EWC and underperformed AFEC1 and AFEC2.The reported AGS-CL results had significantly higher normalized rewards than EWC on Tasks 1 and 7, unlike the reproduced results.Source: §G.2 · p. 20

Related papers

Loading 2110.12187v2…