PaperlayerView exact revision on arXiv

Source-linked AI summary

Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning

Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, Lei Shu

arXiv:2112.02706v1cs.CLcs.AIcs.LGcs.NE

TL;DR

Continual learning must balance catastrophic-forgetting prevention with knowledge transfer, while conventional BERT fine-tuning causes serious forgetting. CTR inserts capsule-based continual-learning plug-ins into BERT, and experiments report strong performance in both forgetting avoidance and knowledge transfer across NLP tasks.

  • Problem

    Existing continual-learning methods often emphasize forgetting prevention without explicit knowledge transfer, while BERT fine-tuning performs poorly because it causes serious forgetting.

  • Method

    CTR inserts capsule-based continual-learning plug-ins into BERT, using transfer routing to identify transferable knowledge and isolate task-specific knowledge.

  • Results

    CTR clearly outperforms all baselines and performs well in both forgetting avoidance and knowledge transfer across the evaluated tasks.

  • Takeaways & Limitations

    CTR provides a way to leverage pre-trained BERT for continual learning while improving both new-task and old-task performance through knowledge transfer.

Abstract

from arXiv · show

Continual learning (CL) learns a sequence of tasks incrementally with the goal of achieving two main objectives: overcoming catastrophic forgetting (CF) and encouraging knowledge transfer (KT) across tasks. However, most existing techniques focus only on overcoming CF and have no mechanism to encourage KT, and thus do not do well in KT. Although several papers have tried to deal with both CF and KT, our experiments show that they suffer from serious CF when the tasks do not have much shared knowledge. Another observation is that most current CL methods do not use pre-trained models, but it has been shown that such models can significantly improve the end task performance. For example, in natural language processing, fine-tuning a BERT-like pre-trained language model is one of the most effective approaches. However, for CL, this approach suffers from serious CF. An interesting question is how to make the best use of pre-trained models for CL. This paper proposes a novel model called CTR to solve these problems. Our experimental results demonstrate the effectiveness of CTR

1 Introduction

The paper frames continual learning as a tension between preventing catastrophic forgetting and enabling forward and backward knowledge transfer. It also asks how pre-trained BERT models can be used without causing severe forgetting.

  • 1 Introduction: Continual learning must prevent catastrophic forgetting while transferring knowledge across tasks, including forward and backward transfer.Knowledge transfer is especially relevant because NLP tasks often share knowledge.
  • 1 Introduction: Methods that address both objectives can suffer severe forgetting when tasks share little knowledge, while forgetting-focused methods lack explicit transfer mechanisms.The paper reports this limitation for existing approaches tested beyond strongly related sentiment-analysis tasks.
  • 1 Introduction: Pre-trained models can improve continual-learning performance, but conventional approaches that add continual-learning components or fine-tune BERT are not consistently suitable.The paper specifically identifies direct continual-learning additions and fine-tuning as problematic choices.
  • 1 Introduction: Fine-tuning BERT causes serious forgetting because task-specific parameters must be updated when learning new tasks.Fine-tuned BERT captures task-specific information that is difficult for other tasks to use.
  • 1 Introduction: CTR is proposed as a neural architecture intended to prevent forgetting, transfer knowledge, and use BERT without fine-tuning it separately for every task.It inserts continual-learning plug-ins into BERT and is presented as distinct from ordinary adapters.

2 Related Work

Related work includes methods for preventing forgetting, transferring knowledge, or adapting pre-trained models, but these approaches generally address only parts of CTR’s target problem. The paper highlights limitations in transfer, forgetting, architecture, or continual-learning applicability.

  • Catastrophic Forgetting: Catastrophic-forgetting methods include regularization, replay or generated data, and parameter isolation approaches.These categories represent major existing strategies for protecting prior-task knowledge.
  • Knowledge Transfer: Knowledge-transfer work includes systems for similar-task transfer, forward transfer, lifelong learning, and continual sentiment classification.The cited methods span Task-CL, Class-CL, and lifelong-learning settings.
  • Knowledge Transfer: SRK and KAN address forgetting and transfer but use RNN architectures and can suffer serious forgetting on tasks with limited shared knowledge.Their architecture also prevents them from using plug-in or adapter modules to tune BERT.
  • Adapter-Based Methods: Adapter-based methods adapt BERT for end tasks, whereas AdapterFusion targets multi-task learning rather than continual learning and therefore does not address catastrophic forgetting.The paper distinguishes CTR’s CL-plugin from adapters designed for task-specific adaptation.

3 CTR Architecture

CTR uses BERT as a base model and inserts continual-learning plug-ins within its transformer layers. The plug-ins combine knowledge sharing, task-specific processing, capsule representations, transfer routing, and task masking.

  • 3 CTR Architecture: CTR uses BERT and inserts a continual-learning plug-in into two locations within each BERT transformer layer.The plug-in is designed to interact with the pre-trained model without fine-tuning BERT for each task.
  • 3 CTR Architecture: The CL-plugin uses a capsule-network-like architecture in which a transfer capsule layer identifies transferable knowledge from previous tasks.Transfer routing explicitly selects transferable features and avoids tuning the number of dynamic-routing iterations.
  • 3 CTR Architecture: Each CL-plugin contains a knowledge sharing module, a task-specific module, and a skip connection.The knowledge sharing and task-specific components are shown in the proposed architecture.
  • 3 CTR Architecture: CTR’s overall architecture separates task-specific knowledge from shareable knowledge while retaining BERT as the underlying pre-trained model.The two-plugin design and its submodules implement this division within BERT.
  • 3 CTR Architecture: The architecture illustrates task masking by assigning neurons to individual tasks or to multiple tasks.Figure 2 distinguishes task-specific and shared neuron usage through task-associated colors.

4 Continual Learning Plug-in (CL-plugin)

The CL-plugin combines task-specific protection with selective knowledge sharing to prevent forgetting while transferring representations between related tasks. It is integrated through task capsules, transfer capsules, similarity estimation, routing, and task masks.

  • The CL-plugin receives transformer hidden states and the task ID, then produces task-specific hidden states for classification.
  • Task Capsule Layer (TK-Layer): Task capsules represent individual tasks and prepare their low-level features using small two-layer fully connected networks.A new discrete capsule is added for each task without sharing parameters.
  • Transfer Routing and Transfer Capsule Layer: Transfer capsules extract transferable representations from task capsules through pre-route vector generation, similarity estimation, and task routing.Similarity estimators compare previous and current task representations, while routers learn differentiable connect-or-disconnect decisions.
  • Transfer Routing and Transfer Capsule Layer: The routing mechanism uses similarity to pass information from related previous tasks while ignoring representations with little similarity to the current task.Low-similarity representations are treated as having little transferable knowledge.
  • Transfer Routing and Transfer Capsule Layer: Connected routes update similar tasks during backpropagation, encouraging backward knowledge transfer while protecting unrelated task capsules from updates.Only task capsules with connected gates are updated.
  • Task-Specific Module and Task Masks: Task masks protect neurons used by previous tasks by blocking their gradient updates when learning a new task.Overlapping masks represent parameter sharing, allowing related old tasks to benefit from shared features.
  • Task-Specific Module and Task Masks: During incremental training, used neurons are masked before each new task, while newly selected neurons become protected for future tasks.The illustrated process applies this masking successively across three tasks.

5 Experiments

CTR is evaluated across three continual-learning applications, including similar-task settings for knowledge transfer and dissimilar-task settings for catastrophic-forgetting resistance. Across these evaluations, CTR outperforms baselines while showing strong transfer, low forgetting, and contributions from all major components.

  • Applications and evaluation: CTR is evaluated on three applications, with similar tasks used to assess knowledge transfer and dissimilar tasks used to assess catastrophic-forgetting resistance.The first two applications contain shared knowledge, whereas the third has very different tasks and little transferable knowledge.
  • Evaluation protocol: Results are averaged over five random task sequences using accuracy and Macro-F1, with Macro-F1 primary because ASC classes are imbalanced.Table 2 reports averages across 19 ASC tasks, 10 DSC tasks, and 20News.
  • Overall performance: CTR clearly outperforms all baselines, achieving strong performance in both forgetting avoidance and knowledge transfer across the evaluated settings.Compared with other methods, CTR is especially stronger when transfer is needed and retains the least forgetting on 20News.
  • Knowledge transfer: CTR’s forward transfer is highly effective on the three similar-task datasets, while backward transfer improves ASC and DSC with small data.For DSC with full data and 20News, backward transfer is slightly weaker; on 20News, forgetting remains below 0.0055.
  • Ablation study: The full CTR system gives the best ablation results, indicating that each component contributes to performance.Removing the knowledge-sharing or task-specific modules, or changing plugin placement, produces poorer results.

6 Conclusion

CTR uses a BERT-based continual-learning architecture to promote knowledge transfer while limiting catastrophic forgetting. Across three NLP applications, it improves new- and old-task performance, though capsule-based processing slows training.

  • CTR inserts a capsule-based CL-plugin with transfer routing into pre-trained BERT to encourage knowledge transfer and isolate task-specific knowledge.
  • Experiments on three NLP applications show that CTR improves performance on both new and old tasks through knowledge transfer.
  • CTR is effective at overcoming catastrophic forgetting while using pre-trained BERT for task continual learning.
  • Capsule representations reduce efficiency because grouping sentence tokens and BERT features creates very large capsules that slow training.For BERTBASE examples, a capsule can combine 128 tokens with 768 features, producing a 128 × 768 representation.
Loading 2112.02706v1…