Source-linked AI summary
Episodic Memory in Lifelong Language Learning
Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, Dani Yogatama
TL;DR
The paper studies lifelong language learning from a stream of examples without dataset identifiers, where distribution shifts make retaining and reusing knowledge difficult. It introduces episodic memory for sparse experience replay and local adaptation, and reports reduced catastrophic forgetting with stronger performance than baselines across text classification and question answering. The authors identify remaining performance gaps and leave relevance-based replay selection for future work.
Problem
Models often require many in-domain examples, overfit to particular datasets, and suffer catastrophic forgetting when learning across shifting data distributions.
Method
An episodic key-value memory stores previously seen examples for sparse experience replay during training and local adaptation during inference.
Results
The proposed method mitigates catastrophic forgetting and outperforms baseline methods on text classification and question answering.
Takeaways & Limitations
Episodic memory is presented as a crucial building block and a first step toward general linguistic intelligence.
Takeaways & Limitations
The setup assumes one pass over examples without dataset identifiers, and the best model remains below the multitask upper bound while relevance-based replay selection is left for future work.
Abstract
from arXiv · showhide
We introduce a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier. We propose an episodic memory model that performs sparse experience replay and local adaptation to mitigate catastrophic forgetting in this setup. Experiments on text classification and question answering demonstrate the complementary benefits of sparse experience replay and local adaptation to allow the model to continuously learn from new datasets. We also show that the space complexity of the episodic memory module can be reduced significantly (~50-90%) by randomly choosing which examples to store in memory with a minimal decrease in performance. We consider an episodic memory component as a crucial building block of general linguistic intelligence and see our model as a first step in that direction.
1 Introduction
The paper defines lifelong language learning as one-pass learning from sequential datasets without dataset identifiers, targeting catastrophic forgetting under distribution shifts. It proposes episodic memory with sparse experience replay and local adaptation, evaluated across text classification and question answering.
- Catastrophic forgetting occurs when models fail to isolate and reuse previously acquired knowledge after data distributions shift.
- During experience replay, randomly selected memory examples retrain the model sparsely to consolidate newly acquired knowledge with existing knowledge.
- During local adaptation, examples retrieved by key similarity update parameters for a test prediction, after which the adapted model is discarded.
- Lifelong language learning requires one-pass learning from sequential examples across datasets without dataset boundaries or dataset identities.
- The episodic memory model augments an encoder-decoder model with key-value memory storing examples for sparse experience replay and local adaptation.
- The method is evaluated against baseline and continual-learning methods on text classification and question answering tasks.
2 Model
The model learns from a stream of examples from multiple datasets without dataset identifiers, using a shared encoder-decoder and episodic memory. The memory stores examples to support sparse replay and local adaptation.
- Setup: The continual-learning setup presents one model with sequential datasets from the same task, without dataset descriptors or known dataset boundaries.
- Architecture: The model combines an example encoder, task decoder, and episodic memory module.
- Encoder and decoder: BERT encodes the input, while the task decoder predicts classes for classification or answer spans for question answering.
- Task decoder: For classification, the decoder applies a linear transformation and softmax to the first BERT token representation and predicts all known classes.
- Task decoder: For question answering, the decoder scores answer start and end positions and selects the highest-probability valid span.
- Episodic memory: The episodic memory uses frozen pretrained-BERT keys and stores each input-label pair as a value; random sampling supports replay and nearest neighbors support local adaptation.
3 Training and Inference
Training uses sparse random replay from episodic memory, while inference retrieves nearby examples for temporary local adaptation. The method limits replay frequency and resets adapted parameters after each prediction.
- Sparse experience replay: Sparse experience replay periodically samples stored examples and updates the encoder-decoder, avoiding replay at every timestep.
- Local adaptation: At inference, the key network retrieves K nearest neighbors for each test example using Euclidean distance.
- Local adaptation: Gradient-based local adaptation updates W into example-specific parameters W_i using retrieved examples while keeping the parameters near W.
- Local adaptation: The adapted parameters W_i are used only for the current prediction, after which the model parameters are reset.
- Future work: Cheaper alternatives to gradient-based local adaptation, such as Hebbian updates, remain future work.
4 Experiments
Experiments evaluate episodic-memory models on text classification and question answering in a one-pass, dataset-agnostic continual-learning setup. Sparse experience replay and local adaptation complement each other, while relevant retrieval and limited memory preserve performance with practical trade-offs.
- Setup: Experiments evaluate episodic-memory models against baselines on text classification and question answering tasks.Models are trained in one pass over concatenated training sets and evaluated across four random dataset orderings.
- Main results: MBPA++ achieves the best performance by combining local adaptation with sparse experience replay, outperforming either component alone against ENC-DEC.A-GEM outperforms ENC-DEC but remains below MBPA; the combined method performs best on both tasks.
- Efficiency and analysis: Relevant retrieval improves local adaptation, more neighbors improve performance in both tasks, and sparse replay keeps training time comparable to the encoder-decoder baseline.Inference is slower because local adaptation is required; approximately 15 adaptation steps are needed for convergence, while memory storage increases space complexity.
- Main results: 70.6 vs. 60.7 averaged performance shows MBPA++ outperforming single-dataset models on text classification, whereas 62.0 vs. 66.0 leaves it behind on question answering.MBPA++ has the smallest gap to the multitask upper bound, but continual models still trail that upper bound.
- Retention: MBPA++ is consistently better at retaining knowledge from the first dataset as training proceeds across additional datasets.Figure 2 evaluates F1 and accuracy on the first dataset's test set as new datasets are learned.
- Memory capacity: 10% memory capacity still maintains reasonably high performance, although performance degrades as fewer stored examples are available.The reduced-capacity variants randomly choose whether to store each training example.
5 Conclusion
The paper presents an episodic memory model for lifelong language learning that combines sparse experience replay and local adaptation. Experiments show mitigation of catastrophic forgetting and baseline improvements on text classification and question answering, with a reported question-answering memory limitation.
- The episodic memory model combines sparse experience replay and local adaptation to continuously learn and reuse previously acquired knowledge.
- Experiments show the proposed method mitigates catastrophic forgetting and outperforms baseline methods on text classification and question answering.
- Question answering results could not be obtained for K = 64 and K = 128 because long inputs caused an out-of-memory issue.
A Dataset Order
The appendix lists randomly chosen dataset orders for evaluating text classification and question answering. The orders vary which datasets are encountered first, supporting evaluation across multiple training sequences.
- Text classification: Three randomly chosen dataset orders are listed for text classification.
- Text classification: The text-classification orders permute Yelp, AGNews, DBPedia, Amazon, and Yahoo across the sequence.
- Question answering: Three randomly chosen dataset orders are listed for question answering.
- Question answering: The question-answering orders permute QuAC, TrWeb, TrWik, and SQuAD across the sequence.
B Full Results
The appendix provides per-dataset breakdowns of the main results in separate tables for text classification and question answering.
- Per-dataset results are provided for each task in dedicated breakdown tables.The text-classification and question-answering breakdowns are identified as Table 4 and Table 5, respectively.
- The breakdowns complement the macro-averaged results reported in the main results table.
- The appendix organizes detailed results by dataset rather than only reporting task-level aggregates.
C Single Dataset Models
The paper includes results for models trained on individual datasets and provides a table of per-dataset text-classification results for each ordering and model.
- Single-dataset models: Single-dataset model results are reported for models trained on particular datasets.
- Full results: Table 4 reports per-dataset text-classification results for each dataset ordering and model.
- Full results: The table layout enables comparison across dataset orderings and model types.
D Analysis of Retrieved Examples
The analysis examines how episodic memory retrieves examples and supports local adaptation, while identifying retrieval failures caused by phrasing differences.
- Retrieved neighbors are generally relevant: question-answering examples are syntactically and semantically related, while classification examples discuss similar topics.The paper uses these retrieval patterns to characterize how episodic memory supports predictions across tasks.
- Local adaptation corrects predictions on two illustrated test examples after retrieving thematically or syntactically related training examples.The examples show different retrieval cues: thematic similarity in one case and syntactic similarity for a shorter query in the other.
- Relevant training examples can remain difficult to retrieve when their phrasing differs substantially from the query, placing them outside the 1,000 nearest neighbors.The paper identifies better embeddings or retrieval methods as potential ways to improve performance.
- Per-dataset question-answering results are reported for each ordering and model, providing a breakdown for comparing model behavior.