Source-linked AI summary
LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5
Chengwei Qin, Shafiq Joty
TL;DR
Lifelong learning models must acquire new tasks from few examples without overfitting or forgetting prior knowledge. LFPT5 uses T5 prompt tuning to learn across domains and task types; experiments report better performance than previous lifelong methods across three task types while retaining learned knowledge.
Problem
Lifelong learning must handle new tasks with few labeled examples while avoiding overfitting and forgetting previously acquired knowledge.
Source: §1 · pp. 1–2Method
LFPT5 tunes prompts on a frozen T5, using them as task solvers and generators to replay pseudo-samples from earlier domains and adding prompts for new task types.
Source: §3.2.1 · pp. 4–5, §3.2 · p. 4Results
LFPT5 achieves much better performance than previous lifelong learning methods on three task types and adapts to new domains or task types while retaining learned knowledge.
Source: §4.3 · p. 8, §6 · p. 9Takeaways & Limitations
The paper presents LFLL as an important step towards general language intelligence.
Source: §6 · p. 9During NER generation, abbreviation substitutions such as “the United States” for “U.S.” unfairly penalize LFPT5’s F1 score.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Existing approaches to lifelong language learning rely on plenty of labeled data for learning a new task, which is hard to obtain in most real scenarios. Considering that humans can continually learn new tasks from a handful of examples, we expect the models also to be able to generalize well on new few-shot tasks without forgetting the previous ones. In this work, we define this more challenging yet practical problem as Lifelong Few-shot Language Learning (LFLL) and propose a unified framework for it based on prompt tuning of T5. Our framework called LFPT5 takes full advantage of PT's strong few-shot learning ability, and simultaneously trains the model as a task solver and a data generator. Before learning a new domain of the same task type, LFPT5 generates pseudo (labeled) samples of previously learned domains, and later gets trained on those samples to alleviate forgetting of previous knowledge as it learns the new domain. In addition, a KL divergence loss is minimized to achieve label consistency between the previous and the current model. While adapting to a new task type, LFPT5 includes and tunes additional prompt embeddings for the new task. With extensive experiments, we demonstrate that LFPT5 can be applied to various different types of tasks and significantly outperform previous methods in different LFLL settings.
1 INTRODUCTION
The introduction frames lifelong few-shot learning as a practical challenge: models should learn new tasks from few examples without forgetting prior knowledge. LFPT5 applies T5 prompt tuning across task domains and types, and the paper reports strong performance against previous baselines.
1 INTRODUCTION
- Earlier lifelong language methods assume plentiful training data, focus largely on question-answering tasks, and fine-tune the entire model, with potential negative transfer across task types.These limitations motivate learning from few-shot examples and addressing sequence labeling as well as other task types.Source: §1 · p. 1
- LFLL asks models to learn new few-shot tasks without forgetting prior knowledge, spanning sequence labeling, text classification, and text generation.The introduction distinguishes learning new domains of the same task type from learning different task types.Source: §1 · p. 2, Figure 1 · p. 2
- LFPT5 uses T5 prompt tuning, generating pseudo-labeled samples from earlier domains for training on a new domain and adding prompts for new task types.It also minimizes a KL divergence loss for label consistency between previous and current models.Source: §1 · p. 2
- LFPT5 outperforms previous baselines by a large margin and generalizes to new few-shot tasks without severe forgetting.The paper presents it as a unified framework for lifelong few-shot language learning.Source: §1 · p. 2
2 RELATED WORK
Related work covers lifelong learning, few-shot learning, and prompt-based learning as separate lines of research. The paper positions LFLL as their intersection: sequential learning from few-shot tasks while avoiding overfitting and forgetting.
2.1 LIFELONG LEARNING
- Lifelong learning methods address catastrophic forgetting through architecture, regularization, or memory, but mostly focus on tasks of the same type.LAMOL is described as a framework for learning across domains and different NLP task types.Source: §2.1 · p. 2
2.2 FEW-SHOT LEARNING
- Few-shot learning addresses learning from scarce labeled examples, with approaches based on model design, data augmentation, or improved hypothesis search.Prompt-based methods such as GPT-3, PET, and LM-BFF are noted for promising few-shot language learning results.Source: §2.2 · pp. 2–3
2.3 PROMPT-BASED LEARNING
- Prompt-based learning modifies inputs with task-specific prompts, which may be discrete or continuous, and has been studied across applications and analyses.The paper combines prompt tuning's few-shot ability with lifelong learning in its LFLL framing.Source: §2.3 · p. 3
3 METHODOLOGY
LFLL formalizes continual learning from few-shot tasks across both domains within one task type and sequences of different task types. LFPT5 uses prompt tuning of T5, task-solving and data-generation prompts, replay with label-consistency training for new domains, and separate prompts for new task types.
3 METHODOLOGY
- The methodology defines continual few-shot adaptation across domains and task types, with a few-shot validation set matching the training set's size.For same-type domains, evaluation uses the combined test sets of learned domains.Source: §3 · p. 3, §3.1 · p. 3
3.1 PROBLEM FORMULATION
- LFLL covers sequential few-shot domains within one task type and sequences of different task types, evaluating learned domains jointly but different task types separately.The validation set is assumed to match the few-shot training set in size.Source: §3.1 · p. 3
3.2 LIFELONG FEW-SHOT LANGUAGE LEARNING WITH PROMPT TUNING OF T5 (LFPT5)
- LFPT5 freezes pretrained T5 and optimizes a small set of task prompt embeddings, drawing on prompt tuning's few-shot learning ability and avoiding full-model fine-tuning on scarce data.The paper notes full-model fine-tuning on small few-shot tasks could lead to overfitting.Source: §3.2 · p. 4
3.2.1 TASK FORMULATION & ADAPTATION
- LFPT5 reframes sequence labeling, classification, and generation as text-to-text tasks, training prompts both to decode task outputs and to generate input-output pseudo-samples.Sequence-labeling outputs use segment-label pairs, while classification labels become natural-language descriptions.Source: §3.2.1 · pp. 4–5
- For a new domain, LFPT5 generates pseudo-labeled samples from earlier domains for replay and minimizes KL divergence between previous and current models to maintain label consistency.The overall new-domain objective combines this with the task loss.Source: §3.2.1 · p. 5
- For a new task type, LFPT5 adds and tunes a dedicated prompt while freezing earlier task prompts; the new prompt can be initialized from a previously learned one.Each task prompt has 300 tunable tokens, about 0.04% of pretrained T5's parameters.Source: §3.2.1 · p. 5
4 EXPERIMENTS
Experiments evaluate LFPT5 across sequence labeling, classification, and summarization, for both new-domain and new-task-type learning. The reported results favor LFPT5 over prior lifelong methods, while the relative benefit of prompt tuning and forward transfer varies across tasks.
4.1 EXPERIMENT SETUP
- Experiments cover NER, text classification, and summarization, using 16-shot training and validation for NER and classification, 64 examples per summarization domain, and three random-seed runs.The study also compares lifelong-learning baselines for new domains and multitask fine-tuning, multitask prompt tuning, and AdapterFusion for new task types.Source: §4.1 · p. 6
4.2 SINGLE TASK RESULTS
- T5 prompt tuning beats fine-tuned BERT-Large on every reported single-task case except OntoNotes NER, and beats T5 fine-tuning on all summarization measures except WikiHow ROUGE-1.The comparison covers NER F1 and classification accuracy in Table 1 and summarization ROUGE scores in Figure 4.Source: Table 1 · p. 6, §4.2 · p. 7, Figure 4 · p. 7
4.3 RESULTS FOR LEARNING NEW DOMAINS
- For NER, LFPT5 outperforms EWC and MAS by a large margin, while those regularization methods improve only slightly over fine-tuning.The comparison reports final F1 after learning all NER domains.Source: §4.3 · p. 7, Table 2 · p. 7
- Across NER and classification, prompt-tuning methods perform better than fine-tuning methods for NER, but fine-tuning methods perform much better for classification.The authors attribute the classification difference to limited label-space overlap across domains and prompt tuning's smaller number of tunable parameters.Source: §4.3 · p. 7, Table 3 · p. 7
- On a nine-task classification sequence, LFPT5 performs much better than previous baselines.The sequence combines five NLI tasks with the original four classification tasks.Source: §4.3 · pp. 7–8
- For summarization, prompt-tuning and fine-tuning methods perform similarly, and LFPT5 outperforms previous lifelong methods.Because generated pseudo summaries were often ambiguous, the authors form pseudo data using the generated document's leading three sentences as its summary.Source: §4.3 · p. 8, Table 4 · p. 8
- LFPT5 performs much better than previous lifelong learning methods across NER, text classification, and summarization.The whole-test results cover all learned NER domains, four classification domains, and three summarization domains.Source: Table 2 · p. 7, Table 3 · p. 7, §4.3 · p. 8
4.4 RESULTS FOR LEARNING NEW TASK TYPES
- Across the three task types considered, both LFPT5 variants achieve better performance than MT-FT and MT-PT, while LFPT5 performs better than AdapterFusion and adds about 0.12% of pretrained T5 parameters versus 21.72%.Forward transfer between classification and summarization is positive, but transfer involving NER negatively affects learning of the new task.Source: §4.4 · p. 8
5 ANALYSIS
Domain order substantially affects classification performance, but LFPT5 outperforms regularization-based lifelong methods across the tested orders. The analysis also finds that label-consistency loss improves summarization results at suitable weights, while NER entity generation can unfairly penalize abbreviation variants.
Influence of Domain Order
- LFPT5 outperforms regularization-based lifelong methods across all tested domain orders, although domain order strongly affects classification performance.PT accuracy is 41.67 on the third run and 18.88 on the first.Source: §5 · p. 9
Influence of KL Loss
- A-RG reaches 17.11 at λkl = 0.10; removing LKL performs worse than other tested variants except λkl = 0.40.The λkl = 0.40 variant scores 16.26.Source: §5 · p. 9, Table 7 · p. 9
Abbreviation Variations
- NER abbreviation variants such as “U.S.” and “the United States” can unfairly lower F1 when LFPT5 generates entities instead of copying them.The authors note that this also indicates T5 can express relevant knowledge in its own way.Source: §5 · p. 9
6 CONCLUSION
LFPT5 is presented as a unified framework for lifelong few-shot learning across new task types and domains. Experiments and analysis indicate adaptation while retaining learned-task knowledge, which the authors regard as a step toward general language intelligence.
6 CONCLUSION
- LFPT5 adapts to new task types and domains while retaining learned-task knowledge, an outcome the authors regard as an important step toward general language intelligence.The authors identify improving generated pseudo-sample quality as future work.Source: §6 · p. 9
A.3 LEARNING FROM A LARGE NUMBER OF DIFFERENT DOMAINS
On a sequence of nine classification tasks, LFPT5 performs much better than previous baselines after learning all tasks.
A.3 LEARNING FROM A LARGE NUMBER OF DIFFERENT DOMAINS
- After learning all 9 classification domains, LFPT5 performs much better than previous baselines.The evaluation combines five NLI tasks with four original classification tasks.Source: §A.3 · p. 13, Table 8 · p. 14
A.4 DIFFERENT BACKBONE MODEL
Across appendix experiments, LFPT5 outperforms comparison methods under varied few-shot counts and prompt-tuning setups, though it trails EWC fine-tuning with a T5-Base backbone. The analysis also identifies a non-monotonic relationship between pseudo-sample count and performance, and explains near-zero prior-task performance for prompt-based EWC and MAS.
A.4 DIFFERENT BACKBONE MODEL
- On T5-Base summarization, LFPT5 beats EWC prompt tuning but is slightly worse than EWC fine-tuning.The authors relate this pattern to prompt tuning performing better with larger pretrained models.Source: §A.4 · p. 14, Table 9 · p. 14
A.5 DIFFERENT NUMBERS OF FEW-SHOT DATA
- With 16 and 32 few-shot examples, LFPT5 consistently outperforms EWC prompt tuning and fine-tuning on summarization.Table 10 reports A-RG scores for these two sample counts.Source: §A.5 · p. 14, Table 10 · p. 14
A.6 INFLUENCE OF THE NUMBER OF PSEUDO SAMPLES
- More pseudo samples do not always improve summarization; 4 pseudo samples yield the best A-RG score, 17.05.The authors note that generating pseudo samples is feasible and cheaper.Source: §A.6 · p. 14, Table 11 · p. 15
A.7 MULTIPLE PROMPTS IN MULTITASK PROMPT TUNING
- LFPT5 outperforms both single- and multiple-prompt multitask tuning, including a comparison with matched tunable-token counts.The multiple prompts share tunable tokens across all three tasks.Source: §A.7 · p. 14, Table 12 · p. 15
A.8 ANALYSIS ON PT-BASED EWC AND MAS
- PT-based EWC and MAS retain almost no previous-task performance; the authors point to few tunable prompt parameters and already suboptimal few-shot training.LFPT5 uses pseudo samples to alleviate forgetting associated with learning new domain knowledge.Source: §A.8 · pp. 14–15