Source-linked AI summary
Continual Learning in Task-Oriented Dialogue Systems
Andrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon, Paul Crook, Bing Liu, Zhou Yu, Eunjoon Cho, Zhiguang Wang
TL;DR
Continual learning could let task-oriented dialogue systems add domains without full retraining, but the area lacks comprehensive evaluation. The paper introduces a 37-task, four-setting benchmark, compares continual-learning strategies, and finds that replay and residual adapters remain below the multitask baseline while exposing resource trade-offs.
Problem
Task-oriented dialogue systems need to acquire new domains and functionalities over time, while existing continual-learning research in this setting is limited and catastrophic forgetting remains a central challenge.
Method
The paper benchmarks 37 sequential tasks across four dialogue settings, compares regularization, rehearsal, and architectural methods, and proposes residual-adapter-based AdapterCL.
Results
REPLAY and AdapterCL perform comparably well, but no continual-learning method reaches the multitask baseline, especially for dialogue state tracking.
Takeaways & Limitations
Continual learning in task-oriented dialogue is challenging, with method choice involving trade-offs among performance, parameter usage, and episodic-memory size.
Takeaways & Limitations
The evaluation is limited to the Avg. Metric and does not include additional transfer metrics because the baselines show a large performance gap.
Abstract
from arXiv · showhide
Continual learning in task-oriented dialogue systems can allow us to add new domains and functionalities through time without incurring the high cost of a whole system retraining. In this paper, we propose a continual learning benchmark for task-oriented dialogue systems with 37 domains to be learned continuously in four settings, such as intent recognition, state tracking, natural language generation, and end-to-end. Moreover, we implement and compare multiple existing continual learning baselines, and we propose a simple yet effective architectural method based on residual adapters. Our experiments demonstrate that the proposed architectural method and a simple replay-based strategy perform comparably well but they both achieve inferior performance to the multi-task learning baseline, in where all the data are shown at once, showing that continual learning in task-oriented dialogue systems is a challenging task. Furthermore, we reveal several trade-offs between different continual learning methods in term of parameter usage and memory size, which are important in the design of a task-oriented dialogue system. The proposed benchmark is released together with several baselines to promote more research in this direction.
1 Introduction
Task-oriented dialogue systems must continually absorb new domains and functionalities, but sequential learning risks forgetting earlier knowledge. The paper addresses this gap with a broad benchmark spanning four dialogue settings and multiple continual-learning approaches.
- Existing dialogue models assume a fixed dataset and require costly full retraining to add domains or functionalities over time.
- Catastrophic forgetting arises from distribution shifts across sequential tasks; common responses include regularization, rehearsal, and architectural methods.
- The benchmark evaluates continual learning across 37 tasks and four settings: intent recognition, dialogue state tracking, natural language generation, and end-to-end dialogue.
- The paper proposes AdapterCL, a residual-adapter method designed to learn tasks without requiring a task classifier at test time.
- The study compares performance, parameter usage, and episodic-memory size across regularization, rehearsal, and architectural continual-learning methods.
2 Background
The paper models task-oriented dialogue as sequence-to-sequence generation and frames continual learning as sequential domain acquisition without task identity at inference. It studies four dialogue mappings and compares regularization, rehearsal, and architectural strategies.
- Task-Oriented Dialogue Modelling: Task-oriented dialogue is modeled as sequence-to-sequence generation of API calls and system responses, supporting intent, state-tracking, natural-language-generation, and end-to-end settings.
- Task-Oriented Dialogue Modelling: Dialogue history is tokenized with speaker markers, while API calls combine an intent or API name with slot-value pairs and may return a speech act.
- Task-Oriented Dialogue Modelling: The four settings map dialogue history or intermediate representations to intents, slot values, speech acts, or complete API-and-response sequences.
- Continual Learning: Continual learning presents domains sequentially and evaluates the model without providing the domain identity at test time.
- Continual Learning: Regularization protects earlier parameters, while rehearsal reuses episodic memories; the study includes L2, EWC, replay, A-GEM, GEM, and LAMOL variants.
3 AdapterCL
AdapterCL is an architectural continual-learning method that assigns each task a residual adapter while keeping the pretrained model’s original weights frozen. During testing, it selects an adapter using input perplexity rather than a separate task classifier.
- AdapterCL: AdapterCL parameterizes each task with residual adapters while leaving the pretrained model’s original weights frozen.A new adapter is spawned and trained for each task, with optimization restricted to that adapter’s parameters.
- Residual adapters: Residual adapters add trainable bottleneck modules on top of transformer layers, steering outputs through layer normalization, linear layers, and a residual connection.The bottleneck dimension b controls adapter capacity for the target task.
- Perplexity-Based Classifier: At test time, AdapterCL predicts which adapter to use by selecting the one with the lowest input perplexity.The selector evaluates adapter confidence from perplexity over the input sequence.
- Perplexity-Based Classifier: The perplexity-based selector requires a number of forward passes linear in the number of adapters but avoids a separate task classifier and its episodic memory.This trades additional inference computation for avoiding another classifier that could suffer catastrophic forgetting.
4 Experimental Settings
The experiments build a 37-domain continual-learning benchmark across four dialogue settings and evaluate metrics, resources, and competing baselines. The evaluation emphasizes end-to-end performance and the trade-offs between continual-learning approaches.
- 4.1 Datasets: 37 domains are evaluated continuously across INTENT classification, dialogue state tracking, natural language generation, and end-to-end settings.The benchmark combines four task-oriented dialogue datasets while avoiding domain overlap.
- 4.2 Evaluation Metric: Evaluation uses intent accuracy, Joint Goal Accuracy, BLEU, and slot error rate, with end-to-end metrics derived from modularized subtasks.When the end-to-end API output is empty, BLEU is used for response evaluation.
- 4.2 Evaluation Metric: Avg. Metric averages test performance across all curriculum tasks after each task is learned, revealing learning dynamics over time.The evaluation constructs a task-by-task performance matrix and reports the average across tasks.
- 4.3 Baselines and Settings: The study compares regularization, rehearsal, architectural, continuously trained VANILLA, and multitask baselines under controlled memory and parameter settings.REPLAY and GEM retain 50 samples per task, while MULTI trains on all curriculum data simultaneously.
- 4.3 Baselines and Settings: Figure 4 tracks Avg. Metric for end-to-end intent accuracy and EER across the continual-learning curriculum and baselines.The figure focuses on two end-to-end metrics over successive learned tasks.
5 Results & Analysis
The experiments compare continual-learning methods across task-oriented dialogue settings and resources. REPLAY and AdapterCL perform relatively well, but no continual-learning method reaches the multi-task baseline, while memory and parameter costs grow with tasks.
- No continual-learning method reaches the multi-task baseline, especially in Dialogue State Tracking.Regularization methods L2/EWC and some rehearsal methods AGEM/LAMOL also suffer greatly, while REPLAY and AdapterCL perform comparably in Intent and DST.
- REPLAY works best in Natural Language Generation, indicating that transferring learning between tasks is needed.
- REPLAY and AdapterCL outperform other baselines over the curriculum, whereas LAMOL matches REPLAY only during the first 12 tasks.LAMOL generates training samples instead of using explicit memory, and generation becomes harder as more tasks are introduced.
- 5.1 No Free Lunch: REPLAY stores 50 samples per task, while AdapterCL adds adapter parameters per task, so both resource requirements grow linearly with the number of tasks.The paper frames this as a no-free-lunch trade-off for avoiding catastrophic forgetting.
- 5.1 No Free Lunch: Memory-based REPLAY preserves parameter count but loses the original pretrained weights, whereas AdapterCL retains them but isolates datasets and limits knowledge transfer.
- 5.2 Analysis: Episodic Memory Size: With around 500 samples per task, REPLAY approaches the multi-task baseline in DST, compared with severe forgetting when storing only 10–50 samples.In the reported setting, 500 samples per task equals 18,500 total samples.
6 Related Work
Prior continual-learning research has mainly focused on computer vision, NLP classification and generation, or limited dialogue settings. Earlier task-oriented dialogue studies used only a few domains or a single setting, motivating broader comparisons.
- Existing continual-learning work spans computer vision, NLP classification, and NLP generation, but some methods are unsuitable for interactive systems.
- LAMOL is used as a baseline after prior studies evaluated it on a subset of DecaNLP.
- Continual learning in dialogue has been studied in limited settings, including three sequential domains with EWC and one-domain-at-a-time NLG in MWoZ.
7 Conclusion
The paper introduces a broad continual-learning benchmark for task-oriented dialogue and evaluates regularization, rehearsal, and architectural approaches. It also proposes residual adapters and analyzes performance-resource trade-offs.
- The benchmark contains 37 continuously learned tasks across Intent Recognition, Dialogue State Tracking, Natural Language Generation, and end-to-end settings.
- The study implements regularization, rehearsal, and architectural continual-learning methodologies.
- Residual adapters with an entropy-based classifier select the adapter used at testing time.
- The analysis reveals a no-free-lunch trade-off among performance, parameter count, and episodic-memory size.
B Supplemental Material
Supplemental material reports modularized results, episodic-memory ablations, and metric plots for response generation and intent accuracy. It also visualizes resource growth across tasks.
- Figure 6 reports average response-generation metrics using BLEU and EER.
- Figures 8 and 9 examine episodic-memory size against intent accuracy and BLEU/EER, respectively.
- Figure 7 presents modularized results, while Table 5 lists all data samples used in the experiments.
- Figure 10 plots the number of tasks against added parameters and episodic-memory size to illustrate resource trade-offs.