PaperlayerView exact revision on arXiv

Source-linked AI summary

Domain Adaptive Dialog Generation via Meta Learning

Kun Qian, Zhou Yu

arXiv:1906.03520v2cs.CLcs.LG

TL;DR

Dialog tasks differ by domain, while collecting real-user dialogs is costly, making adaptation to new domains with little data important. DAML applies meta-learning across dialog tasks and adapts with a small target-domain sample; on a simulated dataset, it reaches state-of-the-art Entity F1 against zero-shot and transfer methods.

  • Problem

    Dialog domains differ, and real-user dialogs are costly to collect, while few studies address domain-adaptive end-to-end dialog training.

    Source: §1 · p. 1

  • Method

    DAML applies model-agnostic meta-learning across multiple dialog tasks to learn a transferable initialization, then fine-tunes it on a small target-domain sample.

    Source: §1 · pp. 1–2

  • Results

    DAML reaches state-of-the-art Entity F1 against zero-shot and transfer methods; on New Domain, it scores 66.2, a 25.8% relative improvement over ZSDG.

    Source: §1 · p. 2, §6 · p. 7, §7 · p. 9

  • Takeaways & Limitations

    The authors conclude DAML is effective and robust for low-resource dialog training; on New Domain, it converges in 5.8 epochs on average, 40% of transfer learning's epochs.

    Source: §6 · p. 8, §7 · p. 9

    DAML still struggles with out-of-domain words: unknown tokens can hinder slot extraction and belief-span copying, while pronoun “ones” can leave the “western” slot unrecognized.

    Source: §6 · pp. 8–9

Abstract

from arXiv · show

Domain adaptation is an essential task in dialog system building because there are so many new dialog tasks created for different needs every day. Collecting and annotating training data for these new tasks is costly since it involves real user interactions. We propose a domain adaptive dialog generation method based on meta-learning (DAML). DAML is an end-to-end trainable dialog system model that learns from multiple rich-resource tasks and then adapts to new domains with minimal training samples. We train a dialog system model using multiple rich-resource single-domain dialog data by applying the model-agnostic meta-learning algorithm to dialog domain. The model is capable of learning a competitive dialog system on a new domain with only a few training examples in an efficient manner. The two-step gradient updates in DAML enable the model to learn general features across multiple tasks. We evaluate our method on a simulated dialog dataset and achieve state-of-the-art performance, which is generalizable to new tasks.

1 Introduction

Dialog systems require substantial task-specific training data, while real-user interactions make collecting it costly. DAML applies MAML across rich-resource dialog domains to support adaptation to new domains with little data.

1 Introduction

  • Because end-to-end dialog systems require thousands of training dialogs and real users must provide them, adapting rich-resource data to new domains is an important task.The passages identify domain-specific knowledge as a reason dialog tasks differ.Source: §1 · p. 1
  • DAML applies MAML across multiple dialog tasks to learn transferable information for adapting to new domains with limited resources.The method seeks an initialization that can adapt accurately and quickly; MAML is described as making small parameter updates yield large improvements on new-task loss.Source: §1 · p. 1
  • The method uses restaurant, weather, and bus-information-search domains as sources and movie information search as the target.It adapts a Sequicity model with two-stage CopyNet, first learning an initialization from source-domain data and then fine-tuning it on a minimal portion of target-domain data.Source: §1 · pp. 1–2
  • DAML outperforms the ZSDG zero-shot baseline and other transfer-learning methods in the reported evaluation.The evaluation uses simulated data, and the paper reports the adapted model on target-domain test data.Source: §1 · p. 2

2 Related Works

Prior work explores dialog adaptation across several components and learning paradigms, but the cited work identifies little seq2seq dialog domain adaptation beyond ZSDG. This paper applies MAML to Sequicity for that setting.

2 Related Works

  • End-to-end dialog systems combine task modules in seq2seq models, and Sequicity uses two-stage CopyNet for belief tracking and response generation.The cited work reports fewer parameters and faster training than its state-of-the-art baselines while outperforming them on two large-scale datasets.Source: §2 · p. 2
  • Prior practice trains task-specific models with abundant annotated data, whereas dialog adaptation seeks to reuse trained models with a small amount of new-task data.The passage frames this as an alternative to training a model from scratch for each task.Source: §2 · p. 2
  • Earlier dialog adaptation work addresses user sentiment, user adaptation, NLG, dialog-state tracking, and policy learning, while seq2seq domain adaptation is described as limited to ZSDG.ZSDG adapts action matching from multiple source domains to a new target using its domain description.Source: §2 · p. 2
  • The paper applies MAML to Sequicity, choosing it as a model-agnostic algorithm that learns an initialization adaptable to new tasks with little data.The passage also describes meta-learning alternatives that optimize the original network’s optimizer.Source: §2 · p. 2

3 Problem Formulation

The formulation treats dialog generation as mapping context to response and defines source- and target-domain training data. Its goal is to fine-tune a source-trained model for the target domain using a small target sample.

3 Problem Formulation

  • Seq2seq dialog models take dialog context c as input and generate response r.The formulation denotes context and system-response sets as C and R.Source: §3 · pp. 2–3
  • The target-domain training data is much smaller than the source data, with N′ set to 1% of N in this setting.The passage characterizes the target sample count as substantially below the source count.Source: §3 · p. 3
  • The adaptation fine-tunes source model Msource on target training data DT_train to obtain Mtarget, with the goal of performing well in the new domain.The supplied Figure 1 caption distinguishes classical gradient updates from the MAML update procedure and says its numbered steps indicate processing order.Source: §3 · p. 3, Figure 1 · p. 3

4 Proposed Methods

DAML applies MAML to dialog generation, using updates across source domains to learn an initialization intended to adapt quickly to a new domain. Its Sequicity-based model represents dialog state with belief spans and generates responses using attention and copying.

Sequicity model

  • The method uses belief spans to represent dialog history and update slot information, with a shared encoder supporting belief-span extraction and response generation.The response is generated using the belief span and current utterance.Source: §4 · pp. 3–5

MAML for dialog generation

  • MAML updates temporary models for each source domain, then uses their losses to update the original model toward representations shared across domains.The meta-objective sums the updated loss values from all source domains; this lets the model learn from temporary models rather than directly from the current model.Source: §4 · pp. 3–4

Response generation

  • The response decoder uses the belief span and a match label to generate a response, while the label indicates whether database results are absent, exact, or multiple.Multiple matches prompt the system to elicit more constraints; an exact match completes the task, while no match restarts the conversation.Source: §4 · pp. 4–5

Copy attention

  • Copy attention combines vocabulary-generation and input-copy probabilities because dialog responses and user utterances often share tokens.The response decoder copies from the recently generated belief span rather than directly from the utterance.Source: §4 · p. 5

5 Experiment

The experiments evaluate DAML on SimDial, using restaurant, weather, and bus as source domains and three target domains. They specify the metrics, comparison baselines, and implementation settings.

5 Experiment

  • The experiment section introduces the dataset, evaluation metrics, compared models, and implementation details.These components define the setup for evaluating DAML.Source: §5 · p. 5

5.1 Dataset

  • The experiments use SimDial, which was introduced to evaluate ZSDG, and designate restaurant, weather, and bus as source domains, with movie, restaurant-slot, and restaurant-style as targets.Movie has different NLG templates and dialog structure from the source domains; target adaptation uses nine dialogs, about 76 system responses, and testing uses 500 dialogs per target model.Source: §5.1 · pp. 5–6

5.2 Metrics

  • The evaluation measures response quality with BLEU, belief-span completeness with entity F1, and adaptation speed by counting training epochs.Adaptation-time comparisons use data of the same size, and results average ten random runs; the study also tests one-shot, 5%, and 10% adaptation-data settings.Source: §5.2 · p. 6, §5.1 · p. 6

5.3 Baseline Models

  • DAML is compared with ZSDG, which uses 100 labeled utterances as domain-description seeds, and transfer learning on Sequicity trained with mixed source-domain data.The transfer-learning baseline also enlarges its vocabulary with target training data, and includes a one-shot version.Source: §5.3 · p. 6

5.4 Implementation details

  • The encoder and decoder are one-layer GRUs with hidden size 50; experiments use pre-trained GloVe word embeddings of dimension 50, Adam with learning rate 0.003, and batch size 32.The dropout rate is 0.5, and the learning rate is halved if validation loss increases.Source: §5.4 · p. 6

6 Results and Analysis

DAML outperforms ZSDG and transfer learning across the reported comparisons, including the new movie domain, and adapts with fewer epochs. Its performance improves with more target data, while unknown domain words and complex utterances remain challenging.

6 Results and Analysis

  • With comparable target data, DAML outperforms ZSDG and transfer learning; on New Domain it reaches Entity F1 66.2, a 25.8% relative improvement over ZSDG.Table 1 also reports a 14.4% improvement over ZSDG for In Domain testing.Source: Table 1 · p. 7, §6 · p. 7
  • DAML also gains 14.4% over ZSDG In Domain, while restaurant-data fine-tuning raises both BLEU and entity F1 on Unseen Slot and Unseen NLG.The reported improvements for Unseen Slot and Unseen NLG appear in parentheses in Table 1.Source: §6 · p. 7, Table 1 · p. 7
  • In one-shot adaptation, DAML beats ZSDG in every reported case except Unseen Slot entity F1, and achieves the highest score for Unseen NLG.The authors suggest Unseen NLG is relatively close to In Domain; the one-shot result is also strong on In Domain.Source: §6 · p. 7
  • DAML needs one epoch to find the In Domain optimum and averages 5.8 epochs for New Domain, 40% of transfer learning’s epochs.The reported epoch values average ten random runs.Source: §6 · pp. 7–8
  • The authors attribute DAML’s results partly to CopyNet copying context tokens, belief spans simplifying slot handling, and meta-learning capturing cross-domain dialog features.The belief span lets the model use previously stored slots rather than extracting slots from all earlier utterances at each turn.Source: §6 · p. 8
  • Across leave-one-out domains, DAML outperforms transfer learning on Entity F1 and BLEU; restaurant is easiest, while bus has lowest entity F1 and movie lowest BLEU.The authors characterize movie as a hard adaptation target.Source: Table 2 · p. 8, §6 · p. 8
  • As target-domain data increases, performance improves, with entity F1 and BLEU nearly converging when 4% of the data, or 90 dialogs, is used.Even the one-shot setting outperforms ZSDG in the new domain.Source: Figure 3 · p. 8, §6 · p. 8
  • Unknown out-of-domain words can impede slot extraction or belief-span copying, and complex corrections or pronouns can lead to incorrect slot interpretation.The examples include failure to replace an earlier year slot after a correction and failure to identify “western” as a slot from “ones.”Source: §6 · pp. 8–9

7 Conclusion and Future Work

DAML uses two-step gradient updates for domain-adaptive dialog generation and reaches state-of-the-art Entity F1 on a simulated dataset. The paper identifies reinforcement-learning-based dialog systems and multi-domain dialog tasks as potential extensions.

7 Conclusion and Future Work

  • DAML reaches state-of-the-art Entity F1 compared with zero-shot and transfer learning methods on a simulated dataset with multiple independent domains.Source: §7 · p. 9
  • DAML uses a two-step gradient update to obtain dialog models that are more sensitive to new domains.Source: §7 · p. 9
  • Potential extensions include applying DAML to reinforcement-learning-based dialog systems and adapting it to multi-domain dialog tasks.Source: §7 · p. 9

A SimDial Dataset Example

Table 3 presents a SimDial-generated restaurant-domain dialog example. The accompanying text notes that domain and sentence complexity can be adjusted and that each turn’s informable and requestable slots are stored in the belief span.

A SimDial Dataset Example

Related papers

Loading 1906.03520v2…