PaperlayerView exact revision on arXiv

Source-linked AI summary

Low-Resource Knowledge-Grounded Dialogue Generation

Xueliang Zhao, Wei Wu, Chongyang Tao, Can Xu, Dongyan Zhao, Rui Yan

arXiv:2002.10348v1cs.CL

TL;DR

Knowledge-grounded dialogues are difficult to collect, limiting training data for response generation. This paper disentangles the response decoder so most parameters can be learned from ungrounded dialogue and documents; on two benchmarks, it reports state-of-the-art performance with only 1/8 training data and generalization to out-of-domain knowledge.

  • Problem

    Knowledge-grounded dialogues are difficult to obtain at scale, and models trained on small datasets can overfit and perform poorly on out-of-domain documents.

    Source: §1 · p. 1

  • Method

    The model disentangles its response decoder so most parameters can be pretrained using ungrounded dialogues and plain text, leaving a small knowledge-dependent component to fit with limited grounded examples.

    Source: §1 · p. 2

  • Results

    With only 1/8 training data, the model achieves state-of-the-art performance on two benchmarks and generalizes well to out-of-domain knowledge.

    Source: §5 · p. 9

  • Takeaways & Limitations

    Pre-trained parameters can be fixed to adapt the model to a new domain with only a little cost.

    Source: §1 · p. 2

    The setup assumes that a dialogue has an associated background document.

    Source: §2.1 · p. 2

Abstract

from arXiv · show

Responding with knowledge has been recognized as an important capability for an intelligent conversational agent. Yet knowledge-grounded dialogues, as training data for learning such a response generation model, are difficult to obtain. Motivated by the challenge in practice, we consider knowledge-grounded dialogue generation under a natural assumption that only limited training examples are available. In such a low-resource setting, we devise a disentangled response decoder in order to isolate parameters that depend on knowledge-grounded dialogues from the entire generation model. By this means, the major part of the model can be learned from a large number of ungrounded dialogues and unstructured documents, while the remaining small parameters can be well fitted using the limited training examples. Evaluation results on two benchmarks indicate that with only 1/8 training data, our model can achieve the state-of-the-art performance and generalize well on out-of-domain knowledge.

1 INTRODUCTION

Knowledge-grounded dialogue generation is constrained by scarce naturally grounded training data, and existing models may overfit small datasets and falter on out-of-domain documents. The paper separates knowledge-dependent decoder parameters so most of the model can be learned from ungrounded dialogue and plain text.

1 INTRODUCTION

  • Naturally grounded dialogue data is difficult to obtain, while existing models can overfit small training sets and lose performance on out-of-domain documents.Collecting enough data for a new domain or language is difficult because human effort is expensive.Source: §1 · p. 1
  • The model disentangles its decoder into language-model, context-processing, and knowledge-processing components coordinated by a decoding manager.The language model and context processor can be pretrained on ungrounded dialogue, while the knowledge encoder can be pretrained on plain text; only the knowledge processor and manager depend on grounded dialogues.Source: §1 · p. 2
  • With only 1/8 training data, the model achieves state-of-the-art performance on Wizard and CMU DoG and generalizes well to out-of-domain knowledge.On Wizard, it significantly outperforms baselines on out-of-domain documents while using 1/16 training data, even though the baselines use all training data.Source: §1 · p. 2
  • The evaluation compares the model with state-of-the-art baselines on Wizard and CMU DoG, using Reddit data and a Wikipedia dump for pretraining.The authors frame the work as exploring low-resource generation, proposing a disentangled-decoder pretraining approach, and empirically testing it on two benchmarks.Source: §1 · p. 2

2 APPROACH

The approach disentangles response generation into language, context, and knowledge processes so most parameters can be learned from ungrounded dialogue and documents. It fits the remaining knowledge-dependent components on grounded dialogues and proposes adapting the knowledge modules for other modalities.

2.1 PROBLEM FORMALIZATION

  • Grounded set DS is paired with much larger ungrounded-dialogue DC and document DP corpora, with N and M both much greater than n.The response model is trained to generate a response from a new document and its associated dialogue context.Source: §2.1 · p. 2
  • Response formation is decomposed into language-model, context, and knowledge actions that can be learned independently.The proposed decomposition reflects that dialogue words are not always related to the background document.Source: §2.1 · pp. 2–3

2.2 GENERATION MODEL

  • The generation model combines context and knowledge encoders, a decoder, and a manager coordinating three independent decoder components.Those components are the language model, context processor, and knowledge processor.Source: §2.2 · p. 3

2.2.1 ENCODERS

  • Context representations feed the context processor, while document representations are encoded separately and knowledge selection is deferred to decoding.Separating the encoders removes the dependency between context and knowledge encoding, facilitating their estimation from DC and DP, respectively.Source: §2.2.1 · p. 3

2.2.2 DISENTANGLED DECODER

  • The language model predicts from decoder state, while context and knowledge processors attend over their respective sources and can copy source words.The knowledge processor uses hierarchical attention over the document; the context processor combines vocabulary prediction with a copy mechanism.Source: §2.2.2 · p. 4

2.2.3 DECODING MANAGER

  • At each response step, the decoding manager selects one of the three components; training uses Gumbel-Softmax, with temperature gradually reduced.At test time, the distribution is discretized as a one-hot vector.Source: §2.2.3 · p. 4

2.3 LEARNING DETAILS

  • Learning assigns context-side and language-model parameters to ungrounded dialogue, knowledge-encoder parameters to documents, and remaining knowledge-processor and manager parameters to grounded dialogues.The authors suggest that other knowledge modalities can use the recipe by modifying the knowledge encoder and processor, and pretraining the encoder on single-modal data if possible.Source: §2.3 · p. 5

3 EXPERIMENTS

The experiments evaluate the model on Wizard and CMU DoG, comparing it with established baselines under reduced training data. Results and follow-up analyses examine its performance, parameter fine-tuning, pre-training, and the role of disentanglement.

3.1 DATASETS AND EVALUATION METRICS

  • Wizard evaluates responses on topics seen or unseen during training, while CMU DoG focuses on movie documents; both tasks generate document-grounded responses from dialogue history.Wizard keeps the latest 128 words of dialogue history; CMU DoG uses the associated document section as knowledge.Source: §3.1 · pp. 5–6
  • The evaluation uses perplexity, BLEU, BOW Embedding, and unigram F1, with human annotators also assessing response quality qualitatively.The human analysis is presented in Appendix C.Source: §3.1 · p. 6

3.2 BASELINES

  • Baselines include TMN (Dinan et al., 2019) and ITDD (Li et al., 2019); TMN is compared end-to-end without knowledge regularization for fair comparison.The authors note that ground-truth knowledge-selection signals are unavailable in many scenarios, including CMU DoG.Source: §3.2 · p. 6

3.3 EVALUATION RESULTS

  • With 1/8 training data, the model achieves state-of-the-art performance on most metrics across all test sets and remains stable on Test Unseen across training sizes.The authors report greater improvement over baselines on Test Unseen and narrowing Seen–Unseen performance gaps as training data shrinks.Source: §3.3 · pp. 6–7, Table 1 · p. 6, Table 2 · p. 7

3.4 DISCUSSIONS

  • With enough data, fine-tuning can improve both test sets, but at small sizes it may reduce performance, especially on Test Unseen; the authors recommend fixing pre-trained parameters below 5k dialogues.The recommendation is to estimate only the remaining 5% of parameters from knowledge-grounded dialogues.Source: §3.4 · p. 7, Figure 2 · p. 7
  • A TMN variant pre-trained on Reddit dialogues and Wikipedia remains much worse than the proposed model, supporting the authors’ view that disentangling is important for leveraging these sources.For this comparison, the TMN encoder is fixed, its decoder is initialized from a pre-trained decoder, and its document representation is pre-trained before grounded-data fine-tuning.Source: §3.4 · pp. 7–8, Figure 3 · p. 8
  • Removing any pre-trained component hurts performance with little training data; the most impactful component differs by test set, with language model > context > knowledge on Test Seen and knowledge > language model > context on Test Unseen.The analysis ablates pre-training for the language model, context components, and knowledge encoder.Source: Figure 2 · p. 7, §3.4 · p. 8

4 RELATED WORK

Prior work develops open-domain dialogue generation and document grounding, while this paper focuses on low-resource learning through pre-training and a disentangled decoder. Its use of disentanglement is a structural adaptation that separates decoder components for asynchronous parameter estimation.

4 RELATED WORK

  • Open-domain dialogue research has extended sequence-to-sequence models for safe responses, multi-turn history, and document-, persona-, image-, or knowledge-base-grounded generation.The cited examples include safe-response work (Li et al., 2015; Xing et al., 2017; Zhao et al., 2017), dialogue-history modeling (Serban et al., 2016; 2017), and grounding work (Ghazvininejad et al., 2018; Zhao et al., 2019; Zhang et al., 2018; Mostafazadeh et al., 2017).Source: §4 · p. 8
  • The authors position their contribution as studying low-resource knowledge-grounded dialogue generation with pre-training, rather than testing new architectures on benchmarks.Their model builds on neural generation techniques such as attention and copying.Source: §4 · p. 8
  • They adapt disentanglement from representation learning to response-decoder structure, producing independent components that allow asynchronous parameter estimation.The approach combines this structural split with pre-training for the low-resource task.Source: §4 · p. 8

5 CONCLUSIONS

The paper studies knowledge-grounded dialogue generation when training data is scarce and decomposes the response decoder so most parameters can be learned from ungrounded dialogues and unstructured documents. On two benchmarks, it reports state-of-the-art performance with 1/8 of the training data and good generalization to out-of-domain knowledge.

5 CONCLUSIONS

  • With only 1/8 training data, the model achieves state-of-the-art performance on two benchmarks and shows good generalization to out-of-domain knowledge.Source: §5 · p. 9
  • The work targets knowledge-grounded dialogue generation in a low-resource setting.Source: §5 · p. 9
  • The response decoder is decomposed into independent components so most parameters can be estimated from large-scale ungrounded dialogues and unstructured documents.Source: §5 · p. 9

A DETAILS OF DATASETS

Table 4 reports statistics for the Wizard and CMU DoG datasets.

A DETAILS OF DATASETS

B MORE IMPLEMENTATION DETAILS

The implementation uses 300-dimensional word embeddings, 1024-dimensional encoder and decoder hidden states, and three-layer context encoder and decoder networks. Training uses Adam, a warm-up and inverse-square-root learning-rate schedule, validation-based early stopping, and beam search for decoding.

B MORE IMPLEMENTATION DETAILS

  • Word embeddings have size 300, while the context encoder, knowledge encoder, and decoder have hidden size 1024.Source: B MORE IMPLEMENTATION DETAILS · p. 13
  • The context encoder and decoder each have three layers, and the similarity functions use single-layer 512-unit feed-forward networks with tanh.Source: B MORE IMPLEMENTATION DETAILS · p. 13
  • Models use Adam with β1 = 0.9, β2 = 0.999, and an initial learning rate of 5e −4, warmed up for 5000 steps and then decayed proportionally to the inverse square root of step number.Source: B MORE IMPLEMENTATION DETAILS · p. 13
  • Training uses batches of 64, validation-based early stopping, and beam search with beam size 5; weak supervision assigns modal verbs to the language-model category.Source: B MORE IMPLEMENTATION DETAILS · p. 13

C HUMAN EVALUATION

The human evaluation compares response fluency, context coherence, and knowledge relevance, with annotators scoring outputs for examples from Wizard Test Seen and Test Unseen. The proposed model leads on all three aspects in both sets with 1/8 training examples, while a Test Unseen example illustrates its knowledge-connected response generation.

C HUMAN EVALUATION

  • With 1/8 training examples, the proposed model achieves the best human-evaluation performance on fluency, context coherence, and knowledge relevance for both Wizard test sets.Table 5 reports average scores on the three aspects; responses were scored from 0 to 2 by annotators.Source: Table 5 · p. 13, C HUMAN EVALUATION · pp. 13–14
  • The evaluation samples 300 examples from each test set and has three native-speaker annotators score each response for fluency, context coherence, and knowledge relevance.Responses are pooled and randomly shuffled; each aspect receives three annotator scores.Source: C HUMAN EVALUATION · p. 13
  • In a Test Unseen case, the model's response connects to the context and adds relevant knowledge, while baseline responses lose the knowledge connection.The accompanying colors identify words from the language model, context processor, and knowledge processor.Source: C HUMAN EVALUATION · p. 14, Table 6 · p. 14

D COMPARISON WITH MASS

The paper compares its knowledge-grounded dialogue model with MASS on Wizard, adapting MASS to take conversational history and knowledge as input. The proposed model outperforms MASS on Test Seen and Test Unseen across all training sizes.

D COMPARISON WITH MASS

  • The proposed model consistently outperforms MASS on Test Seen and Test Unseen across all training sizes.Figure 4 reports the comparison; the authors do not compare PPL because MASS generates with subwords.Source: Figure 4 · p. 14, D COMPARISON WITH MASS · p. 15
  • MASS is adapted by concatenating knowledge sentences and conversational history into one encoder input.The original MASS model is for sequence-to-sequence generation and is pre-trained by reconstructing a sentence fragment from the remainder.Source: D COMPARISON WITH MASS · p. 15
  • The authors suggest MASS's masking-based pre-training may not suit knowledge-grounded generation, which requires operations such as deep copying.They also suggest MASS may be mismatched to the task because it lacks the additional knowledge input.Source: D COMPARISON WITH MASS · p. 15

E ABLATION OVER COMPONENTS

Ablations remove the language model, context processor, or knowledge processor from the decoding manager during both training and testing. All three components are generally useful, with the knowledge processor most important and the language model least important.

E ABLATION OVER COMPONENTS

  • Removing any of the three decoder components generally reduces performance on Test Seen and Test Unseen.Figure 5 reports the ablation results for both test sets.Source: Figure 5 · p. 15, E ABLATION OVER COMPONENTS · p. 15
  • The components rank in importance as knowledge processor, context processor, then language model.The knowledge processor generates 56% of response words, the context processor 27%, and the language model 17%.Source: E ABLATION OVER COMPONENTS · p. 15
  • The language model's impact may be partly covered by the other processors, which also contain language models.The authors note the language model is pre-trained on monolingual Reddit data, unlike the context and knowledge processors.Source: E ABLATION OVER COMPONENTS · p. 15

F COMPARISON WITH NON-PRETRAINING

With full training data, the model without pre-training outperforms both TMN and the pre-trained version on Test Seen. As training data shrinks, pre-training is more effective, including on Test Unseen.

F COMPARISON WITH NON-PRETRAINING

  • With full training data, the model without pre-training outperforms both TMN and the pre-trained version on Test Seen.The authors attribute this to attention and copying operations capturing correlations in the training data.Source: F COMPARISON WITH NON-PRETRAINING · p. 16
  • At training sizes of 1/8 or less, the non-pre-trained model falls below TMN on Test Seen; on Test Unseen, it falls below TMN when training data is halved.The authors say the non-pre-trained model is more prone to overfitting when training data is insufficient.Source: F COMPARISON WITH NON-PRETRAINING · p. 16
  • On Test Unseen, the pre-trained model with 1/8 training data matches the full-data non-pre-trained model.Across all training sizes, the pre-trained version outperforms the non-pre-trained version on Test Unseen.Source: F COMPARISON WITH NON-PRETRAINING · p. 16

Related papers

Loading 2002.10348v1…