Source-linked AI summary
PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable
Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang
TL;DR
Dialogue generation must handle limited conversational data, mismatched training characteristics, and one-to-many response mappings. PLATO uses dialogue-specific pre-training, flexible attention, and jointly learned discrete latent acts, achieving reported improvements across three conversation task types and state-of-the-art comparisons. The authors identify finer-grained latent variables, reinforcement-learning-based latent selection, and multilingual pre-training as future directions.
Problem
Dialogue generation faces limited conversational data, distribution differences from general text, directional-training mismatches, and multiple appropriate replies for one context.
Method
PLATO further pre-trains on Reddit and Twitter, combines uni- and bi-directional processing, and jointly trains response generation with unsupervised discrete latent act recognition.
Results
Experiments on three conversation task types show significant improvements over other state-of-the-art methods, with human evaluations consistently favoring the method across reported metrics.
Takeaways & Limitations
The framework supports chit-chat, knowledge grounded conversation, and conversational question answering within a flexible dialogue-generation pre-training model.
Takeaways & Limitations
The authors identify finer-grained latent variables, reinforcement-learning-based latent selection, and support for other languages as future improvements.
Abstract
from arXiv · showhide
Pre-training models have been proved effective for a wide range of natural language processing tasks. Inspired by this, we propose a novel dialogue generation pre-training framework to support various kinds of conversations, including chit-chat, knowledge grounded dialogues, and conversational question answering. In this framework, we adopt flexible attention mechanisms to fully leverage the bi-directional context and the uni-directional characteristic of language generation. We also introduce discrete latent variables to tackle the inherent one-to-many mapping problem in response generation. Two reciprocal tasks of response generation and latent act recognition are designed and carried out simultaneously within a shared network. Comprehensive experiments on three publicly available datasets verify the effectiveness and superiority of the proposed framework.
1 Introduction
Dialogue generation remains difficult because conversational data, linguistic patterns, generation directionality, and appropriate responses differ from general NLP settings. PLATO addresses these challenges with dialogue-specific pre-training, flexible attention, and unsupervised discrete latent variables, and reports effectiveness across multiple conversation tasks.
- Dialogue generation is challenging because human-conversation data are limited, background knowledge is complex, and utterances have diverse relationships.
- Directly fine-tuning general-text models can underperform because conversation patterns and data distributions differ, generation is uni-directional rather than bi-directional, and one context can have multiple appropriate replies.
- PLATO further pre-trains on large-scale Reddit and Twitter conversations to reduce distribution gaps and uses flexible uni- and bi-directional processing.
- Discrete latent variables model one-to-many responses, with each value representing a particular conversational intent or latent speech act.
- Response generation and latent act recognition are trained simultaneously in a shared network, without requiring human annotations for the latent variables.
- Experiments cover chit-chat, knowledge grounded conversation, and conversational question answering, with results reported as effective and superior to other state-of-the-art methods.
2 Dialogue Generation Pre-training
The framework models diverse responses with discrete latent speech acts and jointly trains response generation with latent act recognition in a shared Transformer network. Flexible attention, structured input embeddings, and auxiliary losses support multi-turn dialogue pre-training.
- Model Architecture: Discrete latent variables model the one-to-many relationship between dialogue contexts and appropriate responses, with each categorical value corresponding to a latent speech act.
- Model Architecture: Response generation conditions on context and a sampled latent speech act, while latent act recognition estimates the posterior speech-act distribution from context and response.
- Model Architecture: A shared Transformer infrastructure supports bi-directional encoding for recognition and uni-directional decoding for generation through task-specific self-attention masks.
- Input Representation: Each input embedding sums token, role, turn, and position embeddings, with special handling for latent variables, utterance boundaries, and conversation participants.
- Training Objectives: Pre-training combines negative log-likelihood, bag-of-words, and response-selection losses; the bag-of-words objective encourages latent variables to capture global response information.
- Pre-training Procedure: The model uses 12 BERTBASE-initialized Transformer blocks and 8.3 million Twitter and Reddit conversation samples, processing each pair for recognition and generation before joint optimization.
3.1 Settings
The experiments compare the proposed model with baselines and state-of-the-art systems across three conversation datasets, using automatic and human evaluation. The evaluation covers response overlap, diversity, knowledge use, response quality, and perplexity.
- Datasets: Experiments use Persona-Chat, Daily Dialog, and DSTC7-AVSD to represent knowledge-grounded conversation, chit-chat, and conversational question answering.Persona-Chat includes persona profiles, Daily Dialog covers daily-life conversations, and DSTC7-AVSD requires answers based on dialogue context and background knowledge.
- Compared Methods: Seq2Seq provides baselines for Persona-Chat and Daily Dialog, while DSTC7-AVSD uses its provided hierarchical recurrent multimodal baseline.Compared state-of-the-art systems include LIC for Persona-Chat and transformer-based approaches.
- Reported Results: Results for Persona-Chat and Daily Dialog are summarized in Table 2, while DSTC7-AVSD results, response examples, and perplexity comparisons are reported in Tables 3–5.The table captions identify the datasets, evaluation type, response-generation examples, and best-value conventions.
- Evaluation Metrics: Automatic evaluation measures n-gram overlap, diversity, knowledge informativeness, and DSTC7-AVSD response quality against six ground-truth responses.The metrics include BLEU, Distinct-1/2, Knowledge Recall/Precision/F1, METEOR, ROUGE-L, and CIDEr.
- Evaluation Metrics: Human evaluation scores 100 dialogue contexts on fluency, coherence, informativeness, and overall quality using three crowd-sourcing workers and a 0–2 scale.Final scores are determined by majority voting, with moderate annotator agreement reported for Persona-Chat and Daily Dialog.
3.2 Experimental Results
The proposed method performs consistently better in human evaluations on Persona-Chat and Daily Dialog, while automatic metrics do not identify a single consistently superior method. On DSTC7-AVSD, response selection with an additional ranking step produces a reported breakthrough.
- Persona-Chat and Daily Dialog: Automatic evaluations on Persona-Chat and Daily Dialog show that no method consistently outperforms all others.The paper also notes weak correlation between automatic metrics and human judgments in open-domain dialogue generation.
- Persona-Chat and Daily Dialog: The proposed method performs better consistently across all human-evaluation metrics on Persona-Chat and Daily Dialog.Its fluency scores approach the upper bound, and workers favor its coherent and more informative responses.
- Ablation Study: Ablation results indicate that incorporating discrete latent variables yields remarkable improvements in dialogue generation.The comparison is between the proposed method and its version without latent variables.
- DSTC7-AVSD: On DSTC7-AVSD, the method uses an additional ranking step to select the highest-scoring candidate response.The reported results are presented in Table 3, together with an upper bound based on selecting the optimal candidate.
3.3 Discussions
The model’s ablations show consistently lower perplexity, supporting flexible attention, dialogue-specific pre-training, and discrete latent variables. Multiple latent values also produce diverse responses.
- Assigning distinct latent-variable values enables multiple diverse and appropriate responses for the same context.
- The proposed method consistently achieves lower perplexity across training-set sizes than the other pre-trained models.
- Flexible attention improves use of bidirectional context while preserving dialogue-generation processing.
- Twitter and Reddit further pre-training outperforms general-text-only pre-training because these sources are closer to human conversations.
- Discrete latent variables boost response-generation quality compared with the model without latent variables.
4 Related Work
Related work covers pre-trained language models and methods for modeling one-to-many response variation. It also notes deficiencies in directly fine-tuning BERT on small conversation datasets and approaches to latent-space control.
- Pre-trained language models are categorized by attention directionality, including uni-directional GPT-family models and bidirectional models such as BERT.
- Directly fine-tuning BERT on small conversation datasets has demonstrated performance deficiencies.
- Dialogue-generation research models one-to-many responses using latent representations, including approaches addressing posterior collapse and diversity–relevance trade-offs.
5 Conclusion
The paper introduces a dialogue-generation pre-training model with discrete latent variables and jointly trains response generation with latent recognition. Experiments across three conversation-task types report significant improvements over state-of-the-art methods, while future work targets finer latent variables and broader language coverage.
- The model uses discrete latent variables to represent one-to-many relationships in dialogue generation.
- Response generation and latent recognition are trained simultaneously on large-scale conversation datasets.
- Experiments on three publicly available conversation-dataset types report significant improvements over other state-of-the-art methods.
- Future work proposes more fine-grained latent variables, reinforcement-learning-based latent selection, and support for additional languages.
A Additional Case Analysis
Case analyses across persona, daily-life, and video-grounded dialogue show that the method produces coherent, informative, diverse, and appropriate responses. The examples also illustrate how latent variables support varied knowledge use.
- On Persona-Chat, the method generates responses that are coherent with context and expressive of background personas.
- The Persona-Chat examples associate latent variables with diverse knowledge selection and utilization.
- On Daily Dialog, the method generates more coherent and higher-quality responses than the compared approaches in the case analysis.
- On DSTC7-AVSD, the method uses video captions and summaries as background knowledge and generates a more appropriate answer than the baseline.
- The case-analysis tables cover Persona-Chat, Daily Dialog, and DSTC7-AVSD response generation.