Source-linked AI summary
Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
Tiancheng Zhao, Ran Zhao, Maxine Eskenazi
TL;DR
Neural encoder-decoder dialog models often generate dull, generic responses, motivating methods that capture the multiple discourse-level possibilities for a given context. The paper adapts CVAEs with latent variables, linguistic knowledge, and a bag-of-word loss to generate diverse responses and improve training. Its models produce appropriate, discourse-level diverse responses and show competence in discourse-level decision-making.
Problem
Encoder-decoder models for open-domain conversation often generate generic and dull responses, while a dialog context may correspond to multiple valid discourse-level responses.
Method
The paper adapts conditional variational autoencoders to dialog, adds linguistic features through Knowledge-Guided CVAE, and uses bag-of-word loss to train latent variables.
Results
The proposed models generate appropriate and discourse-level diverse responses and demonstrate competence in discourse-level decision-making.
Takeaways & Limitations
Modeling latent discourse-level factors can make neural dialog outputs more diverse, explainable, and controllable by humans.
Abstract
from arXiv · showhide
While recent neural encoder-decoder models have shown great promise in modeling open-domain conversations, they often generate dull and generic responses. Unlike past work that has focused on diversifying the output of the decoder at word-level to alleviate this problem, we present a novel framework based on conditional variational autoencoders that captures the discourse-level diversity in the encoder. Our model uses latent variables to learn a distribution over potential conversational intents and generates diverse responses using only greedy decoders. We have further developed a novel variant that is integrated with linguistic prior knowledge for better performance. Finally, the training procedure is improved by introducing a bag-of-word loss. Our proposed models have been validated to generate significantly more diverse responses than baseline approaches and exhibit competence in discourse-level decision-making.
1 Introduction
Open-domain conversations are one-to-many: the same context can support multiple valid discourse-level responses, but encoder-decoder models often produce generic and dull outputs. This paper uses latent-variable CVAE models to represent that diversity and evaluates discourse-level response generation.
- Encoder-decoder models treat dialog history as a source sequence and the next response as a target sequence trained with maximum likelihood.
- Open-domain conversation models often generate generic, dull responses instead of meaningful, specific answers.
- The paper models dialogs as a one-to-many discourse-level problem in which similar histories can have multiple valid responses tied to latent-variable configurations.
- Its CVAE introduces a latent variable that captures discourse-level variation, while responses are generated by sampling latent representations and decoding their words.
- Knowledge-Guided CVAE integrates expert knowledge for improved performance and model interpretability.
- The models are evaluated on human-human conversation data for appropriate, discourse-level diverse responses and training effectiveness.
2 Related Work
Prior work addresses dull dialog responses by enriching encoder inputs or modifying encoder-decoder architectures. This paper adapts conditional variational autoencoders, which generate diverse outputs conditioned on dialog context, while addressing latent-variable training difficulties in language generation.
- Research on dialog diversity seeks responses that are both coherent and diverse, because most models produce generic and dull outputs.
- One research line enriches encoder inputs with speaker characteristics, speaking style, topic encodings, or other context information to encourage specific responses.
- Another line modifies encoder-decoder architectures through mutual-information objectives or search-based losses to reduce generic responses and address decoding bias.
- A VAE encodes an input into a probability distribution over z and reconstructs it through a decoder using samples from z.
- A CVAE generates diverse outputs conditioned on attributes; this paper treats dialog contexts as conditional attributes for response generation.
- Applying VAE-style latent variables to natural-language generation is difficult because recurrent decoders may ignore the latent variable.
3 Proposed Models
The proposed models use conditional latent variables to represent discourse-level response diversity, extend CVAE with linguistic guidance, and address latent-variable collapse with bag-of-word loss.
- Conditional Variational Autoencoder (CVAE) for Dialog Generation: CVAE models the dialog context, response, and latent variable z, approximating p(z|c) and p(x|z,c) with neural networks.The context includes dialog history, conversational floor, and meta features; generation samples z from the prior and decodes the response.
- Conditional Variational Autoencoder (CVAE) for Dialog Generation: The CVAE architecture encodes utterances and context with recurrent networks, then initializes the response decoder from the latent variable and context.At training, z is sampled from the recognition network; at testing, it is sampled from the prior network.
- Knowledge-Guided CVAE (kgCVAE): kgCVAE incorporates linguistic features y into the decoder and predicts y′ from z and c so developers can guide the latent representation with discourse knowledge.During testing, the predicted features replace oracle features, allowing the model to generate responses conditioned on learned linguistic guidance.
- Knowledge-Guided CVAE (kgCVAE): Reconstructing linguistic features helps kgCVAE encode y-related information into z and produce interpretable high-level labels alongside word-level responses.The paper identifies dialog acts as an example of such a high-level label.
- Optimization Challenges: Because RNN decoders can ignore latent variables, the models use bag-of-word loss to force z to capture global response information.The loss predicts an unordered bag of response words in addition to the ordered response, and the authors report that it is effective against vanishing latent variables and complementary to KL annealing.
4 Experiment Setup
The experiments use the Switchboard corpus and automatically assigned dialog-act labels, with specified neural-model hyperparameters selected using validation loss.
- Dataset: The Switchboard Release 2 Corpus contains 2400 two-sided telephone conversations covering 70 discussion topics.The data were randomly split into 2316/60/62 dialogs for training, validation, and testing.
- Dialog-act labeling: Dialog-act labels were generated with a linear-kernel SVM using utterance n-grams and the contextual features of the last 3 utterances.The SVM recognized 42 dialog-act types and achieved 77.3% accuracy on held-out annotated data.
- Model configuration: The models use 200-dimensional shared word embeddings, a bidirectional utterance encoder with hidden size 300 per direction, and a context encoder with hidden size 600.The response decoder has hidden size 400, and hyperparameters were selected according to validation loss.
5 Results
The evaluation compares a baseline encoder-decoder with CVAE and kgCVAE using diversity-sensitive automatic metrics and qualitative analyses. The proposed models improve recall, while kgCVAE achieves the strongest combined precision–recall performance and learns a more informative latent space.
- Quantitative evaluation: The evaluation compares a baseline, CVAE, and kgCVAE using BLEU, bag-of-word embedding, and dialog-act-match metrics.The models generate multiple responses, with baseline diversity coming from softmax sampling and CVAE/kgCVAE diversity coming from sampling latent z followed by greedy decoding.
- Quantitative evaluation: The proposed models significantly outperform the baseline in recall across all reported metrics.This result supports broader coverage of potential responses through discourse-level diversity.
- Quantitative evaluation: kgCVAE achieves the highest precision and recall simultaneously for BLEU1-4 and A-BOW.Its predicted dialog-act label can regularize decoding toward more coherent and precise words.
- Quantitative evaluation: In higher-entropy contexts, CVAE and kgCVAE obtain significantly higher BLEU-4 recall than the baseline.CVAE has lower precision particularly in low-entropy contexts, whereas kgCVAE has higher precision than both alternatives across the full entropy range.
- Qualitative evaluation: Qualitative examples show kgCVAE covering multiple plausible dialog acts, while baseline responses remain closer to local n-gram variations.The examples also indicate that latent z captures lexical diversity in low-entropy contexts and discourse-level diversity in high-entropy contexts.
- Qualitative evaluation: The learned posterior latent space is highly correlated with dialog act and response length in the t-SNE analysis.The visualization uses test responses with the eight most frequent dialog acts, with circle size representing response length.
- Bag-of-word loss: BOW loss produces significantly lower reconstruction perplexity and larger KL cost than the compared training setups.The standard VAE collapses to near-zero KL cost, while BOW loss maintains a non-trivial KL cost without KLA; the authors therefore use BOW loss with KLA in prior experiments.
6 Conclusion and Future Work
The paper concludes that open-domain conversation is inherently one-to-many and that its models generate diverse, appropriate discourse-level responses. Future work extends the latent representation to additional linguistic phenomena and supports data-driven dialog-manager design.
- Conclusion: The proposed models show superior performance in generating diverse and appropriate responses at the discourse level.
- Future Work: Future work will apply kgCVAE beyond dialog acts to phenomena such as sentiment and named entities.
- Future Work: The model’s recognition network is intended to support a data-driven dialog manager that automatically discovers useful high-level intents.
Variational Lower Bound for kgCVAE
The variational lower-bound formulation assumes that, given linguistic feature y regarding x, the prediction of xbow depends only on z and c.
- Given linguistic feature y regarding x, the prediction of xbow is assumed to depend only on z and c.
Collection of Multiple Reference Responses
Multiple reference responses are collected by retrieving responses from training contexts similar in topic, conversation floor, and TFIDF history representation. The similarity function is designed to find diverse responses for the same dialog context.
- Multiple reference responses are collected for each test-set dialog context using information retrieval combined with a traditional machine-learning method.
- Dialog histories are represented as TFIDF-weighted bag-of-words vectors, while topic and conversation floor provide additional context features.
- The context similarity function uses topic agreement and vector similarity between dialog-history representations.
- Because similarity is based only on context and places no constraints on responses, it is suitable for finding diverse responses for the same dialog context.
- For each test context, the procedure retrieves the 10 nearest training neighbors and treats their responses as candidate references.