Source-linked AI summary
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, Joelle Pineau
TL;DR
The paper asks how to build flexible open-domain conversational systems from large dialogue corpora despite limited evaluation and training resources. It extends a hierarchical recurrent encoder-decoder with end-to-end generative dialogue modeling, then improves learning through pretrained embeddings and question-answer pretraining. The resulting hierarchical model outperforms n-gram and baseline neural models on utterance and speech-act modeling, while the authors identify evaluation and data limitations.
Problem
Open-domain dialogue systems need to generate flexible responses from large corpora, but automatic evaluation is not well established and human evaluation is expensive.
Method
The paper extends a hierarchical recurrent encoder-decoder that represents utterances and dialogue context recurrently, and bootstraps training with pretrained embeddings and a 5.5M-pair question-answer corpus.
Results
The hierarchical recurrent neural network generative model outperforms n-gram and baseline neural models, with substantial gains from bootstrapping parameters using related corpora and pretrained embeddings.
Takeaways & Limitations
MovieTriples and related-corpus pretraining support modeling long, open-domain dialogues, while the results point to the need for larger dialogue datasets.
Takeaways & Limitations
Automatic evaluation remains unresolved, and short dialogue contexts can limit topic-specific and diverse response generation.
Abstract
from arXiv · showhide
We investigate the task of building open domain, conversational dialogue systems based on large dialogue corpora using generative models. Generative models produce system responses that are autonomously generated word-by-word, opening up the possibility for realistic, flexible interactions. In support of this goal, we extend the recently proposed hierarchical recurrent encoder-decoder neural network to the dialogue domain, and demonstrate that this model is competitive with state-of-the-art neural language models and back-off n-gram models. We investigate the limitations of this and similar approaches, and show how its performance can be improved by bootstrapping the learning from a larger question-answer pair corpus and from pretrained word embeddings.
Introduction
The paper targets open-domain, non-goal-driven dialogue systems that learn end-to-end from human-human dialogue corpora rather than task-specific objectives or handcrafted state and action representations. It extends hierarchical recurrent models and uses pretraining and pretrained embeddings to improve performance on large-scale dialogue modeling.
- Open-domain dialogue systems support applications such as language learning and entertainment, and may also help train user simulators for goal-driven systems.
- Goal-driven systems remain costly and domain-limited because they often require handcrafted representations and large annotated, task-specific conversation corpora.Recent neural approaches reduce some requirements but still depend on handcrafted features or large annotated simulated-conversation corpora.
- The proposed system is an end-to-end generative probabilistic model trained offline on human-human dialogues to emulate corpus behavior rather than optimize a task-specific objective.It learns dialogue state and action representations directly from corpus examples instead of using predefined state or action spaces.
- The work extends the hierarchical recurrent encoder-decoder for dialogue and compares recurrent neural networks and n-gram models.The experiments introduce the MovieTriples dataset, based on movie scripts, for open-domain dialogue modeling.
- Performance is improved by bootstrapping from pretrained word embeddings and pretraining on a larger question-answer corpus.The paper emphasizes efficient training on large datasets and maintaining state over long conversations.
Related Work
Prior work includes retrieval and generative approaches to open-domain dialogue, especially on micro-blogging data and movie-derived conversational resources. These studies frame response generation as either retrieval or neural sequence modeling over increasingly longer dialogue contexts.
- Ritter et al. framed micro-blog response generation as translation from a post to a response, but found it harder than language translation.The difficulty was attributed to many plausible responses and limited phrase alignment between posts and responses.
- Shang et al. applied recurrent neural networks to micro-blog response generation, while Sordoni et al. extended the setting from status-reply pairs to three consecutive utterances.
- Earlier movie-script work retrieved responses from a movie-script database, and later subtitle-based information retrieval addressed out-of-domain questions.
Models
The paper models dialogue as hierarchically structured sequences, using recurrent networks to encode utterances, track dialogue context, and generate responses token by token. It extends HRED for dialogue and improves initialization through pretrained embeddings and question-answer corpus pretraining.
- A dialogue is modeled as a sequence of utterances, with each utterance represented as a sequence of tokens and speech acts.
- RNN language models represent token history with hidden states and use that context to predict the next token.
- HRED encodes each utterance into a vector, processes utterance vectors with a context RNN, and decodes the next utterance token by token.
- The context RNN represents dialogue history as a continuous state and reduces computational steps between utterances, supporting longer-range information flow.
- A bidirectional encoder is tested because the last state of a unidirectional encoder may omit information from the beginnings of long dialogue utterances.
- Word embeddings are initialized from Google News Word2Vec, while full model parameters are pretrained on approximately 5.5M SubTle question-answer pairs.
Dataset
MovieTriples is constructed from movie-script dialogue to represent three-turn interactions between two interlocutors while reducing sparsity and cross-split dependencies. Preprocessing adds dialogue-boundary tokens and normalizes vocabulary.
- MovieTriples expands and preprocesses the Movie-DiC dataset for generative dialogue modeling.
- Movie scripts provide long interactions with few participants, broad topics, and relatively few spelling mistakes and acronyms.
- Named entities and numbers are replaced with special tokens, text is lowercased, and vocabulary is limited to the 10,000 most frequent tokens plus <unk>.
- Each triple contains three turns, with speakers alternating as A, B, and A, and special tokens marking utterance endings and continued utterances.
- Movies are split into training, validation, and test sets before triples are constructed to avoid dependencies between triples from the same movie.
Experiments
The experiments compare hierarchical and non-hierarchical neural dialogue models with n-gram baselines, using perplexity and word classification error alongside generated-response analysis. Pretraining and pretrained embeddings improve performance substantially, while MAP decoding often produces generic responses despite sensible examples.
- Models and evaluation: The evaluation compares HRED variants, a standard RNN, DCGM-I, and n-gram models on dialogue triples and conditioned response prediction.Test results cover {U1, U2, U3} and U3 conditioned on U1 and U2.
- Metrics: Perplexity measures the probability of regenerating exact reference utterances, while word classification error counts incorrectly predicted words and is less fine-grained.Word classification error is more robust to unlikely words but measures only high-probability regions.
- Results: All neural models substantially outperform state-of-the-art n-gram models on both word perplexity and word classification error.Without bootstrapping, the RNN performs similarly to the more complex DCGM-I and HRED models.
- Results: Bootstrapping produces significant gains, with SubTle pretraining improving perplexity by nearly 10 points over HRED without bootstrapping.SubTle pretraining trains all model parameters, whereas Word2Vec bootstrapping trains only word embeddings.
- Results: Architectural gains are smaller than bootstrapping gains in the relatively data-limited setting, while the bidirectional structure better retains information from earlier utterances.The authors expect architectural differences to grow with more training data and longer dialogues.
- Generated responses: Beam-search MAP decoding often generates sensible but generic responses, whereas stochastic samples contain more topic-specific words and diverse dialogues.The generic-response pattern is observed for both HRED and RNN models.
Conclusion and Future Work
The study finds that hierarchical recurrent generative models outperform n-gram and baseline neural models for modeling utterances and speech acts, with improvements from external pretraining. It also identifies evaluation and scope limitations motivating more diverse metrics, larger datasets, longer contexts, and fuller dialogue modeling.
- A hierarchical recurrent neural network generative model outperformed n-gram and baseline neural network models on modeling utterances and speech acts.
- Pretrained word embeddings and pretraining on a large related non-dialogue corpus were crucial ingredients for improving performance.
- Perplexity may be insufficient for dialogue evaluation because common colloquial words can overwhelm utterances, motivating investigation of other measures.
- Future work should model full-length dialogues, additional speech acts, and longer or multimodal context such as audio and video.