Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

• Open-domain dialogue systems support applications such as language learning and entertainment, and may also help train user simulators for goal-driven systems.Source: Introduction · p. 1
• Goal-driven systems remain costly and domain-limited because they often require handcrafted representations and large annotated, task-specific conversation corpora.Recent neural approaches reduce some requirements but still depend on handcrafted features or large annotated simulated-conversation corpora.Source: Introduction · p. 1
• The proposed system is an end-to-end generative probabilistic model trained offline on human-human dialogues to emulate corpus behavior rather than optimize a task-specific objective.It learns dialogue state and action representations directly from corpus examples instead of using predefined state or action spaces.Source: Introduction · pp. 1–2
• The work extends the hierarchical recurrent encoder-decoder for dialogue and compares recurrent neural networks and n-gram models.The experiments introduce the MovieTriples dataset, based on movie scripts, for open-domain dialogue modeling.Source: Introduction · p. 2
• Performance is improved by bootstrapping from pretrained word embeddings and pretraining on a larger question-answer corpus.The paper emphasizes efficient training on large datasets and maintaining state over long conversations.Source: Introduction · p. 2

• Ritter et al. framed micro-blog response generation as translation from a post to a response, but found it harder than language translation.The difficulty was attributed to many plausible responses and limited phrase alignment between posts and responses.Source: Related Work · p. 2
• Shang et al. applied recurrent neural networks to micro-blog response generation, while Sordoni et al. extended the setting from status-reply pairs to three consecutive utterances.Source: Related Work · p. 2
• Earlier movie-script work retrieved responses from a movie-script database, and later subtitle-based information retrieval addressed out-of-domain questions.Source: Related Work · p. 2

• A dialogue is modeled as a sequence of utterances, with each utterance represented as a sequence of tokens and speech acts.Source: Models · p. 2
• RNN language models represent token history with hidden states and use that context to predict the next token.Source: A recurrent neural network · p. 2
• HRED encodes each utterance into a vector, processes utterance vectors with a context RNN, and decodes the next utterance token by token.Source: Hierarchical Recurrent Encoder-Decoder · p. 3, Figure 1 · p. 3
• The context RNN represents dialogue history as a continuous state and reduces computational steps between utterances, supporting longer-range information flow.Source: Hierarchical Recurrent Encoder-Decoder · p. 3, Figure 1 · p. 3
• A bidirectional encoder is tested because the last state of a unidirectional encoder may omit information from the beginnings of long dialogue utterances.Source: Bidirectional HRED · p. 4
• Word embeddings are initialized from Google News Word2Vec, while full model parameters are pretrained on approximately 5.5M SubTle question-answer pairs.Source: Bootstrapping from Word Embeddings and… · p. 4

• MovieTriples expands and preprocesses the Movie-DiC dataset for generative dialogue modeling.Source: Dataset · p. 4
• Movie scripts provide long interactions with few participants, broad topics, and relatively few spelling mistakes and acronyms.Source: Dataset · p. 4
• Named entities and numbers are replaced with special tokens, text is lowercased, and vocabulary is limited to the 10,000 most frequent tokens plus <unk>.Source: Dataset · p. 4
• Each triple contains three turns, with speakers alternating as A, B, and A, and special tokens marking utterance endings and continued utterances.Source: Dataset · p. 4
• Movies are split into training, validation, and test sets before triples are constructed to avoid dependencies between triples from the same movie.Source: Dataset · p. 4

Models and evaluation

• The evaluation compares HRED variants, a standard RNN, DCGM-I, and n-gram models on dialogue triples and conditioned response prediction.Test results cover {U1, U2, U3} and U3 conditioned on U1 and U2.Source: Experiments · p. 4, Table 2 · p. 6

Metrics

• Perplexity measures the probability of regenerating exact reference utterances, while word classification error counts incorrectly predicted words and is less fine-grained.Word classification error is more robust to unlikely words but measures only high-probability regions.Source: Evaluation Metrics · p. 5

Results

• All neural models substantially outperform state-of-the-art n-gram models on both word perplexity and word classification error.Without bootstrapping, the RNN performs similarly to the more complex DCGM-I and HRED models.Source: Empirical Results · p. 6
• Bootstrapping produces significant gains, with SubTle pretraining improving perplexity by nearly 10 points over HRED without bootstrapping.SubTle pretraining trains all model parameters, whereas Word2Vec bootstrapping trains only word embeddings.Source: Training Procedure · p. 5, Empirical Results · p. 6
• Architectural gains are smaller than bootstrapping gains in the relatively data-limited setting, while the bidirectional structure better retains information from earlier utterances.The authors expect architectural differences to grow with more training data and longer dialogues.Source: Empirical Results · p. 6

Generated responses

• Beam-search MAP decoding often generates sensible but generic responses, whereas stochastic samples contain more topic-specific words and diverse dialogues.The generic-response pattern is observed for both HRED and RNN models.Source: MAP Outputs · pp. 6–7

• A hierarchical recurrent neural network generative model outperformed n-gram and baseline neural network models on modeling utterances and speech acts.Source: Conclusion and Future Work · p. 7
• Pretrained word embeddings and pretraining on a large related non-dialogue corpus were crucial ingredients for improving performance.Source: Conclusion and Future Work · p. 7
• Perplexity may be insufficient for dialogue evaluation because common colloquial words can overwhelm utterances, motivating investigation of other measures.Source: Conclusion and Future Work · p. 7
• Future work should model full-length dialogues, additional speech acts, and longer or multimodal context such as audio and video.Source: Conclusion and Future Work · p. 7