Source-linked AI summary
Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation
Iulian Vlad Serban, Tim Klinger, Gerald Tesauro, Kartik Talamadupula, Bowen Zhou, Yoshua Bengio, Aaron Courville
TL;DR
Dialogue response generators often fail to capture useful high-level context. The paper introduces MrRNN, which jointly models coarse and natural-language sequences hierarchically, and reports dramatic Ubuntu improvements and more relevant, on-topic Twitter responses. Twitter content cannot be redistributed, so only tweet IDs are public.
Problem
End-to-end dialogue models have struggled to generate meaningful context-aware responses and learn useful high-level dialogue abstractions.
Method
MrRNN jointly models parallel coarse-token and natural-language sequences, factorizing their probability hierarchically and optimizing the joint log-likelihood.
Results
MrRNN demonstrates dramatic improvements on Ubuntu and appears to generate more relevant and on-topic responses on Twitter.
Takeaways & Limitations
Explicitly representing multiple abstraction levels and jointly optimizing generation helps model high-level dialogue structure across Ubuntu and Twitter tasks.
Takeaways & Limitations
Twitter content cannot be redistributed; only tweet IDs are publicly available.
Abstract
from arXiv · showhide
We introduce the multiresolution recurrent neural network, which extends the sequence-to-sequence framework to model natural language generation as two parallel discrete stochastic processes: a sequence of high-level coarse tokens, and a sequence of natural language tokens. There are many ways to estimate or learn the high-level coarse tokens, but we argue that a simple extraction procedure is sufficient to capture a wealth of high-level discourse semantics. Such procedure allows training the multiresolution recurrent neural network by maximizing the exact joint log-likelihood over both sequences. In contrast to the standard log- likelihood objective w.r.t. natural language tokens (word perplexity), optimizing the joint log-likelihood biases the model towards modeling high-level abstractions. We apply the proposed model to the task of dialogue response generation in two challenging domains: the Ubuntu technical support domain, and Twitter conversations. On Ubuntu, the model outperforms competing approaches by a substantial margin, achieving state-of-the-art results according to both automatic evaluation metrics and a human evaluation study. On Twitter, the model appears to generate more relevant and on-topic responses according to automatic evaluation metrics. Finally, our experiments demonstrate that the proposed model is more adept at overcoming the sparsity of natural language and is better able to capture long-term structure.
1 Introduction
The paper extends deterministic sequence-to-sequence modeling to multiple parallel stochastic sequences, with high-level sequences guiding natural-language generation. It targets dialogue systems whose end-to-end models struggle to produce contextually meaningful responses.
- Research direction: MrRNNs generalize sequence-to-sequence modeling to multiple parallel sequences, each with its own stochastic process.The model factorizes their joint probability and imposes a hierarchy in which abstract information flows to lower-level natural-language sequences.
- Training objective: Training maximizes the joint log-likelihood over all observed sequences rather than the likelihood of a single output sequence.This objective biases the model toward representing high-level abstractions.
- Motivation: End-to-end dialogue models have struggled to generate meaningful responses that account for dialogue context, suggesting inadequate high-level dialogue abstractions.The paper applies MrRNNs to Ubuntu technical support and Twitter conversations to address this problem.
2 Model Architecture
MrRNN models parallel coarse and natural-language token sequences with a hierarchical probabilistic factorization. A coarse sub-model supplies high-level information to the natural-language sub-model, while generation uses approximate MAP inference with beam search.
- RNN language model: An RNN language model factorizes sequence probability into conditional next-token probabilities and typically learns parameters by maximizing training-set log-likelihood.Its hidden state is updated recurrently, and the output distribution is commonly parameterized with a softmax RNN.
- Multiresolution representation: MrRNN jointly models utterance-level coarse tokens and word tokens as parallel hierarchical sequences.In the experiments, w denotes utterance words and z denotes coarse tokens such as nouns from the same utterance.
- Probabilistic architecture: The joint distribution factorizes into a coarse sub-model conditioned on prior coarse sequences and a natural-language sub-model conditioned on prior words and coarse sequences.This structure transmits high-level information from z to the generation of w.
- Neural implementation: The coarse sub-model and natural-language sub-model are parameterized using HRED-style recurrent modules.A coarse-prediction encoder summarizes previously generated coarse tokens and feeds that representation to the natural-language decoder.
- Inference: At test time, MrRNN generates responses by approximating the maximum a posteriori estimate and using beam search for each constituent sequence.The probabilistic factorization first supports coarse-sequence generation and then conditions natural-language generation on it.
3 Tasks
The paper evaluates conditional dialogue response generation in goal-driven Ubuntu technical support and non-goal-driven Twitter conversations. These domains differ in structure, with Ubuntu centered on technical problems and Twitter being noisier and less topic-focused.
- Task definition: The dialogue task generates responses conditioned on a context containing one or more utterances.The evaluation covers both goal-driven and non-goal-driven dialogue settings.
- Ubuntu technical support: Ubuntu contains about 0.5 million technical-support dialogues extracted from the Ubuntu IRC channel.Users describe technical problems, after which other users attempt to resolve them.
- Twitter conversations: Twitter provides a non-goal-driven dialogue corpus whose conversations are often noisier and not centered on a single topic.The dataset was split into training, validation, and test sets containing 749,060, 93,633, and 10,000 dialogues, respectively.
4 Coarse Sequence Representations
The paper extracts coarse sequences from utterances to represent high-level discourse information alongside natural-language tokens. It uses noun representations generally and activity-entity representations specifically for Ubuntu.
- Two procedures extract coarse sequence representations from dialogue utterances.
- Noun Representation: Noun representations extract nonrepeated nouns after POS tagging and stop-word removal, using separate predefined stop-word sets for Ubuntu and Twitter.
- Activity-Entity Representation: Activity-entity representations target Ubuntu technical support by retaining verbs from an activity set and technical entities linked to problem-solving discourse.
- Activity-Entity Representation: Ubuntu activity-entity sequences also encode missing activities, terminal-command detection, and utterance tense.
- Both procedures operate at the utterance level, aligning each coarse sequence with one natural-language utterance and coarse tokens with corresponding language tokens.
5 Experiments
Experiments compare MrRNN variants with recurrent and latent-variable baselines on Ubuntu and Twitter using automatic metrics, human judgments, and qualitative examples. MrRNN shows strong Ubuntu gains and more topic-oriented Twitter responses.
- Models are trained with joint log-likelihood over coarse and natural-language sequences, using Adam, early stopping, gradient clipping, and beam search.
- Baseline Models: The comparison includes RNNLM, HRED, VHRED, and an Ubuntu HRED variant conditioned on past activity-entity features.
- Ubuntu: Ubuntu evaluation combines activity, entity, tense, and command precision, recall, F1, and accuracy with human fluency and relevancy scores.
- Ubuntu: 2x−3x higher entity scores are achieved by the noun-representation MrRNN, alongside substantially higher human fluency and relevancy ratings than other models.
- Ubuntu: The activity-representation MrRNN achieves 2x−3x higher activity scores and nearly 2x higher entity scores than baselines, with substantially higher human fluency.
- Ubuntu: MrRNN responses are described as more coherent, topic-oriented, and goal-oriented than competing Ubuntu responses, especially with activity-entity representations.
- Twitter: Twitter noun-representation MrRNN leads on noun precision, recall, F1, and tense accuracy, while matching VHRED on embedding-based metrics.
6 Related Work
Related work spans hierarchical, latent-variable, and dialogue-specific architectures. MrRNN differs by automatically modeling sequences of high-level tokens rather than manually annotated or continuous high-level representations.
- Ji et al. jointly model natural language and discourse phenomena, but use one manually annotated discrete class per sentence rather than an automatically extracted token sequence.
- Twitter evaluation includes noun precision, recall, F1, tense accuracy, and embedding-based metrics, which are reported in Table 3.
- Variational recurrent models learn high- and low-level generative processes jointly while typically assuming continuous Gaussian high-level latent variables.
- Dialogue-specific architectures may require domain-specific hand-crafted, human-labelled dialogue-state representations and multiple components trained with different objectives.
7 Discussion
MrRNN models sequential data at multiple abstraction levels and improves dialogue response generation across Ubuntu and Twitter evaluations.
- MrRNN jointly models coarse and natural-language sequences to generate more fluent, relevant, coherent, and goal-oriented responses.The low-level abstraction supports fluency, while the high-level abstraction supplies semantic structure for coherence and relevance.
- On Ubuntu, MrRNN demonstrates dramatic improvements compared to competing models in human and automatic evaluations.
- On Twitter, MrRNN appears to generate more relevant and on-topic responses according to automatic evaluation metrics.
8 Task Details
The experiments use Ubuntu technical-support dialogues and Twitter conversations as distinct evaluation domains.
- The Ubuntu experiments use the Ubuntu Dialogue Corpus v2.0.
- The Twitter experiments preprocess conversations with the Moses tokenizer.
Nouns
The noun-based coarse representation extracts ordered discourse tokens intended to capture topic structure, with preprocessing and language-model statistics reported for Ubuntu and Twitter.
- Noun extraction uses the hypothesis that dialogue topics are characterized by nouns and evolve as conversations progress.
- The procedure applies POS tagging, removes non-nouns, non-alphabetic words, stop words, and repeated tokens while preserving order.It uses 84 Ubuntu stop words and 795 Twitter stop words.
- A no_nouns token prevents empty coarse sequences, while tense tokens encode detected past, present, and future tenses.
- Coarse tokens align one-to-many with natural-language tokens, apart from special-token exceptions.
- 25 coarse tokens occur per Twitter training dialogue on average, versus 36 for Ubuntu.
- Twitter preprocessing replaces unknown tokens, numbers, URLs, and heart emoticons with standardized placeholders; Ubuntu commands and entities are similarly replaced.
- Noun representations show a substantial unigram–bigram bits-per-word difference, suggesting significant correlation among nouns.
Activity-Entity Pairs
The activity-entity representation encodes Ubuntu technical assistance through activities, technical entities, commands, and tense, with extracted sequences evaluated and modeled.
- The activity-entity procedure targets Ubuntu’s domain-specific problem-solving dialogues, which commonly center on activities and technical entities.
- It builds activity and technical-entity dictionaries, including 192 manually inspected activity verbs with synonyms, conjugations, and misspelled variants.
- The extraction pipeline identifies entities, extracts activities, removes repeated items, preserves order, and marks whether commands occur.
- Tense information is appended to the coarse sequence, while none_activity and no_cmd tokens represent missing activities and commands.
- The models use maximum log-likelihood for noun representations, while scripts for representation generation and activity-entity evaluation are available online.
- The Ubuntu experiments report activity-entity sequence examples and train models with closely matched hyperparameters across datasets.
- Manual inspection finds technical entities identified with very high accuracy and activities capturing the main intended action in most utterances.
- Beam-search candidates are scored by length-normalized token log-likelihood, and LSMT remains worse than HRED across metrics except command accuracy.