Source-linked AI summary
A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, Bill Dolan
TL;DR
Open-domain response generation must incorporate previous dialog utterances despite sparsity in classic statistical models. The paper addresses this with end-to-end neural dynamic-context models that encode context for response generation. These models consistently outperform context-sensitive and context-independent Machine Translation and Information Retrieval baselines, with relative BLEU improvements up to 11% for MT and 24% for IR.
Problem
Classic statistical models struggle to integrate contextual information for response generation, while previous utterances are important for context-sensitive dialog.
Method
The paper uses neural dynamic-context generative models that encode context and message representations to condition a recurrent language model’s response generation.
Results
Up to 11% relative improvement in BLEU in the MT setting and 24% in the IR setting was achieved over context-independent and context-sensitive baselines.
Takeaways & Limitations
The context-sensitive models consistently outperform both context-sensitive and context-independent baselines while using a minimal number of features.
Takeaways & Limitations
The models lack mechanisms for representing agent intent and maintaining consistency with sentiment polarity, while longer contexts can produce off-topic or incoherent responses.
Abstract
from arXiv · showhide
We present a novel response generation system that can be trained end to end on large quantities of unstructured Twitter conversations. A neural network architecture is used to address sparsity issues that arise when integrating contextual information into classic statistical models, allowing the system to take into account previous dialog utterances. Our dynamic-context generative models show consistent gains over both context-sensitive and non-context-sensitive Machine Translation and Information Retrieval baselines.
1 Introduction
Open-domain conversational response generation became more feasible with large social-media conversation datasets, but generating replies sensitive to previous utterances remained difficult. The paper proposes embedding-based neural models trained end to end to address this challenge.
- Twitter and Reddit provide vast conversational exchanges that enable data-driven models for open-domain response generation.
- Context-sensitive response generation must use previous utterances to keep conversations active and engaging.The example “good luck” is motivated by an earlier reference to “your game.”
- Traditional machine-translation approaches struggle to capture conversational context because phrase pairs do not share statistical weights despite semantic commonality.
- The proposed models use continuous word and phrase representations to encode semantic and syntactic similarity and model transitions between consecutive utterances.
- The paper introduces a context-sensitive, data-driven neural architecture trainable end to end on massive social-media data.It also introduces a multi-reference extraction technique for automated evaluation.
2 Related Work
The paper extends data-driven response generation toward jointly modeling dialog context and response production. It uses learned continuous context representations within conditional recurrent language models rather than relying on separately pretrained context models.
- The work extends earlier Twitter response generation by exploiting larger conversational context.
- Its holistic approach is presented as an initial attempt to accomplish dialog management and response generation jointly.
- The framework uses a completely latent network model directly optimized for end-to-end performance.
- The models extend recurrent neural network language models by encoding past utterances in a learned continuous context vector that helps generate responses.
- Unlike approaches using pretrained topic models, the paper learns context encodings jointly with the conditional recurrent language model.The encodings can assign discriminative power to stop words in contentful dialog utterances.
3 Recurrent Language Model
The recurrent language model generates sentences by processing tokens sequentially, maintaining a hidden history and predicting each next word. Its recurrent architecture uses learned embeddings, recurrent state transitions, and an output probability distribution.
- An RLM is a generative model that estimates sentence probabilities from sequences of tokens.
- The architecture uses input, recurrent, and output weight matrices, with input rows providing word embeddings.
- The recurrent matrix maintains the processed subsequence history, while the output matrix projects the hidden state to vocabulary scores for next-word prediction.
- The forward recurrence applies an elementwise logistic sigmoid, starts from a zero hidden state, and uses softmax to obtain the next-word distribution.
- Training minimizes the negative log-likelihood of the training sentence and uses back-propagation through time to accumulate gradients across multiple steps.
- Figure 2 contrasts the RLM’s compact architecture with its unrolled form across two time steps.
4 Context-Sensitive Models
The paper defines context-sensitive response models that encode prior utterances and condition an RLM decoder on those representations. RLMT concatenates utterances, while DCGM-I and DCGM-II use fixed-length context encodings, with DCGM-II preserving context-message order.
- The models estimate response generation conditioned on a sequence of past exchanges c and a message m.
- RLMT: RLMT concatenates c, m, and r into one sentence and computes response likelihood from the hidden state after processing c and m.
- RLMT: RLMT is difficult to model when multiple utterances make the concatenated sentence long and create long-range dependencies.
- DCGM-I: DCGM-I encodes the combined bag-of-words representation of c and m into a fixed-length vector that biases the decoder RLM.
- DCGM-I: DCGM-I keeps the context representation constant across response generation, encouraging a general representation and helping the decoder retain context information.
- DCGM-II: DCGM-II concatenates separate representations of c and m before the feed-forward encoder to preserve order information and address DCGM-I’s inability to distinguish them.
5 Experimental Setting
The experiments use Twitter context-message-response triples, multi-reference evaluation, and log-linear systems combining MT, IR, exact-match, and neural features. Proposed models are trained on a 4M subset and evaluated by rescoring candidate responses from MT and IR systems.
- Dataset: 127M Twitter context-message-response triples were mined from June through August 2012, retaining triples with same-user context and response and a frequent bigram.
- Automatic Evaluation: The evaluation uses BLEU and METEOR, supplemented by targeted human pairwise comparisons because reasonable responses can be diverse.
- Automatic Evaluation: The multi-reference procedure retrieves 15 candidate triples using message-response similarity, retains responses rated at least 4 out of 5, and yields 3.58 references per example on average.
- Feature Sets: The systems include phrase-based MT features, an IR feature, context-and-message exact-match features, and neural likelihood features from context-sensitive RLMs.
- Training: The proposed neural models are trained on a 4M subset with a 50K-word vocabulary and Noise-Contrastive Estimation to avoid repeated summations over the vocabulary.
- Evaluation Procedure: Models are evaluated by rescoring n-best candidate responses from phrase-based MT and IR systems, followed by MERT estimation of log-linear feature weights.
6 Results
Across automatic and human evaluations, context-sensitive neural models outperform baseline systems, while combining semantic DCGM representations with exact CMM matches yields further gains. The experiments also identify stronger contextual retention and limitations involving response consistency and context length.
- 6.1 Lower and Upper Bounds: The ranking experiments establish unambiguous separation between HUMAN, phrase-based MT, and random-response bounds.Table 2 uses BLEU scores to define expected upper and lower bounds for the task.
- 6.2 BLEU and METEOR: Phrase-based MT outperforms IR, CMM features improve both baselines, and DCGM models deliver more consistent gains than RLMT.These patterns hold across BLEU and METEOR evaluations on MT and IR n-best lists.
- 6.2 BLEU and METEOR: 0.38 BLEU points improve MT+CMM over baseline MT, while IR+CMM gains 1.8 BLEU points and 1.5 METEOR points over IR.MT+CMM uses 17 features and achieves a 9.5% relative BLEU improvement; IR+CMM uses 10 features.
- 6.2 BLEU and METEOR: DCGM-{I-II}+CMM systems gain up to 0.48 BLEU over MT+CMM and up to 0.88 BLEU over the Ritter et al. model.METEOR improvements align with BLEU improvements for both MT and IR lists.
- 6.3 Human Evaluation: Human preferences favor context-sensitive systems, with gains up to approximately 5.3% versus IR and about 3.1% versus MT for DCGM-II+CMM.DCGM systems also outperform non-DCGM context-sensitive systems by 1.5% on MT and 2.3% on IR.
- 6.4 Discussion: Generated responses average 8.95 tokens versus 11.5 for human tuning responses and are often plausible, but longer outputs can conflict with context or become incoherent.The model lacks mechanisms for agent intent and sentiment consistency, while longer contexts may encourage off-topic responses.
7 Conclusion
The paper formulates a data-driven neural architecture that conditions response generation on past dialog utterances and evaluates it with automated multi-reference methods. Its context-sensitive models outperform context-independent and context-sensitive baselines, while future work targets richer neural models and improved evaluation metrics.
- The proposed architecture generates responses conditioned on past dialog utterances, making the system context-sensitive and data-driven.It is trained from social media conversations.
- The models consistently outperform context-independent and context-sensitive baselines by up to 11% relative BLEU improvement in MT and 24% in IR.These gains use a minimal number of features.
- Future work should examine more complex models that represent word order and improve automated evaluation metrics.The paper also identifies direct neural-network generation as a promising next step.