Source-linked AI summary
Augmenting End-to-End Dialog Systems with Commonsense Knowledge
Tom Young, Erik Cambria, Iti Chaturvedi, Minlie Huang, Hao Zhou, Subham Biswas
TL;DR
Open-domain dialogue models must integrate commonsense knowledge to respond in engaging ways, but Seq2Seq approaches rely on the message itself. This paper adds commonsense as external memory and proposes Tri-LSTM for retrieval-based response selection; its experiments report better performance for knowledge-augmented models than knowledge-free counterparts.
Problem
Open-domain dialogue systems are expected to respond in interesting and engaging ways, while message-only models lack the recalled commonsense that can inform human responses.
Source: Introduction · p. 1Method
The paper augments end-to-end dialogue systems with commonsense external memory and uses Tri-LSTM to encode assertions for retrieval-based response selection.
Source: Introduction · p. 1, Commonsense Knowledge Retrieval · p. 3, Tri-LSTM Encoder · p. 3Results
Tri-LSTM outperforms Dual-LSTM; LSTMs outperform word embeddings, and max-pooling over response-related assertions performs better than attention over memory in this setting.
Source: Results and Analysis · p. 5Takeaways & Limitations
The experiments support using commonsense knowledge as external memory to enhance response selection in open-domain chit-chat.
Source: Results and Analysis · p. 5, Conclusion and Future Work · p. 6The reported gains are not spectacular according to Recall@k, and the work focuses on retrieval-based dialogue rather than generation.
Source: Conclusion and Future Work · p. 6
Abstract
from arXiv · showhide
Building dialog agents that can converse naturally with humans is a challenging yet intriguing problem of artificial intelligence. In open-domain human-computer conversation, where the conversational agent is expected to respond to human responses in an interesting and engaging way, commonsense knowledge has to be integrated into the model effectively. In this paper, we investigate the impact of providing commonsense knowledge about the concepts covered in the dialog. Our model represents the first attempt to integrating a large commonsense knowledge base into end-to-end conversational models. In the retrieval-based scenario, we propose the Tri-LSTM model to jointly take into account message and commonsense for selecting an appropriate response. Our experiments suggest that the knowledge-augmented models are superior to their knowledge-free counterparts in automatic evaluation.
Introduction
Data-driven dialogue models can select or generate responses from the message, while human conversation also draws on relevant recalled information. This paper investigates augmenting end-to-end dialogue systems with commonsense knowledge in external memory.
Introduction
- Seq2Seq models produce responses based solely on the message, without a memory module.The framework relies on large numbers of message-response pairs.Source: Introduction · p. 1
- Open-domain dialogue agents are expected to respond in interesting and engaging ways, requiring effective integration of commonsense knowledge.Human speakers may recall relevant concepts, personal experience, recent events, and commonsense when responding.Source: Introduction · p. 1
- The paper investigates augmenting end-to-end dialogue systems with commonsense knowledge integrated as external memory.The introduction speculates that external memory may suit the vastness of commonsense better than encoding it in model parameters.Source: Introduction · p. 1
Related Work
Prior conversational work explored response selection and generation, including external memory in limited settings. This paper distinguishes its approach by using a large heterogeneous commonsense knowledge base in open-domain retrieval, drawing on everyday relations represented in ConceptNet.
Conversational Models
- Conversational systems include retrieval methods that select from a response repository and generation methods that encode a message and generate a response.Generation is described as more natural but prone to dull or vague responses and generally requiring substantial training data.Source: Conversational Models · p. 1
- Earlier dialogue work used fact triples or local restaurant information in limited settings; this work targets a large heterogeneous commonsense base for open-domain retrieval.Prior Seq2Seq approaches incorporated topic words or domain-specific food tips, while this model shares an RNN encoding approach with Lowe et al. (2015a).Source: Conversational Models · pp. 1–2
Commonsense Knowledge
- ConceptNet captures objective facts and informal everyday relations, a feature suited to the open-domain conversations considered here.Commonsense knowledge bases can be represented as semantic networks, with concepts as nodes and relations as edges; triples are assertions.Source: Commonsense Knowledge · p. 2
Model Description
The model selects a response from candidates using message–response compatibility, then augments that scoring with commonsense assertions retrieved for message concepts. Its Tri-LSTM combines message, response, and assertion encodings, using the best-matching assertion for each response and reverting to Dual-LSTM when no assertions are available.
Task Definition
- The retrieval task selects the most compatible response from a candidate set, training on message–response–label triples with binary labels for real or randomly combined pairs.The paper focuses on retrieval-based models, leaving generation-based modeling to future work.Source: Task Definition · p. 2
Dual-LSTM Encoder
- Dual-LSTM represents the message and response as fixed-size embeddings using the last hidden states of the same LSTM.The compatibility function scores those two embeddings.Source: Dual-LSTM Encoder · p. 2
Commonsense Knowledge Retrieval
- The retrieval procedure finds message concepts by n-gram matching and adds to Ax all knowledge-base assertions associated with each matched concept.The method uses n-grams up to N=5 and chooses this approach for speed and recall.Source: Commonsense Knowledge Retrieval · p. 3
Tri-LSTM Encoder
- Tri-LSTM encodes each retrieved assertion as concept–relation–concept text and scores its match with a response; it selects the highest-scoring assertion because most retrieved assertions are irrelevant to a given response.When Ax is empty, the model reduces to Dual-LSTM.Source: Tri-LSTM Encoder · pp. 3–4
Supervised Word Embeddings
- The supervised-word-embedding baseline scores message–response pairs using bag-of-words embeddings rather than the Tri-LSTM’s recurrent encodings.The embeddings are trained directly for the scoring task.Source: Supervised Word Embeddings · p. 4
Memory Networks
- The memory-network variant uses attention over Ax to form a weighted memory representation, with attention determined by the message x.The authors argue that response y may better determine attention in this setting because knowledge retrieval already uses the message context.Source: Memory Networks · p. 5
Experiments
The experiments evaluate commonsense-augmented response selection on Twitter dialogues using ConceptNet and compare knowledge-augmented models with baselines. The reported comparisons favor Tri-LSTM over word embeddings and Dual-LSTM, while response-conditioned max-pooling performs better than memory attention in this setting.
Twitter Dialogue Dataset
- The Twitter response-selection experiments use 1.4M message-response pairs, with 1M training pairs and matched positive and random-response negative instances.Validation and test sets each contain 10K pairs; each instance is evaluated against nine random response candidates.Source: Twitter Dialogue Dataset · p. 5
ConceptNet
- ConceptNet yields an average of 2.8 matched concepts and 150 assertions per message, while 15.5% of assertions are judged false or vague.The authors say max-pooling is partly designed to alleviate this knowledge-base noise.Source: ConceptNet · p. 5
Parameter Settings
- The models use pretrained GloVe embeddings, 256-unit LSTMs, 100-dimensional word embeddings, and stochastic gradient descent with batch size 64 and learning rate 0.001.Source: Parameter Settings · p. 5
Results and Analysis
- Table 2 cases show commonsense assertions supplying clues missed by Dual-LSTM, while other examples show both models succeeding or Dual-LSTM selecting correctly without a helpful assertion.For “bonjour madame,” Tri-LSTM selects a response about improving French; in another case, both models select a shoe-color response despite Dual-LSTM lacking a relevant assertion.Source: Results and Analysis · pp. 5–6, Table 2 · p. 6
- Tri-LSTM outperforms Dual-LSTM, and word embeddings outperform memory networks, supporting commonsense integration and response-conditioned max-pooling over attention in this setting.The authors also report that LSTMs model dialogues better than word embeddings; Table 1 summarizes model evaluation.Source: Results and Analysis · p. 5, Table 1 · p. 6
Conclusion and Future Work
The paper presents commonsense knowledge as external memory for open-domain chit-chat response selection, with LSTMs encoding assertions to enhance selection. The authors describe the Recall@k gains as not spectacular and propose broader knowledge coverage, improved utilization, and longer conversational memory as future directions.
Conclusion and Future Work
- The paper uses commonsense knowledge as external memory and LSTMs to encode assertions for open-domain chit-chat response selection.Source: Conclusion and Future Work · p. 6
- The authors describe the method’s Recall@k gains as not spectacular, while presenting large heterogeneous knowledge bases as a promising direction for conversational models.They note that such knowledge could potentially condition response-generation decoders toward more interesting and relevant responses.Source: Conclusion and Future Work · p. 6
- Future work includes expanding knowledge coverage, improving on max-pooling, using memory across multiple message-response pairs, and integrating affective knowledge.Source: Conclusion and Future Work · p. 6