Source-linked AI summary
Learning to Select Knowledge for Response Generation in Dialog Systems
Rongzhong Lian, Min Xie, Fan Wang, Jinhua Peng, Hua Wu
TL;DR
The paper addresses the difficulty of selecting appropriate external knowledge when dialogue models must generate informative responses. It separates response-conditioned posterior knowledge selection from an utterance-only prior and trains the prior to approximate the posterior. The model outperforms existing baselines in automatic and human evaluations, with improved knowledge incorporation and response quality.
Problem
Dialogue models can generate uninformative responses, while selecting knowledge from utterances alone may fail when multiple knowledge items fit the same utterance.
Method
The model uses a posterior distribution inferred from utterances and responses during training, then minimizes KL divergence so an utterance-only prior can approximate it during inference.
Results
The model significantly outperforms existing models in comprehensive automatic and human evaluations by incorporating knowledge more properly and generating appropriate, informative responses.
Takeaways & Limitations
Approximating response-informed posterior knowledge selection with an utterance-only prior supports appropriate knowledge selection when responses are unavailable.
Abstract
from arXiv · showhide
End-to-end neural models for intelligent dialogue systems suffer from the problem of generating uninformative responses. Various methods were proposed to generate more informative responses by leveraging external knowledge. However, few previous work has focused on selecting appropriate knowledge in the learning process. The inappropriate selection of knowledge could prohibit the model from learning to make full use of the knowledge. Motivated by this, we propose an end-to-end neural model which employs a novel knowledge selection mechanism where both prior and posterior distributions over knowledge are used to facilitate knowledge selection. Specifically, a posterior distribution over knowledge is inferred from both utterances and responses, and it ensures the appropriate selection of knowledge during the training process. Meanwhile, a prior distribution, which is inferred from utterances only, is used to approximate the posterior distribution so that appropriate knowledge can be selected even without responses during the inference process. Compared with the previous work, our model can better incorporate appropriate knowledge in response generation. Experiments on both automatic and human evaluation verify the superiority of our model over previous baselines.
1 Introduction
Knowledge-grounded dialogue systems can improve informativeness, but selecting knowledge from utterances alone may choose information mismatched with the response. The paper proposes separating posterior and prior knowledge distributions so training can use responses while inference can operate without them.
- Seq2Seq dialogue models often generate fluent but uninformative responses such as “I don’t know” and “That’s cool”.
- External knowledge, including commonsense, personal profiles, and conversation-related information, has been used to improve informative response generation.
- Semantic similarity between utterances and knowledge provides a prior distribution, but multiple relevant knowledge items can support diverse responses.
- Selecting knowledge without response information can choose knowledge irrelevant to the true response, hindering proper knowledge use during training.
- The proposed model uses a response-conditioned posterior during training and an utterance-only prior during inference, minimizing their KL divergence.
- Comprehensive experiments report significant outperformance over existing models through more appropriate knowledge incorporation and more informative responses.
2 Model
The model selects knowledge through prior and posterior distributions, then uses the selected knowledge with encoder context to generate responses. Training aligns the prior with the posterior, while decoding combines knowledge through hard or soft mechanisms and optimizes multiple losses.
- Architecture: The model encodes the utterance, each knowledge item, and the response when available before knowledge selection and response generation.The utterance and knowledge encoders use bidirectional RNNs; response encoding is available to the posterior module.
- Knowledge Manager: The knowledge manager contains prior and posterior modules that select knowledge using the utterance alone or the utterance and response, respectively.The selected knowledge vector and attention-based context are passed to the decoder.
- Knowledge Manager: The prior distribution measures utterance–knowledge association, whereas the posterior uses both utterance and response to capture knowledge relevant to the true response.Because multiple knowledge items may be relevant to an utterance, prior-only selection is difficult during training.
- Knowledge Manager: During training, knowledge is sampled from the posterior; during inference, it is sampled from the prior because the response is unavailable.Gumbel-Softmax re-parametrization enables backpropagation through the sampling process.
- Knowledge Manager: KL divergence trains the prior to approximate the posterior, allowing prior-based knowledge selection when responses are unknown at inference.The model treats the posterior distribution as guidance for learning the prior distribution.
- Decoder and Loss Function: The decoder offers hard concatenation or hierarchical gated fusion, while NLL and BOW losses complement KL divergence during training.The hard decoder forces selected knowledge into decoding; the fusion unit allows flexible knowledge incorporation.
3 Experiments
Experiments evaluate PostKS on Persona-chat and Wizard-of-Wikipedia against knowledge-free and knowledge-grounded baselines. Across automatic, human, case-study, and transfer evaluations, the model selects and incorporates knowledge more effectively.
- Datasets and setup: Experiments use Persona-chat and Wizard-of-Wikipedia, where persona profiles or conversation-related knowledge support response generation.Persona-chat contains 151,157 dialogue turns; Wizard-of-Wikipedia includes train, validation, and test splits with seen and unseen topics.
- Models and baselines: PostKS is evaluated in concatenation and fusion variants against Seq2Seq and memory-network baselines.PostKS(concat) uses a hard GRU decoder, while PostKS(fusion) uses soft HGFU-based knowledge incorporation.
- Automatic evaluation: PostKS models significantly outperform all baselines, achieving the highest scores on most automatic metrics.Compared with Seq2Seq on Persona-chat, Distinct-1/2 rises from 0.026/0.074 to 0.048/0.126 for PostKS(concat).
- Automatic evaluation: PostKS(fusion) obtains higher knowledge R/P/F1 than all MemNet-based models on Wizard-of-Wikipedia and usually slightly exceeds PostKS(concat).The results indicate stronger appropriate-knowledge selection and more effective incorporation into generated responses.
- Human evaluation: Human ratings show that both PostKS variants are remarkably better than existing baselines across the evaluated datasets.Three annotators rated 300 responses per model per dataset on a 0–2 scale; Fleiss’ kappa was 0.48 and 0.41.
- Case study and further evaluation: Case studies show PostKS avoids bland or irrelevant responses by using correct knowledge, while PostKS(fusion) can support more evolving conversations.On the music example, PostKS mentions rock music and can ask about a favorite band; the transfer to LIC produced a threefold improvement in knowledge R/P/F1 on Persona-chat.
4 Related Work
Related work improves dialogue informativeness through diversity techniques and external knowledge, but commonly selects knowledge from conversation history alone. The paper distinguishes this prior-only approach from posterior knowledge selection and from CVAE’s discourse-diversity objective.
- Response generation: Earlier work targeted generic Seq2Seq responses through diversity promotion and unknown-word handling, but lacked external information.The generic-response problem therefore remained in these approaches.
- Knowledge incorporation: Knowledge-grounded methods incorporate texts, memory networks, diffusion, commonsense bases, and domain-specific knowledge bases into neural dialogue models.These approaches use external information to improve response generation.
- Knowledge selection: Most existing knowledge-grounded models condition knowledge on conversation history alone, treating this as a prior distribution.Compared with posterior knowledge that also considers the knowledge used in true responses, the prior has larger variance.
- Relation to CVAE: The paper differs from CVAE by addressing knowledge selection in knowledge-grounded conversations rather than discourse-level diversity.CVAE learns a distribution in latent space, whereas this work focuses on selecting conversational knowledge.
5 Conclusion
The paper presents a knowledge selection mechanism using both prior and posterior distributions, and reports effectiveness across automatic and human evaluations.
- The model uses both prior and posterior distributions over knowledge to facilitate knowledge selection.It approximates the posterior with the prior for inference, when responses are unavailable.
- Approximating the posterior with the prior enables appropriate response generation during inference.
- Extensive automatic and human evaluations demonstrate the model’s effectiveness and usefulness.