Source-linked AI summary
Retrieve and Refine: Improved Sequence Generation Models For Dialogue
Jason Weston, Emily Dinan, Alexander H. Miller
TL;DR
Dialogue generators can be generic and unengaging, whereas retrieval models may be erroneous or unable to adapt beyond their retrieval set. The paper combines retrieval with sequence generation by refining a retrieved response as additional context, and reports more human-engaging conversations on ConvAI2.
Problem
Dialogue generation is challenged by generic, unengaging generator responses and retrieval responses that can be erroneous or unavailable for the specific context.
Method
RetrieveNRef concatenates a retrieved response with dialogue input and uses a sequence generator to refine it, with variants addressing retrieval reliance and copy errors.
Results
RetNRef++ has human-like word-use statistics and provides more engaging conversations according to human judgments.
Takeaways & Limitations
Retrieval can improve dialogue generation while allowing the model to generate content that a standard retrieval model cannot.
Takeaways & Limitations
Future work should replace the heuristics and concatenation-based integration with more sophisticated retrieval-generation approaches, including independent input treatment or joint training.
Abstract
from arXiv · showhide
Sequence generation models for dialogue are known to have several problems: they tend to produce short, generic sentences that are uninformative and unengaging. Retrieval models on the other hand can surface interesting responses, but are restricted to the given retrieval set leading to erroneous replies that cannot be tuned to the specific context. In this work we develop a model that combines the two approaches to avoid both their deficiencies: first retrieve a response and then refine it -- the final sequence generator treating the retrieval as additional context. We show on the recent CONVAI2 challenge task our approach produces responses superior to both standard retrieval and generation models in human evaluations.
1 Introduction
Dialogue generation remains difficult because many valid replies make context underconstrained: generators often produce generic responses, while retrieval systems can return engaging but contextually erroneous replies.
- Dialogue contexts permit many valid interpretations, making strong performance harder than in constrained tasks such as translation and summarization.
- Sequence generation models often produce short, generic “I don’t know” responses that are uninformative and unengaging.
- Retrieval models can avoid generic replies but may produce completely erroneous responses when the retrieval set lacks a valid context-specific reply.
- The paper proposes Retrieve and Refine to combine the advantages of retrieval and generation while avoiding both disadvantages.
2 Retrieve and Refine
RetrieveNRef concatenates a retrieved utterance with dialogue history and uses a generator to refine it. The paper also introduces variants that increase reliance on retrieval and correct near-copy errors, while noting unresolved design alternatives.
- RetrieveNRef concatenates a retrieval model’s output with the generator’s usual input, then trains a standard generative model to produce the response.
- The implementation uses a two-layer attention-based LSTM Seq2Seq generator and a Key-Value Memory Network retriever, whose top utterance is prepended with a separator token.
- Training reranks each dialogue turn’s top 100 retrieval predictions by embedding-space similarity to the label before appending the selected utterance to Seq2Seq inputs.
- RetrieveNRefine+ clips initial ConvAI2 profile sentences so the generator relies more strongly on retrieval, which retains those profile sentences.
- RetrieveNRefine++ exactly copies the retrieval when generated text exceeds 60% word overlap; otherwise, it leaves the generation unchanged.
- The approach leaves alternative architectural changes, explicit copying, and BPE tokenization for future work.
3 Experiments
Experiments on ConvAI2 evaluate automatic statistics, retrieval overlap, human judgments, and paired comparisons of retrieval-refinement variants against Seq2Seq and retrieval baselines.
- Dataset: ConvAI2 contains around 160,000 utterances in around 11,000 persona-based dialogues, with 2,000 validation and test dialogues using non-overlapping personas.
- Automatic Evaluation: Perplexity shows RetNRef improves with label neighbors or the label itself, while no retrieval, random labels, and the best retriever perform similarly.RetNRef++ reaches perplexity 48.4 with the Memory Network retriever, but poor perplexity does not preclude better human judgments.
- Word Statistics: Seq2Seq responses are shorter and use fewer rare words than human responses, while RetNRef improves rare-word usage and RetNRef++ moves statistics closer to human output.These statistics do not measure semantic coherence.
- Retrieval Overlap: RetNRef++ has more than 80% word overlap with retrieved output 53% of the time, compared with 3% for Seq2Seq and 8% for RetNRef.This indicates RetNRef++ uses retrieval while retaining the ability to generate novel content.
- Human Evaluation: Human evaluations score engagingness, consistency, fluency, and persona detection; all RetNRef variants exceed Seq2Seq in engagingness, with RetNRef++ slightly exceeding its retriever.RetNRef remains able to generate text unavailable to a retrieval model, although persona use is weaker than Seq2Seq.
- Qualitative Analysis: Example dialogues show RetNRef++ can attend to retrieved content for longer, nuanced replies or generate shorter context-fitting replies independently.Remaining issues include repeated phrases and copying the speaking partner’s phrases.
4 Conclusion
The paper shows that retrieval can improve dialogue generation, helping avoid short, frequent-word responses that are not engaging. RetNRef++ achieves human-like word statistics and more engaging conversations, while future work should reduce reliance on heuristics and simple concatenation.
- Retrieval models can improve dialogue generation by helping avoid short, frequent-word responses that are ultimately not engaging.
- RetNRef++ has word statistics similar to human utterances and provides more engaging conversations according to human judgments.
- Future work should improve retrieval incorporation beyond the heuristics and concatenation-plus-attention approach used here.The paper suggests treating inputs as independent sources more clearly or training the models jointly.