Learning Retrieval Augmentation for Personalized Dialogue Generation

1 Introduction

• Persona profiles typically contain only four to five sentences, limiting the background detail available for highly personalized, contextually rich responses.Existing models are described as restricted to static profiles and unable to incorporate more intensive extra persona information during decoding.Source: §1 · p. 1
• Story data offers life events, traits, motivations, and experiences, while its inherent structure may support persona consistency.The paper selects stories to support more engaging and contextually meaningful dialogue.Source: §1 · p. 2
• Using external knowledge raises challenges because retrieval annotations are absent, content usefulness is unclear, and dense retrieval may repeatedly select similar passages.The paper notes that retrieval based on predicted probability distributions may not directly align with personalized-response objectives.Source: §1 · p. 2
• LAPDOG retrieves story content to enrich persona profiles and jointly trains a retriever and generator using non-differentiable generation metrics.The retriever uses metrics such as BLEU, F1, and ROUGE-L to guide retrieval; the generator also uses persona information and dialogue context.Source: §1 · p. 2
• LAPDOG consistently improves over baseline models of varying sizes on CONVAI2 with ROCStory as external knowledge.The introduction also reports that retrieved content offers insight into the generator's response rationale and that joint objective guidance outperforms either individual objective.Source: §1 · p. 2
• Candidate augmentation is introduced to improve performance and increase the diversity of candidates selected at inference.The contribution summary also describes LAPDOG as retrieving relevant external content for persona profiles using a non-differentiable objective.Source: §1 · p. 2

2.1 Personalized Dialogue Generation

• Personalized dialogue approaches include GPT2 fine-tuning, reinforcement learning with BERT models or manually designed rewards, data augmentation, and adaptive attention.LAPDOG contrasts with these approaches by using an end-to-end dense retriever to augment generator inputs from an external source while tuning the retriever.Source: §2.1 · p. 3

2.2 Retrieval-Augmented Text Generation

• Retrieval-augmented generation systems use varied strategies, including retrieval annotations, bootstrapped data, prompts, probability distillation, and joint retriever-language-model training.The passage names DocPrompting, Toolformer, FLARE, RePlug, and RAG as examples.Source: §2.2 · p. 3
• LAPDOG targets desired generation objectives directly, avoids retrieval annotations, and adds candidate augmentation to broaden potential dialogue contexts.The paper also notes that RePlug, Toolformer, and FLARE are excluded from comparison because they rely on large language models or their API calls.Source: §2.2 · p. 3

3.1 Task Formulation

• Persona profiles contain only four to five sentences, so the method uses a story corpus to enrich them despite the lack of explicit persona-story retrieval annotations.The response is generated from the persona and dialogue context, with retrieved stories added as supplementary context.Source: §3.1 · p. 3

3.2 The Architecture

• A dual-encoder retriever selects stories from an external corpus, while a transformer encoder-decoder generator combines retrieved stories, persona, and dialogue history using FiD.FiD encodes each story-persona-context combination separately, concatenates the encoded contexts, and uses decoder cross-attention to generate the response.Source: §3.2 · p. 4

3.3 Training Process

• LAPDOG first supervises the generator, then tunes retrieval using generator metrics, explores candidates, trains on retrieved content, and jointly updates both components.The two-stage training process is illustrated in Figure 1.Source: §3.3 · p. 4, Figure 1 · p. 4

3.3.1 Supervised Training

• The initial generator is trained on persona and dialogue context to predict the ground-truth response without retrieval results.This stage minimizes negative log-likelihood; the resulting model is denoted Gsup.Source: §3.3.1 · pp. 4–5

3.3.2 Learning Retrieval Augmentation

• Generator metric scores on retrieved stories become a probability-distribution target, and the retriever is trained to align its similarity scores with that target.A higher metric value indicates better performance; the metric is negated when lower values indicate better performance.Source: §3.3.2 · p. 5

3.3.3 Retrieval Candidate Augmentation

• Candidate augmentation replaces retrieved stories with randomly sampled candidates to encourage exploration and reduce the risk of retrieval stagnation.Without exploration, a retriever may repeatedly select a narrow set, leaving the generator to disregard retrieved content.Source: §3.3.3 · p. 5

3.3.4 Training Retrieval-Augmented Generator

• The retrieval-augmented generator is trained on persona, dialogue context, and retrieved content, adding retrieval content to the supervised generator input.Its loss is combined with the retriever loss for joint training, with the two losses treated equally.Source: §3.3.4 · p. 5, §3.3.5 · p. 6

3.4 Inference Process

• At inference, ROCStory passages aligned with the persona are integrated with dialogue context through FiD to generate the response autoregressively.The authors omit a weighting hyperparameter between retriever and generator losses to avoid the computational cost of tuning it.Source: §3.4 · p. 6, §3.3.5 · p. 6

4.1 Dataset

• CONVAI2 contains crowd-sourced multi-turn conversations paired with persona descriptions, with 8,939/1,000 conversations and 1,155/100 personas in train/dev.Each persona is described by approximately five profile sentences.Source: §4.1 · p. 6

4.2 Retrieval Corpus

• Because CONVAI2 lacks a paired annotated retrieval corpus, the experiments use ROCStory as auxiliary retrieval data to supplement personas.The corpus is preprocessed toward persona-style narratives and contains 98,161 five-sentence stories.Source: §4.2 · p. 6

4.3 Experimental Settings

• T5 small, base, and XL models initialize the generator, while a Contriever-initialized dense retriever uses persona profiles as queries.The models are first supervised on CONVAI2, then tuned for retrieval augmentation.Source: §4.3 · p. 6

4.4 Evaluation Metric

• The overall training metric sums F1, BLEU, and ROUGE-L, which measure word overlap, n-gram precision, and longest-common-subsequence similarity, respectively.Source: §4.4 · p. 6

4.5 Baseline

• Comparisons include supervised T5 foundations, fixed-retriever and reinforcement-learning variants, and RAG tuning based on generator probabilities rather than desired metrics.Source: §4.5 · pp. 6–7

4.6 Results

• Fixed retrieval improves F1 and BLEU but lowers ROUGE-L; reinforcement-learning tuning degrades all metrics, while RAG raises F1 but lowers BLEU and ROUGE-L.Source: §4.6 · p. 7
• LAPDOG delivers the best improvements over supervised foundations across model sizes, including small, base, and XL models.The reported gains span all evaluated metrics for small models and remain consistent for larger models.Source: §4.6 · p. 7

5.1 Analysis on Metrics

• BLEU removal yields BLEU 3.07, ROUGE-L 14.35, and F1 14.29; removing F1 causes a larger performance reduction, and removing ROUGE-L lowers all three metrics.The authors conclude that each metric contributes uniquely to the combined optimization objective.Source: §5.1 · pp. 7–8

5.2 Number of the Candidates

• As K rises from 2 to 6, metrics generally improve but fluctuate rather than increasing monotonically, so candidate count requires careful selection.Too few candidates may provide insufficient information, while too many may introduce irrelevant content.Source: §5.2 · p. 8, Figure 2 · p. 8

5.3 Candidate Augmentation

• Candidate augmentation improves LAPDOG across F1, BLEU, and ROUGE-L compared with using no augmentation.Source: §5.3 · p. 8, Table 3 · p. 8
• With candidate augmentation, LAPDOG retrieves 1,570 unique stories, versus 738 without augmentation; RAG retrieves 25 and the fixed retriever 1,351.Figure 3 compares the unique story counts across models.Source: §5.3 · p. 8, Figure 3 · p. 8

5.4 Training Strategy

• Training LAPDOG from scratch lowers all metrics and takes longer to converge than the two-stage training process.The ablation describes two-stage training as important for both performance and efficiency.Source: §5.4 · p. 8, Table 3 · p. 8

• LAPDOG jointly tunes a retriever and generator to retrieve useful ROCStory stories and consistently improves performance over language models of varying sizes.Source: §6 · p. 9

Limitations

• Experiments use T5 rather than currently prevalent large language models because of resource constraints.The authors suggest that tuning retrieval content with large language models could yield advantages.Source: Limitations · p. 9
• The study uses only 2–6 retrieved passages and a 512-token context, while larger story sets and longer contexts are proposed for future work.The authors anticipate these expansions could further enhance performance.Source: Limitations · p. 9
• The training objective sums F1, ROUGE, and BLEU; the authors suggest a more diverse objective could better support engaging dialogue.They also note that a generator designed beyond conventional T5 could further improve LAPDOG.Source: Limitations · p. 9

Ethics Statement

• Experiments use fictional persona profiles and neither handle nor store real personal data.For real-world applications, the authors call for ethical sourcing and informed consent, as well as respect for privacy, dignity, and cultural sensitivities.Source: Ethics Statement · p. 9

A.1 Detail Settings for Training

• Table 4 is identified as the source of LAPDOG’s detailed training settings.The supplied passage does not specify the individual settings.Source: Table 4 · p. 12

A.2 Query Analysis

• Persona-only queries achieve the highest BLEU, while Persona+Dialogue marginally improves ROUGE-L and F1 at a slight BLEU cost.Generated-output queries are less competitive, and a randomly selected single persona sentence performs worse on all metrics.Source: §A.2 · p. 12, Table 5 · p. 12

A.3 Evaluation Results on CONVIAI2 Revised Dataset

• LAPDOG consistently improves performance on the paraphrased CONVAI2-Revised dataset, which the paper describes as more difficult than the original.Source: §A.3 · p. 12, Table 6 · p. 12

A.4 Case Study

• In case studies, retrieved stories and persona details align responses with a divorce intention and enrich a bland gym greeting with workout context.A further example contrasts a generic reply with one that includes a grade-3 persona’s Disney World aspiration.Source: §A.4 · p. 12

A.5 Comparison to Traditional Knowledge Dialogue Approaches

• LAPDOG outperforms ITDD in the reported comparison; ITDD uses paired documents and conversations, which an off-the-shelf retriever creates for the comparison.The paper says LAPDOG trains its retriever and generator unsupervised on an external corpus, unlike ITDD’s use of preannotated pairs.Source: §A.5.2 · p. 14, §A.5.1 · p. 14, §A.5 · p. 14

A.6 Complete Training Procedure; A.7 Pre-process ROCStory Coprus

• Algorithm 2 describes the complete training procedure, while Algorithm 1 describes ROCStory preprocessing.Source: §A.6 · p. 14, §A.7 · p. 14

A.8 Extended Evaluation Metrics

• LAPDOG excels over baselines on METEOR, while BERT-score variance is minimal; the authors tentatively connect this to optimization for traditional metrics.The appendix adds METEOR and BERT to the evaluation based on F1, BLEU, and ROUGE-L.Source: §A.8 · p. 14

A.9 Additional Related Work on Large Language Models (LLMs)

• The related-work discussion contrasts LAPDOG’s retrieval augmentation with scaling language models, suggesting it could complement existing LLMs.The authors characterize LAPDOG as producing more diverse and interpretable results with smaller model parameters and corpus size.Source: §A.9 · p. 14

A.10 Human Evaluation

• 56.32% of evaluators preferred LAPDOG over fine-tuned T5, though the preference difference was relatively narrow.The authors note that both models may have merits in certain conversational contexts.Source: §A.10 · p. 14, Table 10 · p. 15