BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data

1 Introduction

• Personas combine identity elements such as profiles and background personal facts, and condition responses alongside dialogue context.Source: §1 · p. 1
• Limited crowd-sourced persona-dense data and persona-sparse everyday conversations make it difficult to train models that sufficiently understand persona consistency.Persona-dense datasets are costly to annotate, while less than 10% of Twitter messages reveal personal anecdotes or activities at home or work.Source: §1 · p. 1
• The paper separates persona-based dialogue generation into consistency understanding and dialogue generation, drawing on non-dialogue inference and persona-sparse dialogue data, respectively.SNLI and MNLI are examples of non-dialogue inference resources.Source: §1 · pp. 1–2
• BoB uses a BERT-based encoder, an autoregressive response-generation decoder, and a bidirectional consistency-understanding decoder.The encoder and first decoder produce a coarse response representation, which the second decoder maps to final response representations using the persona.Source: §1 · p. 2
• Unlikelihood training on non-dialogue inference data makes contradicted cases less likely to strengthen the consistency-understanding decoder.Source: §1 · p. 2
• Across two limited-data scenarios, BoB generalizes well and outperforms strong baselines on most metrics, especially persona consistency.The scenarios are low-resource persona-dense data and persona-sparse data.Source: §1 · p. 2

Persona-based Dialogues

• Persona-dialogue work learns persona features through embeddings, profiles, or personal facts, with later models exploring mutual personas and multi-stage generation (Li et al., 2016b; Qian et al., 2018; Mazaré et al., 2018; Liu et al., 2020; Song et al., 2020a).Source: §2 · p. 2
• Pretrained approaches include GPT fine-tuning, persona-information attention routing, BERT dialogue generation, and large-scale pretrained chatbots (Wolf et al., 2019; Zheng et al., 2020; Lin et al., 2020; Roller et al., 2020; Madotto et al., 2020).Source: §2 · p. 2

Disentangled Representation

• Disentangled representations transform some underlying-model properties while leaving others invariant; variational autoencoders are one framework used for this purpose (Higgins et al., 2018; Kingma and Welling, 2013; Kim and Mnih, 2018).Source: §2 · p. 2

Unlikelihood Training

• Prior dialogue work used unlikelihood training to reduce contradicted-response probability, whereas BoB’s decoder design supports learning from large-scale non-dialogue inference data (Welleck et al., 2019a; Li et al., 2020; Welleck et al., 2019b).Source: §2 · p. 2

3.1 Overview

• BoB uses an encoder, response decoder, and consistency decoder to transform persona and query inputs into a persona-consistent response.The consistency decoder processes the response representation together with persona information before producing the final response representation.Source: §3.1 · p. 3

3.2 Disentangling

• Mapping the response representation R1 to R2 independently of the query lets BoB train generation on personalized dialogue data and consistency understanding on non-dialogue inference data.The paper treats non-dialogue premises and hypotheses as approximations of persona and response, respectively.Source: §3.2 · p. 3

3.3.1 Encoder

• The encoder converts persona and query text into embeddings and uses bidirectional multi-head attention to produce hidden vectors for downstream tasks.A special token separates persona from query, and their token, type, and position embeddings are combined.Source: §3.3.1 · p. 4

3.3.2 Response Generation Decoder

• The BERT-initialized response decoder uses cross-attention to access encoder context and a left-to-right mask to generate responses autoregressively.Its output representation R1 is passed to the consistency decoder.Source: §3.3.2 · p. 4

3.3.3 Consistency Understanding Decoder

• The consistency decoder attends to persona and R1, fuses their information into final representations R2, and uses an output layer to generate the response.D2 is initialized from BERT and learns consistency understanding from non-dialogue inference data.Source: §3.3.3 · p. 4

3.4 Training Objectives

• Positive inference pairs use NLL, while contradicted pairs receive unlikelihood loss to reduce their likelihood; response-generation and consistency losses are summed for optimization.The procedure trains response generation on personalized dialogue data and consistency understanding on non-dialogue inference data.Source: §3.4 · p. 5

4.1 Datasets

• The two dialogue datasets are persona-dense PersonaChat, grounded in personal facts, and persona-sparse PersonalDialog, whose dialogues are mostly not persona-related.PersonalDialog includes random and manually selected biased test sets; MNLI and CMNLI provide auxiliary non-dialogue inference data, while DNLI and KvPI support evaluation.Source: §4.1 · p. 5

4.2 Compared Methods

• Comparisons include Transformer and LIC on both datasets, CMAML and GDR on PersonaChat, AttentionRouting on PersonalDialog, and GPT-2 on PersonaChat.CMAML and GDR are not used for the persona-sparse scenario.Source: §4.2 · pp. 5–6

4.3 Evaluation Metrics

• Evaluation combines five-point human ratings of fluency, informativeness, relevance, and persona consistency with automatic PPL, Distinct-1/2, C.Score, and ∆P metrics.C.Score uses a referee model; ∆P compares model perplexity on entailed versus contradicted dialogue tuples.Source: §4.3 · p. 6

4.4 Persona-Dense Results

• On full PersonaChat, BoB performs better across all automatic and human metrics, with significant improvements in PPL and ∆P.Table 3 reports the full PersonaChat results.Source: Table 3 · p. 7, §4.4 · p. 7
• With only 1/8 of the PersonaChat training data, BoB outperforms most baselines’ best results and maintains stable ∆P across training-set sizes.The authors attribute this consistency stability to learning from non-dialogue inference data rather than persona-dense dialogue data.Source: §4.4 · p. 7, Table 4 · p. 7

4.5 Validations on Persona-Sparse

• On the biased PersonalDialog test set, BoB achieves the best results on most metrics, while no method consistently leads on the random test set.The authors suggest the random test may resemble ordinary dialogue generation, limiting BoB’s advantages.Source: §4.5 · p. 7, Table 5 · p. 8

4.6 Analysis and Ablation Study

• Removing unlikelihood training degrades consistency-related metrics, while removing D2 causes the largest PPL increase across the ablations.The authors also report that BERT initialization contributes to strong performance across metrics.Source: §4.6 · p. 8, Table 6 · p. 8

4.7 Reproducibility

• The authors released the BoB implementation.The release is identified in the reproducibility subsection.Source: §4.7 · p. 8

• The proposed BERT-based dialogue model learns from limited personalized data by disentangling response generation and consistency understanding.Source: §5 · p. 8
• The paper introduces unlikelihood training with non-dialogue inference data as part of its approach.Source: §5 · p. 8
• Experiments on two publicly available datasets show significant improvements over strong methods while using limited personalized dialogue data.Source: §5 · p. 9

• The data resources come from published works and do not involve privacy issues related to data collection.Source: Ethical Statement · p. 9
• The work does not automatically infer or attribute identity characteristics to participants.Source: Ethical Statement · p. 9
• The work does not categorize participants in the training datasets.Source: Ethical Statement · p. 9