Source-linked AI summary
A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data
Yinhe Zheng, Rongsheng Zhang, Xiaoxi Mao, Minlie Huang
TL;DR
Personalized dialogue generation must represent personas despite sparse persona-related turns in real-world conversations. This paper combines pretrained encoder-decoder models, attribute embeddings, and dynamically weighted attention routing to use such data. Automatic and manual evaluations report more coherent and persona-consistent responses than state-of-the-art baselines, while the method’s weighting choices include documented implementation boundaries.
Problem
Real-world dialogue is often persona-sparse, making it difficult for dialogue models to retain persona information when most responses are unrelated to speakers’ personas.
Method
The model initializes an encoder and decoder with a pretrained language model, adds persona attribute embeddings, and dynamically routes persona and dialogue features during decoding.
Results
Automatic and manual evaluations show richer persona-related features and more coherent, persona-consistent responses than state-of-the-art baselines on persona-sparse data.
Takeaways & Limitations
Dynamic weighting lets the model use persona-sparse dialogues during training while controlling the amount of persona-related content expressed during decoding.
Takeaways & Limitations
Hard script-based weighting can introduce bias and lead to sub-optimal results, while masked self-attention is used to avoid feeding the golden-truth token.
Abstract
from arXiv · showhide
Endowing dialogue systems with personas is essential to deliver more human-like conversations. However, this problem is still far from well explored due to the difficulties of both embodying personalities in natural languages and the persona sparsity issue observed in most dialogue corpora. This paper proposes a pre-training based personalized dialogue model that can generate coherent responses using persona-sparse dialogue data. In this method, a pre-trained language model is used to initialize an encoder and decoder, and personal attribute embeddings are devised to model richer dialogue contexts by encoding speakers' personas together with dialogue histories. Further, to incorporate the target persona in the decoding process and to balance its contribution, an attention routing structure is devised in the decoder to merge features extracted from the target persona and dialogue contexts using dynamically predicted weights. Our model can utilize persona-sparse dialogues in a unified manner during the training process, and can also control the amount of persona-related features to exhibit during the inference process. Both automatic and manual evaluation demonstrates that the proposed model outperforms state-of-the-art methods for generating more coherent and persona consistent responses with persona-sparse data.
Introduction
The paper addresses persona sparsity in real-world dialogue, where persona-relevant turns are limited and may be overlooked. It proposes a pre-training-based model with explicit persona representations and dynamic attention routing, and reports stronger coherent, persona-related responses than prior methods.
- Introduction: Real-world conversations are persona-sparse because speakers often reveal their personas in only limited turns.Most dialogue turns are not persona-related, so directly fine-tuning on such data may cause models to ignore persona information.
- Introduction: The paper proposes a pre-training-based method that uses persona-sparse real-world dialogue data to build personalized dialogue agents.Unlike approaches relying on persona-dense crowdsourced data, the method targets daily social-media conversations where personas are not intentionally revealed.
- Introduction: Attribute embeddings encode speakers’ personas with dialogue histories, while decoder attention routing balances target-persona and contextual features dynamically.The decoder uses routes for target-persona features, dialogue histories, and previously decoded tokens, with predicted weights controlling their contributions.
- Introduction: The method is designed to preserve coherent, diverse dialogue while leveraging persona-sparse data to generate persona-related responses.The stated contributions include exploiting pretrained models and controlling the amount of persona-related information expressed in responses.
- Introduction: Both automatic and manual evaluations show that the method outperforms previous methods in producing more coherent and persona-related responses.The reported result is qualitative rather than tied to a specific metric in the supplied passage.
Related Work
Prior personalized dialogue work includes psychology-inspired models, data-driven sequence-to-sequence approaches, and pretrained dialogue systems. The paper distinguishes its approach by targeting persona-sparse data and explicitly modeling speaker attributes during context encoding.
- Related Work: Psychology-inspired personalized dialogue models use speaker traits such as the Big Five, but subjective metrics and difficult data collection limit large-scale application.The limitation concerns both modeling the traits and obtaining corresponding dialogue data.
- Related Work: Data-driven approaches learn persona-related features from movie dialogues or large-scale social-media dialogue datasets using sequence-to-sequence models.These studies represent an alternative to explicitly psychology-based personalization.
- Related Work: Pretraining has produced strong results across natural-language-processing tasks and has been applied to dialogue modeling through specialized objectives and pretrained GPT2 adaptation.The related approaches include dialogue-specific pretraining and pretrained language-model adaptation.
- Related Work: Compared with the most relevant prior work, this study handles persona-sparse dialogues and adds explicit attribute embeddings for modeling speakers’ personas in dialogue contexts.The cited distinction is both the training-data setting and the explicit context representation.
Model
The model uses a shared-parameter Transformer encoder-decoder to represent dialogue context and target persona separately, then dynamically routes persona, context, and decoding features. Attribute embeddings enrich context representations, while pre-training and auxiliary losses support persona-sparse dialogue generation.
- Architecture: The encoder and decoder share parameters and independently encode dialogue context C and target persona T before autoregressive decoding.The decoder receives context encoding EC and persona encoding ET through attention routing modules.
- Persona Encoding: Each context-token embedding sums word, positional, and speaker-attribute embeddings for gender, location, and interest tags.Interest-tag embeddings average the tag representations for speakers with multiple tags.
- Attention Routing: Three attention routes separately attend to target-persona, dialogue-context, and previously decoded-token features.The first two routes use unmasked bidirectional attention, while the previous-token route uses masked self-attention.
- Attention Routing: A persona weight α ∈[0, 1] controls target-persona contribution while preserving dialogue-context and previous-token features in decoder outputs.Larger α values are intended to produce responses with more persona-related features.
- Dynamic Weight Prediction: A neural predictor estimates α from context encodings, replacing impractical manual annotation and avoiding the bias of hard heuristic weights.The predictor is trained jointly using script-generated noisy labels; soft weights outperform heuristic hard weights in the reported experiments.
Dataset
The study uses PersonalDialog conversations from Chinese social media, where speaker profiles provide structured persona attributes. It evaluates both persona-sparse random contexts and persona-dense biased contexts.
- Source and Structure: The dialogue data come from PersonalDialog, sampled from Chinese Weibo posts and replies with structured speaker profiles.The study uses gender, location, and interest tags as persona attributes.
- Dataset Statistics: About 0.88M dialogues were labeled persona-related by the heuristic process.The supplied passage reports this figure while truncating the surrounding dataset statistics.
- Splits: The validation set contains 10K randomly sampled dialogue sessions.The study constructs separate random and biased test sets to examine behavior across contexts.
- Test Sets: The random test set contains 10K sessions and is persona-sparse because most Weibo dialogues do not intentionally reveal personas.These contexts represent common conversations with limited persona-related content.
- Test Sets: The biased test set contains 521 manually labeled dialogues selected for contexts where speakers tend to reveal personas.Examples include questions and replies explicitly revealing gender.
Experiments
The experiments include a manually labeled response-persona classifier to assess whether generated responses exhibit the specified persona.
- Persona Evaluation: A binary classifier predicts whether a response R exhibits persona T from their concatenated token sequence.The evaluation data include 1,044 manually labeled positive response-persona pairs and an equal number of negative pairs.
Implementation Details
The model is pre-trained on a large Chinese-novel corpus and then fine-tuned for dialogue generation using the stated Transformer configuration.
- Pre-training: The pre-training corpus contains about 0.5 billion tokens from Chinese novels spanning Comedy, Romance, and Mystery genres.A character-level language model with vocabulary size 13,084 was trained on this corpus.
- Training Schedule: Pre-training lasted 70 epochs, followed by 30 epochs of fine-tuning.These durations are reported for the pre-training and dialogue fine-tuning stages, respectively.
Baselines
The study compares recurrent, Transformer, and pretrained Transformer baselines, including variants trained on persona-labeled subsets, and tests ablations of the proposed model.
- Att+PAB is an RNN-based personalized dialogue model that fuses persona input and uses a persona-aware decoding bias.
- Trans. is a Transformer baseline that generates responses from concatenated dialogue histories without persona-related features.
- TTransfo directly fine-tunes a pretrained TransferTransfo model on persona-sparse dialogues, while TTransfo + P uses only heuristically persona-related dialogues.
- LConv uses pretrained encoder and decoder components and is fine-tuned directly on persona-sparse dialogues, with LConv + P restricted to heuristically persona-related dialogues.
- Ablations remove pre-training, encoder attribute embeddings, or dynamic weight prediction to test the contribution of each component.
- Inference additionally varies persona weight α across no-persona, full-persona, and dynamically predicted settings.
Automatic Evaluation
Automatic evaluation measures persona consistency, response overlap, and diversity across random and biased test sets, including the effect of persona-weight control.
- Metrics: Persona Accuracy measures consistency with the target persona, while BLEU and F1 assess overlap with reference responses.
- Results: Our model outperforms all baselines on every reported metric except perplexity across the random and biased test sets.
- Persona-weight analysis: Higher persona weight α increases persona accuracy, while its effects on BLEU, F1, and distinct differ between persona-sparse random and persona-dense biased contexts.
- Persona-weight analysis: On the random test set, BLEU, F1, and distinct decrease as α increases because stronger persona features reduce overlap with human responses in persona-sparse dialogues.
- Persona-weight analysis: On the biased test set, BLEU, F1, and distinct generally increase with α but drop when α reaches 1, indicating a trade-off between persona consistency and other dimensions.
Manual Evaluation
Manual evaluation finds that the proposed model produces fluent, persona-consistent, and context-coherent responses, while excessive persona expression creates a quality trade-off.
- Metrics: Three annotators rated generated and human responses for utterance fluency, persona consistency, and context coherency.
- Results: The evaluation sampled 200 dialogue sessions from each test set and generated 3.2K responses, with substantial agreement for fluency and persona consistency.
- Results: Our model outperforms all baselines on every manual measure, and the full-persona model significantly leads on persona consistency.
- Trade-offs: Exhibiting too many persona-related features hurts fluency and context coherency, while dynamic weighting better balances persona consistency and context coherency on the biased test set.
- Results: The full-persona model’s persona consistency surpasses the human-generated response on the random test set.
Case Study
A sampled case shows that the model can produce coherent, grammatically fluent responses while controlling whether persona information appears through α.
- With full persona (α = 1), the model reveals the speaker’s location attribute, whereas without persona (α = 0), it omits persona-related features.
- In the sampled comparison, baseline responses either lack persona-related features or are not grammatically fluent.
Conclusion
The model generates coherent, persona-consistent responses from persona-sparse dialogue data by representing speakers’ personas and dynamically balancing persona-related features during decoding.
- The model produces coherent, persona-consistent responses conditioned on explicitly represented personas.
- It effectively utilizes persona-sparse dialogue data during fine-tuning.
- Automatic and manual evaluations show richer persona-related features than state-of-the-art baselines on persona-sparse dialogues.