Source-linked AI summary
Training Millions of Personalized Dialogue Agents
Pierre-Emmanuel Mazaré, Samuel Humeau, Martin Raison, Antoine Bordes
TL;DR
End-to-end dialogue systems often lack engagement, while earlier persona-based evidence came from an artificial dataset with limited coverage. This paper builds a Reddit-derived dataset of millions of personas and trains persona-conditioned retrieval models. Persona conditioning improves prediction, and pretraining followed by fine-tuning achieves state-of-the-art results on PERSONA-CHAT.
Problem
End-to-end dialogue systems show limited engagement in chit-chat, while PERSONA-CHAT is artificial and covers only a bit more than 1k personas.
Method
The paper extracts user-specific text personas from Reddit conversations and combines encoded personas with context to retrieve candidate responses.
Results
Persona conditioning improves prediction, and pretraining on Reddit followed by fine-tuning on PERSONA-CHAT raises hits@1 from 35.4% to 60.7%.
Takeaways & Limitations
Large-scale persona training improves end-to-end dialogue performance and transfers effectively to PERSONA-CHAT.
Takeaways & Limitations
Extracted personas may omit the discussion topic, contain contradictions, or fail to represent general user traits.
Abstract
from arXiv · showhide
Current dialogue systems are not very engaging for users, especially when trained end-to-end without relying on proactive reengaging scripted strategies. Zhang et al. (2018) showed that the engagement level of end-to-end dialogue models increases when conditioning them on text personas providing some personalized back-story to the model. However, the dataset used in Zhang et al. (2018) is synthetic and of limited size as it contains around 1k different personas. In this paper we introduce a new dataset providing 5 million personas and 700 million persona-based dialogues. Our experiments show that, at this scale, training using personas still improves the performance of end-to-end systems. In addition, we show that other tasks benefit from the wide coverage of our dataset by fine-tuning our model on the data from Zhang et al. (2018) and achieving state-of-the-art results.
1 Introduction
End-to-end dialogue systems can learn across domains from dialogue logs but show limited engagement in chit-chat. Persona conditioning improves engagement, motivating a larger, less limited persona-based dataset.
- End-to-end dialogue systems learn from large dialogue sources without expert knowledge, but exhibit limited engagement and lack consistency in chit-chat.
- PERSONA-CHAT pairs dialogue agents with text personas, and conditioning models on those personas improves dialogue-agent engagement.
- PERSONA-CHAT's artificial collection mechanism limits representativeness and coverage to a bit more than 1k personas.
- This paper builds a Reddit-derived dataset with over 5 million personas and more than 700 million conversations, then trains persona-based end-to-end models.
2 Related work
Prior personalized dialogue work has modeled speaker traits using latent variables, structured knowledge, or explicit text personas. Text personas are especially suited to user-interpretable personalization across varied situations.
- Prior approaches learn latent variables representing speaker bias or personality in dialogue models.
- Other personalization strategies extract explicit variables from structured knowledge bases or symbolic sources.
- Text personas describe speakers' habits and preferences and improve dialogue modeling when used as conditioning information.
- Text is preferable for personal chatbots because users can generate and interpret it, including when knowledge-base facts are unavailable or insufficiently varied.
- REDDIT is used as a massive online message board and as a response-prediction resource for dialogue and natural-language-inference research.
3 Building a dataset of millions of persona-based dialogues
The paper constructs persona-based dialogue examples from Reddit by extracting user-specific persona sentences and pairing them with conversational contexts and responses. The resulting dataset covers millions of users and links most training examples to personas.
- Each example contains a persona, a context utterance, and a response to be predicted.The persona represents the responding agent's personality, while context is the utterance receiving the response.
- Persona extraction: Persona sentences are selected from a user's comments using length, first-person wording, verb, and noun, pronoun, or adjective requirements.
- Persona extraction: Four persona setups vary rule-based, classifier-scored, user-random, and dataset-random sentence selection; the last provides a control for user-specific information.
- Persona extraction: Extracted personas may omit the discussion topic, contain contradictions, or include sentences that do not represent general user traits.
- Reddit comments are paired as successive context-response examples, with personas extracted using training data only to prevent test responses from appearing explicitly.
- Dataset creation: 4.6m users are covered in rule-based setups and 7.2m in random setups, with 97–99.4% of training examples linked to a persona.
4 End-to-end dialogue models
The models retrieve a response by comparing candidate representations with a joint context-persona representation, using several encoder architectures and persona encoding choices.
- Architecture: Responses are retrieved from candidate sets by maximizing the dot product with the joint context-persona representation.Training applies a softmax to candidate dot products and uses other batch responses as negatives.
- Architecture: The architecture combines encoded context and persona through a 1-hop memory network with a residual connection.The context serves as the query, while persona sentences form the memory.
- Context and response encoders: Three encoder architectures are trained: bag-of-words, a 2-layer bidirectional LSTM, and a Transformer encoding module.The bag-of-words encoder uses projections and position averaging, the LSTM uses its last hidden state, and the Transformer averages representations across positions.
- Evaluation: Table 1 reports test results for selecting the correct answer from 100 possible candidates.
- Persona encoder: The persona encoder processes each persona sentence separately, applies a linear layer over shared word embeddings, and sums sentence representations.A simpler architecture is chosen because each batch encodes an order of magnitude more personas than training examples.
5 Experiments
The experiments evaluate persona-based dialogue models on Reddit and PERSONA-CHAT, showing that personas improve Reddit prediction and that Reddit pretraining transfers strongly to PERSONA-CHAT.
- Experimental details: The experimental models are trained on the persona-based dataset and evaluated both on the original Reddit task and after transfer to PERSONA-CHAT.Training uses Adamax with a learning rate of 8e−4 and mini-batches of size 512, with training distributed across 8 GPUs for one epoch.
- Impact of personas: Persona conditioning improves prediction performance across the evaluated encoder architectures on the Reddit task.The comparison includes the different architectures reported in Table 1.
- Persona extraction: Increasing the maximum persona size improves prediction performance, while adding the persona classifier degrades results under the evaluated extraction setup.The authors suggest the classifier may be too narrow because it was trained on revised PERSONA-CHAT personas.
- Transfer learning: 42.1% hits@1 is achieved by the best Transformer architecture, up from 35.4% for the results of Zhang et al. (2018) on PERSONA-CHAT.The evaluation uses different candidate counts across datasets for comparability with prior state-of-the-art results.
- Transfer learning: 60.7% hits@1 is reached after pretraining on Reddit and fine-tuning on PERSONA-CHAT.Fine-tuning on PERSONA-CHAT reduces performance on Reddit, while direct Reddit-model testing on PERSONA-CHAT yields a very low result.
6 Conclusion
The paper presents a very large persona-based dialogue dataset and finds that aligning responses with both persona and context improves prediction performance, including state-of-the-art transfer results on PERSONA-CHAT.
- 6 Conclusion: The paper creates a very large dataset for persona-based dialogue and trains models to align answers with both author persona and conversational context.The authors identify fine-tuning for various dialogue systems and selecting limited persona sets as directions for future work.
- 6 Conclusion: State-of-the-art transfer results on PERSONA-CHAT provide evidence of broad dataset coverage.The conclusion attributes the promising coverage to the trained models' transfer results.