Source-linked AI summary
You Impress Me: Dialogue Generation via Mutual Persona Perception
Qian Liu, Yihong Chen, Bei Chen, Jian-Guang Lou, Zixuan Chen, Bin Zhou, Dongmei Zhang
TL;DR
Personalized chit-chat systems have largely emphasized human-like responses while leaving interlocutor understanding less directly modeled. P2 BOT addresses this gap with a transmitter-receiver framework using mutual persona perception, and experiments on PERSONA-CHAT report improvements over baselines across automatic metrics and human evaluations.
Problem
Existing chit-chat work focuses mainly on mimicking human-like responses, leaving explicit modeling of understanding between interlocutors understudied despite its relevance to high-quality conversation.
Method
P2 BOT uses a transmitter-receiver framework that models mutual persona perception and incorporates its relevance scores as rewards for personalized dialogue generation.
Results
P2 BOT outperforms baselines on PERSONA-CHAT across automatic metrics and human evaluations.
Takeaways & Limitations
Explicitly modeling mutual persona perception provides the paper’s supported approach to enhancing personalized dialogue generation.
Takeaways & Limitations
Self-play may enter repeated cycles, and reward-shaping hyperparameters may lead to redundant questions.
Abstract
from arXiv · showhide
Despite the continuing efforts to improve the engagingness and consistency of chit-chat dialogue systems, the majority of current work simply focus on mimicking human-like responses, leaving understudied the aspects of modeling understanding between interlocutors. The research in cognitive science, instead, suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P^2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding. Specifically, P^2 Bot incorporates mutual persona perception to enhance the quality of personalized dialogue generation. Experiments on a large public dataset, Persona-Chat, demonstrate the effectiveness of our approach, with a considerable boost over the state-of-the-art baselines across both automatic metrics and human evaluations.
1 Introduction
Personalized chit-chat systems often mimic human-like responses without explicitly modeling interlocutors’ understanding. P2 BOT addresses this gap with mutual persona perception and outperforms baselines on automatic and human evaluations.
- Motivation: Current chit-chat systems can generate uninformative responses and inconsistent personality traits because training dialogues span diverse speakers.These limitations hinder personalized conversation quality.
- Research gap: Prior personalized dialogue methods improve response style or use predefined personas, but leave explicit modeling of interlocutor understanding understudied.The paper positions its contribution against approaches focused primarily on human-like response imitation.
- Motivation: Cognitive-science findings suggest that understanding between interlocutors is an essential signal of high-quality chit-chat conversation.Persona-related questions and self-disclosures are presented as ways interlocutors build understanding and keep conversation flowing.
- Proposed approach: P2 BOT models understanding with a transmitter-receiver framework centered on mutual persona perception.The framework uses supervised training and self-play fine-tuning guided by rewards characterizing mutual persona perception.
- Results: P2 BOT outperforms baselines in both automatic metrics and human evaluations on PERSONA-CHAT.The introduction reports superiority over baselines without giving the underlying metric values here.
2 Methodology Overview
P2 BOT alternates dialogue generation with perception of how each interlocutor’s persona is represented by the other. Receiver-derived relevance scores become mutual persona perception rewards for training Transmitter.
- Dialogue process: P2 BOT models a dialogue between interlocutors who each have persona profile sentences and generate alternating utterances from dialogue history.The same generation process applies to both interlocutors, allowing the conversation to continue across turns.
- Mutual persona perception: As dialogue proceeds, utterances build impressions of each interlocutor that should correspond to the interlocutor’s actual persona.The overview illustrates this with an utterance implying that one interlocutor is a writer.
- Mutual persona perception: Receiver projects impressions and personas into a latent space and measures their relevance to quantify mutual persona perception.The resulting relevance scores are used as rewards for training the dialogue generator.
- Transmitter: The Transmitter architecture uses stacked Transformer layers, position encoding, and masked-token training objectives.The figure description identifies the blocks as Transformer Blocks and states that [MASK] tokens are ignored in the training objective.
3 Transmitter
Transmitter generates personalized responses using a pretrained transformer, supervised learning, and self-play reinforcement learning. Its rewards target language style, discourse coherence, and mutual persona perception.
- 3.1 Supervised Dialogue Generation: Transmitter is initialized with GPT and generates responses from persona, dialogue history, and special tokens using a stacked transformer architecture.Supervised training maximizes the conditional log-likelihood of responses.
- 3.1 Supervised Dialogue Generation: An auxiliary Next Utterance Prediction task trains a classifier to distinguish the correct response from a randomly sampled distractor.During inference, the classifier ranks response candidates together with the generation score.
- 3.2 Self-play Model Fine-tuning: Self-play fine-tuning simulates multi-turn communication between two Transmitters, with one frozen user and one learnable agent optimized by policy gradient.Multinomial sampling and a minibatch reward baseline are used during reinforcement learning.
- 3.2 Self-play Model Fine-tuning: The reward combines language style, discourse coherence, and mutual persona perception to guide response generation.The persona-perception reward is supplied by Receiver and can reflect effects that occur several turns after an action.
- 3.2 Self-play Model Fine-tuning: Mutual persona perception is modeled as a long-term reward, so earlier questions can contribute to later persona-relevant responses.The discount factor γ controls how far ahead the agent looks when assigning this reward.
4 Receiver
Receiver estimates mutual persona perception by comparing dialogue-derived impressions with actual and distractor personas. It uses contextual encoders, relevance aggregation, negative sampling, and sparse marginal-loss training.
- 4.1 Training: Negative sampling trains Receiver to identify the real persona w_A from the pair {w_A, w_Z}, where w_Z is a randomly sampled distractor.At inference, the resulting relevance score models mutual persona perception and contributes to Transmitter’s self-play reward.
- 4 Receiver: Receiver uses separate BERT-initialized encoders for impressions and personas, averaging token representations into fixed d-dimensional sentence vectors.Sequential utterance encoding produces an impression matrix H_A, while persona sentences produce W_Δ.
- 4 Receiver: A scaled dot product computes fine-grained relevance scores between each impression and persona sentence.The resulting matrix U_Δ is aggregated into a cumulative score c_Δ.
- 4.1 Training: The marginal loss encourages the real-persona score c_A to exceed the distractor score c_Z by margin m, while L1 regularization encourages sparse relevance matrices.Sparsity reflects the assumption that an utterance generally relates to zero or one profile.
- 4.1 Training: A temperature-controlled weighted aggregation shifts from average pooling toward the highest relevance score as τ anneals, reducing noise during training.This avoids maximizing unrelated profile–utterance scores that could introduce unnecessary noise into Transmitter training.
5 Experiment
Experiments on PERSONA-CHAT evaluate P2 BOT through automatic metrics, human judgments, variant analysis, persona-perception probing, and qualitative examples. The results support its effectiveness while revealing self-play cycles and redundant questions as limitations.
- 5.2 Methods Comparison: P2 BOT achieves new state-of-the-art performance on Perplexity and F1, with highly competitive Hits@1 and a 13.4% relative F1 improvement in revised mode.The comparison uses Hits@1, Perplexity, and F1; lower Perplexity is better, while F1 measures word-level precision and recall.
- 5.3 Model Analysis: P2 BOT responses are more consistent, engaging, and informative in sampled examples, including responses that explain behavior while revealing persona information.One example links not exercising to being busy with a robot project, consistent with the speaker’s persona.
- 5.3 Model Analysis: Self-play may enter repeated cycles after training, and reward-shaping hyperparameters may contribute to redundant questions.These issues were observed despite the approach working well in most cases.
- 5.4 Persona Perception Probing: Compared with IR, P2 BOT improves original-mode Hits@1 by 26.3% and performs strongly in revised mode, indicating Receiver can perceive rephrased personas.The probing dataset adds 31 persona distractors per dialogue and evaluates Hits@1 and MRR.
- 5.4 Persona Perception Probing: Receiver assigns higher relevance scores to related profile sentences and dialogue utterances, such as linking volunteering at a pool with loving water.Figure 6 visualizes relevance scores between a sampled dialogue and its revised persona; darker colors indicate higher scores.
6 Related Work
Related work spans retrieval-based and generative dialogue systems, reinforcement learning, personalization, diversity, and goal-oriented adaptation. P2 BOT differs by explicitly modeling understanding through mutual persona perception.
- Dialogue Systems: Retrieval-based systems rank response candidates against dialogue context, whereas generative systems commonly encode context and decode responses with sequence-to-sequence models.The paper places its architecture within the generative dialogue tradition while also using reinforcement-learning-based fine-tuning.
- Personalized Dialogue: Prior personalization methods re-rank candidates, preserve names, adapt to speakers, or extract user attributes, while P2 BOT enhances persona-based dialogue generation through understanding modeling.The cited approaches include post-processing, adversarial training, meta-learning, and attribute extraction.
- Personalized Dialogue: Other work targets diverse persona-conditioned responses, while goal-oriented personalization adjusts responses to user profiles rather than maintaining persistent personalities.P2 BOT is positioned around persistent personality in open-domain chit-chat.
7 Conclusion & Future Work
The paper concludes that P2 BOT explicitly models interlocutor understanding through mutual persona perception and demonstrates effectiveness on PERSONA-CHAT. It proposes extending Receiver to conversational recommendation by inferring user personas after dialogue.
- Conclusion: P2 BOT combines a transmitter-receiver framework with mutual persona perception as a reward signal for personalized dialogue generation.The framework explicitly models understanding between interlocutors.
- Future Work: Future work will extend Receiver to conversational recommender systems that infer user personas from dialogue for personalized content recommendation.The proposed setting involves persona inference after multiple turns of chatting.