PaperlayerView exact revision on arXiv

Source-linked AI summary

Personalized Dialogue Generation with Diversified Traits

Yinhe Zheng, Guanyi Chen, Minlie Huang, Song Liu, Xuan Zhu

arXiv:1901.09672v2cs.CL

TL;DR

The paper addresses how dialogue systems can express explicit personality traits despite limited persona-labeled data and the subtlety of trait expression. It constructs PersonalDialog and develops persona-aware sequence-to-sequence models, which experiments show can address appropriate traits across contexts.

  • Problem

    Personalized dialogue generation lacks large-scale persona-labeled data, while expressing personality through language is challenging and often subtle.

  • Method

    The paper constructs PersonalDialog and uses trait fusion with persona-aware attention and bias in sequence-to-sequence dialogue generation.

  • Results

    The model significantly outperforms all baselines on both evaluation metrics, indicating more fluent and appropriate responses with incorporated personality traits.

  • Takeaways & Limitations

    PersonalDialog and the proposed models provide baselines for personalized dialogue research and can support related sociolinguistic and social-science research.

  • Takeaways & Limitations

    Multi-turn dialogue generation is left for future work because the experiments do not yet encode additional contexts.

Abstract

from arXiv · show

Endowing a dialogue system with particular personality traits is essential to deliver more human-like conversations. However, due to the challenge of embodying personality via language expression and the lack of large-scale persona-labeled dialogue data, this research problem is still far from well-studied. In this paper, we investigate the problem of incorporating explicit personality traits in dialogue generation to deliver personalized dialogues. To this end, firstly, we construct PersonalDialog, a large-scale multi-turn dialogue dataset containing various traits from a large number of speakers. The dataset consists of 20.83M sessions and 56.25M utterances from 8.47M speakers. Each utterance is associated with a speaker who is marked with traits like Age, Gender, Location, Interest Tags, etc. Several anonymization schemes are designed to protect the privacy of each speaker. This large-scale dataset will facilitate not only the study of personalized dialogue generation, but also other researches on sociolinguistics or social science. Secondly, to study how personality traits can be captured and addressed in dialogue generation, we propose persona-aware dialogue generation models within the sequence to sequence learning framework. Explicit personality traits (structured by key-value pairs) are embedded using a trait fusion module. During the decoding process, two techniques, namely persona-aware attention and persona-aware bias, are devised to capture and address trait-related information. Experiments demonstrate that our model is able to address proper traits in different contexts. Case studies also show interesting results for this challenging research problem.

1 INTRODUCTION

The paper frames personalized dialogue generation around explicit, diversified personality traits and introduces a large-scale corpus and persona-aware generation models to study it.

  • 1 INTRODUCTION: Explicit, diversified traits should guide which aspects of a persona a dialogue system expresses in different contexts.The paper illustrates that language can reveal traits such as gender and location without stating them directly.
  • 1 INTRODUCTION: The corpus represents speaker personae as explicit key-value trait pairs, making them more interpretable and allowing data sharing across speakers with the same trait values.
  • 1 INTRODUCTION: Persona-aware models fuse trait embeddings and use persona-aware attention and bias during decoding to address trait information.
  • 1 INTRODUCTION: PersonalDialog contains about 20.83 million dialogue sessions from about 8.47 million speakers, with traits including Age, Gender, Location, and Interest Tags.More than 3.43 million sessions are multi-turn, each containing at least four utterances.
  • 1 INTRODUCTION: The paper introduces a task for incorporating explicit personality traits into conversation generation using large-scale real social conversations.
  • 1 INTRODUCTION: Experiments show that the model addresses proper personality traits in different contexts.

2 RELATED WORK

Prior personalized dialogue systems model personality implicitly through speaker representations or explicitly through persona descriptions, but face interpretability, data sparsity, and data-collection constraints.

  • 2 RELATED WORK: Existing personalized dialogue research includes Big Five models, movie-character styles, speaker embeddings, and social-media-based neural models.
  • 2 RELATED WORK: Embedding-based personalized models require abundant dialogue data from each speaker, while grouping speakers by gender can alleviate data sparsity.
  • 2 RELATED WORK: Earlier explicit-persona work uses key-value profiles and decoders that ensure a selected persona value appears in the response.
  • 2 RELATED WORK: Crowdsourcing large-scale dialogue data for explicit persona models is expensive.

3 MODEL

The model represents explicit speaker traits as a unified persona vector and incorporates it into sequence-to-sequence decoding through persona-aware attention or bias. These mechanisms condition context selection or generation probabilities on personality information, with persona-aware bias generally outperforming attention in experiments.

  • Task and architecture: The model encodes key-value traits, fuses their embeddings into persona representation vp, and uses vp to condition response generation.The task generates a response embodying responder traits without requiring their exact values to appear in the response.
  • Persona-aware decoding: Persona-aware attention uses vp to generate decoding-time attention weights, making each context vector dependent on both decoder state and persona.The resulting context representation is then used by the decoder during response generation.
  • Trait fusion: The trait fusion module supports attention, averaging, and concatenation strategies for combining individual trait representations.Traits attention weights representations according to context, averaging weights them equally, and concatenation requires compatible vector dimensions.
  • Persona-aware decoding: Persona-aware bias adds vp directly to the output layer and uses a gate to balance persona-related and semantic information at each decoding step.The gate controls how much trait-related information enters the generation distribution over time.
  • Persona-aware decoding: Persona-aware bias generally outperforms persona-aware attention across all reported metrics.The paper attributes this to bias influencing the decoding process more directly.

4 PERSONALDIALOG DATASET

PersonalDialog is a large-scale Weibo dialogue corpus pairing multi-turn conversations with explicit, diverse speaker traits for personalized dialogue and sociolinguistic research. Its construction combines publicly available persona information, dialogue-tree extraction, filtering, and trait-based analysis resources.

  • 4.1.1 Personality Traits: The corpus forms dialogue sessions from Weibo posts and comment branches, preserving informal interactions with publicly available speaker traits.An original post and one comment branch constitute a session, enabling conversations to be organized from reply-to relations.
  • 4.1.1 Personality Traits: Each speaker is represented with five traits: Gender, Age, Location, Interest Tags, and Self Description.Age values span 8–48 when retained, locations cover 35 Chinese provinces or urban districts, and infrequent interest tags are removed.
  • 4.1.2 Corpus Size: 20.83M dialogues and 56.25M utterances make PersonalDialog a large-scale corpus for personalized dialogue research.The corpus also contains 3.43M multi-turn sessions, supporting multi-turn open-domain dialogue research.
  • 4.1.4 Sociolinguistics Phenomena: PersonalDialog supports sociolinguistic research because its large, socially situated dialogues pair language use with diversified speaker traits.The corpus also offers a setting where findings from audience-specific sociolinguistic datasets can be examined when posts do not specify a particular audience.
  • 4.2 Data Collection and Filtering: Data collection began with roughly 60 million raw sessions and 12 million speakers before filtering and cleaning.Processing removed abusive or unsuitable utterances, reduced noise, and filtered likely spammers using user activeness levels.

5 EXPERIMENTS

The experiments evaluate persona-aware dialogue generation on unbiased and trait-biased test sets, using classifier-based procedures to construct biased examples and assess trait expression.

  • 5.1 Data Preparation for Experiments: The evaluation uses 10M single-turn dialogue sessions, three traits, coarse Age and Location labels, and 20,000 validation sessions.Multi-turn generation is left for future work.
  • 5.1 Data Preparation for Experiments: Four 10,000-session test sets cover unbiased, gender-biased, age-biased, and location-biased contexts.Biased sets target contexts in which speakers tend to reveal particular traits.
  • 5.1 Data Preparation for Experiments: Individual response bias is estimated by averaging classifier confidence over multiple 20-utterance inputs containing that response, then selecting the 10,000 highest-scoring responses.The approximation samples N = 50,000 post-response pairs and m = 1,000 classifier inputs per response.
  • 5.1 Data Preparation for Experiments: The constructed biased sets contain rich trait-related features, as their concatenated responses achieve high classifier accuracy.The authors use N = 50,000 and m = 1,000 because the resulting accuracy scores are near-perfect.

5.2 Implementation Details

The model is implemented with recurrent encoder-decoder components and a 100-dimensional persona representation, trained using Adam.

  • 5.2 Implementation Details: The encoder and decoder are 2-layer GRUs with 512 hidden units per layer, while the vocabulary contains 40,000 words.Word vectors have dimension 100 and are shared between encoder and decoder.

5.3 Baselines

The experiments compare persona-free, single-trait, and diversified-trait systems, including six variants of the proposed fusion and decoding mechanisms.

  • 5.3 Baselines: The baselines include a persona-free Seq2Seq model and three GLBA models targeting Gender, Age, or Location individually.The GLBA baselines incorporate one personality trait at a time.
  • 5.3 Baselines: The proposed system varies three trait-fusion methods and two decoding schemes, producing six tested variants.Fusion uses attention, averaging, or concatenation; decoding uses Persona-Aware Attention or Persona-Aware Bias.
  • 5.3 Baselines: GLBA models are treated as stronger baselines than speaker models because gated bias generally outperforms trait embeddings in the cited comparison.Speaker models were also excluded because they require substantial dialogue data per speaker.
  • 5.3 Baselines: The evaluation reports perplexity, distinct-1, distinct-2, and trait accuracy on the unbiased set, with trait accuracy also reported on biased sets.The supplied table captions identify the metric scope for Tables 5 and 6.

5.4 Automatic Evaluation

Automatic evaluation measures fluency, diversity, and trait incorporation across unbiased and contextually biased test sets; the attention-fusion plus PAB variant performs best overall.

  • 5.4 Automatic Evaluation: Perplexity measures content-level fluency, Distinct measures response diversity, and trait accuracy measures agreement between supplied traits and classifier-predicted traits.Higher trait accuracy indicates stronger incorporation of the specified trait.
  • 5.4 Automatic Evaluation: The attention-fusion model with Persona-Aware Bias achieves the best performance on almost all metrics, particularly on biased test sets.The result indicates that trait attention helps model diversified traits and select appropriate traits across contexts.
  • 5.4 Automatic Evaluation: Persona-Aware Bias generally outperforms Persona-Aware Attention across all reported metrics.The authors attribute this pattern to PAB influencing decoding more directly.
  • 5.4 Automatic Evaluation: GLBA models perform well for their single target trait but degrade on the other traits, whereas the proposed models achieve higher trait accuracy across all traits.This comparison supports using a trait-fusion module for diversified personality modeling.

5.5 Manual Evaluation

Manual evaluation compares baseline and proposed responses on fluency and personality-trait appropriateness. The proposed model significantly outperforms all baselines on both metrics, while some GLBA models underperform Seq2Seq in appropriateness.

  • Evaluation Criteria: Annotators rated grammatical quality and fluency separately from whether personality-trait usage was logical and natural.Both dimensions used five-star scales, where 1 indicated poor performance and 5 indicated excellent performance.
  • Evaluation Setup: 100 posts from each of four test sets produced 2,000 responses across five models for manual evaluation.Inter-rater agreement was κ=0.82 for Fluency and κ=0.53 for Appropriateness.
  • Results: The proposed model significantly outperforms all baselines in both Fluency and Appropriateness.Three annotators scored responses on five-star scales, and the comparison was statistically significant at p-value < 0.05.
  • Results: Seq2Seq outperforms some GLBA models in Appropriateness.The authors attribute this pattern to GLBA models emphasizing a single trait in each utterance, even though different contexts may require different traits or none.

5.6 Case Study

Case studies show that the proposed model selects and expresses different personality traits according to context. Trait-attention visualizations associate gender-related language with Gender and location-related content with Location.

  • Sample Responses: The proposed model incorporates suitable traits for different contexts, unlike Seq2Seq’s universal responses and GLBA’s single-trait focus.Table 8 compares sampled responses from the proposed model and baselines, with response words color-coded by corresponding traits.
  • Trait Attention: Gender attention is higher for “come on come on,” while Location attention is higher for “still waiting for you (in Yunnan).Figure 5 visualizes attention scores from the Att. + PAB model for the generated response.

6 PRIVACY PROTECTION

PersonalDialog uses anonymization schemes intended to prevent speaker identities from being traced. Access is restricted to academic researchers who agree not to de-anonymize the data.

  • Anonymization: Speaker and post IDs are masked, dialogues containing explicit user references are removed, and additional anonymization measures are applied.The schemes follow the principle that a speaker’s identity should not be traceable.
  • Access Control: Use of PersonalDialog is restricted to academic researchers who sign a contract prohibiting de-anonymization attempts.The authors state that these restrictions further protect speakers’ privacy.

7 CONCLUSION AND FUTURE WORK

The paper introduces PersonalDialog and persona-aware generation models for dialogue with explicitly represented traits. Its evaluations show context-sensitive trait selection, while the corpus and simple models are positioned as resources for future research.

  • Contributions: PersonalDialog contains 20.83M sessions from 8.47M speakers, and the models use trait fusion, persona-aware attention, and persona-aware bias.The corpus supports explicitly represented personality traits in personalized dialogue generation.
  • Findings: Automatic and manual evaluations show that the models incorporate richer traits and select proper traits in different contexts.The paper frames these models as simple baselines because personalized dialogue research remains in its infancy.
  • Future Research: PersonalDialog is intended to support research in personalized dialogue systems, sociolinguistics, and social science.The authors present the corpus as useful beyond the specific generation task.
Loading 1901.09672v2…