ConvoDrift: A Multi-Turn Conversational Dataset for Modeling Stylistic Tone Evolution

1 Introduction

• Existing style-transfer benchmarks mainly rewrite isolated sentences, leaving conversational context and progressive tone changes underrepresented (Rao and Tetreault, 2018; Roy et al., 2023).The gap is a lack of datasets tracking tone drift across turns while maintaining context and semantic intent.Source: §1 · p. 1
• CONVODRIFT separates semantic content from stylistic modification and supports both multi-turn drift supervision and persona-conditioned, pluralistic preference study.The same conversational trajectories can be reused for stylistic-dynamics modeling or preference supervision.Source: §1 · pp. 1–2
• CONVODRIFT supports persona-conditioned preference learning and training preference models that capture diverse stylistic preferences (Poddar et al., 2024).The paper reports a downstream preference-modelling task using Variational Preference Learning.Source: §1 · pp. 1–2

2.1 Stylistic Tone Variation and Drift in Text

• Sentence-level style transfer preserves content while changing attributes, but treats transformation as context-independent; dialogue-wide transfer adds context without planned turn-by-turn drift (Rao and Tetreault, 2018; Roy et al., 2023).Conversational style transfer also offers limited supervision for when changes occur and how tone evolves progressively.Source: §2.1 · p. 2

2.2 Personalised Dialogues and Alignment

• Persona-conditioned dialogue models capture user attributes, but style is generally static, while preference datasets rarely label tone or model its changes across turns (Zhang et al., 2018; Zhang et al., 2024).Existing alignment datasets often emphasize global qualities and aggregate preferences into a single reward signal.Source: §2.2 · pp. 2–3

3 Methodology

• CONVODRIFT comprises a multi-turn conversation dataset and a persona-conditioned pairwise preference dataset, reusing prompts and responses to connect drift trajectories with preference supervision.The RLHF subset restructures conversational responses into style-contrasting pairs.Source: §3 · p. 3
• 0.579 accuracy with GPT-2 representations and 0.65 with Llama 3.1 8B representations were achieved in the downstream VPL preference-modelling task.The task provides empirical evidence that the dataset supports downstream preference modelling.Source: §3 · p. 3

3.1.1 Model Selection and Generation Setup

• The generation setup selected GPT-4.1-mini for responses and GPT-5-nano for drift and direction labels, producing 15,727 conversations with six prompt–response pairs each.The models were selected after comparison across 12 LLMs on output quality, instruction adherence, latency, and estimated token cost.Source: §3.1.1 · p. 3

3.1.2 Prompt Design and Quality Control

• Prompts span professional and informal genres and enforce semantic preservation alongside controlled stylistic variation; rule-based filtering checks structural consistency and drift events.The pipeline also includes human validation and LLM-as-a-judge validation.Source: §3.1.2 · p. 3, Figure 1 · p. 4

3.1.3 Multi-turn Conversational Structure

• Each conversation has six context-conditioned prompt–response turns; the first prompt is Drift=False, and later prompts are Drift=True only when they explicitly request a style change.Semantic content is preserved across turns so the dataset captures drift trajectories rather than isolated rewrites.Source: §3.1.3 · p. 4

3.1.4 Drift, Direction Labels, and Metadata

• Each pair receives a binary Drift label for a requested tone change and a Direction label for the response’s shift relative to the previous response.Direction 1 denotes casual/expressive tone and Direction 2 formal/precise tone; the labels are evaluated independently.Source: §3.1.4 · p. 4

3.2.1 Pairwise Preference Construction

• Response pairs contrast the initial answer with a later answer showing an explicitly tagged drift direction, with adjacent or non-adjacent turns allowing different degrees of stylistic contrast.Both responses share semantic intent, so preference distinctions arise from stylistic variation rather than content differences.Source: §3.2.1 · p. 5

3.2.2 Persona-based Preference Learning

• Drift and direction tags allow stylistic preferences to be grouped and evaluated without relying on implicit preference inference.The tags include directions such as casual/expressive and formal/precise.Source: §3.2.2 · p. 5
• Five communication personas encode consistent, interpretable stylistic biases, and each independently labels which response best matches its style.Keeping persona-specific labels separate preserves differences in preferences and treats multiple communication styles as valid at once.Source: §3.2.2 · p. 5

4.1.1 Human Evaluation

• The human-validation design balances agreement estimation and coverage across 850 unique conversations, with 1,500 total conversation-level ratings.The gold set had 175 conversations rated by all annotators; the remaining subsets included 300 rated by two annotators and 375 by one.Source: §4.1.1 · p. 6
• Drift and direction labels achieve inter-annotator agreement of Drift α: 0.7628 and Direction α: 0.7772 on the common subset.Three annotators rated conversations using seven ordinal criteria; agreement metrics were computed on the 175-conversation gold set.Source: §4.1.1 · p. 6, Table 3 · p. 6

4.1.2 LLM-as-a-Judge Evaluation

• The LLM judge agrees with human references at Drift of 0.888 and Direction of 0.930 (Cohen’s κ).The remaining dataset portion was judged using the same annotation guidelines, with human majority votes as the reference.Source: §4.1.2 · p. 6

4.1.3 Automatic Evaluation

• Mean adjacent-turn similarity is 0.899, anchor similarity is 0.890, and end-to-start similarity is 0.852, indicating semantic retention across refinement turns.The three views measure local continuity, similarity to the initial response, and similarity between the final and initial responses.Source: §4.1.3 · pp. 6–7, Table 4 · p. 7
• Across genres, lexical overlap is consistently higher within non-drift transitions than at drift boundaries, while semantic intent remains preserved.Boundary drops in ROUGE-L, Jaccard-1, and Jaccard-2 indicate lexical rephrasing; semantic similarity is evaluated through adjacent-turn, anchor, and end-to-start views.Source: §4.1.3 · p. 7

4.2.1 Experimental study

• Claude Haiku 4.5 offers the reported balance of very high stability, strongest persona sensitivity, relatively low global stylistic bias, and sufficient contextual flexibility.It was selected to generate persona-conditioned preference labels; examples with semantic similarity below 0.6 were removed during pairwise dataset construction.Source: §4.2.1 · pp. 7–8

4.2.2 Relation to Agreement Analysis

• Personas A, C, and D show positive agreement with one another, while B and E show positive agreement with each other and disagreement with the other group.The differing agreement-score distributions indicate that each persona retains distinct evaluative characteristics.Source: §4.2.2 · p. 8, Figure 3 · p. 8

5 Conclusion

• CONVODRIFT is a benchmark for modelling progressive stylistic tone drift in multi-turn conversations.Source: §5 · p. 8
• Separating semantic intent from stylistic variation enables multi-turn drift trajectories and persona-conditioned preference datasets for pluralistic alignment.Source: §5 · p. 9
• Human assessment, LLM-as-a-judge, and lexical metrics confirm semantic preservation alongside structured stylistic dynamics.Source: §5 · p. 9

6 Limitations and Future Directions

• Synthetic conversations lack the variability and spontaneity of organic human–LLM interactions.The authors suggest incorporating real human conversations to improve realism.Source: §6 · p. 9
• Tone variation is limited to formal and casual styles, leaving more complex emotions and subtle shifts unexplored.Source: §6 · p. 9
• Conversations are fixed at six turns, limiting analysis of longer-range stylistic dependencies and naturally varying dialogue lengths.Source: §6 · p. 9
• Predominantly synthetic generation and LLM annotation may introduce model-specific stylistic patterns and annotation biases.Source: §6 · p. 9
• Persona conditioning uses five predefined personas, limiting the range of user preferences represented.The authors propose expanding the persona space.Source: §6 · p. 9

A Downstream Validation with VPL

• VPL treats preference learning as a latent-variable problem, conditioning reward predictions on hidden user or annotator context rather than averaging preferences.Source: A Downstream Validation with VPL · p. 10
• The experiment paired prompts and two responses with Persona A and Persona B as distinct preference contexts for training and prediction.Source: A Downstream Validation with VPL · p. 11
• 0.65 accuracy with Llama 3.1 8B representations exceeded the GPT-2-based model's 0.579 accuracy in persona-conditioned preference prediction.Source: A Downstream Validation with VPL · p. 11
• The Llama 3.1 8B t-SNE visualization shows clearer clustering into two persona-specific groups than the GPT-2 visualization.The authors interpret this separation as evidence of structured differences between persona-conditioned preference distributions.Source: Figure 4 · p. 10, A Downstream Validation with VPL · p. 11, Figure 5 · p. 11

B Prompts used for dataset generation

• Each domain uses a dedicated template to generate six prompt–response pairs, rewriting the full response at every turn based on incremental user feedback.Source: B Prompts used for dataset generation · p. 11
• The first prompt requests a no-drift draft; later revision requests introduce two to three tone-drift transitions across turns two to six.Drift is annotated at transitions and allowed only for noticeable tone changes, such as formal to casual or vice versa.Source: B Prompts used for dataset generation · p. 11
• Templates cover business email, casual email, LinkedIn posts, quotes and wishes, and tweet posts; prompts require a fixed JSON schema and prohibit placeholders.The appendix also includes a prompt template for LLM-as-a-judge evaluation.Source: Figure 6 · p. 11, B Prompts used for dataset generation · p. 11, Figure 7 · p. 12

• Positive κ values indicate similar stylistic preferences between persona pairs.Source: C Pairwise Inter-Persona Agreement · p. 12
• Negative κ values indicate consistent preferences for opposing stylistic choices between persona pairs.Source: C Pairwise Inter-Persona Agreement · p. 12
• Table 8 reports the complete pairwise Cohen’s κ agreement scores between personas.These scores underlie the persona-level distributions shown in Figure 3.Source: Table 8 · p. 12, C Pairwise Inter-Persona Agreement · p. 12

• 74.8% of conversations drift in turn two; rates fall to 52.8% in turn three, rise to 60.3% in turn four, then are roughly balanced in turns five and six.All first turns are labeled non-drift because they establish the initial conversational context.Source: Figure 12 · p. 13, D Dataset Distribution Statistics · p. 13
• Direction 1 accounts for 56.4% of conversations in turn two, while Directions 1 and 2 each represent approximately 30% in turn four.The first turn predominantly has no direction because it contains no drift.Source: Figure 13 · p. 13, D Dataset Distribution Statistics · p. 13
• Median response lengths are 127 words for business emails, 114 for LinkedIn posts, 103 for casual emails, 33 for quotes and wishes, and 39 for tweets.These differences are consistent with the genres’ expected structural and stylistic requirements.Source: Figure 14 · p. 13, D Dataset Distribution Statistics · p. 13