PaperlayerView exact revision on arXiv

Source-linked AI summary

SweetieChat: A Strategy-Enhanced Role-playing Framework for Diverse Scenarios Handling Emotional Support Agent

Jing Ye, Lu Xiang, Yaping Zhang, Chengqing Zong

arXiv:2412.08389v1cs.CL

TL;DR

LLM emotional-support responses can be verbose and formulaic, while building diverse multi-turn datasets manually is challenging. The paper uses three-role strategy-enhanced role-playing to construct ServeForEmo and fine-tune SweetieChat; experiments and human evaluations report supportive responses across diverse open-domain scenarios aligned with human preference.

  • Problem

    LLM responses can be verbose and formulaic, and constructing high-quality multi-turn emotional-support datasets manually is challenging, limiting scenario scale and diversity.

    Source: §1 · p. 1

  • Method

    The framework simulates Seekers, Strategy Counselors, and Supporters to generate ServeForEmo, then fine-tunes LLaMA to create SweetieChat.

    Source: §1 · p. 2

  • Results

    Role-playing dialogues significantly enhance supportive-response ability in alignment with human preference, and SweetieChat excels across diverse, open-domain scenarios.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The framework offers a procedure for creating diverse, effective emotional-support conversations.

    Source: §6 · p. 8

    Data construction can include strategy-following and character-background consistency errors, and scaling supervised fine-tuning data is only a temporary solution.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Language Models (LLMs) have demonstrated promising potential in providing empathetic support during interactions. However, their responses often become verbose or overly formulaic, failing to adequately address the diverse emotional support needs of real-world scenarios. To tackle this challenge, we propose an innovative strategy-enhanced role-playing framework, designed to simulate authentic emotional support conversations. Specifically, our approach unfolds in two steps: (1) Strategy-Enhanced Role-Playing Interactions, which involve three pivotal roles -- Seeker, Strategy Counselor, and Supporter -- engaging in diverse scenarios to emulate real-world interactions and promote a broader range of dialogues; and (2) Emotional Support Agent Training, achieved through fine-tuning LLMs using our specially constructed dataset. Within this framework, we develop the \textbf{ServeForEmo} dataset, comprising an extensive collection of 3.7K+ multi-turn dialogues and 62.8K+ utterances. We further present \textbf{SweetieChat}, an emotional support agent capable of handling diverse open-domain scenarios. Extensive experiments and human evaluations confirm the framework's effectiveness in enhancing emotional support, highlighting its unique ability to provide more nuanced and tailored assistance.

1 Introduction

The introduction identifies limited diversity and strategic guidance in emotional-support conversations and proposes strategy-enhanced role-playing to build ServeForEmo and train SweetieChat. The authors report that SweetieChat provides emotional support across diverse, open-domain scenarios.

1 Introduction

  • Existing LLM responses can be verbose and formulaic, while manual construction of high-quality multi-turn support datasets limits their scale and scenario diversity.The paper also identifies repetitive dialogues and insufficient strategic guidance as challenges in LLM-augmented datasets.Source: §1 · pp. 1–2
  • The framework uses Seekers, Strategy Counselors, and Supporters to generate emotional-support dialogues, then fine-tunes LLaMA on the resulting dataset.The roles are intended to support diverse interactions and well-reasoned strategies.Source: §1 · p. 2
  • ServeForEmo contains 3.7K+ dialogues and 62.8K+ utterances across a wide range of emotional-support scenarios.The paper reports its scale as approximately three times that of ESConv.Source: §1 · p. 2
  • SweetieChat excels at providing emotional support across diverse, open-domain scenarios, according to experiments and comprehensive human evaluations.The introduction does not provide specific metric values for this result.Source: §1 · p. 2

2 Related Work

Prior emotional-support datasets face scale or diversity and strategy-guidance challenges, while role-playing research offers a way to broaden dialogue data. The paper situates its work across these two lines.

2.1 Emotional Support Conversation

  • The first manually curated emotional-support dataset was limited in size and scope because its creation required substantial expertise and time.Later LLM-based augmentation approaches modify, complete, or generate dialogues, but often retain limited diversity or strategic guidance.Source: §2.1 · p. 2
  • LLM-based augmentation includes rewriting single-turn data, completing dialogues, imitating existing datasets, and adding emotion and strategy reasoning.These approaches can still face limited diversity or insufficient strategic guidance.Source: §2.1 · p. 2

2.2 Role-playing

  • Role-playing agents have been used for personalized dialogue generation and evaluation; the authors draw on diverse roles to enhance emotional-support data diversity.Examples include persona-driven casual dialogue datasets and role-play agents interacting with emotional-support models.Source: §2.2 · p. 3

3 Method

Section 3 describes a three-role framework for generating emotional-support dialogues and training SweetieChat. It diversifies Seeker scenarios, uses a Counselor to select strategies from dialogue history, and guides a Supporter’s responses with those strategies.

3.1 Role Construction

  • The Seeker represents the user, the Supporter provides emotional support, and the Strategy Counselor selects strategies to guide the Supporter.Source: §3.1 · p. 3

3.1.1 Seeker Construction

  • Seeker diversity comes from expanding 13 ESConv problem types to 45, generating specific scenarios, and tailoring profiles to each problem and scenario.The method uses 1,000 seed triplets and 100 handcrafted profiles, then generates new scenarios and profiles with GPT-4o.Source: §3.1.1 · pp. 3–4

3.1.2 Counselor Construction

  • Because LLMs tend to favor reassurance and suggestions over deeper interaction, a Counselor fine-tuned on manually annotated ESConv selects strategies from dialogue history.The paper frames emotional-support conversations as exploration, comforting, and action stages.Source: §3.1.2 · p. 4

3.1.3 Supporter Construction

  • GPT-4o generates emotionally supportive replies using the dialogue history and case dialogues, guided by the Counselor’s selected strategy.Source: §3.1.3 · p. 4

3.2 Role-playing Dialogue Generation

  • The role-playing interactions pair Seekers from diverse backgrounds with a Supporter, while the Counselor provides strategy guidance and a self-iterative process extends the seed pool.The generation description identifies a randomly selected problem type, a newly generated scenario, and a simulated emotional-support conversation.Source: §3.2 · p. 4

3.3 SweetieChat Agent

  • SweetieChat is built by fine-tuning LLaMA on ServeForEmo for emotional support across diverse seekers and scenarios.Source: §3.3 · p. 5

4 Data Analysis

ServeForEmo combines broad dataset scale with diverse dialogue and strategy patterns: its dialogues span varied scenarios, and strategy use shifts across conversation phases while remaining varied within individual dialogues.

4.1 Statistics (Q1)

  • ServeForEmo contains 3.7K+ dialogues and 62.8K+ utterances, approximately 3× the scale of ESConv.Post-processing filters low-quality dialogues, removes redundant greetings, and manages dialogue length; utterances are limited to three sentences, making dialogues and utterances generally shorter than ESConv. Table 1’s ESConv statistics exclude initial greetings and merge consecutive utterances from the same speaker.Source: §4.1 · p. 5, Table 1 · p. 5

4.2 Diversity Analysis (Q2)

  • ServeForEmo shows greater inter-dialogue diversity than crowdsourced ESConv, while directly prompted ExTES displays minimal diversity.The analysis measures cosine similarity between dialogue pairs using TF-IDF features; lower similarity indicates greater diversity.Source: §4.2 · p. 5, Figure 3 · p. 5
  • Responses guided by the same strategy still have low cosine similarity, which the authors interpret as evidence of scenario tailoring.Source: Figure 4 · p. 5, §4.2 · p. 6

4.3 Strategy Analysis (Q3)

  • ServeForEmo and ESConv have similar strategy distributions, concentrated in Affirmation and Reassurance, Providing Suggestions, and Questioning.The authors take this similarity as evidence that the Strategy Counselor selects and assesses strategies effectively.Source: §4.3 · p. 6, Table 2 · p. 6
  • Questioning is used mainly early, Providing Suggestions grows more prominent later, and Affirmation and Reassurance and Reflection of Feelings persist throughout.This pattern is described as reflecting Hill’s (2009) three-stage helping skill model.Source: §4.3 · p. 6, Figure 5 · p. 6

Distribution of Unique Strategies in Dialogues

5 Experiments

SweetieChat is evaluated against LLaMA and dataset-finetuned baselines using automatic and human assessments. It performs strongly across automatic metrics and generally outperforms the compared systems in human evaluation, while new-scenario ratings reveal areas for improvement.

5.1 Baseline Models

  • The baselines are LLaMA3-8B-Instruct, AUGESC, ExTES, and ESConv; SweetieChat is LLaMA fine-tuned on ServeForEmo.Source: §5.1 · p. 7

5.2 Implementation Details

  • Training uses LoRA with rank 8, AdamW at 3 × 10−5, five epochs, and early stopping; reported results average three runs.The best validation-set model is selected for testing, and experiments use eight Tesla V100 GPUs.Source: §5.2 · p. 7

5.3 Evaluation Settings

  • Evaluation uses the manually annotated ESConv test set, with generated-context and reference-context interactive settings and automatic and human metrics.Human dimensions include Empathy, Informativeness, Coherence, Suggestion, Understanding, Helpfulness, and Overall Quality.Source: §5.3 · p. 7, §5.4.1 · p. 7

5.4.1 Automatic Evaluation Results

  • ServeForEmo is stronger overall across both automatic evaluation settings, with the highest BLEU-2 and BLEU-4, BERT-Score 85.81, and Distinct2 and Distinct3 scores of 99.59 and 99.89.ExTES is competitive on R-2 and R-L, but ServeForEmo is reported to perform better overall across both settings.Source: §5.4.1 · pp. 7–8, Table 3 · p. 8

5.4.2 Human Evaluation Results

  • Human evaluators generally rate ServeForEmo responses above AUGESC, ESConv, and ExTES, consistent with the automatic evaluation pattern.The human comparison uses responses to the same dialogue history and randomizes response order; 50 dialogues were sampled.Source: §5.4.2 · p. 8

5.5 Evaluation on New Scenarios

  • In interactions with 10 volunteers on new scenarios, ServeForEmo performs slightly better than ExTES, while both show weaknesses in Suggestion, Helpfulness, and Informative ratings.Ratings use a five-point Likert scale, with higher scores indicating better quality.Source: §5.5 · p. 8

6 Conclusion

The paper concludes that strategy-enhanced role-play generates diverse emotional-support dialogues and improves downstream support, while identifying limitations in data quality, evaluation, alignment, and modality.

6 Conclusion

  • Role-play dialogues significantly enhanced supportive responses aligned with human preference, according to the paper’s conclusion.Figure 6 summarizes human-evaluation win, tie, and loss rates on ESConv across 10 volunteers; Fleiss’ kappa indicates fair or moderate agreement (0.4 < k < 0.8).Source: §6 · p. 8, Figure 6 · p. 9

Limitations

  • The authors identify data-construction errors, evaluation subjectivity, reliance on supervised fine-tuning, and text-only interactions as limitations.They cite inconsistent strategy and character adherence, variation in evaluators’ perceptions, and evaluators’ lack of firsthand experience; they plan to explore human-chatbot preference alignment and speech.Source: Limitations · p. 9

A Details of Problem Types

  • ServeForEmo expands ESConv’s 13 problem types to 45 across five categories, with at least seven types per category and random scenario selection.The listed categories include emotional and mental health issues, life and work stress, and interpersonal relationships; selection simulates a pre-counseling questionnaire.Source: A Details of Problem Types · p. 11

B Details of Strategy Counselor Construction

The strategy counselor is designed to guide supporters, addressing differences between LLM and human strategy preferences; the dataset also undergoes postprocessing for consistency and length.

B Details of Strategy Counselor Construction

  • In 100 GPT-4o counselor dialogues, LLMs favored offering hope, affirmation and reassurance, and suggestions, unlike the more nuanced human approach.The authors describe this tendency to rush solutions and empathy as contributing to an “AI flavor” in emotional-support interactions.Source: B Details of Strategy Counselor… · pp. 11–12
  • The authors fine-tune Meta-Llama3-8B-Instruct on manually annotated ESConv to improve its strategy-selection capability.They use a question-and-indefinite-choice task with prompts matching the Counselor’s view.Source: B Details of Strategy Counselor… · p. 12

C.1 Dataset Postprocessing

  • Postprocessing removes role-inconsistent and greeting-heavy dialogues, filters conversations that are too short or long, and limits utterances to three sentences.Role inconsistencies are addressed through systematic sampling and inspection across problem types.Source: §C.1 · p. 12

C.2 Dataset Quality Evaluation

Human evaluation reports that ServeForEmo dialogues maintain role responsibilities and conversational quality, and a scenario comparison indicates alignment between dialogue content and settings.

C.2 Dataset Quality Evaluation

  • The dataset evaluation used seven quality metrics on a five-level Likert scale, with three annotators assessing 50 randomly selected dialogues.Higher scores indicate superior quality; the metrics include coherence, consistency, helpfulness, informativeness, rationality, understanding, and safety.Source: §C.2 · pp. 12–13, Table 5 · p. 13
  • Evaluation of 50 dialogues found that seekers articulated concerns, counselors gave strategic guidance, and supporters followed strategies in coherent, informative, empathetic conversations.Three annotators with psychology and computational linguistics backgrounds rated dialogues on seven metrics using a five-level scale.Source: §C.2 · p. 13
  • TF-IDF cosine similarities between scenarios and their dialogues indicate that dialogue content correlates highly with scenario settings.The authors interpret this as evidence that dialogues adhere closely to prescribed scenarios.Source: §C.2 · p. 13

C.3 Data Example from ServeForEmo

  • ServeForEmo examples label each dialogue’s problem type, scenario, seeker profile, and supporter strategies.Source: §C.3 · p. 13

D Dataset Lexical Diversity Analysis

Distinct-n analysis finds ServeForEmo’s lexical diversity comparable to ESConv and higher than ExTES, under a comparison using equal-sized dialogue samples.

D Dataset Lexical Diversity Analysis

E Additional Experiments

Training on both ESConv and ServeForEmo produced the highest scores on nearly all metrics, while the human evaluations assessed dialogue quality and supporter responses.

E Additional Experiments

  • Models trained on both ESConv and ServeForEmo scored highest on nearly all metrics, and ServeForEmo-only training transferred well to ESConv.The authors report that training on both datasets outperformed training on either one or neither; the comparisons are in Table 7.Source: E Additional Experiments · p. 13

F Human Evaluation Metrics

  • Human evaluation assessed both dataset quality and response effectiveness.The evaluation included separate dataset and supporter-response metrics.Source: F Human Evaluation Metrics · p. 13

F.1 Dataset Evaluation Metrics

  • The dialogue-quality evaluation added Rationality to assess the Strategy Counselor’s performance.It follows Zheng et al. (2023) and adds Rationality as a criterion.Source: §F.1 · p. 13

F.2 Response Evaluation Metrics

The supplementary materials describe response-evaluation criteria and role-playing prompts, then present case studies and examples of chatbot interactions. The case-study passage reports that SweetieChat’s responses are concise and use varied supportive strategies.

F.2 Response Evaluation Metrics

  • The response questionnaire evaluates Informativeness, Suggestion, Empathy, Understanding, Helpfulness, Coherence, and Overall Quality.Figure 11 shows an example questionnaire comparing different chatbots.Source: §F.2 · p. 13, Figure 11 · p. 19

G Prompt Demonstration

  • The role-playing prompts specify situation, Seeker-profile, Counselor, and Supporter construction, and cap dialogue history at six utterances.Figure 12 depicts the scenario/profile construction and interaction among the three roles.Source: G Prompt Demonstration · p. 14, Figure 12 · p. 20

H Case study

  • SweetieChat is described as excelling in supportive responses and empathy on ESConv test examples, with concise, strategy-varied open-domain interactions.The discussion points to Figures 13–16 for the evaluation and volunteer interaction examples.Source: H Case study · p. 14, Figure 13 · p. 21, Figure 14 · p. 22

Example Dialogue from ServeForEmo

Guideline of Dialogue Quality Evaluation

  • Figure 10 presents the guideline used for dialogue-quality evaluation.Source: Figure 10 · p. 18

Chatbot Performance Evaluation Survey

Interaction with ExTES

  • In the ExTES interaction, the bot validates relationship-related frustration and suggests communication, cooling down, and considering relationship health.The transcript also shows repeated prompts for the user to explain their experience.Source: Interaction with ExTES · p. 23, Figure 15 · p. 23

Related papers

Loading 2412.08389v1…