Modeling Multiple Support Strategies within a Single Turn for Emotional Support Conversations

1 Introduction

• Conventional ESConv work typically generates one response associated with one strategy, but this misses how a single supportive utterance can combine strategies (Tu et al. 2022; Peng et al. 2022; Cheng et al. 2022; Chen et al. 2025).The paper defines an utterance as the supporter’s full message in one turn and a response as the segment reflecting a specific strategy.Source: §1 · p. 1
• All-in-One generates all strategy-response pairs in one decoding step, while One-by-One generates them iteratively until stopping or reaching a predefined limit.Both methods use cognitive reasoning and reinforcement learning to improve strategy selection and response quality.Source: §1 · p. 2
• Utterance-level evaluation predicts the next supporter utterance from gold dialogue history, while dialogue-level evaluation tests full conversations with a simulated seeker.The dialogue-level setting assesses support maintenance, strategy adaptation, and progress toward emotional relief.Source: §1 · p. 2
• The paper argues that multi-strategy utterances are feasible and beneficial for emotional support systems, based on experiments at both utterance and dialogue levels.The experiments assess modeling multiple strategy-response pairs in supportive conversation.Source: §1 · p. 2

2.1 Multi-Strategy Utterances in ESConv

• ESConv contains 1,300 dialogues annotated with eight support-strategy types, split into 1,040 training, 130 validation, and 130 test conversations.Source: §2.1 · p. 2
• 17.7% of 15,325 ESConv supporter utterances use two or more strategies per turn, motivating multi-strategy generation rather than one-strategy-per-turn modeling.Across 1,300 conversations, supporter utterances average 11.8 turns per dialogue.Source: Table 1 · p. 2, §2.1 · p. 2

2.2 Problem Formulation

• The formulation generates an entire supportive utterance that may contain multiple strategy-aligned responses, unlike prior work predicting one response for one strategy per turn.Each response segment is associated with a specific support strategy.Source: §2.2 · p. 2

3.1 All-in-One Method

• All-in-One emits the complete utterance in one decoding process, placing each strategy before its response and concatenating the pairs.It is trained autoregressively to maximize the likelihood of the target sequence.Source: §3.1 · p. 3

3.2 One-by-One Method

• One-by-One iteratively generates a strategy, response, and termination indicator until completion or a maximum of three steps.The model conditions on the seeker’s current utterance, dialogue history, and previously generated pairs; K = 3 because nearly all validation utterances contain at most three strategies.Source: §3.2 · p. 3

3.3 Cognitive Reasoning and Reinforcement Learning

• Both methods add a four-part reasoning chain—Context, Cognition, Emotion, and Support Plan—to prompt reflection before generating the supportive utterance (Chen et al., 2025; Zhu et al., 2025).Reasoning annotations are distilled from DeepSeek-R1 (Guo et al., 2025), Qwen3-235B-A22B-Instruct-2507 (Yang et al., 2025), GPT-5, and Gemini-2.5-Flash.Source: §3.3 · p. 4
• GRPO uses a format-validity reward and strategy-sequence Levenshtein Ratio, with an additional correct-stop-flag reward for One-by-One (Shao et al., 2024; Bai et al., 2025).The rewards encourage accurate strategy prediction and, for One-by-One, appropriate termination behavior.Source: §3.3 · p. 4

4.1 Experimental Settings

• On ESConv, LLaMA-3.1-8B-Instruct is trained with SFT using LoRA and RL using GRPO; utterance quality uses BLEU-1/2/4, ROUGE-1/2/L, and BERTScore, while strategy prediction uses EMR, LR, and ADL.The Single-Strategy baseline predicts one strategy-response pair; instruction-following LLM baselines use the One-by-One method.Source: §4.1 · p. 4, §4.2 · p. 5

4.2.1 Main Results.

• One-by-One achieves the strongest utterance-level performance; its EMR is 24.99 versus 25.21 for Single-Strategy, while All-in-One reaches 23.61.Both multi-strategy methods outperform Single-Strategy overall. Cognitive reasoning improves both methods across metrics; RL improves All-in-One across metrics and most One-by-One metrics, though One-by-One BLEU scores fall slightly.Source: Table 2 · p. 5, §4.2.1 · p. 5

4.2.2 Discussion

• On single-strategy utterances, All-in-One and One-by-One slightly underperform Single-Strategy, while reasoning improves all metrics and RL further stabilizes and enhances generation.On multiple-strategy utterances, the Single-Strategy baseline performs poorly, whereas both multi-strategy methods gain clearly; reasoning and RL consistently improve strategy prediction and generation quality.Source: §4.2.2 · p. 6
• Multi-strategy outputs rise from base rates of 2.7% for All-in-One and 1.1% for One-by-One to 8.4% and 7.7% with reasoning and RL, respectively.ESConv references contain 18.9% multi-strategy utterances. Distilling from any single LLM improves All-in-One over no reasoning; no teacher leads every metric, while combining four teachers gives the most consistent and strongest overall performance.Source: §4.2.2 · p. 6, Table 4 · p. 6, Table 5 · p. 7

4.3.1 Main Results

• Fine-tuned multi-strategy models improve dialogue success over Single-Strategy without increasing average turns or strategies, and One-by-One with reasoning and RL achieves the lowest turns and highest success rate.Instruction-only LLMs have zero success within the 10-turn limit; reasoning and RL further improve the fine-tuned methods’ dialogue results.Source: Table 6 · p. 7, §4.3.1 · p. 7

4.3 Experimental Results: Dialogue-level Evaluation

• Dialogue-level self-play uses GPT-5 as seeker simulator and reward model, with average turns, success rate, and average strategies per dialogue as metrics.Success is determined by a final-turn distress score threshold, with dialogues capped at 10 turns.Source: §4.3 · p. 7

4.3.2 Discussion

• Human annotators rank both multi-strategy methods above Single-Strategy across all four dialogue dimensions; in the All-in-One RL curve, multi-strategy use rises from 7.05 to 12.50 as performance rises from 32.31 to 34.62.The RL curve first shows a slight decline in multi-strategy usage and dialogue success through step 4, followed by increases in both.Source: Table 7 · p. 8, Figure 3 · p. 8, §4.3.2 · p. 8

Related Work

• ESConv, proposed by Liu et al. (2021), is a widely used ESC benchmark, and most prior work combines strategy prediction with strategy-constrained response generation.These formulations associate each supporter turn with a single strategy.Source: §5 · p. 8
• Prior approaches use commonsense knowledge, multi-turn strategy planning, explicit reasoning, or synthetic data augmentation to improve emotional understanding, support planning, and model robustness.These lines include Tu et al. (2022), Peng et al. (2022), Cheng et al. (2023), Hao and Kong (2025); Cheng et al. (2022), Zhao et al. (2023); Chen et al. (2025), Zhu et al. (2025); Zheng et al. (2023), and Ye et al. (2025).Source: §5 · p. 8
• Nearly all prior studies restrict each turn to one strategy-response pair; Bai et al. (2025) is identified as the only work explicitly examining multi-strategy generation within a turn.Whether multiple strategies benefit emotional support conversations remains unclear, motivating this paper’s research question.Source: §5 · p. 8

• Across utterance-level and dialogue-level ESConv evaluations, both methods consistently outperform the Single-Strategy baseline.The results support the feasibility and benefit of modeling multiple strategies within a supportive utterance.Source: §6 · p. 8
• All-in-One generates a complete supportive utterance in one pass, while One-by-One iteratively predicts strategy-response pairs.Both methods are enhanced with cognitive reasoning and reinforcement learning.Source: §6 · p. 8

• Generated utterances with multiple strategies remain less frequent than in ESConv, potentially limiting response diversity and richness.Source: Limitations · p. 9
• Dialogue-level assessment relies on simulated seekers, which may not capture the nuances of real human emotional interactions.Source: Limitations · p. 9
• The study focuses on ESConv, so evaluation on other emotional support datasets or real-world deployment is needed to assess generalizability.Source: Limitations · p. 9

Ethical Considerations

• The system is not designed or validated for clinical use and must not replace professional care; deployment requires safeguards for high-risk or crisis situations.The authors caution against misuse in these settings.Source: Ethical Considerations · p. 9
• Reasoning and reinforcement learning do not guarantee safety from inappropriate, biased, or misleading outputs, so real-world use requires content moderation and human oversight.Developers should also account for biases inherited from training data and use outputs responsibly.Source: Ethical Considerations · p. 9
• Experiments used only the open-access ESConv benchmark, which contains no personally identifiable or sensitive user information; no additional user data were collected.The study states that it followed standard privacy and data protection principles.Source: Ethical Considerations · p. 9
• Three trained institutional annotators with dialogue-assessment experience participated voluntarily, received fair compensation under local guidelines, and provided no personal or sensitive information.The authors characterize payment as fair for the time required.Source: Ethical Considerations · p. 9
• Although composing multiple support strategies may improve supportive dialogue, the authors caution against undue reliance on automated systems for emotional or mental health needs.Source: Ethical Considerations · p. 9

A Detail of Cognitive Reasoning

• Following Zhu et al. (2025), the reasoning chain models context, cognition, emotion, and support planning, grounding circumstances, appraisals, affect, and intended supportive action.The support plan translates psychological understanding into a concrete response.Source: A Detail of Cognitive Reasoning · pp. 10–11
• The structured reasoning chain guides both strategy selection and response generation in the models.Source: A Detail of Cognitive Reasoning · p. 11

B Training Details

• Table 8 reports the number of instances collected for SFT and RL, but the supplied passage does not state the counts.Source: B Training Details · p. 11
• SFT uses LoRA rank 8 and scaling factor 16 for All-in-One, versus rank 16 and scaling factor 32 for One-by-One.Source: B Training Details · p. 11
• SFT runs for 5 epochs at a 3e-5 learning rate on four NVIDIA A100 80GB GPUs, with per-device batch size 4 and gradient accumulation over 2 steps.Source: B Training Details · p. 11
• RL further optimizes the SFT models with VERL and GRPO for 7 epochs, using batch size 1024, rollout size 16, learning rate 1e-6, and KL-penalty coefficient 0.01.Source: B Training Details · p. 11

C Performance of Instruction-following LLMs using the All-in-One Method

• The utterance-level evaluation of instruction-following LLMs using All-in-One is presented in a table, but the supplied passage gives no results.Source: C Performance of Instruction-following… · p. 11

D Case Study

• DeepSeek-R1 uses validation without visible deliberate reasoning, Single-Strategy gives advice without reflection, and the reasoning-enhanced methods combine node-based reasoning with empathy and action guidance.All-in-One integrates situational, interpretive, and emotional considerations into a cohesive support plan; One-by-One separates emotional alignment from problem-solving, potentially improving receptivity and trust.Source: D Case Study · p. 11
• In friendship-conflict dialogues, the reasoning-enhanced methods deliver comparable or greater strategic diversity in fewer turns while maintaining empathetic, constructive engagement.The case study holds the help-seeker scenario constant to make differences in strategy use and reasoning apparent.Source: D Case Study · p. 11

E Prompt Details

• The appendix provides prompts used across models and experimental evaluation settings.Source: E Prompt Details · p. 12

E.1 Single-Strategy Baseline

• The Single-Strategy baseline prompt predicts one support strategy and its corresponding response for each supporter turn.Source: §E.1 · p. 12

E.2 All-in-One Method

• The All-in-One prompts cover generation without and with cognitive reasoning, plus distillation of reasoning chains from teacher LLMs.Source: §E.2 · p. 12

E.3 One-by-One Method

• The One-by-One prompts cover generation without and with cognitive reasoning, plus reasoning distillation from large LLMs.Source: §E.3 · p. 12

E.4 Dialogue-Level Evaluation

• Dialogue-level self-play prompts simulate the seeker and have a critic assess dialogue progress and success.Source: §E.4 · p. 12

E.5 Preparation for Human Evaluation

• Human-evaluation preparation uses prompts for profile extraction and seeker simulation, adapted from Zhang et al. (2025).Source: §E.5 · p. 12

F Guideline of Human Evaluation

• Figure 16 presents the human-evaluation guideline adapted from Zhang et al. (2025).The supplied passages identify the guideline but do not describe its evaluation criteria.Source: F Guideline of Human Evaluation · p. 12, Figure 16 · p. 22