CARE: Cognitive-reasoning Augmented Reinforcement for Emotional Support Conversation

1 Introduction

• ESC is a dialogue-generation task in which a supporter helps a help-seeker alleviate emotional distress through understanding, empathy, and appropriate guidance or comfort.Source: §1 · p. 1
• Existing ESC datasets often provide surface-level information without capturing the deeper cognitive reasoning needed for effective support.Prior augmentation approaches are described as relying on simple scenarios and template-based dialogues that limit simulation of complex social interactions.Source: §1 · pp. 1–2
• CARE uses the original ESC training set to guide multi-step cognitive reasoning and refines that reasoning with reinforcement learning.It aims to generate logically coherent, supportive responses without relying on large-scale synthetic data.Source: §1 · p. 2
• Experiments report that CARE outperforms strong baselines in both automatic and human evaluations, improving logical consistency and supportive quality.The paper presents these results as evidence that effective reasoning can be achieved without large-scale synthetic corpora.Source: §1 · p. 2

2 CARE Framework

• CARE enriches existing ESC conversations with structured cognitive reasoning chains and uses multi-dimensional reinforcement learning to strengthen reasoning and response quality.It builds on the original ESConv training set rather than creating new dialogues or relying on large-scale synthetic expansion.Source: §2 · p. 2

2.1 Task Formulation

• At each turn, the supporter receives the seeker’s utterance and dialogue history, then generates a response augmented with a reasoning chain and final supportive answer.The response is intended to address the seeker’s needs and provide empathetic, supportive value.Source: §2.1 · p. 3

2.2 Cognitive Reasoning Guidance

• CARE’s four reasoning nodes represent the seeker’s context, cognition, resulting emotion, and a support plan, traversed in sequence to ground the response psychologically and logically.The Support Plan node selects a supportive intention and strategy, such as reassurance, perspective, or coping mechanisms.Source: §2.2 · p. 3

2.3 Construction of CARE Reasoning Data

• DeepSeek-R1 generates four-node reasoning chains for existing ESC turns, and chains whose Support Plan conflicts with the gold strategy are discarded.This produces reasoning-augmented data without large-scale synthetic data expansion.Source: §2.3 · p. 3

2.4 Reinforcement Learning for Cognitive Reasoning

• CARE’s scalar RL reward integrates multiple signals hierarchically, and positive reinforcement is reserved for outputs with correct structure, cognitive flow, and supportive strategy.The criteria include structured reasoning/answer tags, cognitive coherence, and the selected support plan’s agreement with the gold strategy.Source: §2.4 · p. 4
• The format and coherence checks require correctly formatted output and all four valid reasoning nodes in order; the strategy reward compares the plan with the gold strategy.The cognitive-coherence reward checks node completeness and ordering, while the strategy target is annotated in the dataset.Source: §2.4 · p. 4

3.1 Experimental Setups

• Using ESConv, 8,186 reasoning-chain instances train SFT and 4,573 hard cases train RL; CARE is compared with models trained on ESConv, AugESC, and ExTES.Evaluation uses the standard ESConv test set and automatic metrics covering overlap, semantic similarity, diversity, and strategy accuracy.Source: §3.1 · pp. 4–5

3.2 Main Results

• Both CARE variants beat all baselines on most metrics; CARE (SFT-RL) leads BLEU-2, ROUGE-L, METEOR, BERTScore, and strategy accuracy, while SFT leads BLEU-1 and diversity.Table 1 reports results on the ESConv test set.Source: Table 1 · p. 5, §3.2 · p. 5

3.3 Ablation on Cognitive Reasoning Nodes

• Dropping Support Plan causes the largest diversity losses (D-1 4.73 → 3.92; D-2 27.80 → 22.58), while removing Context lowers BERTScore to 16.02.Without Context, D-1 is 4.47, D-2 is 27.07, and ACC Stra. is 30.05.Source: §3.3 · p. 5
• Removing Cognition or Emotion modestly lowers BLEU/BERTScore and ACC Stra.; Cognition’s D-2 rise to 28.43 reflects superficial variety, not better strategies.Emotion ablation yields BERTScore 16.06 and ACC Stra. 30.12.Source: §3.3 · pp. 5–6
• Removing all four reasoning nodes lowers ACC Stra. from 30.29 to 26.32 and BERTScore from 16.75 to 15.13; the full model performs best overall.The ablation removes one node at a time while keeping prompts, decoding, and data fixed.Source: §3.3 · p. 5, Table 2 · p. 6

3.4 Human Evaluation

• CARE wins against ESConv in 84.33% of comparisons, AUGESC in 91.33%, and ExTES in 68.42% of comparisons by three expert annotators.Annotators assessed 100 randomly sampled test cases; Figure 2 shows win, tie, and loss percentages.Source: Figure 2 · p. 6, §3.4 · p. 6
• The three expert annotators’ overall Fleiss’ Kappa is 0.6789, indicating substantial agreement and supporting the reported evaluation’s reliability.Source: §3.4 · p. 6

3.5 Case Study

• In the job-loss case, reference/ESConv risk invalidation, AugESC offers platitudes, ExTES stops at empathy, while CARE disputes the belief and offers recovery-oriented support.CARE’s response combines validation with a cognitive challenge and a forward-looking support plan.Source: §3.5 · pp. 6–7, Figure 3 · p. 7

• CARE strengthens ESC reasoning by using the original training data to guide logically coherent and supportive response generation.Source: §4 · p. 7
• Reinforcement learning further refines the reasoning process in CARE.Source: §4 · p. 7
• The reported improvements concern both reasoning soundness and the quality of emotional support.Source: §4 · p. 7