Source-linked AI summary
Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach
Ashish Sharma, Inna W. Lin, Adam S. Miner, David C. Atkins, Tim Althoff
TL;DR
Online mental health support needs more empathic conversations, yet highly empathic interactions are rare. The paper introduces empathic rewriting and PARTNER, a reinforcement-learning system for sentence-level edits. Automatic and human evaluations show more empathic, specific, and diverse responses than related methods, with safety and framework scope remaining important boundaries.
Problem
Highly empathic conversations are rare online, motivating methods that improve empathy while preserving fluency and conversational specificity.
Method
PARTNER uses reinforcement learning to generate and position sentence-level additions or replacements, trained with rewards for empathy, fluency, specificity, and diversity.
Results
PARTNER generates more empathic, specific, and diverse responses and outperforms related baselines, including by >35% in empathy improvement.
Takeaways & Limitations
Empathic rewriting is presented as a proof of concept for computational feedback and support in online peer-support settings.
Takeaways & Limitations
The approach uses a theoretically grounded empathy framework and excludes predefined unsafe posts, while future deployment requires further safety and bias assessment.
Abstract
from arXiv · showhide
Online peer-to-peer support platforms enable conversations between millions of people who seek and provide mental health support. If successful, web-based mental health conversations could improve access to treatment and reduce the global disease burden. Psychologists have repeatedly demonstrated that empathy, the ability to understand and feel the emotions and experiences of others, is a key component leading to positive outcomes in supportive conversations. However, recent studies have shown that highly empathic conversations are rare in online mental health platforms. In this paper, we work towards improving empathy in online mental health support conversations. We introduce a new task of empathic rewriting which aims to transform low-empathy conversational posts to higher empathy. Learning such transformations is challenging and requires a deep understanding of empathy while maintaining conversation quality through text fluency and specificity to the conversational context. Here we propose PARTNER, a deep reinforcement learning agent that learns to make sentence-level edits to posts in order to increase the expressed level of empathy while maintaining conversation quality. Our RL agent leverages a policy network, based on a transformer language model adapted from GPT-2, which performs the dual task of generating candidate empathic sentences and adding those sentences at appropriate positions. During training, we reward transformations that increase empathy in posts while maintaining text fluency, context specificity and diversity. Through a combination of automatic and human evaluation, we demonstrate that PARTNER successfully generates more empathic, specific, and diverse responses and outperforms NLP methods from related tasks like style transfer and empathic dialogue generation. Our work has direct implications for facilitating empathic conversations on web-based platforms.
1 INTRODUCTION
The paper introduces empathic rewriting to improve empathy in online mental health support while preserving conversational quality. PARTNER uses reinforcement learning to make sentence-level edits, and evaluations show stronger empathy and quality than related baselines.
- Online mental health support platforms serve millions, but highly empathic conversations remain uncommon despite empathy’s association with positive support outcomes.
- Empathic rewriting transforms low-empathy responses into higher-empathy responses by adding or replacing sentences that communicate understanding and felt emotion.The motivating example adds recognition of the seeker’s experience and an expression of compassion.
- The task requires understanding nuanced feelings while maintaining fluency, context specificity, diversity, and changes beyond simple word-level transformations.
- PARTNER is a deep reinforcement learning agent that identifies edit positions and generates empathic sentences for insertion or replacement.
- >35% empathy improvement over related baselines, while producing fluent, specific, and diverse rewritings.Experts in clinical psychology also preferred PARTNER’s rewritings over baselines on empathy, specificity, and fluency.
2 RELATED WORK
Prior work studies supportive-conversation strategies, empathic dialogue generation, and general text rewriting. This paper distinguishes empathic rewriting by editing existing mental-health responses and jointly selecting edit locations and generated sentences.
- NLP research on online mental health support has examined conversational support, therapeutic actions, counselor language, engagement, moderation, and cognitive restructuring.
- Empathic rewriting modifies existing responses rather than generating new responses from scratch, focusing on online mental health support instead of mainly open-domain chit-chat.
- Unlike approaches centered on emotional grounding or emotion mimicking, this work leverages emotional and cognitive aspects of empathy valued in mental health support.
- The task differs from style transfer and other rewriting tasks because it requires contextual empathy understanding alongside fluency, specificity, and diversity.
- The proposed RL agent jointly identifies where improvements are needed and generates empathic sentences for insertion or replacement.
3 DATASET DESCRIPTION
The dataset is built from TalkLife conversations, filtered for mental-health relevance and annotated with computational empathy measurements. It combines a large non-parallel corpus with a small expert-rewritten parallel evaluation set, while applying safety and privacy safeguards.
- TalkLife is a peer-to-peer mental health platform containing seeker posts and peer-supporter response posts in thread-like conversations.
- A BERT classifier trained on approximately 3k manually annotated posts filtered the corpus to 3.33M interactions from 1.48M seeker posts.
- The authors create a large non-parallel dataset with empathy measurements because a large expert-labeled parallel corpus would be difficult to collect.
- Empathy labels cover Emotional Reactions, Interpretations, and Explorations, while high levels of empathy are especially rare for Interpretations and Explorations.
- The evaluation includes 180 pairs of low-empathy and expert-rewritten high-empathy responses produced by six graduate students in clinical psychology.
- The study removes unsafe posts using predefined expressions, removes personally identifiable information, obtains institutional review approval, and makes no treatment or diagnostic claims.
4 PROBLEM DEFINITION AND GOALS
Empathic rewriting transforms existing low-empathy responses into more empathic ones while preserving conversational quality and making precise, useful changes.
- 4.1 Empathic Rewriting: Empathic rewriting transforms existing low-empathy conversational posts into higher-empathy posts rather than generating responses from scratch.The task is intended for realistic support settings that augment rather than replace human contributions.
- 4.2 Goals: The goals require theoretically grounded empathy because empathy includes multiple aspects, including emotional reactions and cognitive understanding of others’ feelings and experiences.The paper contrasts warmth and compassion with communicating understanding of feelings and experiences.
- 4.2 Goals: The task must balance empathy with fluency, sentence coherence, context specificity, and response diversity.Generic empathic responses may lack specificity, while poorly integrated edits can produce ungrammatical or incoherent posts.
- 4.2 Goals: Rewriting should make minimal, precise changes that remain suitable for the original response and support machine-in-the-loop feedback and training.A special stopping action lets the agent learn when to stop editing, avoiding unnecessarily large transformations.
5 PARTNER: EMPATHIC REWRITING USING REINFORCEMENT LEARNING
PARTNER uses reinforcement learning to edit responses sentence by sentence, selecting edit locations and generating empathic sentences while rewarding quality-preserving transformations.
- 5 PARTNER: PARTNER is a reinforcement learning model that learns to add empathic sentences or replace existing response sentences with more empathic ones.The agent operates on seeker-response pairs and simultaneously identifies where edits are needed and generates empathic sentences.
- States and actions: The agent represents states with a seeker post and a fixed-length response span, enabling context-sensitive transformations and a static action set.Insertion, replacement, and deletion are sentence-level actions; deletion is implemented by replacing a sentence with an empty string.
- Policy: At each step, PARTNER selects an insertion or replacement position and generates a candidate empathic sentence using a transformer language model based on GPT-2.The position action includes insertion, replacement, and a stopping action; the sentence action generates arbitrary-length candidates.
- Rewards: The reward design favors increased empathy while maintaining fluency, sentence coherence, context specificity, and diversity.Empathy is scored with a classifier on a 0–6 scale, fluency uses inverse perplexity, and coherence measures average coherence between the candidate and response sentences.
- Rewards: Sentence coherence is rewarded separately because fluent individual sentences may still be incoherent with the surrounding response.The coherence reward estimates the probability that candidate sentences form coherent pairs with existing response sentences.
6 EXPERIMENTS
The experiments evaluate Partner against dialogue-generation and sequence-to-sequence baselines, ablations, and expert rewritings using automatic and human measures. Partner generally improves empathy while preserving specificity, diversity, and fluency, adapts edits to initial empathy, and benefits from reinforcement learning.
- Baseline Results: Partner achieves the largest empathy improvement, 35% more than MIME, while producing more specific outputs than all baselines.Table 2 evaluates empathy improvement, fluency, specificity, diversity, and edit rate.
- Baseline Results: BART has the lowest perplexity and highest diversity but produces a 0.06 overall drop in empathy.Its fluent and diverse outputs therefore do not satisfy the empathic rewriting objective.
- Adaptability of rewritings to original post: Partner improves low-empathy responses, avoids deteriorating highly empathic ones, and achieves these changes with the fewest edits across both empathy levels.Figure 4 analyzes empathy change and edit counts by original response empathy.
- Ablation Results: RL training improves empathy improvement by +0.8929 over warm-start-only training, while coherence and mutual-information rewards add approximately +0.03 in empathy.Warm-start alone is +0.2783 above DialoGPT, but remains much worse than the full Partner model.
- Results: Baselines and ablations: Human evaluators prefer Partner over baselines and ablations for empathy and specificity, while judging it competitive in fluency.Partner has approximately 55% preference for empathy over ablations without coherence and mutual-information rewards (p<0.01).
- Results: Expert rewritings: Experts are preferred approximately 80–90% of the time over Partner in empathy, fluency, and specificity, although Partner is preferred in 10–20% of cases.The expert comparison uses 180 empathic rewritings as a gold-standard evaluation set.
7 DISCUSSION AND CONCLUSION
The paper presents empathic rewriting and PARTNER as a proof-of-concept for supporting peer supporters and providing more empathic responses online. It frames this machine-in-the-loop approach as a way to combine human context with AI assistance while recognizing safety risks in high-stakes mental health settings.
- 7 DISCUSSION AND CONCLUSION: Online mental health platforms offer one route to scaling support, but effectively training or scaffolding peer supporters remains a major challenge.The paper motivates empathic rewriting as one computational approach to this training and scaffolding problem.
- 7 DISCUSSION AND CONCLUSION: Machine-in-the-loop rewriting can provide peer supporters with actionable, real-time feedback and support seekers with more empathic responses.The approach combines human knowledge of context and experience with computational rewriting assistance.
- 7 DISCUSSION AND CONCLUSION: The approach is intended to balance potential benefits of artificial intelligence in mental health with risks related to toxicity and safety.The paper specifically discusses settings involving suicidal ideation, self-harm, and insensitive comments related to race, ethnicity, or gender.
- 7 DISCUSSION AND CONCLUSION: The paper proposes empathic rewriting and PARTNER as a foundational proof-of-concept for scaling support on online mental health platforms.PARTNER makes sentence-level edits to transform low-empathy posts into more empathic ones and outperforms baseline methods in automatic and human evaluations.