Understanding Client Reactions in Online Mental Health Counseling

1 Introduction

• Feedback helps communicators understand their partners’ states and adjust strategies to the communication context.This is presented as important to successful communication.Source: §1 · p. 1
• Prior counseling research mainly studies counselors’ strategies, leaving client reactions and their role in conversation trajectories less examined.Assessing interventions from the client’s perspective is motivated by the difficulty of identifying reactions in counseling.Source: §1 · pp. 1–2
• The paper introduces a theoretically grounded framework mapping conversation turns to counselors’ intentions and clients’ reactions.It applies the framework to a large-scale text-based Chinese counseling dataset collected over two years from an online welfare platform.Source: §1 · p. 2
• The study examines how counselor strategies elicit client reactions, how reactions relate to conversation effectiveness, and how counselors can adjust strategies.It also analyzes strategies for encouraging positive behaviors and responding to behaviors that negatively affect effectiveness.Source: §1 · p. 2

2 Related Work

• Prior NLP studies commonly focus on consultation skills, while large-scale studies of human-human interaction are scarce.Available mental-health conversation datasets often come from public social networks or crowdsourcing, limiting their use for studying interaction patterns.Source: §2 · p. 2
• Compared with the related text-based crisis counseling corpus, the paper reports longer conversations—78 versus 42 utterances—and emphasizes client reactions.Althoff et al. (2016) is described as focusing mostly on counselors’ utterances.Source: §2 · p. 2
• Unlike client-response classification limited to motivational interviewing’s three-class scheme, this work presents a genre-independent, finer-grained annotation approach.The paper relates its approach more closely to psychotherapy research.Source: §2 · p. 3
• Psychotherapy research widely investigates counselor-client interactions and working alliance, but many analyses use small samples or session-level data.This paper instead conducts moment-by-moment analysis on a large-scale dataset and pursues an automatic solution.Source: §2 · p. 3

3 Annotation Framework

• The framework adapts established face-to-face counseling taxonomies to online text conversations, with counselor interventions linked to client reactions and feedback informing strategy adjustment.Its development involved counseling psychology experts and three developers using a consensual qualitative research method.Source: §3 · p. 3, Figure 2 · p. 3

3.1 Counselor Intentions and Conversational Strategies

• Counselor intentions are classified as Supporting or Challenging, and the strategy taxonomy contains eight Supporting and four Challenging strategies.The paper describes counselors as alternating supportive and challenging strategies in practice.Source: §3.1 · pp. 3–4
• Supporting strategies include reflecting clients’ content or feelings and affirming strengths, while challenging strategies can confront beliefs or invite exploration of new actions.These examples illustrate different ways counselors convey their intentions.Source: §3.1 · p. 4
• The framework distinguishes strategies that share a broad aim by how counselors provide insight: Interpretation offers an explanation, while Invite to Take New Perspectives asks questions.Source: §3.1 · p. 4

3.2 Client Reactions and Behaviors

• Client reactions are categorized as Positive or Negative feedback on whether counselor intentions and strategies achieve their intended purposes.Negative reactions indicate that intentions may not have succeeded as expected and can signal a need to adjust subsequent strategies.Source: §3.2 · p. 4
• The taxonomy includes five positive and six negative client behavior types; examples include Confirming or Extending versus Defending or Changing Topics.These behaviors can follow an Interpretation strategy.Source: §3.2 · p. 4

4 Data Collection

• The framework maps each conversation turn to counselor intentions and client reactions and is applied to a large-scale text-based Chinese counseling dataset.The corpus was collected from an online welfare counseling platform, and the dataset is intended to be available under ethical guidelines.Source: §4 · p. 4

4.1 Data Source

• The corpus contains 2,382 sessions, including 479 with client self-reported effectiveness scores, and averages 78.49 utterances versus 29.8 in ESConv.Clients average 32.48 characters per utterance, compared with 24.11 for counselors.Source: §4.1 · p. 5

4.2 Annotation Process

• Twelve trained annotators labeled 520 randomly selected sessions using a training-in-the-loop process, with average agreement of 0.67 for counselors and 0.59 for clients.Client scores reflect self-reported clarity about problem-solving approaches, including new perspectives and confidence in coping.Source: §4.2 · p. 5, §4.1 · p. 5

4.3 Data Characteristics

• Counselors use supporting strategies about four times as often as challenging ones; Inquiring Subjective Information is the most frequent strategy.Among challenging strategies, Confrontation is less common than Interpretation and Invite to Explore New Actions.Source: §4.3 · pp. 5–6
• Positive reactions outnumber negative ones; Giving Information is the most frequent behavior, while Defending is the most common negative behavior.Self-criticism and Hopelessness are rare, limiting the available examples for understanding and classifying them.Source: §4.3 · p. 6

5.1 How Client Reactions Indicates Counseling Effectiveness

• ρ = −0.2080 (p = 1.7591e−5) linked a higher proportion of clients’ Negative reactions with lower perceived conversation effectiveness.Figure 3 shows the relationship between negative reactions and perceived conversation-level scores.Source: Figure 3 · p. 6, §5.1 · p. 6
• A lasso model identified Extending and Defending as the most influential positive and negative behaviors, respectively, for conversation effectiveness.The paper relates Extending to clients experiencing changes and Defending to clients insisting on mistaken beliefs [Hill et al., 1992; Ribeiro et al., 2013].Source: §5.1 · pp. 6–7
• 15.79 was the average score for conversations with at least one negative behavior, about 2% below the overall average; Defending occurred in most such sessions.Table 3 covers the effect of each negative behavior’s occurrence on conversation effectiveness.Source: §5.1 · p. 7, Table 3 · p. 7

5.2 Similar Counseling Strategies Leads to Similar Client Reactions

• Challenging strategies elicited more Negative behaviors than Supporting (4.77% vs. 2.87%) and brought more Extending behaviors as well as Defending.Figure 4 presents the distributions of Extending and Defending after Supporting and Challenging strategies.Source: §5.2 · p. 7, Figure 4 · p. 7

5.3 Appropriate Strategy Utilization

• Across the analyzed middle three stages, both Extending and Defending increased after Challenging as counselors’ use of Challenging rose.The first and last stages were excluded because much of their content was irrelevant to counseling topics; Figure 5 shows the behaviors and the proportion of Challenging by stage.Source: §5.3 · p. 7, Figure 5 · p. 8
• After prior positive reactions to Challenging, using Challenging again increased the probability of Extending; after negative reactions, the probability shifted the other way.The authors advise addressing negative reactions before further Challenging and continuing Challenging after positive reactions; Table 4 reports coefficients predicting Extending rather than Defending.Source: §5.3 · p. 8, Table 4 · p. 8

5.4 Baseline Classifiers for Automatic Label Prediction

• Negative reactions had F1-value = 34.78%, and client reactions and behaviors were harder to identify than counselors’ intentions and strategies.The classifiers use Chinese RoBERTa-large with conversation history and jointly train with masked language modeling as an auxiliary task; Table 5 describes test-set results across four tasks.Source: §5.4 · p. 8, Table 5 · p. 8
• A major behavior-labeling error was confusing Reformulating with Extending; Defending was also difficult to identify because clients express resistance in diverse ways.Clients may resist by expressing opinions different from counselors’ rather than directly denying them.Source: §5.4 · p. 8

• The study develops a theoretically grounded framework for counselors’ strategies and clients’ behaviors, validated using a large-scale, high-quality counseling dataset.The dataset was collected over two years.Source: §6 · p. 8
• Positive client reactions boost counseling-effectiveness ratings, while negative reactions undermine them.Source: §6 · p. 8
• Clients are more likely to extend after counselors challenge their beliefs.Source: §6 · pp. 8–9
• Client reactions and behaviors are harder for automatic annotation models to predict than counselors’ intentions and strategies.The authors describe their analysis as relatively shallow because of data complexity and limited labels for rare cases.Source: §6 · p. 9

• Rare behavior classes have fewer than 50 instances, so analysis focuses mostly on Extending and Defending, and rare-case classifiers may be inaccurate enough to mislead counselors.Source: §7 · p. 9
• A single short post-survey limits the study to coarse-scale analysis of counseling outcomes.Source: §7 · p. 9
• Because the analysis covers only psycho-counseling, transfer to everyday conversations remains untested and findings should not be overgeneralized.The authors caution that overgeneralization may harm the naturalness of daily conversations.Source: §7 · p. 9

Ethics Statement

• The study received ethics approval; clients and counselors consented, data were manually de-identified, and raw-data access is restricted to approved academic or research institutions under confidentiality requirements.Source: Ethics Statement · p. 9

A.1 Framework Development Process

• Three taxonomy developers, including two with clinical or emotional-support experience, iteratively refined the framework through independent annotation and consensus on 30 conversations.Across five iterations, Fleiss’ kappa rose from 0.6255 to 0.7233.Source: §A.1 · p. 12

A.2 Comparison with Existing Frameworks

• Unlike MISC-based frameworks limited to motivational interviewing, this genre-nonspecific framework provides a more comprehensive, finer-grained, intention-based counselor-skills classification: 12 strategies versus ESC’s 7 strategies based on three counseling stages [Liu et al., 2021].Source: §A.2 · pp. 12–13

A.3 Definitions of Strategies

• Counselor-strategy categories range from reflecting client meaning and feelings to answering, interpreting, confronting, and inviting new perspectives or actions.Source: §A.3 · p. 13

A.4 Definitions of Behaviors

• Client behaviors distinguish agreement, information-giving, and elaboration from reformulation, confusion, defensiveness, disengagement, and other responses.Source: §A.4 · pp. 13–14

B.1 Post-survey Scales

• The post-counseling assessment uses five response options, from seldom to always, scored from 1 to 5.Source: §B.1 · p. 14

B.2 Annotators Selection and Training

• Twelve annotators were selected from 30 candidates after three exams, and expert review of difficult samples across six training stages was associated with higher agreement in later stages.Source: §B.2 · p. 14

B.3 Data Quality Control

• Annotators labeled sentence-level counselor skills and client behaviors, and Fleiss’ kappa measured agreement beyond chance.The study treated 0.61–0.80 as substantial agreement.Source: §B.3 · p. 14
• Agreement averaged 0.67 for counselors’ utterances and 0.59 for clients’ utterances, with annotators finding client reactions harder to identify.Disagreements included Interpretation versus Invite to Take New Actions and Extending versus Reformulation; majority voting determined final labels.Source: §B.3 · pp. 14–15, Figure 6 · p. 15

C.1 Experimental Details

• Annotated data for both speakers were randomly split into training (70%), validation (15%), and test (15%) sets, with all utterances from each conversation kept together in a single set.Source: §C.1 · p. 15
• Models used cross-entropy loss and Adam, with the best validation macro-F1 checkpoint selected for testing.Training ran for ten epochs; domain post-training used four NVIDIA A100 GPUs and fine-tuning used one.Source: §C.1 · p. 15

C.2 Experimental Results

• Table 9 reports precision, recall, and macro-F1 by category for predicting counselors’ intentions and strategies and clients’ reactions and behaviors.The table covers four tasks.Source: §C.2 · p. 15, Table 9 · p. 15

D Application to Counseling

• Table 12 presents clients’ follow-up behavior distributions after different counselor strategies.The provided passage identifies the distribution but gives no comparison or values.Source: D Application to Counseling · p. 16

D.1 Correlation Between Clients’ Reactions and Conversation Outcomes

• Conversations were grouped by the proportion of clients’ Negative reactions, and mean self-reported conversation scores were calculated for each group.Table 10 reports group averages with standard deviations.Source: §D.1 · p. 16, Table 10 · p. 16

D.2 Which behavior influences conversation effectiveness the most?

• Table 11 lists behavior types selected by Lasso as important independent variables affecting clients’ self-reported conversation-effectiveness evaluations.The table tracks selection as the L1 regularization coefficient increases from 0.001 to 0.1.Source: Table 11 · p. 16

D.3 Clients Reactions and Behaviors towards Counselors’ Strategies

• Challenging strategies are more likely than Supporting strategies to be followed by clients’ Negative behaviors.Figure 7 shows the proportion of each immediate client behavior type after each counselor strategy.Source: §D.3 · p. 16, Figure 7 · p. 17
• Table 12 compares strategies using Euclidean distances between their follow-up behavior distributions.It reports average distances within Supporting and Challenging intentions, with lower values bolded.Source: §D.3 · p. 16, Table 12 · p. 17