PaperlayerView exact revision on arXiv

Source-linked AI summary

Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions

Huachuan Qiu, Zhenzhong Lan

arXiv:2408.15787v2cs.CLcs.IR

TL;DR

High-quality counseling dialogue data is difficult to collect because real-session collection is costly, privacy-sensitive, and hard to scale, while reconstruction approaches do not capture interactive dynamics. Interactive Agents uses prompted LLM client-counselor role-play to generate synthetic data. Systems fine-tuned on that data significantly outperform strong baselines, including a model trained on real counseling dialogues.

  • Problem

    High-quality counseling dialogue datasets remain difficult to build because real-session collection is costly, privacy-sensitive, and hard to scale, while reconstruction-based approaches miss interactive dynamics.

    Source: §1 · p. 1

  • Method

    Interactive Agents prompts LLMs to role-play a client and counselor, using client role cards and a counselor guided through exploration, insight, and action.

    Source: §1 · p. 2

  • Results

    The fine-tuned dialogue system significantly outperforms strong baselines, including a model trained on real-life counseling dialogues, and the authors report strongest overall performance among state-of-the-art mental-health dialogue models.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results demonstrate the potential of LLM role-playing simulation for scalable, privacy-preserving counseling dialogue construction and downstream dialogue-system development.

    Source: §6 · p. 9

    The authors identify client resistance behaviors as a direction for future work in client simulation.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Creating effective dialogue systems for mental health support requires high-quality multi-turn counseling dialogue data, yet collecting real counselor-client conversations presents significant challenges, including privacy concerns, high costs, and limited scalability. We present \textbf{Interactive Agents}, a novel framework that simulates naturalistic counseling dialogues through controlled LLM-to-LLM interactions. The framework introduces two key innovations: (1) a personalized client agent that maintains consistent psychological characteristics throughout a session, and (2) a counselor agent that implements a theoretically grounded three-stage therapeutic model comprising the exploration, insight, and action phases. Through rigorous evaluation using both automatic metrics and professional-counselor assessments based on the Working Alliance Inventory, we demonstrate that our framework generates therapeutically valid dialogues that are comparable in quality to human-generated sessions. Models fine-tuned on our proposed synthetic dataset (SimPsyDial) achieve state-of-the-art performance in a standard pairwise chatbot-arena evaluation of LLM-based counselors. Our framework provides a scalable, privacy-preserving method for generating high-quality counseling dialogue data while maintaining professional therapeutic standards.

1 Introduction

The introduction presents Interactive Agents as an LLM-to-LLM framework for generating counseling dialogue data in response to limitations of existing data-collection approaches. It evaluates the simulated clients and counselor dialogues, then tests models trained on the synthetic data.

1 Introduction

  • Real counselor-client data collection is costly, time-consuming, privacy-sensitive, and difficult to scale, while reconstruction-based dialogues lack real-session interaction dynamics.The introduction identifies high-quality counseling dialogue datasets as a central challenge for psychological counseling systems.Source: §1 · p. 1
  • Interactive Agents uses role-playing LLM-to-LLM interactions to create a scalable counseling dataset, with a help-seeking client and a counselor guiding exploration, insight, and positive change.The paper names its synthetic dataset SimPsyDial.Source: §1 · p. 2
  • Simulated-client diversity is comparable to real-client diversity, and role cards substantially influence generated client utterances.The client simulation is evaluated for role-playing fidelity and diversity.Source: §1 · p. 2
  • The work evaluates simulated-dialogue quality with professional counselors and benchmarks systems fine-tuned on synthetic data against state-of-the-art mental-health dialogue models.The counselor evaluation uses the Observer-rated Short version of the Working Alliance Inventory.Source: §1 · p. 2
  • The fine-tuned dialogue system significantly outperforms strong baselines, including a model trained on real-life counseling dialogues.The paper also reports strongest overall performance against state-of-the-art mental-health dialogue models.Source: §1 · p. 2

2 Related Work

Related work covers interactive simulacra across domains and prior approaches to mental-health dialogue systems. The authors position their work as an annotator-free counselor-client simulator using client role cards sampled from a real-life pool.

2.1 Interactive Simulacra

  • Prior interactive-simulacra research spans multiple domains, and counseling studies include client evaluation and CBT training; the authors claim their work is the first annotator-free simulator for dialogue generation at scale.The role cards are sampled from a real-life client pool.Source: §2.1 · pp. 2–3

2.2 Conversational Agents for Mental Health

  • Mental-health dialogue systems include models trained on interviews, rewritten single-turn dialogues, and counseling reports; this work explores simulation as a privacy-preserving construction approach.The paper focuses on conversational agents for mental health.Source: §2.2 · p. 3

3 Method

The method simulates counseling as an interaction between an LLM client and counselor, then uses the resulting dialogues for evaluation and model training. Client role cards, a three-stage counselor model, and GPT-4 prompting structure the simulation.

3.1 Problem Definition

  • A simulated conversation is an ordered sequence of client and counselor utterances, initiated by the client and ending with the counselor’s final response.The client and counselor are denoted Ω and Ψ, respectively.Source: §3.1 · p. 3

3.2 Task Formulation

  • Client role cards guide simulations in which clients maintain a consistent style and discuss specified concerns and life events for up to 50 turns or until a counselor end token.The client begins with “Hello.”Source: §3.2 · p. 3

3.3 Simulation Framework Overview

  • Figure 1 presents client-pool construction, interactive data collection, and model training as the framework’s main stages.Generated dialogues are evaluated and used to train dialogue systems for downstream effectiveness.Source: §3.3 · pp. 3–4, Figure 1 · p. 4

3.4 Client Simulation

  • Four professional counselors design role cards based on PsyQA; the pool contains 1,000 cards plus 100 held-out cards, each describing nine client attributes.The attributes include demographics, relationships, Big Five traits, resistance, and chief complaint.Source: §3.4 · p. 4

3.5 Counselor Simulation

  • The counselor follows an integrative exploration-insight-action model; response checks terminate interactions and reject outputs exceeding 200 Chinese characters or containing newlines or enumerated items.The three stages draw on client-centered, psychodynamic, and cognitive behavioral therapy, respectively.Source: §3.5 · pp. 4–5

3.6 Experimental Setup

  • GPT-4 is used for both simulated roles because the other tested models produced overly lengthy utterances or prematurely short interactions.The preliminary alternatives included GPT-3.5, GLM-4, DeepSeek-V2-Chat, and Qwen1.5-110B-Chat.Source: §3.6 · p. 5

4 Simulation Evaluation

SimPsyDial contains 1,000 simulated counseling dialogues and is evaluated for client role fidelity, client diversity, and working alliance. It shows stronger role-card consistency than a random-mapping control, broadly similar client-topic distributions to RealPsyDial, and higher WAI-O-S scores across all three dimensions.

4 Simulation Evaluation

  • SimPsyDial comprises 1,000 dialogues averaging 13 turns per conversation, collected by simulating both counselor and client agents with GPT-4.Source: §4 · p. 5

4.1 Client Evaluation

  • Mapping-group clients show higher vocabulary overlap (0.406 vs. 0.284) and semantic similarity (0.791 vs. 0.570) than random-mapping controls (both p < 0.001).The authors interpret both comparisons as evidence that client utterances are conditioned on the assigned role card.Source: §4.1 · pp. 5–6, Figure 2 · p. 6
  • SimPsyDial and RealPsyDial have broadly similar client-topic categories and frequencies; topic entropy is 4.526 vs. 4.875, a nonsignificant difference (p = 0.055).The reported entropy standard deviations are 0.009 for SimPsyDial and 0.020 for RealPsyDial.Source: §4.1 · p. 6, Figure 3 · p. 6

4.2.1 Analysis of Working Alliance Inventory

  • SimPsyDial scores higher than RealPsyDial on Goal (6.045 vs. 5.505), Task (6.191 vs. 5.695), and Bond (5.953 vs. 5.807) (all p < 0.001).The comparison reports lower score variability for SimPsyDial across all three dimensions.Source: §4.2.1 · pp. 6–7, Figure 4 · p. 7

5 Dialogue System

The section describes supervised fine-tuning of counselor models on synthetic dialogue, the comparison systems and implementation setup, and a multi-counselor evaluation. In that evaluation, simpsybot_D and simpsybot_Q perform best under LLM-based and professional-counselor judgments.

5.1 Mathematical Formulation

  • Training instances pair dialogue history with the counselor response, and the model is fine-tuned on synthetic data using maximum likelihood estimation.The model predicts response r_t from history h_t; π_θ is initialized from π_0.Source: §5.1 · p. 7

5.2.1 Comparison Models

  • Comparison systems use rewritten single-turn counseling data, RealPsyDial, or counseling reports as training sources.MeChat and SoulChat use rewritten single-turn corpora, PsyChat uses RealPsyDial, and CPsyCounX uses counseling reports.Source: §5.2.1 · p. 7

5.2.2 Implementation Details

  • The study fine-tunes Qwen2-7B-Instruct and DeepSeek-LLM-7B-Chat using full fine-tuning and counselor-response training samples.Training uses two epochs, a 1×10−5 learning rate, and 16-bit half precision.Source: §5.2.2 · p. 7

5.3 Interaction with Multiple Counselors

  • simpsybot_D and simpsybot_Q achieve the best performance under both LLM-based and professional-counselor judgments.Figure 6 reports Elo ratings for six LLM-based counselors across three evaluation methods.Source: Figure 6 · p. 8, §5.3 · p. 8
  • Three professional counselors each annotate 97 dialogues, while an LLM judge can select responses automatically until dialogue termination criteria are met.Two shuffled candidate responses are compared given the same dialogue history.Source: §5.3 · pp. 7–8

6 Conclusion

Interactive Agents uses two prompted LLMs to simulate counselor-client interactions, with a client role card and a counselor guided by a three-stage therapeutic model. The paper compares simulated dialogues with real counselor-client dialogues and benchmarks systems trained on its synthetic data, concluding that the approach has potential for scalable, privacy-preserving dialogue construction and system development.

  • Interactive Agents uses a role-playing setup in which one prompted LLM simulates a role-card client and another acts as an experienced counselor guided by an integrative three-stage therapeutic model.Both roles are implemented through GPT-4 prompting.Source: §6 · p. 9
  • The evaluation compares simulated dialogues with real counselor-client dialogues involving professional counselors and benchmarks dialogue models trained on synthetic data against state-of-the-art mental health dialogue models.Source: §6 · p. 9
  • The results indicate potential for scalable, privacy-preserving counseling-dialogue construction and downstream dialogue-system development.Source: §6 · p. 9

Limitations

The authors identify client resistance, follow-up sessions, and greater realism in counseling dialogue data as directions for future work.

  • Future client simulations could incorporate resistance behaviors and examine their influence on simulated dialogues more comprehensively.Source: Limitations · p. 9
  • Simulating follow-up sessions could examine changes in clients’ behavior, concerns, and emotional states across sessions.Source: Limitations · p. 9
  • Improving dataset realism remains a goal, with planned prompt optimization and retrieval-augmented generation using real-life counseling sessions.Source: Limitations · p. 9

Ethics Statement

The ethics discussion warns that simulated counseling can produce inappropriate advice and omit important client contexts or counseling depth. It also marks professional supervision as a boundary and describes implementation materials for client and counselor simulation.

Ethics Statement

  • The study does not recommend using simulated counseling systems as substitutes for psychological treatment without professional supervision.Source: Ethics Statement · p. 9
  • LLMs may generate inaccurate, inappropriate, or unprofessional advice that could reinforce maladaptive behaviors without safeguards.A reading recommendation, for example, may not suit a client with a visual impairment.Source: Ethics Statement · p. 9
  • Simulated clients may omit social and clinical context, including family relationships, employment status, and suicide risk, limiting realism and applicability.Source: Ethics Statement · p. 9
  • LLM-based counselors may not capture the depth of real counseling, where counselors gradually explore clients’ thoughts, emotions, and underlying concerns.Source: Ethics Statement · p. 9
  • Training and inference with LLMs require substantial computation and carry energy-consumption and environmental-impact concerns.Source: Ethics Statement · p. 9
  • The authors say the ethical implications of scalable LLM-based counselor-client simulation warrant consideration and future attention.Source: Ethics Statement · p. 9

A Client Simulation

B Counselor Simulation

  • Counselor-simulation materials specify dialogue termination criteria and a WAI-based dialogue evaluation prompt following the WAI-O manual.The termination criteria are listed in Table 2.Source: B Counselor Simulation · p. 13, Table 2 · p. 13

C Experiments

The experiments were conducted on NVIDIA A100 80G GPUs.

C.1 Generation Phase

The paper reports inference hyperparameters for LLM-based counselors in Table 3.

C.2 Chatbot Arena

The chatbot-arena experiments used professional counselors to design response-selection prompts and annotate counselor-response comparisons. Prompt 2 achieved higher reported selection accuracy than Prompt 1 with both judge models evaluated.

C.2 Chatbot Arena

  • 64.4% accuracy was achieved by GPT-4o judging Prompt 2, compared with 61.8% for Qwen1.5-110B-Chat judging Prompt 2 and 60.8% for Qwen1.5-110B-Chat judging Prompt 1.Source: §C.2 · p. 13, Figure 12 · p. 17, Figure 13 · p. 17
  • Three professional counselors each annotated 97 multi-turn dialogues in a human arena comparing shuffled candidate responses given dialogue history, yielding 291 annotated dialogues.Source: §C.2 · p. 13, Table 4 · p. 14
  • Four professional counselors collaborated to design two response-selection prompts, and Prompt 2 was used as the guideline for automatic multi-response competition.Source: §C.2 · p. 13, Figure 12 · p. 17, Figure 13 · p. 17
Loading 2408.15787v2…