Source-linked AI summary
AutoCBT: An Autonomous Multi-agent Framework for Cognitive Behavioral Therapy in Psychological Counseling
Ancheng Xu, Di Yang, Renhao Li, Jingwei Zhu, Minghuan Tan, Min Yang, Wanxin Qiu, Mingchen Ma, Haihong Wu, Bingyu Li, Feng Sha, Chengming Li, Xiping Hu, Qiang Qu, Derek F. Wong, Ruifeng Xu
TL;DR
Existing counseling systems need more adaptive, context-aware therapeutic approaches. AutoCBT introduces dynamic routing and supervision in a multi-agent CBT framework, and the paper reports improved dialogue quality over purely prompt-based counseling frameworks. Its routing can also face unresolved loops tied to limits in LLM semantic understanding.
Problem
Psychological counseling systems still need more adaptive, context-aware approaches that better reflect complex human interaction and cognitive processes.
Source: §1 · p. 1Method
AutoCBT is a general multi-agent CBT counseling framework that uses dynamic routing and supervisory mechanisms, with agents organized through static or dynamic communication topologies.
Source: §3.1 · pp. 3–4, §5 · pp. 8–9Results
AutoCBT significantly improves dialogue quality compared with other purely prompt-based counseling frameworks.
Source: §5 · p. 9Takeaways & Limitations
The framework provides assistance in identifying and challenging users’ cognitive distortions in CBT-based counseling.
Source: §5 · p. 9LLMs can enter unresolved dynamic-routing loops because their semantic understanding and logic processing remain limited, making a complete solution challenging.
Source: Simultaneous Routing · p. 7
Abstract
from arXiv · showhide
Traditional in-person psychological counseling remains primarily niche, often chosen by individuals with psychological issues, while online automated counseling offers a potential solution for those hesitant to seek help due to feelings of shame. Cognitive Behavioral Therapy (CBT) is an essential and widely used approach in psychological counseling. The advent of large language models (LLMs) and agent technology enables automatic CBT diagnosis and treatment. However, current LLM-based CBT systems use agents with a fixed structure, limiting their self-optimization capabilities, or providing hollow, unhelpful suggestions due to redundant response patterns. In this work, we utilize Quora-like and YiXinLi single-round consultation models to build a general agent framework that generates high-quality responses for single-turn psychological consultation scenarios. We use a bilingual dataset to evaluate the quality of single-response consultations generated by each framework. Then, we incorporate dynamic routing and supervisory mechanisms inspired by real psychological counseling to construct a CBT-oriented autonomous multi-agent framework, demonstrating its general applicability. Experimental results indicate that AutoCBT can provide higher-quality automated psychological counseling services.
1 Introduction
AutoCBT is presented as a flexible multi-agent framework for improving single-turn counseling responses, with routing and memory mechanisms that support structured reasoning and editing. The introduction situates this approach against limitations in existing CBT systems and describes its evaluation and routing-related challenges.
1 Introduction
- Prior CBT approaches use prompt-based therapy or single-agent memory, while reported limitations include weak personalization and adaptation to changing patient needs.CBT interventions identify and challenge cognitive distortions to mitigate adverse emotions and behaviors.Source: §1 · pp. 1–2
- The introduction frames AutoCBT as a general autonomous multi-agent framework addressing shortcomings of existing LLM-based CBT approaches.Source: §1 · p. 2
- Dynamic routing and memory support agent autonomy, while generated messages undergo structured reasoning and editing.The framework also allows its topology to adapt as therapeutic techniques evolve.Source: §1 · p. 2
- The evaluation compares the framework and its realizations with existing LLM-based approaches using a bilingual dataset.The introduction says each realization's performance is analyzed across the dataset.Source: §1 · p. 2
- AutoCBT is presented as delivering superior single-turn counseling responses and enhancing response quality in other prompt-based counseling frameworks.Source: §1 · p. 2
- The authors discuss challenges from dynamic routing and supervision, including Llama's over-protection issue.Source: §1 · p. 2
2 Related Work
Related work describes cognitive distortions as biased thinking patterns and traces efforts to apply AI and LLMs to CBT. It covers earlier computer-based systems and newer prompt-based and memory-based approaches, while motivating evaluation of CBT conversational agents.
2 Related Work
- Cognitive distortions are biased or irrational thinking patterns that can mislead reality judgments and result in negative emotions and behaviors.CBT interventions seek to help users recognize and correct these thinking traps.Source: §2 · pp. 2–3
- AI-related CBT work includes annotated datasets and ontologies, while earlier computer-based systems often relied on predefined responses and lacked natural conversation.Source: §2 · p. 3
- CBT-LLM uses prompt-based learning, while CoCoA uses memory and identifies distortions in user statements.Source: §2 · p. 3
- Other work assesses LLM therapists' conversational behavior, motivating evaluation of CBT-based conversational agents in therapeutic settings.Source: §2 · p. 3
3 Methodology
AutoCBT combines a configurable counsellor-supervisor framework with CBT-specialized supervision and a bilingual consultation dataset. Its methodology specifies agent communication and routing, maps five CBT standards to supervisors, and constructs Chinese and English dataset splits.
3.1 AutoCBT
- AutoCBT models a counsellor, supervisor set, communication topology, and permitted routing strategies as a general multi-agent framework.Its structure is represented by a counsellor agent, supervisors, their communication topology, and the strategies agents may use to communicate.Source: §3.1 · p. 3
- The counsellor routes user messages among configurable LLM supervisors until sufficiently confident, while memories retain recent messages and sliding-window summaries.Agents use short-term memory for recent messages and long-term memory for summaries; the counsellor can seek information from supervisors before replying.Source: §3.1 · pp. 4–5
3.2 Decomposition of CBT Core Principles
- Following CBT-LLM, the framework assigns five CBT standards to separate supervisors, which the counsellor consults selectively using the message and its memory.The standards are validation and empathy, identifying key thoughts or beliefs, challenging or reflecting, providing strategy or insight, and encouragement and foresight.Source: §3.2 · p. 5
3.3 Bilingual Dataset
- The study compiles a bilingual dataset from PsyQA and TherapistQA3, classifies instructions into 10 cognitive-distortion categories, and randomly selects 10 items per category for 100-item Chinese and English splits.For PsyQA, questions and descriptions were merged into unified instructions.Source: §3.3 · p. 5
4 Experiments
Across the bilingual evaluation, AutoCBT outperformed the baselines, including in human evaluations and tests of generalizability. The experiments also exposed routing and role-confusion challenges and Llama-specific refusals.
4.1 Setting
- Evaluation used Qwen and Llama, direct-generation and CBT-prompt baselines, a draft-then-supervisor refinement process, repeated GPT-4o-mini scoring, and professional review.Source: §4.1 · pp. 5–6
4.2 Results
- Across English, AutoCBT outperformed baselines on six metrics and overall with Llama and Qwen; in Chinese, it exceeded PromptCBT on 5 of 6 metrics.Source: §4.2 · p. 6
- Compared with PromptCBT, AutoCBT offered warmer, more flexible support shaped by cultural differences, with a softer, context-specific tone.Source: §4.2 · p. 6
- Over 70% of bilingual questions received AutoCBT’s best answer in human evaluation, and professionals preferred it across all seven cognitive-distortion dimensions.Source: §4.2 · p. 6, Figure 2 · p. 7
4.3.1 Validating Generalizability of AutoCBT
- AutoCBT’s draft responses approximated PromptCBT, while final responses scored significantly higher than drafts on every metric.Source: §4.3.1 · p. 7
4.3.2 Three Challenges In AutoCBT Simultaneous Routing
- LLMs produced conflicting routing targets, repeated routing loops, and supervisor replies in place of requested advice; role confusion was not fully eliminated.Source: Simultaneous Routing · pp. 7–8
- The system treats simultaneous user-and-supervisor targets as an exit signal, cues supervisors to identify their role, and removes used edges to cap routing at N+1 operations.Source: Simultaneous Routing · pp. 7–8
4.3.3 Over-Protection of Llama
- Llama refused nine English questions concerning minors, sex, and suicide; excluding them left 91 questions for its effectiveness analysis.Source: §4.3.3 · p. 8
5 Conclusion
AutoCBT is a general multi-agent framework for CBT-based psychological counseling that introduces dynamic routing and supervisory mechanisms. The paper reports improved dialogue quality and assistance in identifying and challenging cognitive distortions, while also noting LLM instruction-following and semantic-understanding deficiencies.
- The framework introduces dynamic routing and supervisory mechanisms to CBT-based counseling and assists in identifying and challenging users’ cognitive distortions.Source: §5 · p. 9
- AutoCBT significantly improves dialogue quality over other purely prompt-based counseling frameworks.Source: §5 · p. 9
- Experiments identified deficiencies in LLM instruction-following and semantic understanding, which the paper analyzes and addresses with proposed solutions.Source: §5 · p. 9
A Evaluation Metrics of Detailed Sampling Evaluation
Detailed Sampling Evaluation retains four metrics from the automatic evaluation and introduces three new metrics.
- Identify-CD and Challenge-CD update Identification and Reflection, while Presentation evaluates the counsellor response overall.Source: A Evaluation Metrics of Detailed… · p. 11
- The evaluation retains four metrics from the automatic evaluation and adds three new metrics for Detailed Sampling Evaluation.Source: A Evaluation Metrics of Detailed… · p. 11
A.1 Consistent Metrics
The section identifies four metrics as consistent with those used in the previous automatic evaluation.
- Four metrics are consistent with the metrics used in the previous automatic evaluation.Source: Table 7 · p. 11
A.2 New Metrics
The section identifies three metrics as inconsistent with those used in the previous automatic evaluation.
- Three metrics are inconsistent with the metrics used in the previous automatic evaluation.Source: Table 8 · p. 11
B Human Analysis of Detailed Sampling Evaluation
Six psychological experts compare AutoCBT with baselines in the Detailed Sampling Evaluation using cognitive-distortion principles; Table 9 presents their human analysis.
B Human Analysis of Detailed Sampling Evaluation
- Six psychological experts analyze differences between AutoCBT and baselines in the Detailed Sampling Evaluation using cognitive-distortion principles.Their analysis is presented in Table 9.Source: B Human Analysis of Detailed Sampling… · p. 11, Table 9 · p. 12