PaperlayerView exact revision on arXiv

Source-linked AI summary

Psyche-R1: Towards Reliable Psychological LLMs through Unified Empathy, Expertise, and Reasoning

Chongyuan Dai, Jinpeng Hu, Hongchang Shi, Zhuo Li, Dan Guo, Xun Yang, Meng Wang

arXiv:2508.10848v3cs.CL

TL;DR

Psychological LLM research has emphasized empathy and expertise while giving limited attention to complex reasoning. Psyche-R1 combines rationale-rich psychological QA and empathetic dialogues in a hybrid training strategy, and its 7B model performs comparably to DeepSeek-R1.

  • Problem

    Psychological LLMs have limited capacity for complex reasoning, while reasoning-augmented LLMs have limited psychological-domain performance when empathy and expertise are not integrated.

    Source: §1 · p. 2

  • Method

    Psyche-R1 combines psychological questions with detailed rationales and empathetic dialogues, using challenging examples for GRPO and remaining data for SFT.

    Source: §1 · p. 2

  • Results

    Psyche-R1 outperforms existing psychological LLMs and achieves performance comparable to DeepSeek-R1.

    Source: §5 · p. 9

  • Takeaways & Limitations

    Psyche-R1 shows capability in counseling tasks requiring emotional empathy and cognitive empathy.

    Source: §3.4 · p. 8

    Cross-cultural generalization remains challenging, and multimodal psychology and affective computing warrant further study.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Amidst a shortage of qualified mental health professionals, the integration of large language models (LLMs) into psychological applications offers a promising way to alleviate the growing burden of mental health disorders. Recent reasoning-augmented LLMs have achieved remarkable performance in mathematics and programming, while research in the psychological domain has predominantly emphasized emotional support and empathetic dialogue, with limited attention to reasoning mechanisms that are beneficial to generating accurate responses. Therefore, in this paper, we propose \logopsyche\textit{Psyche-R1}, the first Chinese psychological LLM that jointly integrates empathy, psychological expertise, and reasoning, built upon a novel data curation pipeline. Specifically, we design a comprehensive data synthesis pipeline that produces over 75k high-quality psychological questions paired with detailed rationales, generated through an iterative prompt-rationale optimization procedure, along with 73k empathetic dialogues. Subsequently, we employ a hybrid training strategy wherein challenging samples are identified through a multi-LLM cross-selection strategy for group relative policy optimization (GRPO) to improve reasoning ability, while the remaining data are used for supervised fine-tuning (SFT) to enhance empathetic response generation and psychological domain knowledge. Extensive experiment results demonstrate the effectiveness of \textit{Psyche-R1} across several psychological benchmarks, where our 7B \textit{Psyche-R1} achieves comparable results to 671B \texttt{DeepSeek-R1}.

1 Introduction

The introduction positions Psyche-R1 as a response to psychological LLMs’ limited integration of empathy, expertise, and complex reasoning, and previews a hybrid system with competitive benchmark performance. The methods combine curated psychological questions and empathetic dialogues, then allocate data to supervised fine-tuning and reinforcement learning.

1 Introduction

  • 7B Psyche-R1 significantly outperforms similar-scale models and competes with substantially larger DeepSeek-R1, addressing the gap in unified empathy, expertise, and reasoning.The introduction identifies limited complex reasoning and integration of empathy and expertise as gaps in prior psychological LLMs.Source: §1 · p. 2

2.1 Data Collection

  • Collection combines textbooks, question banks, synthetic QA, and support-community dialogues; cleaning standardizes text and filters counselor replies lacking substantive advice.The source types feed psychological reasoning QA or the empathetic-dialogue dataset.Source: §2.1 · pp. 2–3

2.2 Psychological Reasoning Data Synthesis

  • A 210k QA pool is filtered to 90k, expanded with answer-checked CoT rationales refined iteratively, then split by three-LLM failures into 19k challenging cases.Rationales are retained when their predicted answers match the ground truth; the three models are Qwen, Llama, and Phi.Source: §2.2 · pp. 3–4

2.3 Empathetic Dialogue Synthesis

  • 73k dialogues are revised to combine empathetic language with evidence-based guidance and concrete coping steps.The dialogue synthesis aims to move beyond surface-level empathy toward practical, issue-specific support.Source: §2.3 · p. 4, §2.4 · p. 4

2.4 Data Split

  • SFT receives non-challenging reasoning data, empathetic dialogues, and PsycoLLM data; GRPO receives challenging cases plus 8k CPsyExam examination questions.The split aligns the two data subsets with distinct training objectives.Source: §2.4 · pp. 4–5

2.5 Model Training

  • SFT trains Qwen2.5-7B-Instruct to emit a rationale followed by an answer; GRPO rewards parsable formatting and answer accuracy, including partial credit for overlap.The reward combines format and accuracy signals.Source: §2.5 · p. 5

3 Experiments

The experiments evaluate Psyche-R1 on psychological knowledge and counseling tasks, then examine its training components, data composition, and performance across difficulty levels.

3.1 Experimental Setting

  • The evaluation covers PCEB and CPsyExam, using zero-shot and five-shot settings with shared exemplars for the latter.PCEB includes multiple-choice and open-ended case-analysis items; CPsyExam spans 39 psychological subfields.Source: §3.1 · p. 5

3.2 Overall Results

  • Psyche-R1 performs strongly across PCEB multiple-choice and subjective tasks, while Psyche-R1 and DeepSeek-R1 show superior performance on CPsyExam.The cited tables compare models on PCEB and CPsyExam; the text also notes distinct strengths and limitations across model types.Source: §3.2 · p. 6, Table 1 · p. 6, Table 2 · p. 7

3.3 Ablation Study

  • SFT improves performance across three task categories; GRPO without prior SFT degrades SMCQ case-task performance, while SFT plus GRPO yields further gains, especially on case tasks.The paper attributes the degradation to insufficient domain knowledge and empathy in the base model, and describes unstable training dynamics.Source: §3.3 · pp. 6–7
  • Rationale-augmented training outperforms option-only QA, and iterative prompt–rationale optimization adds further gains over rationales alone.The reported comparisons include +QA, +Rat., and +Rat.+Iter. training variants.Source: §3.3 · p. 7, Table 3 · p. 7
  • Multi-LLM cross-selection yields markedly superior outcomes across tasks compared with random selection of challenging questions.The comparison is presented in Figure 3.Source: §3.3 · p. 7, Figure 3 · p. 7
  • PRD or empathetic dialogues alone produce marginal gains or slight declines on some tasks, whereas combining them improves performance; adding public datasets improves it further.The paper links the combined data to integrating domain-specific knowledge and emotional understanding.Source: §3.4 · p. 8, Table 4 · p. 8

3.4 Discussion

  • On counseling tasks, Psyche-R1 improves over its base model; evaluation sampled 200 PsyDT test instances and used GPT-4o as evaluator.The paper attributes performance to empathetic dialogues and reasoning mechanisms.Source: §3.4 · p. 8, Table 5 · p. 8
  • Psyche-R1 improves on both challenging and non-challenging PCEB subsets, with more pronounced gains on challenging questions.Non-challenging gains concentrate mainly in theory and moral dimensions; Figure 4 distinguishes standard from elastic accuracy.Source: §3.4 · p. 8, Figure 4 · p. 8

4 Related Work

Prior work applied LLMs to mental-health access, emotional support, and specialized psychological applications. Reasoning methods have advanced in other domains, but psychology has seen limited research on reasoning; Psyche-R1 combines reasoning with empathy and domain expertise.

4.1 LLMs for Psychology

  • Psychology-focused LLM work spans emotion recognition, empathetic dialogue, interactive psychometric tools, and applications grounded in therapies or psychological concepts.Earlier work also sought to improve access to mental-health services.Source: §4.1 · pp. 8–9

4.2 LLM Reasoning

  • LLM reasoning research progressed from CoT and STaR to structured reasoning architectures and reinforcement-learning-trained models, with applications extending to medicine and finance.The cited examples include Tree of Thoughts, which explores multiple paths with self-evaluation, and PAL, which uses program generation with external tools.Source: §4.2 · p. 9
  • Despite these advances, limited research has investigated reasoning in psychology; Psyche-R1 unifies empathy, domain-specific expertise, and reasoning in a psychological LLM.The paper identifies Psyche-R1 as the first psychological LLM to combine these capabilities.Source: §4.2 · p. 9

5 Conclusion

Psyche-R1 combines empathy, psychological expertise, and reasoning, supported by synthesized reasoning samples and empathetic dialogue. Experiments report that it outperforms existing psychological LLMs and performs comparably to DeepSeek-R1.

5 Conclusion

  • Psyche-R1 outperforms existing psychological LLMs and achieves performance comparable to DeepSeek-R1, according to the reported experiments.The paper also reports ablation studies and analyses of individual components and strategies.Source: §5 · p. 9
  • The system combines empathy, psychological expertise, and reasoning as a Chinese psychological LLM.The conclusion describes it as the first Chinese psychological LLM to jointly integrate these capabilities.Source: §5 · p. 9
  • Its multi-stage synthesis pipeline produces psychological reasoning samples with detailed rationales and empathetic dialogues, refining rationales through iterative prompt–rationale optimization.Multi-LLM cross-selection identifies challenging examples for GRPO, while remaining data are used for SFT.Source: §5 · p. 9

Limitations

The paper identifies language and cultural specificity, model scale, and the need for further multimodal work as limitations or future research boundaries.

Limitations

  • Psyche-R1 and its training corpus are predominantly tailored to Chinese language and cultural context, and cross-cultural generalization remains a future challenge.The authors frame this tailoring as a step toward local applicability.Source: Limitations · pp. 9–10
  • The authors identify multimodal psychology and affective computing as topics warranting further study.Source: Limitations · p. 10
  • Constrained computational resources limited Psyche-R1 to a 7B-parameter backbone; the authors posit that a larger base model would yield superior performance.They suggest more efficient fine-tuning as a way to facilitate scaling under limited resources.Source: Limitations · p. 10

Ethical Considerations

The paper calls for ethical standards in mental-health LLM deployment and describes limits on Psyche-R1’s role alongside privacy and safety measures in data curation.

Ethical Considerations

  • The paper states that mental-health LLM development and deployment require rigorous adherence to ethical standards.Source: Ethical Considerations · p. 10
  • Psyche-R1 is a supportive tool for mental-health support and education, not a replacement for qualified professionals or a tool for diagnosis, treatment, or crisis intervention.People facing severe crises should seek human professionals or emergency services.Source: Ethical Considerations · p. 10
  • The data curation process de-identifies social-media data, avoids reconstructing private interactions, and filters toxic content and harmful suggestions.Source: Ethical Considerations · p. 10

A Case Study

In a career-transition case, Psyche-R1 combines empathetic language, psychological concepts tailored to the user, and step-by-step reasoning. Qwen2.5-72B-Instruct offers less natural empathy, generic suggestions, and no theoretical contextualization of the dilemma.

A Case Study

  • In a career-transition dilemma involving a company manager, Psyche-R1 begins with empathy and then applies relevant psychological concepts tailored to the user’s situation, demonstrating emotional attunement and domain-specific expertise.By contrast, Qwen’s empathetic expressions are described as less natural and engaging, and it does not apply theoretical knowledge to contextualize or explain the user’s dilemma.Source: A Case Study · p. 14
  • In the company-manager career-transition case, Psyche-R1 moves from surface-level observations to in-depth analysis, while Qwen offers generic suggestions without step-by-step or in-depth reasoning.The authors characterize Psyche-R1’s reasoning path as clear and efficient.Source: A Case Study · p. 14

B.1 Details of Baselines

The baseline comparison covers general, reasoning-augmented, closed-source, and psychological LLMs. Qwen3-series models and Claude3.7-Sonnet are evaluated in reasoning mode.

B.1 Details of Baselines

  • The baseline set spans general LLMs, reasoning-augmented LLMs, closed-source LLMs, and psychological LLMs.Named examples include Qwen2.5, DeepSeek-R1, GPT-4o, and CPsyCounX, respectively.Source: §B.1 · p. 14
  • Qwen3-series models and Claude3.7-Sonnet are set to reasoning mode to stimulate their best performance.Detailed baseline information is provided in Table 7.Source: §B.1 · p. 14

B.2 Implementation Details

The experiments use SFT followed by GRPO, with specified learning rates, batch sizes, and two epochs for each phase. Evaluation uses fixed decoding settings and runs use eight RTX A6000 GPUs.

B.2 Implementation Details

  • SFT uses a 1e-5 learning rate and batch size 256; GRPO uses a 1e-6 learning rate and batch size 128, with two epochs for each phase.The SFT and GRPO setups are described in the experimental implementation details.Source: §B.2 · p. 14
  • All experiments run on eight RTX A6000 GPUs, each with 48GB of memory.Source: §B.2 · p. 15
  • Evaluation uses temperature 0.0, a maximum sequence length of 1024, and top-p 0.95.The authors state these settings are used to ensure evaluation fairness.Source: §B.2 · p. 15

B.3 Details of Model Training

The appendix specifies SFT and GRPO training settings and documents prompts for generating, cleaning, selecting, and reasoning about psychological questions, as well as synthesizing empathetic dialogue.

B.3 Details of Model Training

  • SFT uses a 1e-05 learning rate, batch size 256, two epochs, AdamW, and cosine scheduling with a 0.1 warmup ratio; GRPO uses a 1e-06 learning rate, batch size 128, and two epochs.GRPO also uses PPO mini-batches of 32, micro-batches of 20 per GPU, and KL regularization with coefficient 1e-03.Source: §B.3 · p. 15

C Details of Prompts

  • Question-generation prompts require challenging, clear questions with supported answers and evidence grounded in the source text.The prompt also specifies a JSON format and requests a target number of questions.Source: C Details of Prompts · pp. 15–16
  • Question-control prompts retain only questions candidates can answer from the question and their knowledge, without needing the original text.They classify questions as reasonable or unreasonable based on sufficient information for solving them.Source: C Details of Prompts · pp. 16–17
  • Rationale-generation prompts request detailed, step-by-step psychological reasoning and a final predicted answer.The prompt asks for analysis, verification, reflection, and revision of reasoning.Source: C Details of Prompts · p. 17
  • Candidate-prompt generation seeks improved prompts for more accurate and detailed reasoning on a given psychological question.Source: C Details of Prompts · p. 17
  • Rationale-comparison prompts ask evaluators to choose the best response to a question and explain their choice.The selection is returned in a specified JSON format.Source: C Details of Prompts · pp. 15, 17
  • Question-selection prompts ask a model to choose an answer option and output its letter without explanation.Source: C Details of Prompts · pp. 17–18
  • Empathetic-dialogue prompts ask a counselor to organize events, identify psychological confusion, and refine the question from the user’s first-person perspective.A separate prompt evaluates whether counselor responses are helpful and positively significant to users.Source: C Details of Prompts · pp. 15, 18

Related papers

Loading 2508.10848v3…