PaperlayerView exact revision on arXiv

Source-linked AI summary

MindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients' Journaling

Taewan Kim, Seolyeong Bae, Hyun Ah Kim, Su-woo Lee, Hwajung Hong, Chanmo Yang, Young-Ho Kim

arXiv:2310.05231v2cs.HCcs.AIcs.CL

TL;DR

MindfulDiary addresses the challenge of supporting psychiatric patients’ journaling while controlling LLM behavior in clinical settings. It combines an MHP-informed, stage-based conversational app with clinician-facing summaries and was evaluated in a four-week field study. The study found richer patient records and improved clinician understanding and communication, within a limited single-hospital and model-specific scope.

  • Problem

    LLMs enable naturalistic mental-health conversations but raise safety and controllability concerns, while psychiatric patients may struggle to document and express daily experiences.

  • Method

    MindfulDiary uses an MHP-informed, stage-based mobile conversational AI for journaling alongside a clinician dashboard that summarizes patient dialogues.

  • Results

    In a four-week field study, MindfulDiary supported 28 patients with major depressive disorder in enriching daily records and helped psychiatrists understand patients’ thoughts and contexts.

  • Takeaways & Limitations

    LLM-driven journaling can serve as a supportive bridge between patients’ daily experiences and clinical consultations.

  • Takeaways & Limitations

    Findings may not generalize beyond the young, mostly adolescent participants recruited from one university hospital, and later GPT versions may behave differently.

Abstract

from arXiv · show

In the mental health domain, Large Language Models (LLMs) offer promising new opportunities, though their inherent complexity and low controllability have raised questions about their suitability in clinical settings. We present MindfulDiary, a mobile journaling app incorporating an LLM to help psychiatric patients document daily experiences through conversation. Designed in collaboration with mental health professionals (MHPs), MindfulDiary takes a state-based approach to safely comply with the experts' guidelines while carrying on free-form conversations. Through a four-week field study involving 28 patients with major depressive disorder and five psychiatrists, we found that MindfulDiary supported patients in consistently enriching their daily records and helped psychiatrists better empathize with their patients through an understanding of their thoughts and daily contexts. Drawing on these findings, we discuss the implications of leveraging LLMs in the mental health domain, bridging the technical feasibility and their integration into clinical settings.

1 INTRODUCTION

MindfulDiary addresses limitations of conventional journaling chatbots by using an LLM-based, expert-collaboratively designed system for psychiatric patients. A four-week study found that it enriched patients’ records and supported psychiatrists’ understanding and communication.

  • Existing rule-based and retrieval-driven chatbots have limited capability for open-ended conversations, motivating LLM-based journaling support.LLMs offer more naturalistic conversation, but their uncertain response control requires safety precautions and domain-expert collaboration in mental health settings.
  • MindfulDiary combines a mobile conversational AI for patients with a clinician dashboard for reviewing summarized dialogue histories.The LLM generates stage-sensitive prompts and automatically summarizes conversation records for mental health professionals.
  • A four-week field study involved 28 patients with major depressive disorder and five psychiatrists using MindfulDiary during daily journaling and clinical visits.The multidisciplinary team iteratively designed the system with HCI researchers, AI engineers, and psychiatrists.
  • MindfulDiary helped patients consistently enrich daily records and helped psychiatrists monitor and comprehend patient states.The system supported detailed record-keeping and expression while improving the quality of patient-provider communication.
  • The work contributes a collaborative design case and implications for deploying controllable, expert-informed LLM conversational AI in clinical mental health.

2 RELATED WORK

Prior research establishes journaling and conversational agents as tools for self-reflection, patient-generated health data, and mental-health support. MindfulDiary extends this work by using interactive LLM dialogue to lower journaling barriers while addressing the safety and controllability challenges of LLMs.

  • Journaling and patient-generated health data: Journaling can support emotional expression, reflection, and mental wellness, while recording contextual symptoms and daily experiences useful in clinical care.Patient-generated health data can provide information outside regular clinical visits and support communication with mental health professionals.
  • Journaling and patient-generated health data: Technology-enhanced journaling research has explored life-logging, emotional articulation, and more comfortable, engaging, and honest self-expression.
  • Journaling and patient-generated health data: MindfulDiary extends journaling research by using interactive LLM dialogue and contextualized prompts to encourage psychiatric patients’ deeper documentation and reflection.
  • Conversational agents for mental health: Mental-health chatbots have supported therapy, expressive writing, behavioral reinforcement, sensitive disclosure, and access beyond temporal and spatial constraints.
  • Conversational agents for mental health: Rule-based and retrieval-based chatbots offer conversational control but struggle with open-ended, serendipitous, and versatile responses.
  • Conversational agents for mental health: LLM-driven chatbots improve context-aware open-domain interaction but introduce explainability, predictability, harmful-output, and conversational-consistency concerns.
  • Conversational agents for mental health: The paper responds by combining LLM-powered patient journaling with LLM-based dashboard summarization and classification developed with clinicians.

3 FORMATIVE STUDY: FOCUS GROUP INTERVIEW

The formative study used mental health professionals’ perspectives to identify barriers to patient disclosure and shape MindfulDiary’s supportive, stage-based journaling design. Participants saw LLMs as useful for eliciting richer self-reflection and strengthening clinical dialogue, but not as a standalone intervention.

  • 3 FORMATIVE STUDY: FOCUS GROUP INTERVIEW: The team conducted a focus group interview to understand MHPs’ expectations, values, and precautions for using LLMs in clinical mental health.
  • 3.1 Procedure and Analysis: Six MHPs participated, including four clinical psychologists and two psychiatrists with 1–11 years of experience.
  • 3.1 Procedure and Analysis: Two one-hour remote sessions introduced LLM capabilities and limitations before discussing treatment challenges and possible clinical uses.
  • 3.2.1 Challenges in Eliciting Responses from Patients with Depression: Patients with depression may remain silent or struggle to express feelings during brief consultations, forcing providers to rely on standardized, repetitive questions.
  • 3.2.1 Challenges in Eliciting Responses from Patients with Depression: Paper diaries showed low participation and engagement, sometimes omitting significant events later revealed during consultation.
  • 3.2.1 Challenges in Eliciting Responses from Patients with Depression: Participants envisioned flexible LLM conversations as a way to bridge communication gaps and obtain more nuanced accounts from patients who have difficulty articulating feelings.
  • 3.2.2 LLMs as a Bridge for Enhanced Patient Communication: Participants wanted LLMs to trace thought continuums and prompt deeper examination of underlying emotions, enriching therapeutic dialogue.
  • 3.2.3 LLMs for Analytical Insights and Personalized Mental Health Support: Participants also proposed longitudinal analysis of themes, words, and sentiments to provide feedback on emotional patterns and support patient self-management.

4 MINDFULDIARY

MindfulDiary combines a patient-facing conversational journaling app with clinician access to daily records. Its state-based, expert-informed design supports free-form reflection while structuring dialogue and handling sensitive topics.

  • System Overview: MindfulDiary provides patients with an LLM-driven mobile journaling app and clinicians with a dashboard for accessing daily records.The app supports daily record-keeping, while the dashboard enables professionals to review patient dialogue history.
  • Conversation Design: A journaling session moves through rapport building, exploration, and wrap-up, with open- and closed-ended questions supporting reflection and detailed descriptions.A separate Sensitive Topic state handles self-harm or suicidal ideation and can notify psychiatrists for intervention.
  • Conversation Design: The system uses a state machine with dedicated prompts to improve compliance with the intended conversational design.The authors use state-specific prompts instead of one lengthy master prompt because complex prompts can impair LLM task performance.
  • Conversational Pipeline: Each response is generated through a pipeline in which a dialogue analyzer recommends stage transitions and summarizes context before a response generator produces the next message.The response generator combines the current-stage prompt, dialogue summary, and recent conversation turns; both modules use GPT-4.
  • Pilot Evaluation: Pilot evaluation involved psychiatrists and clinical psychologists reviewing prompts and chatbot behavior, followed by a lab study with five recently discharged patients.These evaluations were conducted to assess clinical relevance and safety before field deployment.

5 FIELD DEPLOYMENT STUDY

The field deployment examined MindfulDiary in outpatient care over four weeks, combining patient use of the app with clinician use of its dashboard. The study used mixed quantitative and qualitative analyses to examine engagement and clinical experiences.

  • Study Design: The study deployed MindfulDiary for four weeks with 28 outpatients diagnosed with major depressive disorder.Participants were recruited from a university hospital and the protocol received institutional review board approval.
  • Participants: Initially, 36 patients enrolled, but 28 completed the study after eight participants failed to meet the minimum data-collection requirement.The reported reasons for disengagement were lack of time or decreased interest.
  • Study Procedure: Participants used the patient app for daily journaling, while five psychiatrists received the clinician dashboard for use during clinical care.Clinicians were advised to verify LLM-driven analyses because they could contain inaccuracies.
  • Study Procedure: The patient protocol included an introductory session, four-week deployment, and interviews conducted after the second and fourth weeks.Participants were encouraged to use MindfulDiary when they had noteworthy experiences, especially at the end of the day.
  • Analysis: Researchers analyzed usage with descriptive statistics and repeated-measures ANOVA, and analyzed interaction logs and interviews through open coding and thematic analysis.Psychiatrists reviewed coded data to provide clinical insight and strengthen interpretation of qualitative findings.

6 RESULTS

The results section reports findings on journaling adherence, dialogue patterns, patients’ perspectives, and mental health professionals’ perspectives on clinical use.

  • Results Overview: Field-study results are organized into journaling adherence, dialogue patterns, patients’ perspectives, and mental health professionals’ clinical perspectives.This structure covers both usage behavior and participants’ experiences with MindfulDiary.

6.1 Journaling Adherence

Participants engaged with MindfulDiary regularly across the four-week deployment, producing hundreds of journal entries through sessions that typically lasted several minutes. The generated dialogue primarily used questions to elicit reflection across multiple journaling strategies.

  • Engagement: 501 journal entries were submitted by 28 participants over four weeks, averaging 17.90 entries per participant and 0.62 entries per day.Twenty-two participants used MindfulDiary more than once every two days.
  • Engagement: Each journaling session lasted an average of 438 seconds, with an average dialogue length of 105.6 syllables.Session duration varied substantially across participants, with a standard deviation of 225.97 seconds.

6.2 Dialogue Patterns

MindfulDiary supported sustained, exploratory journaling through question-driven conversations that captured patients’ emotions, activities, thoughts, and health-related experiences.

  • Conversation volume and stages: 4,410 messages were exchanged across the field study, with most conversation focused on exploring patients’ daily lives, emotions, and casual topics.Each session averaged 10.82 messages (SD=2.70).
  • Conversation volume and stages: 62% of messages belonged to Exploration and 30% to Rapport building, while only small numbers were used for Wrapping up or left unselected.Wrapping up accounted for 62 messages, and 14 messages were not selected.
  • Question-driven strategies: 72% of AI messages were questions designed to elicit patients’ daily experiences and emotions.The system used Emotional Exploration, Activity/Behavior Exploration, In-depth Follow-up & Countermeasures, and Future Plan Exploration.
  • Patient response characteristics: Participants’ responses averaged 29.42 syllable counts, with substantial variation from one character to 559 syllable counts.The median was 20 syllable counts and the standard deviation was 35.9.
  • Content of daily records: Daily records covered emotional states, events and activities, thoughts and beliefs, and patterns linked to mental health.Participants described both negative and positive emotions alongside routines, beliefs, and perceived health-related experiences.

6.3 Patients’ Perspectives on MindfulDiary

Patients viewed MindfulDiary’s conversational format as engaging and supportive, helping them share stories and sustain regular journaling.

  • Engagement and sustained use: Participants described MindfulDiary as a space to open up and share their stories while feeling empathy from the system.The dialogue-driven interaction was viewed as more engaging than recording activities or emotions in a conventional diary.
  • Engagement and sustained use: Dialogue with the AI helped participants maintain consistent records and form a journaling habit.Participants linked continued use to the conversational and enjoyable nature of the interaction.

Diverse Interests.

MindfulDiary’s flexible, question-driven conversations accommodated diverse interests and helped patients elaborate experiences, reconsider difficulties, and structure daily narratives.

  • Diverse interests: Participants discussed diverse subjects, including games, webcomics, novels, movies, and hobbies such as Pilates.The LLM’s flexibility allowed documentation tailored to individual interests and preferences.
  • Varied perspectives: AI-generated perspectives helped participants view struggles and emotions from different angles and diverge from ruminating on depressive feelings.One dialogue connected pride in game progress with satisfaction from meeting study goals and planning further study.
  • Detailed reflection: Question-driven journaling made daily reflection less daunting and encouraged more detailed entries about particular events, thoughts, and feelings.Participants contrasted this with writing alone, where they could become stuck or record only brief content.
  • Building narratives: Contextualized follow-up questions and conversation summaries helped participants organize disjointed thoughts into persistent, structured diary narratives.Participants reported that they could tell the story of their day while the system continued and wrapped it into a well-structured entry.
  • Building narratives: The generated narrative connected a difficult day, a father’s praise, and feelings of joy and pride into a coherent reflection.The entry described how making and sharing food transformed an exhausting experience into a joyful one.

6.4 MHPs’ Perspectives on MindfulDiary for Clinical Mental Health Settings

Psychiatrists used MindfulDiary data to understand patients’ everyday experiences and progress more deeply, while identifying concerns about clinical integration, tone, and misuse.

  • Dashboard use: Psychiatrists reviewed MindfulDiary entries alongside medical charts, spending about 5 to 10 minutes per patient depending on severity and focal concerns.They checked PHQ-9 trends, then read summaries of events and documented emotions.
  • Clinical insight: Psychiatrists valued consistent daily entries because they provided more detailed patient data for outpatient visits and supported understanding of patient progress.They viewed ongoing records as useful for observing condition trends between appointments.
  • Clinical insight: MindfulDiary helped psychiatrists gain deeper understanding and empathy by eliciting responses and asking questions that could be sensitive or burdensome to ask directly.The machine-mediated format was perceived to reduce concerns that patients might interpret questioning as a provider taking someone else’s side.
  • Everyday perspectives: Psychiatrists gained insights into patients’ positive feelings and everyday experiences that are difficult to obtain during outpatient consultations.Patients commonly discuss bad experiences in consultations, making positive experiences less visible there.
  • Clinical limitations: MHPs raised concerns that summarized data may omit tone, pace, and other nuances relevant to the Mental Status Examination.They identified raw patient data as an opportunity for further linguistic analysis.
  • Clinical limitations: Clinicians also warned that MindfulDiary could be misused for manipulation or amplify delusions when patients revisit and organize past entries.These concerns were raised for some patients with borderline personality disorders, schizophrenia, or delusions.

7 DISCUSSION

MindfulDiary suggests that LLM-assisted journaling can support adherence, self-reflection, and patient-provider communication, while clinical integration requires expert-aligned evaluation and safeguards. The study also identifies limits involving subjective assessment, hallucination risks, provider-patient dynamics, and generalizability.

  • Supporting Patient Engagement: MindfulDiary may support adherence by helping patients record daily experiences despite depression-related barriers to following interventions.The discussion connects this potential to challenges such as low energy, difficulty with mental processes, and apathy.
  • Supporting Self-Reflection: Varied chatbot perspectives may help patients with depression revisit fixed viewpoints and consider alternative interpretations.The authors relate this possibility to depression’s negative and rigid thought patterns.
  • Patient-Provider Communication: LLM-assisted journaling can lower barriers to daily health-data collection and foster collaborative reflection between patients and providers.MindfulDiary is positioned as supporting both deeper patient records and shared understanding of mental health conditions.
  • Patient-Provider Communication: Clinical integration must account for the possibility that sharing journal content with mental health professionals could lead some patients to exaggerate conditions or needs.This concern was raised by mental health professionals, although the study did not observe such exaggeration.
  • Clinical Evaluation: Mental-health LLM performance is difficult to quantify because success includes subjective outcomes such as emotional well-being, psychological improvement, rapport, and emotional observation.The authors contrast this with medical-imaging evaluation, where objective accuracy metrics are available.
  • Clinical Evaluation: MHP involvement is needed to test sensitive outputs, anticipate hallucinations, and monitor deployment across pre-use, mid-use, and later phases.The paper specifically warns that hallucinations could reinforce or expand delusions and calls for evidence-based tests developed with MHPs.
  • Clinical Safeguards: LLM-generated summaries may contain inaccuracies, biases, or misinterpretations that could affect treatment decisions and patient well-being.The authors recommend careful expert review and suggest comparing generated summaries with phrases directly expressed by patients.
  • Limitations and Future Work: Findings have limited generalizability because participants came from one university hospital, were mostly adolescents, and consulted a fixed number of psychiatrists.The authors call for studies involving subjects from varied backgrounds and note that later GPT versions may produce different conversational behavior.

8 CONCLUSION

The paper presents MindfulDiary as an LLM-driven journaling system for psychiatric outpatients, developed with mental health professionals to support daily reflection within a clinical setting.

  • 8 CONCLUSION: MindfulDiary uses a stage-based LLM chatbot to guide psychiatric patients through daily journaling while following mental-health professional guidelines.The system was developed collaboratively from initial concept building through conversation-flow design and evaluation.
  • 8 CONCLUSION: The system is intended to assist outpatient psychiatric patients with recording daily experiences through prompt questions and answers.Its design keeps the clinical mental-health setting central to the journaling interaction.
  • 8 CONCLUSION: MindfulDiary illustrates how LLM-driven journaling can be designed around clinical mental-health requirements through collaboration with MHPs.The collaboration covered the system’s concept, conversation flow, and evaluation.
Loading 2310.05231v2…