Responsible Evaluation of AI for Mental Health

1 Introduction

• Current evaluations often rely on generic metrics, involve few mental health professionals, and give insufficient attention to safety, equity, and user experience.The authors argue that responsible evaluation should make clinical validity and therapeutic appropriateness visible, even when tools are not intended for clinical deployment.Source: §1 · p. 1
• Technical benchmark wins do not capture the outcomes clinicians, users, and implementation scientists prioritize, including symptoms, functioning, safety, workflow fit, and equitable reach.The paper argues for a common evaluative language pairing psychometric constructs with human-centered and implementation measures.Source: §1 · pp. 1–2
• The proposed taxonomy distinguishes assessment, intervention, and information synthesis, aligning evaluation with clinical goals and each tool type’s risks.The paper illustrates the taxonomy through five case studies and offers guiding principles for responsible evaluation.Source: §1 · p. 2
• The paper targets mental health’s clinical, ethical, and implementation challenges, complementing broader generative-AI evaluation work.It contrasts its normative framework with surveys that catalog tasks and capabilities and reviews focused on chatbot effectiveness or clinician-like abilities.Source: §1 · p. 2

2 Observed Practices

• The review quantitatively analyzed 135 ACL Anthology papers from the past five years; 36% appeared in 2025, and half were double-annotated (Cohen’s kappa=0.67).Two annotators coded the papers, with a senior annotator reviewing disagreements.Source: §2 · p. 2, Table 2 · p. 3
• Across tasks from condition detection to therapeutic chatbots, evaluations often rely on narrow, model-centric criteria.The surveyed literature spans a wide range of mental health tasks, while current evaluation remains limited in scope and rigor.Source: §2 · p. 3
• 50% of papers used only standard AI/NLP metrics; 54% had no human evaluation, and 29% of those with human evaluation involved no mental health experts.Nearly one-fifth omitted evaluation guidelines, and roughly a third did not discuss evaluation limitations.Source: §2 · p. 3
• Generic NLG metrics may not reflect clinical standards or user needs, motivating shared, rigorous evaluation practices rather than criticism of individual papers.The following sections develop a taxonomy, case studies, and guiding principles in response.Source: §2 · p. 3

3 Proposed Taxonomy

• Validity, reliability, implementation, and maintenance form a multidimensional evaluation space; no single score captures AI mental health tools’ full opportunities and risks.The taxonomy maps these principles onto three common applications: assessment, intervention, and information synthesis.Source: Table 3 · p. 4, §3 · p. 5
• Assessment tools require evaluation of convergent, discriminant, and criterion validity; reliability across time and populations; and implementation equity, feasibility, and bias mitigation.Maintenance includes monitoring generalizability, performance drift, population outcomes, unintended consequences, and evolving language norms.Source: §3 · p. 5
• Interventions require evaluation of therapeutic ingredients, efficacy and harm, durable effects across settings, real-world symptom improvement, user engagement, and risks such as overuse.Implementation also considers clinician usability and off-label use; maintenance tracks persistent benefits and risks of avoiding human care.Source: §3 · p. 5
• Information-synthesis tools require evaluation of clinical accuracy, effects on clinician time or documentation, reproducibility across scenarios, workplace usefulness, and patient impact.Maintenance also considers tool-level effects such as reduced burnout, over-reliance, or skill atrophy.Source: §3 · p. 5

4.1 Study I (Assessment): LLM rating scales for psychometric assessment of patient engagement

• Eberhardt et al. (2025) reported internal consistency (McDonald’s ω = 0.953) and good CFI and SRMR fit, though RMSEA = 0.108 indicated unexplained variance.The study analyzed 1,131 sessions from 155 patients; future work should test robustness, fairness, and safety across populations and contexts.Source: §4.1 · p. 6

4.2 Study II (Assessment): Natural language response formats for assessing depression and worry

• Gu et al. (2025) found r = .60-.79 convergent validity, but combined models' r = .88-.95 indicated reduced discriminant validity; open responses were richer yet up to 4 times slower.Two-week test-retest correlations showed moderate to strong stability, and prospective accuracies matched cross-validated estimates.Source: §4.2 · pp. 6–7

4.3 Study III (Intervention): Evaluating the capabilities of LLMs vs. human therapists to generate personalized interventions

• Bar-Shachar et al. (2025) found AI interventions clinically relevant and sometimes near human quality, but less deep and personalized; ecological validity, fairness, and implementation remained unaddressed.The system used specialized agents and a Judge-LLM to select interventions based on dialogue and the patient’s evolving state.Source: §4.3 · p. 7

4.4 Study IV (Intervention): A clinically-grounded framework for evaluating LM-assisted cognitive restructuring

• After finding reduced effectiveness for adolescents aged 13–17, a follow-up trial found targeted adaptations—simpler, more casual reframes—improved helpfulness without affecting other groups.In a field study on the MHA website (N = 15,531), the majority of participants reported reduced negative emotion and helpfulness of reframes for overcoming negative thoughts. Flagged content was 0.65%, and the tool has since served over 160,000 users.Source: §4.4 · pp. 7–8

4.5 Study V (Information synthesis): Hierarchical LLM-VAE tool for clinically meaningful timeline summarization

• Song et al. (2024) paired first-person evidence summaries with clinical summaries, but social-media-only data limited ecological validity and clinical generalizability.The evaluation also left safety risks, demographic and linguistic fairness, clinical usability, practitioner acceptance, and long-term stability unexamined.Source: §4.5 · p. 8

Evaluation foundations

• Evaluation standards differ by tool type: assessments need clinical validity, interventions need benefit, safety, and acceptability evidence, and synthesis tools need workflow or decision improvements.The paper identifies relatively few tools reaching implementation or maintenance.Source: §5 · p. 8

Evaluation maturity

• The proposed pipeline increases evaluation depth across exploratory technical validation, human-centered validation, and deployment-stage contextual assessment.The layers cover retrospective technical testing, prospective or external validation and user studies, then workflow integration, long-term impact, equity, safety, and monitoring.Source: §5 · pp. 8–9

Safety, fairness, and maturity

• 68% of 60 sampled papers were exploratory and 32% validation-stage; most case studies lacked explicit safety or fairness evaluation.The paper calls for proactive stress tests of hallucinations, inappropriate reassurance, and biased outputs, including across demographic, cultural, and linguistic groups.Source: §5 · p. 9

Practical implications across maturity stages

• Evaluation expectations should scale with system claims and intended use, while remaining adaptable as clinical theories and constructs evolve.The paper notes a shift toward dimensional and dynamic frameworks for mental health states.Source: §5 · p. 9

Early-stage evaluation

• Researchers without clinical access can use simulations, guideline-based scenarios, demographic bias audits, expert-informed annotation, and clinically aligned LLM-as-a-Judge assessments as early proxies.These proxies are not substitutes for real-world validation, so claims should remain calibrated to system maturity.Source: §5 · p. 9

6 Conclusion

• Analysis of 135 *CL publications found generic metrics, limited mental health professional involvement, and insufficient attention to safety, equity, and real-world use.The paper characterizes these practices as misaligned with responsible deployment requirements.Source: §6 · p. 9
• The proposed framework introduces assessment-, intervention-, and information synthesis-oriented AI support types.Each type has distinct risks and evaluation needs.Source: §6 · p. 9
• The taxonomy is intended to align evaluation practices more closely with each system’s intended use and risks.Source: §6 · p. 9

• The taxonomy and evaluation pathways are conceptual rather than empirically validated, and their applicability may vary across clinical, cultural, and linguistic contexts.Source: Limitations · p. 9
• The analysis draws on published case studies that may not represent the breadth of ongoing work or emerging AI mental health tools.Source: Limitations · p. 9
• The paper outlines evaluation principles but provides no detailed operational metrics, leaving refinement and adaptation for future work.Source: Limitations · p. 9

• Psychologically grounded metrics draw on psychological theory, clinical research, or expert input, whereas AI/NLP metrics measure computational performance and are largely agnostic to therapeutic soundness.Source: A Details about the surveyed papers · p. 21
• Tables 4–17 list surveyed papers and observed practices, with the first five installments shown in Tables 4–8.Source: A Details about the surveyed papers · p. 21, Table 4 · p. 22, Table 5 · p. 23
• Table 18 provides an overview of the diverse tasks addressed by the surveyed papers.Source: A Details about the surveyed papers · p. 21, Table 18 · p. 36
• The surveyed papers address conditions ranging from anxiety and depression to suicide ideation, PTSD, ADHD, OCD, and addiction.Source: A Details about the surveyed papers · p. 21