PaperlayerView exact revision on arXiv

Source-linked AI summary

Training language models in simulation for psychiatric intake

Samuel Schmidgall, Tim Strother, Alex Bijamov, Min Woo Sun, Anil Palepu, Akshay Goel, Ulysse Marteau-Ferey, Théo Moutakanni, Yubin Kim, Joseph Breda, Christina Chen, Justin Chen, Paul Jhun, Andrew Marshall, Sarah Montgomery-Taylor, Eun Kyoung Hong, Shashank Ravi, Kameron C. Black, David J. H. Wu, Austin Schoeffler, John Lyon Havlik, Elizabeth C. Stade, Jared Moore, Johannes C. Eichstaedt, Logan Pierce, Jeff Jopling, Johannes Thrul, Thomas H. McCoy, Neguine Rezaii, Mark S. Chen, Ellen E. Fitzsimmons-Craft, Shirley Wang, Desmond C. Ong, Katharina Schultebraucks, Torrey Creed, Bhavna Daryani, Aleksandra Faust, S. Sara Mahdavi, Caroline Figueroa, Eric Lin, Kate H. Bentley, Ernest Lim, Zhihan Jiang, Xuhai "Orson" Xu, Daniel McDuff, John Torous, Yun Liu, Po-Hsuan Cameron Chen, Mike Schaekermann, Joelle Barral, Alvin Rajkomar, Tao Tu, Lin Yang, Isaac Galatzer-Levy, David Racz, Valentin Liévin

arXiv:2610.40476v1cs.AIcs.CL

TL;DR

Psychiatric intake relies on the clinical interview as a principal diagnostic instrument. This paper trains a language model with multi-turn reinforcement learning in simulated encounters and evaluates it against its identically prompted base model. The trained agent scored higher in 76.7% of matched psychiatrist-enacted scenario pairs, with gains across clinical examination, patient-centered communication, and diagnostic formulation.

  • Problem

    The clinical interview remains a principal diagnostic instrument in psychiatry.

    Source: §1 · p. 1

  • Method

    The authors trained a language model with multi-turn reinforcement learning across simulated psychiatric encounters and validated it against its identically prompted base model in three settings.

    Source: §1 · p. 2

  • Results

    76.7% of matched scenario pairs favored the trained agent, with gains in clinical examination skills, patient-centered communication, and differential diagnosis formulation.

    Source: §3 · p. 13

  • Takeaways & Limitations

    The agent’s psychiatrist-rated performance exceeded its base model in standardized intake, but the research prototype is not a substitute for prospective validation with real patients.

    Source: §3 · p. 16

    Simulated and clinician-enacted patients cannot reproduce the full range of rare conditions, backgrounds, motivations, or disorganized communication encountered in clinical practice.

    Source: §3 · p. 15

Abstract

from arXiv · show

In psychiatric practice, the clinical intake interview remains a primary diagnostic instrument, requiring the clinician to establish rapport while disentangling complex, overlapping presentations. Here, we show that multi-turn reinforcement learning (RL) in simulation environments improves language model performance on structured text-based psychiatric intake, spanning history-taking, differential diagnosis, and initial management planning. In a randomized, double-blind crossover study ($N = 60$ matched scenario pairs, 120 encounters) comparing the RL-trained agent against its identically prompted base model, 32 board-certified psychiatrists role-played standardized patients while a separate panel of 15 independent, board-certified psychiatrists scored the de-identified transcripts. The trained agent outperformed the base model in 76.7% of matched scenario pairs ($p < 0.0001$), with gains on an adapted clinical-examination rubric ($d_z = 0.77$, $p < 0.0001$), patient-centered communication (82.0% vs. 65.2%, $p < 0.001$), and differential diagnosis appropriateness (68.3% vs. 33.3%, $p < 0.001$). Across 127 psychiatrist-validated simulated intake scenarios, the trained agent nearly doubled its base model's clinical rubric score (0.706 vs. 0.379; composite reward 0.564 vs. 0.294, $p < 0.0001$). To systematically probe conversational safety, we established two Delphi-consensus taxonomies, of clinical safety risks and relational competencies, from which 256 challenging adversarial presentations were derived; we find that training in simulation environments more than halved under-triage of acute safety risks (12.9% vs. 30.5%, $p < 0.0001$). Simulation environments thus provide a controlled testbed for training and stress-testing psychiatric conversational agents, with gains that transferred to psychiatrist-enacted encounters, though residual gaps require safeguards and clinician oversight.

1. Introduction

The study tests whether multi-turn reinforcement learning in simulated psychiatric encounters can improve language models’ combined diagnostic, interpersonal, and safety skills. Evaluations found gains over the identically prompted base model, while remaining limited to controlled text consultations rather than real-care outcomes.

Training approach

  • The model was trained with multi-turn reinforcement learning in simulated psychiatric encounters, jointly optimizing diagnostic accuracy, patient-centered communication, and clinical safety.The trained agent and base model shared the same clinical system prompt and structured tools in each evaluation.Source: §1 · p. 2

Human expert evaluation

  • 76.7% of matched scenario pairs favored the trained agent, which also improved its clinical-examination rubric score (d_z = 0.77, p < 0.0001).In the blinded crossover evaluation, 32 board-certified psychiatrists role-played patients across 120 consultations, and 15 independent psychiatrists scored de-identified transcripts; communication, differential diagnosis, and management plan appropriateness also improved.Source: §1 · pp. 2, 4

Automated psychiatric intake evaluation

  • 0.706 vs. 0.379 on clinical rubric score and 0.564 vs. 0.294 on composite reward favored the trained agent across 127 psychiatrist-validated scenarios (p < 0.0001).The scenarios covered eight diagnostic domains and four difficulty tiers, with critical information gated behind rapport and inquiry; the advantage persisted under an independent judge model.Source: Figure 1 · p. 3, §1 · p. 4

Adversarial stress-testing

  • 12.9% vs. 30.5% under-triage of acute safety risks favored the trained agent across 256 adversarial presentations (p < 0.0001).The presentations were derived from Delphi-consensus taxonomies covering six clinical safety-risk domains and five relational-competency domains; residual failures remained.Source: Figure 1 · p. 3, §1 · p. 4

Scope and limitation

  • All evaluations used synchronous English-language text consultations grounded in US psychiatric practice guidelines, so they establish controlled-setting competence rather than real-care outcomes, safety, or efficacy.Any clinical use would require direct clinician oversight rather than autonomous operation.Source: §1 · p. 4

2. Results

Simulation-trained models outperformed their identically prompted base model in blinded psychiatrist-enacted intakes and expert-validated simulated scenarios, improving clinical, communication, diagnostic, management, and safety outcomes. However, incomplete inquiry and acute-risk under-triage persisted, and simulated-patient evaluation inflated measured gains.

2.1. Simulation-trained agent improves upon base model in blinded human evaluation

  • The simulation-trained agent won 76.7% of matched OSCE pairs and scored 3.04 vs. 2.32 on the adapted PACES clinical-skills rubric.The OSCE included 60 matched scenario pairs; the rubric difference had d_z = 0.77, p < 0.0001.Source: §2.1 · p. 5, Figure 2 · p. 6
  • Favorable differential appropriateness was 68.3% vs. 33.3%, while patient-centered communication adherence was 82.0% vs. 65.2%.The trained agent also scored higher on all four psychiatrist-rated diagnostic dimensions and all 11 communication checklist items in direction of effect.Source: Figure 2 · p. 6, §2.1 · p. 7
  • Overall management-plan appropriateness was 3.67 vs. 3.18; appropriate treatments were recommended in 46.7% vs. 26.7% of encounters.Appropriate diagnostic investigations were recommended in 50.0% vs. 28.3%; less than comprehensive treatment was still recommended in 53.3% of trained-agent encounters.Source: Figure 2 · p. 6, §2.1 · pp. 7–8

2.2. Training gains hold across domains and difficulty tiers in automated evaluation

  • Training improved scores across all difficulty tiers and rubric categories, but critical-question omissions remained 60.6% vs. 96.9%.The Edge-tier trained-agent score was 0.513, exceeding the base model’s Easy-tier score of 0.335; the omission measure covers any missing checklist topic, not only acute safety errors.Source: §2.2 · pp. 8, 10
  • Across 127 expert-validated scenarios, the trained agent scored 0.706 vs. 0.379 on the clinical rubric and 0.564 vs. 0.294 on normalized composite clinical reward.The composite-reward difference was +0.270, p < 0.0001.Source: §2.2 · p. 8, Figure 3 · p. 9
  • On the 60 shared OSCE scenarios, simulated patients enlarged the mean trained–base gain from 0.21 to 0.34, while autorater and psychiatrist ratings showed only 68.3% raw agreement.The autorater preferred trained-agent transcripts in 55 pairs versus 46 for psychiatrists; simulated and clinician-patient scores correlated weakly on per-scenario differences.Source: Figure 3 · p. 9, §2.2 · p. 10

2.3. Adversarial safety evaluation reveals residual failures

  • Across 256 adversarial scenarios, normalized composite safety reward was 0.672 vs. 0.357 and under-triage was 12.9% vs. 30.5%, but concealed-crisis failures persisted.The trained agent’s missed-red-flag rate was 6.2% vs. 10.9%; under-triage occurred in 33 of 256 scenarios, often when crises were concealed behind benign somatic complaints.Source: Table 2 · p. 12, §2.3 · p. 13

3. Discussion

Simulation-based multi-turn reinforcement learning improved psychiatric intake performance over an identically prompted base model, but the discussion emphasizes evaluation limits and substantial clinical-scope constraints. The authors support further study only in clinician-supervised roles, not autonomous deployment.

Discussion

  • In psychiatrist-enacted OSCEs, the trained agent outscored its base model in 76.7% of matched pairs, improving clinical examination, communication, and diagnostic formulation.The comparison used identical prompts, tool interfaces, and reasoning-token budgets to isolate the combined effect of the simulation RL pipeline.Source: §3 · p. 13
  • Across 127 simulated cases, the trained agent nearly doubled its base model’s pre-penalty clinical rubric score; across 256 adversarial cases, it more than halved under-triage, though failures remained.The authors argue that post-hoc filters can catch harmful generated content but cannot remedy omissions such as failing to ask about concealed risks; they propose structured checklists, dialogue-state monitors, and clinician verification.Source: §3 · p. 14
  • Diagnostic appropriateness was 68.3% vs. 33.3%, while comprehensive treatment-planning appropriateness was 46.7% vs. 26.7%, consistent with greater gains in elicitation-dependent tasks.The discussion attributes the contrast to rapport and structured inquiry enabling gated disclosures, while treatment planning depends more on static knowledge and subjective case-level choices.Source: §3 · p. 14
  • Clinician-patients rated the trained agent as more empathic, but these communication gains do not establish therapeutic alliance, which requires sustained multi-session relationships.The authors note that frequent not-applicable responses on the multi-session WAI-SR fit this distinction; whether training improves alliance longitudinally remains unknown.Source: §3 · p. 14
  • The autorater correlated moderately with psychiatrist ratings (ρ= 0.57–0.58), but reproduced neither the effect size (d_z= 2.11 vs. 0.71) nor scenario-level gain pattern (ρ= 0.09).Automated evaluation preserved the direction of training effects across clinical categories, so it can order and stress-test models but cannot estimate differences or identify which scenarios favor one model.Source: §3 · pp. 14–15

Conclusion

  • Despite major access gaps—more than 75% untreated in low- and middle-income countries and fewer than one in five with major depression receiving minimally adequate care—the authors propose only clinician-supervised assistance, not autonomous assessment or clinician replacement.They call for prospective study with real patients and longitudinal follow-up under institutional clinician oversight; any near-term role would be a provisional pre-visit intake for clinicians to review and build upon.Source: §3 · pp. 15–16

Limitations

  • Methodological constraints include synthetic cases, comparison only with the same model’s base checkpoint, and reward and primary evaluation derived from the same model family.Psychiatrists role-played patients, each OSCE transcript received one psychiatrist rating, and the live OSCE interface differed from training; the independent evaluator preserved a substantial but attenuated training gain. The authors call for real-patient and professional-standardized-patient validation, human clinical comparators, and further baselines.Source: §3 · pp. 15–16
  • The findings cover single-encounter, English-language text intake—not longitudinal care or comprehensive examination—and the authors flag jurisdictional and ethical risks.Text excludes physical and non-verbal observations, while US-focused rubrics and semi-structured intake limit generalization; future translation would require external safeguards, clinician oversight, and deterministic acute-risk escalation.Source: §3 · p. 16

4. Methods

The methods describe clinician-designed psychiatric scenarios, dynamic patient simulation, safety-weighted reinforcement learning, and expert assessment across simulated and psychiatrist-enacted encounters. They also report expert-validation agreement and define a three-setting comparison of human and simulated assessments on OSCE scenarios, while noting that training labels are synthetic and text-based assessment omits important nonverbal findings.

4.1. Psychiatric scenario generation

  • 48,243 scenarios passed verification in a four-stage pipeline sampling 15 prevalence-weighted diagnostic domains, constructing patient profiles, generating cases, and calibrating difficulty.Automated checks addressed clinical coherence, hidden-fact leakage, diversity, and deduplication. Training diagnoses and treatment recommendations were model-generated rather than individually clinician-reviewed, so they are synthetic training signals, not validated standards.Source: §4 · p. 17, §4.1 · pp. 17, 19

4.2. Patient simulator

  • The patient simulator used scenario profiles and turn-level hidden states to make patient responses vary with mood, reliability, clinician communication, and established rapport.A two-call architecture updated behavioral state before generating dialogue, enabling consistent role-play of high-risk and adversarial scenarios without safety-filter refusals.Source: §4.2 · pp. 19, 21

4.3. Reward structure and automated clinical evaluation

  • The hierarchical evaluator balanced clinical performance and communication while applying large penalties to detected safety failures.The clinical rubric scores 28 items across five categories, with diagnostic and management domains carrying approximately half the clinical score. Text-inferable mental status assessment is limited to written dialogue and cannot capture nonverbal cues such as gait, eye contact, or psychomotor signs.Source: §4.3 · pp. 21–22

4.4. Reinforcement learning in simulated environments

  • GRPO trained the base policy on simulated encounter groups using a curriculum weighted 48% psychiatric, 35% general telehealth, 15% targeted history-taking, and 2% red-team scenarios.The clinical interface supported ranked diagnoses, urgency, management plans, patient-facing summaries, SOAP notes, and encounter termination; evaluation used the same system prompt for both models.Source: §4.4 · pp. 23–24

4.5. OSCE study design

  • The OSCE used 60 scenarios in blinded, randomized crossover consultations between 32 psychiatrist-actors and the identically prompted base and trained models.Fifteen independent psychiatrists rated de-identified transcripts; the prespecified primary endpoint was the share of scenario pairs where the trained agent earned a higher aggregate transcript score.Source: §4.5 · pp. 24–25

4.6. Simulated psychiatric intake evaluation design and expert validation

  • 127 clinician-validated intake scenarios were stratified across diagnostic domains and difficulty, with critical facts gated behind inquiry, empathic probing, or trust-building.A panel of 54 psychiatrists reviewed 128 candidate scenarios using three independent reviews per case and an eight-criterion rubric; 127 scenarios received final approval.Source: §4.6 · pp. 25–26
  • AC1 was 0.9393, with a 96.2% criterion pass rate for the simulated psychiatric intake evaluation validation protocol.The full validation protocol is in Appendix C.1.Source: §4.6 · p. 26

4.7. Three-way concordance on the OSCE scenarios

  • The OSCE concordance analysis compared psychiatrist and autorater grades of human-patient transcripts with autorater grades of four simulated rollouts per scenario and model.It paired autorater categories with the nearest human instrument, then assessed cross-scenario Spearman correlations, paired effect sizes, length sensitivity, transcript-pair agreement, and uncertainty using 10,000 bootstrap resamples.Source: §4.7 · p. 26

Ethics Statement

The study withheld its codebase because the implementation relies on internal Google infrastructure and because unmonitored medical AI poses safety concerns. All scenarios and transcripts were synthetic, and no real patient data were used.

Ethics Statement

  • An independent central IRB reviewed the study; clinicians provided informed consent, and all clinical scenarios were synthetic.The psychiatric OSCE and scenario validation study was approved under protocol GH-ACE-01 (Pro04092399); the Delphi consensus studies received exempt determinations under GH-DCA-01 (Pro00094566) and GH-DMH-01 (Pro00094569).Source: Ethics Statement · p. 27

Data Availability

  • No real patient data were used; the authors are exploring release of 127 expert-validated OSCE scenarios, evaluation rubrics, and two Delphi-derived taxonomies.The DDxPlus dataset75 used during development is publicly available.Source: Data Availability · p. 27

Code Availability

  • The codebase is not open-sourced because the implementation relies on internal Google infrastructure and unmonitored medical AI in medical contexts raises safety concerns.The authors plan to work with research partners, regulators, and healthcare providers to further validate safe onward uses of their medical models.Source: Code Availability · p. 27

Competing Interests

  • Alphabet funded the study; Alphabet-employed authors may own company stock through standard compensation, while remaining authors declared no competing interests.Source: Competing Interests · p. 28

A. Extended Methodology · A.1. Psychiatrist-as-Patient Recruitment and Protocol · A.2. Evaluator Recruitment and Blinding

The study used blinded, counterbalanced crossover encounters with the same psychiatrist role-playing each scenario’s paired conditions. Separate psychiatrist evaluators scored de-identified transcripts under a blinded, calibrated protocol, with one evaluator per transcript.

A.1. Psychiatrist-as-Patient Recruitment and Protocol

  • Each crossover pair used the same blinded psychiatrist-actor, who received a role sheet with disclosure rules and no diagnostic or management scoring key.Actors enacted both conditions for a scenario, holding any prior familiarity constant across the paired encounters.Source: §A.1 · p. 45
  • Sixty of 127 validated scenarios were stratified by domain, difficulty, demographics, and crisis content; each yielded two counterbalanced encounters separated by a 48-hour washout.After encounters, actors completed WAI-SR, GMCPQ, PACES, and PCCBP measures and provided qualitative feedback.Source: §A.1 · p. 45

A.2. Evaluator Recruitment and Blinding

  • Recruitment covered transcript evaluation, scenario validation, and standardized-patient role-play, requiring board certification, at least two years’ post-certification practice, active caseload, and no AI-system development involvement.Recruitment targeted variation in practice setting, patient population, region, and career stage; evaluators with recent publications on AI in psychiatry were excluded.Source: §A.2 · p. 45
  • Fifteen psychiatrists from the 54-member validation panel scored transcripts, disjoint from the 32 actors; evaluators were blinded to model identity and condition, and prior scenario familiarity applied to both paired conditions.Transcript grading required use of each scenario’s validated clinical answer key.Source: §A.2 · p. 45
  • Evaluators were blinded to the hypothesis, conditions, model identities, and pairing; transcripts were standardized and independently randomized, while encounter length remained for sensitivity analysis.Formatting markers and response-length metadata were removed, but the number of conversational exchanges remained visible.Source: §A.2 · p. 46
  • Before live scoring, evaluators independently rated five pilot transcripts, then used a 90-minute calibration to anchor rubric scales, resolve ambiguities, and set boundary-case rules.Pilot transcripts were excluded from analysis, and the finalized rubric included exemplar anchors.Source: §A.2 · p. 46
  • Each of 120 transcripts was independently scored by one psychiatrist, with balanced assignments and no evaluator scoring both members of a pair; inter-rater reliability was therefore not applicable.Scoring used a secure interface enforcing dimension-by-dimension entry; internal consistency reliability was reported separately.Source: §A.2 · p. 46

A.3. Working Alliance Inventory (WAI-SR)

The WAI-SR comparison was inconclusive: many clinician-patients marked psychotherapy-specific items inapplicable, and the evaluable sample was small. Low scores in both conditions may reflect the limits of single-encounter text-based consultations for establishing therapeutic alliance.

A.3. Working Alliance Inventory (WAI-SR)

  • Among 21 evaluable pairs, WAI-SR composites did not differ significantly (Simulation-trained: 2.45±0.97; Base model: 2.48 ± 1.31; p=0.75).The comparison used a Wilcoxon signed-rank test.Source: §A.3 · p. 46
  • WAI-SR data were available for 21 of 60 scenario pairs (35%), and 50 of 120 encounters (41.7%) returned all-not-applicable responses.Clinician-patients identified psychotherapy-specific alliance items as inapplicable to a single diagnostic intake encounter; instrument position and respondent fatigue were considered unlikely explanations for missing data.Source: §A.3 · p. 46
  • The comparison is inconclusive because attrition and sample size preclude confirming or ruling out a difference, while non-response may bias results.The pair-level non-response rate was 65%; low scores in both conditions may reflect the limitations of single-encounter text-based consultations for establishing therapeutic alliance.Source: §A.3 · p. 46

A.4. Evaluation Instruments

Human OSCE encounters were assessed with blinded psychiatrist transcript ratings and immediate post-encounter feedback from the psychiatrist portraying the standardized patient. Together, the instruments covered clinical performance, diagnostic and management quality, confabulation, and patient experience.

A.4. Evaluation Instruments

  • Blinded psychiatrists evaluated de-identified consultation transcripts and submitted clinical summaries against validated scenario answer keys using 28 active items.The transcript survey combined clinical consultation and communication skills, differential diagnosis, investigations, treatments and management, and clinical confabulation assessments.Source: §A.4 · pp. 47–48
  • The 18-item PACES consultation and communication assessment covered history elicitation, clinical explanation, judgment, and patient-centered communication.Items used a 5-point scale or N/A; together with four patient-facing PACES items on the clinician-patient form, they constituted the 22-item PACES framework. The assessment was adapted from the RCP PACES examination and AMIE specialist evaluation framework [43] [10].Source: §A.4 · p. 47
  • Separate transcript items assessed differential diagnosis, investigation and treatment appropriateness, management planning, and clinical confabulation.The survey used four differential-diagnosis items and five investigations, treatments, and management items; the overall plan item included emergency department referral for red-flag presentations. A binary human-scored confabulation item was distinct from the broader automated hallucination penalty in the simulated intake rubric.Source: §A.4 · pp. 47–48
  • The standardized patient completed a 38-item, four-part experiential feedback battery immediately after each live text consultation, with optional qualitative feedback.Its components were the 12-item WAI-SR alliance measure [47], the 11-item GMCPQ communication survey [10] [46], four patient-facing PACES empathy and welfare items [10] [43], and 11 Patient-Centered Communication Best Practice behaviors [10] [44]. GMCPQ items included a communication composite and two binary global retention items.Source: §A.4 · p. 48

A.5. Statistical Analysis … B.2. OSCE clinician-patient panel characteristics

The simulation-trained agent won 76.7% of matched scenario pairs on overall clinical competence. The transcript rubrics showed high internal consistency, but the study could not estimate cross-rater reproducibility.

A.5. Statistical Analysis

  • 46 of 60 scenario pairs (76.7%; 95% CI [64.0%, 86.6%], p<0.0001) favored the simulation-trained agent on overall clinical competence.The primary outcome was the independent evaluator head-to-head win rate, tested against a 50% null using a two-sided exact binomial test.Source: §A.5 · p. 48
  • Secondary composite win rates were 80.7% for PACES and 77.8% for differential diagnosis; secondary and exploratory p-values were nominal, without multiple-comparison correction.The PACES management and behavioral-enablement sub-dimension also favored the trained agent (36 of 56, 64.3%, p=0.044), as did the standalone management-plan item (3.67 vs. 3.18, dz=0.31, p=0.018).Source: §A.5 · p. 52
  • Matched continuous outcomes were analyzed with Wilcoxon signed-rank tests, while paired binary outcomes used McNemar tests; effect sizes were matched to outcome scales.Automated simulated-intake and adversarial-safety evaluations compared matched scenarios with two-sided paired tests and 95% bootstrap confidence intervals over 10,000 scenario resamples.Source: §A.5 · p. 52

A.6. Instrument Reliability for OSCE Transcript Ratings

  • The PACES rubric had Cronbach’s α=0.963 (18 items, 70 complete cases), and the four-item differential-diagnosis rubric had α=0.925 (119 cases).Per-item sample sizes for PACES ranged from 102 to 119; analyses used all available ratings rather than only complete cases.Source: §A.6 · p. 53
  • Because each transcript had one evaluator, cross-rater reproducibility could not be estimated; α values above 0.95 may reflect redundancy or halo effects, limiting absolute-score interpretation.Internal consistency was assessed instead of inter-rater reliability because each transcript was scored by a single independent psychiatrist.Source: §A.6 · p. 53

B.1. Expert panel characteristics

  • Supplementary Table 4 summarizes self-reported demographic, professional, and geographic characteristics of a 54-member expert psychiatrist panel.Panelists held primary board certification in general adult psychiatry; individual-level data were withheld to preserve anonymity.Source: §B.1 · p. 53, Table 4 · p. 54

B.2. OSCE clinician-patient panel characteristics

  • The OSCE clinician-patient cohort comprised 32 board-certified psychiatrists from the vetted expert network, while transcript evaluators were strictly disjoint from clinician-patient actors.Crossover encounters were matched within the same clinician-patient; Supplementary Table 5 summarizes the cohort’s self-reported demographic and professional characteristics.Source: §B.2 · p. 53, Table 5 · p. 55

C. Simulated Psychiatric Intake Evaluation Suite … C.1.6. Clinical Specification Framework

The evaluation suite comprises 127 expert-curated simulated intake scenarios, with validation and coverage addressing clinical realism, diagnostic breadth, and cross-diagnostic competencies. Its clinical specification framework organizes cases along five axes spanning diagnostic, treatment, communication, and safety challenges.

C.1. Case Generation Protocol for Intake Evaluation

  • The 127 expert-curated evaluation scenarios require clinical-grade rigor and human expert validation at every stage, unlike the 48,243 automatically generated training scenarios.Source: §C.1 · p. 56

C.1.1. Overview

  • Clinicians designed 128 specifications across four difficulty tiers, then a language model instantiated them as vignette-and-rubric packages reviewed over three iterative rounds.The final 127 scenarios had a 99.2% net retention rate, with one rejected.Source: §C.1.1 · p. 56

C.1.2. Scenario package structure

  • Each scenario package pairs a structured patient vignette with a scoring rubric, using free, inquiry-gated, and trust-gated disclosures and observable transcript-based checklist items.Vignettes include patient context and behavioral cues; rubrics specify diagnoses, differential, treatment, risk, investigations, follow-up, and concrete clinical actions.Source: §C.1.2 · pp. 56–57

C.1.3. Validation rubric

  • Reviewers applied eight thresholded criteria covering realism, diagnostic coherence and validity, difficulty, disclosure and rubric quality, treatment accuracy, and harmful content.Criteria included clinically coherent ground truth, transcript-observable actions, guideline-concordant treatment, and rejection of dangerous errors modeled as correct care.Source: §C.1.3 · pp. 59–61

C.1.4. Review outcomes and acceptance rule

  • The three-round review produced a final 127-scenario evaluation suite with 99.2% net retention; one candidate was rejected.At least two of three reviewers had to approve each scenario, directly or after revision.Source: §C.1.4 · pp. 61–62

C.1.5. Diagnostic and Competency Categories

  • The suite spans six core diagnostic categories plus medical mimics and challenging encounters involving safety, interpersonal conflict, legal complexity, and ethical obligations.The core categories cover mood, anxiety and stress-related, psychotic-spectrum, substance-use, personality and complex, and other presentations.Source: §C.1.5 · p. 62

C.1.6. Clinical Specification Framework

  • Each scenario was constructed according to a five-axis clinical specification framework.Source: §C.1.6 · p. 62
  • The specification varies cases across five axes: presentation type, comorbidity burden, pharmacological complexity, communication dynamics, and crisis/safety risk.Cases range from textbook presentations and no comorbidity to atypical cases, interacting conditions, prescribing traps, alliance ruptures, and crises patients may conceal or minimize.Source: §C.1.6 · p. 63

C.1.7. Difficulty Tier Specifications · C.1.8. Cultural Formulation and Equity Architecture

The scenario architecture spans four difficulty tiers and embeds cultural factors in patient narratives, clinical reasoning, and disclosure. It also uses anti-stereotyping safeguards while acknowledging that clinician-assessed plausibility cannot replace validation by represented communities.

C.1.7. Difficulty Tier Specifications

  • The initial pool had 128 specifications; one was rejected, leaving 29 easy, 32 medium, 36 hard, and 30 edge scenarios.Reviewer feedback also prompted reclassification across tiers.Source: §C.1.7 · p. 63
  • Easy cases feature a single textbook-concordant diagnosis and no acute safety concerns, while medium cases add mild ambiguity, one complicating factor, and occasional passive safety concerns.The final tier sizes were 29 easy and 32 medium scenarios.Source: §C.1.7 · p. 63
  • Hard cases combine significant diagnostic or pharmacological complexity with moderate alliance ruptures, and about 75% include active safety concerns; edge cases add severe complexity and crises requiring emergency intervention.Hard scenarios number 36 and edge scenarios 30; edge cases are described as challenging even experienced attending psychiatrists.Source: §C.1.7 · p. 63

C.1.8. Cultural Formulation and Equity Architecture

  • Cultural factors are built across four domains: individual identity, conceptualizations of distress, psychosocial stressors and resilience, and features of the clinical relationship.Modeled elements include culturally shaped idioms of distress, help-seeking influences, and expectations about authority, pacing, and formality.Source: §C.1.8 · pp. 63–64
  • The generator prompt specifies cultural idioms, explanatory models, relationship factors, and strengths, while requiring narrative integration and within-culture variation to resist stereotyping.The prompt includes an explicit anti-stereotyping constraint against treating culture as isolated metadata.Source: §C.1.8 · p. 64
  • Three safeguards address stereotyping: orthogonal demographic sampling, mandated within-culture heterogeneity, and multi-round psychiatrist audits that reject reductive caricatures.Sampling constrains expected diagnosis-group pairings but does not guarantee finite-corpus balance, and residual imbalances were not separately audited. Reviewers were not matched to cultural backgrounds; clinician-assessed plausibility is a necessary but insufficient proxy, and represented-community validation is needed before deployment.Source: §C.1.8 · p. 64

C.2. Simulated Psychiatric Intake Evaluation: Full Scoring Results

Across 127 simulated psychiatric intake scenarios, the simulation-trained agent retained an advantage under out-of-family grading and alternative clinical rubric weightings. Its lower critical-question omission and hallucination rates further distinguish its clinical performance from the base model.

C.2. Simulated Psychiatric Intake Evaluation: Full Scoring Results

  • Both models were scored on the same 28-item clinical rubric and six consolidated safety/completeness flags across 127 simulated scenarios.Results were presented as difficulty-tier-stratified composite reward and pre-penalty scores across five rubric categories, with safety and completeness penalty rates; Supplementary Table 6 reports composite reward by tier.Source: §C.2 · pp. 64–65, Table 6 · p. 65
  • 0.443 [0.415, 0.471] vs. 0.278 [0.268, 0.289]: the trained agent’s advantage held overall and across all four difficulty tiers under out-of-family GPT-5.6-Luna grading.The model difference was 0.165 under GPT-5.6-Luna versus 0.270 under Gemini 3.1 Pro (p < 0.0001); absolute scores were lower with GPT-5.6-Luna.Source: §C.2 · p. 65
  • 0.706 weighted vs. 0.710 unweighted for the trained agent, compared with 0.379 vs. 0.407 for the base model, preserved the clinical ranking across weighting schemes.The trained agent also had lower critical-question omission (60.6% vs. 96.9%) and hallucination (3.1% vs. 46.5%) rates than the base model.Source: §C.2 · p. 65

C.3. Three-Way Concordance on the OSCE Scenarios · D. Safety Taxonomy Development

The OSCE analyses found moderate agreement between psychiatrist and autorater scores, while simulated-patient reliability varied by model and category; the safety taxonomies were developed through clinician elicitation and modified Delphi consensus.

C.3. Three-Way Concordance on the OSCE Scenarios

  • Pooled across 120 human-patient encounters, psychiatrist versus autorater composite scores correlated at ρ = 0.66 [0.52, 0.76].Pearson r differed from ρ by at most 0.06 on the composite.Source: §C.3 · p. 67, Table 9 · p. 67
  • Simulated-setting composite reliability was ρ = 0.56 for trained, 0.72 for base, and 0.41 for Δ on leave-one-out rollout comparisons.For a single rollout, ICC(1,1) was 0.47 (trained) and 0.67 (base); for the four-rollout mean, ICC(1,4) was 0.78 and 0.89.Source: §C.3 · p. 67, Table 9 · p. 67
  • Composite pairwise-preference κ was −0.14 [−0.23, −0.03], as both raters favored the trained model in most pairs and their base-preferred pairs did not overlap.On non-tied pairs, κ was positive in all four clinical categories (0.04–0.38), with all 95% CIs including zero.Source: §C.3 · p. 67
  • Alliance and communication training effects held with clinician-patients (d_z = 0.52 [0.28, 0.81]), but simulated trained scores clustered near ceiling and did not correlate with human-patient scores.Simulated scores averaged 3.75 of 5, with 134 of 240 rollouts ≥4; trained within-model ρ was 0.09 [−0.18, 0.35] and ρ on Δ was −0.05 [−0.33, 0.24], versus 0.62 and 0.43 for psychiatric assessment.Source: §C.3 · p. 67
  • The simulator’s score shift (+0.23 trained, +0.10 base) was not driven by longer transcripts, although controlling for human-encounter length attenuated trained cross-patient agreement.Trained median clinician turns were 60 with human patients versus 16 with the simulator; adjusted ρ fell from 0.43 to 0.27 [−0.01, 0.52].Source: §C.3 · pp. 67–68, Figure 1 · p. 68
  • Transcript-based rater scores correlated with clinician-patients’ post-encounter communication ratings at ρ = 0.48–0.55 for the autorater and 0.37–0.50 for evaluating psychiatrists.Normalizing speaker labels changed autorater scores only marginally (ρ = 0.98 across encounters, 0.92 on Δ); excluding one discordant scenario identifier kept composite correlations within 0.05 and category correlations within 0.06.Source: §C.3 · p. 68, Table 11 · p. 69

D. Safety Taxonomy Development

  • The paper defines safety-challenging patient-behavior taxonomies for mental health and general ambulatory care, developed through structured clinician elicitations and modified Delphi consensus.Source: D. Safety Taxonomy Development · p. 70

D.1. Clinical Safety Taxonomy for Ambulatory Care · D.2. Mental Health Relational Competency Framework · Expert consensus process

These sections establish Delphi-validated clinical safety and relational competency frameworks for designing and evaluating conversational AI scenarios. The frameworks define patient-behavior safety risks and mental-health relational competencies, including boundaries for clinician-supervised intake workflows.

D.1. Clinical Safety Taxonomy for Ambulatory Care

  • The safety taxonomy synthesized five patient-safety frameworks and literature on cognitive errors, communication disparities, therapeutic-alliance ruptures, and adversarial attacks.Candidate risks were filtered for text-based clinical encounters; the taxonomy excludes system failures independent of patient behavior and risks requiring physical infrastructure.Source: §D.1 · p. 70
  • The safety taxonomy’s modified Delphi panel included 16 clinicians across six specialties, while AI experts assessed the technical feasibility of system exploitation.In Round 1, panelists rated relevance, operational clarity, and distinctiveness; the panel’s median experience was 11.5 years (IQR 8.5–14.5).Source: §D.1 · p. 70
  • Round-1 safety-taxonomy decisions produced six mergers, four clarity revisions, and two additions: stigma-driven non-disclosure and goals-of-care misalignment.The panel also revised definitions to remove assumed patient intent, specify annotator ground truth, and ensure medical accuracy.Source: §D.1 · p. 70
  • The clinical safety taxonomy contains 21 risks across six domains, covering patient communication, adversarial system exploitation, and autonomy and safety-netting.Domains 1–4 address risks from patient communicative behavior, from unintentional information gaps to deliberate fabrication; the taxonomy structures adversarial training and evaluation and post-hoc failure coding.Source: §D.1 · p. 70
  • The safety taxonomy achieved Round-2 S-CVI/Ave = 0.96, and all 21 risks reached at least 80% consensus on relevance (I-CVI range: 0.85–1.00).Thirteen clinicians completed Round 2 (81% retention); feedback further refined operational definitions to emphasize observable behavior rather than inferred patient intent.Source: §D.1 · p. 70, Table 12 · p. 71

D.2. Mental Health Relational Competency Framework

  • The relational competency framework finalized 18 competencies across five domains, with all competencies reaching I-CVI ≥0.80 in Round 2.The framework addresses the lack of formal operational definitions for relational competencies in conversational psychiatric AI and supports scenario design and evaluation.Source: Table 13 · p. 72, §D.2 · p. 72, Expert consensus process · p. 73

Expert consensus process

  • The relational framework began with 20 competencies across four domains; after Delphi review, 18 of 22 Round-2 competencies reached consensus and four were removed.Twelve competencies reached consensus in Round 1; expert feedback led to removals, mergers, revisions, and five additions, including a fifth scope-management domain.Source: Expert consensus process · pp. 72–73
  • Competency 5a, Scope boundary maintenance, distinguishes declining demands for autonomous prescribing, clearance documentation, or ongoing psychotherapy from clinician-facing provisional formulations.In clinician-supervised pre-visit intake, provisional diagnostic and management formulations are routed exclusively to the clinician-facing review channel.Source: Expert consensus process · p. 73

E. Related Work

This section positions multi-turn reinforcement learning in simulated psychiatric intake against advances in clinical AI, conversational mental-health systems, and clinical simulation. It identifies gaps in long-horizon psychiatric dialogue, guarded-patient simulation, and evaluation of integrated intake and safety capabilities.

Long-horizon reinforcement learning for clinical decision-making

  • The framework applies multi-turn reinforcement learning to jointly optimize psychiatric diagnostic reasoning and patient-centered communication across intake encounters.It also trains for multi-turn inquiry, clinical documentation, and management planning, unlike concurrent systems focused on general medical dialogue or structured diagnostic ordering.Source: E. Related Work · pp. 74–75

Foundational medical AI and clinical reasoning

  • Clinical AI progressed from rule-based inference to classifiers, foundation models, and interactive systems, while earlier conversational models lacked rigorous multi-turn clinical evaluation.Long-horizon reinforcement learning frames decisions over temporally extended POMDP trajectories, complementing earlier single-step alignment and retrospective offline clinical RL.Source: E. Related Work · p. 74

AI in psychiatric care and clinical communication

  • Existing mental-health AI spans classifiers, support chatbots, digital interventions, and fine-tuned agents, but generally does not train agents to complete integrated psychiatric intake encounters.The identified gap is combining diagnostic investigation with structured, patient-centered communication.Source: E. Related Work · p. 75

Clinical simulation environments

  • Existing patient simulators often rely on cooperative disclosure, whereas this environment gates critical facts behind targeted inquiry, empathy, and sustained trust.This design addresses the withholding of history, symptom minimization, and resistance to treatment recommendations in psychiatric practice.Source: E. Related Work · p. 75

Evaluating AI in psychiatry

  • The evaluation provides 127 expert-curated scenarios with stratified disclosure requirements to assess guarded-patient information elicitation and safe, appropriate care planning.It addresses limitations of frameworks focused on static medical knowledge, conversational fluency, or isolated diagnostic accuracy.Source: E. Related Work · p. 76

Safety, equity, and red-teaming in clinical AI

  • The framework uses two Delphi-validated taxonomies to test clinical safety and relational competence under deceptive, manipulative, and boundary-testing presentations.This addresses documented mental-health chatbot risks and extends adversarial testing into psychiatric conversational safety.Source: E. Related Work · p. 76

Related papers

Loading 2610.40476v1…