PaperlayerView exact revision on arXiv

Source-linked AI summary

K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations

Laura M. Vowels, Matthew J. Vowels, Shivali Sharma, Apoorv Jha, Rehnuma Choudhury, Wasseem El Sarraj, Rachel Francois-Walcott, Aruba Hussain, Sarah Ingram, Angela Loulopoulou, Adva Segal, Elena Volkova

arXiv:2609.15855v3cs.CLcs.AIcs.LG

TL;DR

Safety in evolving, high-risk mental health conversations remains poorly characterised, despite LLM use for support. K-Bench evaluates configurations in multi-turn, clinically calibrated conversations and finds strong leading-model performance alongside substantial variation in risk exploration among weaker systems.

  • Problem

    Evaluations must test whether models recognize, explore, and manage risk as it develops across turns while keeping the broader conversation appropriate.

    Source: Main · p. 2

  • Method

    K-Bench combines synthetic patient vignettes, controlled multi-turn conversations, a clinician-developed rubric, consensus ratings, and an automated judge calibrated against that ground truth.

    Source: Online Methods · p. 19

  • Results

    Leading systems scored above 95 on the combined-risk measure, while weaker models showed persistent gaps in eliciting risk-relevant information and identifying concerns.

    Source: Discussion · p. 12

  • Takeaways & Limitations

    K-Bench’s public leaderboard provides an ongoing accountability tool for tracking progress, configuration-specific regressions, and work needed for safer deployment.

    Source: Discussion · p. 14

    The evaluation covers API configurations and rubric-concordant conversation, not complete consumer products or clinical outcomes such as help-seeking and symptom change.

    Source: Discussion · p. 14

Abstract

from arXiv · show

People increasingly use large language models (LLMs) for mental health support, yet their safety in evolving, high-risk conversations remains poorly characterised. We developed K-Bench, a clinician-calibrated, protected benchmark evaluating 125 model configurations representing 33 base models from 14 providers across a fixed cohort of 200 multi-turn vignettes involving suicide, self-harm, domestic violence, substance misuse, and no-risk presentations. Synthetic patient conversations showed substantial distributional overlap with real human-AI conversations. A frozen GPT-4o judge achieved 94.2% exact agreement with clinician consensus across 6,751 eligible item comparisons from 151 clinician-rated transcripts. Leading models combined strong supportive conversation with combined-risk scores above 95, whereas risk exploration exposed substantial variation among lower-performing configurations. Therapeutic prompting produced configuration-specific gains concentrated among weaker models, while elevated reasoning produced no average improvement. K-Bench combines broader clinical coverage and configuration-scale comparison with a continuously updated public leaderboard whose operational test materials are protected from direct optimisation. The leaderboard is available at www.k-bench.ai.

Main

Existing benchmarks leave gaps in sustained, co-occurring risk conversations and broad, clinically granular model comparison; K-Bench combines these capabilities with protected operational test materials.

Main

  • Prior benchmarks do not combine multiple co-occurring risks, detailed assessment, and a large versioned model space, while open materials permit optimisation against known tests.K-Bench was designed around clinical breadth, enough conversational time for risks to emerge, clinician-grounded scalable scoring, and protected operational materials.Source: Main · p. 3
  • K-Bench evaluates a fixed cohort of 200 vignettes in conversations of up to 20 turns, covering suicide, self-harm, domestic violence, substance misuse, and no-risk presentations.Its 122-variable factorial design varies clinical and psychosocial features, and risks can emerge indirectly, escalate, or co-occur.Source: Main · p. 3
  • The benchmark separates risk recognition, risk exploration, and broader therapeutic conduct across seven dimensions and 47 rubric components, calibrated against clinician ratings.Clinicians rated 151 transcripts to create 16,157 consensus item-level ground-truth cells.Source: Main · p. 3
  • The public benchmark compares 125 model–configuration entries spanning 33 base models and 14 providers on the same vignette cohort.It publishes versioned methods, metadata, aggregate findings, and a continuously updated leaderboard while retaining operational prompts, disclosure schedules, scoring examples, and transcripts.Source: Main · p. 3

Results

K-Bench’s validation found substantial overlap between synthetic and real-user conversations and strong clinician-calibrated scoring agreement. Across evaluated configurations, overall and combined-risk performance varied, with risk exploration especially discriminating.

RQ1: Validation of K-Bench as a clinically grounded evaluation system

  • Synthetic patient turns showed substantial overlap with real-user turns across four patient agents, supporting their use as a realistic, controlled basis for comparison.The real-user reference comprised no-risk Amanda conversations; GPT-4o was selected for public evaluation because its realism was comparable at lower cost.Source: RQ1: Validation of K-Bench as a… · p. 4, Figure 1 · p. 5
  • 94.2% exact judge–consensus agreement across 6,751 eligible item comparisons exceeded clinician–clinician agreement by 4.9 percentage points across D1–D7.The frozen judge applied the shared rubric across risk identification, exploration, and broader therapeutic competencies.Source: RQ1: Validation of K-Bench as a… · pp. 4, 6, Table 1 · p. 7

RQ2: Model performance across risk and clinical complexity

  • 81.19–98.96 overall scores and 52.39–96.11 combined-risk scores spanned the 125 entries; GPT-5.5 led overall and Claude Fable 5 led combined risk.The overall-score leader differed from the combined-risk leader, and the combined-risk range spanned 43.72 points.Source: RQ2: Model performance across risk and… · p. 6, Figure 2 · p. 8
  • Risk exploration averaged 79.9 (range, 22.1–93.1), separating providers more than near-ceiling clinical judgement and broader therapeutic dimensions.Provider-level profiles were selected from each namespace’s highest-overall-scoring configuration.Source: RQ2: Model performance across risk and… · pp. 6, 9, Figure 3 · p. 9
  • Mean overall scores declined 0.43 points from low- to imminent-risk cases and 0.49 points from one to four concurrent risks, but the differences were heterogeneous.The authors caution against interpreting these averages as uniform severity or comorbidity penalties.Source: RQ2: Model performance across risk and… · p. 9
  • Combined-risk performance generally increased with inference cost, yet similarly strong scores appeared across a broad price range and some lower-cost configurations approached pricier systems.Models at similar cost sometimes differed appreciably, so price alone did not determine clinical capability.Source: RQ2: Model performance across risk and… · pp. 9–10, Figure 4 · p. 10

RQ3: Prompting and reasoning were configuration-specific rather than uniformly beneficial

Therapeutic prompting and elevated reasoning did not improve combined-risk performance uniformly across configurations. Therapeutic prompts produced larger gains for some weaker systems, while elevated reasoning showed no average benefit.

RQ3: Prompting and reasoning were configuration-specific rather than uniformly beneficial

  • Across 61 matched prompt pairs, therapeutic prompting improved combined-risk scores in 24 and reduced them in 37, with a mean difference of +0.11 points.Effects varied substantially by configuration, with the largest gains among lower-scoring configurations.Source: RQ3: Prompting and reasoning were… · p. 10
  • At the clean no/low baseline, the 95% CI was −0.54 to +2.61; under elevated reasoning, the therapeutic-minus-default difference was −0.25 points (95% CI, −1.17 to +0.65).Source: RQ3: Prompting and reasoning were… · p. 11
  • Elevated reasoning improved combined-risk scores in 9 matched pairs and reduced them in 21, with a mean difference of −0.58 points and configuration-specific gains and declines.Provider-exposed reasoning level did not operate as a monotonic safety control.Source: RQ3: Prompting and reasoning were… · p. 11

Discussion

K-Bench finds strong performance among leading models in sustained, high-risk conversations, while risk exploration separates stronger from weaker configurations. Its broad clinical and model coverage supports comparisons across configurations, but simulated, capped conversations and rubric-based outcomes bound what the evaluation establishes.

Discussion

  • Leading models scored above 95 on the combined-risk measure and assessed suicide, self-harm, domestic violence, and substance misuse, including escalating or co-occurring risks.The strongest systems combined supportive dialogue with detailed assessment across these domains.Source: Discussion · p. 12
  • Risk exploration created the clearest separation: weaker models omitted essential questions or missed concerns, while leading models elicited risk factors and appropriate next steps.Other dimensions, including supportive conversation, ethical reasoning, psychological knowledge, cultural competence, and autonomy, were consistently strong across many configurations.Source: Discussion · p. 12
  • K-Bench evaluated 125 configurations from 33 base models and 14 providers on a common cohort of 200 vignettes spanning four risk domains, with co-occurring risks and conversations of up to 20 talk turns.Source: Discussion · p. 12
  • Synthetic conversations closely approximated real human–AI interactions, and the GPT-4o judge agreed with clinician consensus on 94.2% of eligible items across seven clinical dimensions.The study reports this as the first mental health benchmark to validate synthetic patient conversations through direct comparison with real human–AI interactions.Source: Discussion · pp. 12–13
  • Therapeutic prompting produced configuration-specific gains concentrated among weaker systems, whereas additional reasoning produced no average improvement in risk performance.The authors recommend evaluating model version, prompt, and reasoning setting together as the clinical deployment configuration.Source: Discussion · p. 13
  • Careful use for low-risk support and identifying concerns for human review would require a clear response pathway, which general-purpose chatbots may lack the capacity to carry out.Implementation must establish who responds to escalations, when responsibility passes to a clinician, and how follow-up and sensitive disclosures are handled.Source: Discussion · pp. 13–14
  • The evaluation used simulated English conversations in a single episode of up to 20 turns, and some complex cases reached the cap before safety planning or referral.K-Bench measures rubric-concordant conversation, not help-seeking, symptom change, or other clinical outcomes.Source: Discussion · p. 14

Online Methods

The Online Methods describe how K-Bench’s vignettes, multi-turn conversations, clinician-developed rubric, ratings, consensus, and calibrated automated judge were developed and assessed. They also specify analyses of benchmark scores and matched configuration comparisons, alongside procedures for reproducible tracking and protecting operational test materials.

Online Methods

  • The clinician-developed rubric contains seven competency dimensions and 47 subdimensions, with domain-specific risk items and global transcript-level items.Risk items assess severity and, where applicable, exploration; global items assess conversational quality and contextual sensitivity, with not-applicable responses excluded from derived-score denominators.Source: Online Methods · pp. 19–20
  • The vignette library builds on lived-experience-derived cases and expands them across 48 cells, yielding 14,400 schema-valid rows.The factorial design crossed 16 risk combinations, including no-risk, with three levels of cognitive coherence; rows were validated against the schema and conditional rules.Source: Online Methods · p. 21

Synthetic conversation generation and realism assessment

  • Synthetic patient turns were compared with an independent reference corpus, and manual curation produced a clinician-rated calibration set of 151 transcripts.Realism was assessed using embedding-based nearest-neighbour cosine distance, maximum mean discrepancy, and Fréchet distance; curation balanced realised risk coverage and excluded transcripts without an interpretable evaluation opportunity.Source: Synthetic conversation generation and… · pp. 21–22

Clinician rating protocol

  • Six clinician raters completed formal ratings after rubric training, with each formal transcript receiving at least three independent ratings.The dataset contained 478 clinician-rating rows, and item-level consensus was reviewed against each transcript and the rubric.Source: Clinician rating protocol · pp. 22–23

Automated judge calibration and scoring

  • The automated judge was calibrated against all 151 clinician-consensus transcripts using deterministic scoring, with prompt-tuning transcripts separated from a validation holdout.The judge prompt included the rubric, item-level anchors, transcript, and applicability instructions; exact prompts and operational rules remain non-public.Source: Automated judge calibration and scoring · p. 23

Model evaluation conditions and configurations

  • The evaluation cohort comprised 200 benchmark vignettes and 125 model–configuration entries representing 33 base models across 14 provider namespaces.Models were evaluated under default and, where applicable, therapeutic prompts; reasoning settings were compared only where matched conditions were available within model families.Source: Model evaluation conditions and… · p. 24

Data analysis and reporting

  • 22,816 valid clinician-pair item comparisons and 6,751 judge–consensus item comparisons formed the pooled D1–D7 agreement denominators.Clinician–clinician agreement used unordered pairs, whereas judge agreement compared the single judge score with clinician consensus; eligibility rules varied by rubric item.Source: Automated judge calibration and scoring · pp. 23–24, Data analysis and reporting · p. 24
  • Overall scores summarize all rubric dimensions on a 0–100 scale, while combined-risk scores combine D1 clinical judgement and D2 risk exploration.Analyses also summarized overall, risk-domain, and rubric-dimension scores and examined variation by risk severity, co-occurrence, and vignette metadata.Source: Data analysis and reporting · p. 25
  • Matched prompt and reasoning effects were expressed as score differences, with model-cluster bootstrap 95% confidence intervals where reported; demographic and ethnicity-stratified analyses were excluded.Elevated-reasoning comparisons used a true no-reasoning or explicit low-reasoning baseline where available, excluding provider-default settings.Source: Data analysis and reporting · p. 25

Reproducibility and protection of a living benchmark

  • The versioned public leaderboard supports repeated independent evaluation by tracking configuration performance across risk domains and rubric dimensions, comparative cost, and progress or regressions over time.Results can be traced and regenerated from exported item- and dimension-level data.Source: Reproducibility and protection of a… · p. 25
  • Following AEF-1 minimum operating conditions, K-Bench releases its constructs, methods, metadata, and aggregate results but withholds operational prompts, disclosure schedules, tuning examples, judge examples, and transcripts to limit direct test optimisation.This division supports scrutiny while preserving unseen evaluation material for future model releases and benchmark refreshes.Source: Reproducibility and protection of a… · p. 25

Funding

The funding, competing interests, data availability, and code availability statements describe Kivira Health’s role and the benchmark’s protected operational materials.

Funding

  • The study was funded by the ESRC Digital Good Network and Kivira Health; Kivira Health funded benchmark-platform engineering and implementation.The contributions of Kivira Health-affiliated authors are described in the Author contributions statement.Source: Funding · p. 26

Competing interests

  • M.J.V. is Kivira Health’s CTO, and A.J. consulted for Kivira Health during development before joining OpenAI; OpenAI had no involvement in the study.L.M.V. is married to M.J.V. and declares no financial or other competing interests; the remaining authors declare no competing interests.Source: Competing interests · p. 26

Data availability

  • Versioned aggregate results and benchmark documentation are public, but vignettes, prompts, disclosure policies, transcripts, and clinician-level data are withheld to protect benchmark validity.The stated purpose is to limit test-set contamination and direct optimisation or gaming against the evaluation set.Source: Data availability · pp. 26–27

Code availability

  • The benchmark’s operating code is not public because its release would expose operational features and enable benchmark-specific optimisation.Public documentation, evaluated configurations, scoring dimensions, and aggregate results are available.Source: Code availability · p. 27

Supplementary Tables of Results

The supplementary tables cover leaderboard configurations, score distributions, severity and co-occurring risks, matched prompt effects, and elevated-reasoning comparisons.

Supplementary Tables of Results

  • Supplementary Table 1 lists public leaderboard configurations; each row is a distinct model–configuration entry, with scores normalised to 0–100.Prompt and reasoning labels record the tested conditions.Source: Table 1 · p. 28, Supplementary Tables of Results · p. 30
  • Supplementary Table 2 summarizes score distributions by risk domain and rubric dimension.Supplementary Table 3 summarizes overall scores by risk severity and number of active risk domains.Source: Table 2 · p. 31, Table 3 · p. 31
  • The reported values summarize 125 model–configuration entries, and the stated ranges describe spread across evaluated configurations rather than population-level clinical effects.The entries were evaluated on a fixed vignette cohort.Source: Supplementary Tables of Results · p. 31
  • Supplementary Tables 4 and 5 report matched therapeutic-prompt effects, with positive differences favoring the therapeutic prompt and comparisons paired within model conditions.Table 4 presents selected effects, while Table 5 presents the full matched effects.Source: Table 4 · p. 31, Supplementary Tables of Results · pp. 31–32, Table 5 · p. 32
  • Supplementary Table 6 compares elevated reasoning with a clean no/low-reasoning baseline, excluding provider-default settings.A clean baseline is a true no-reasoning setting where available, or an explicit low-reasoning setting when none is offered.Source: Table 6 · p. 33, Supplementary Tables of Results · p. 33

Related papers

Loading 2609.15855v3…