PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling Clinical Judgment to Evaluate Medical AI

Thomas A. Buckley, Zahir Kanjee, Peter G. Brodeur, Byron Crowe, Anthony M. Pettinato, Aashna P. Shah, Adrian D. Haimovich, Liam G. McCoy, Daniel Restrepo, Jason A. Freed, Ethan Goh, Jonathan H. Chen, Laura Zwaan, Katherine E. Goodman, Daniel J. Morgan, Raja-Elie E. Abdulnour, Adam Rodman, Arjun K. Manrai

arXiv:2609.12822v2cs.AI

TL;DR

Expert physician grading of lengthy LLM outputs is slow and resource-intensive, while existing LLM judges often fall short through disagreement. The paper introduces PrecepTron, a fine-tuned judge, and GRAND-ROUNDS, a physician-annotated benchmark; frontier rubric-prompted models disagree, while a reported task agreement measure rises from 46% to 61%.

  • Problem

    Expert physician grading of lengthy LLM outputs is slow, expensive, and resource-intensive, while existing LLM-as-a-judge approaches often fall short by disagreeing.

    Source: MAIN · p. 3, DISCUSSION · p. 14

  • Method

    The authors introduce PrecepTron, a fine-tuned LLM judge for open-ended clinical reasoning, alongside GRAND-ROUNDS, a physician-annotated benchmark.

    Source: DISCUSSION · p. 11

  • Results

    Frontier LLMs prompted only with published rubrics disagreed with one another; a task's agreement rose from 46% to 61% (κ from 0.62 to 0.80).

    Source: DISCUSSION · pp. 11–12

  • Takeaways & Limitations

    Calibration should be checked on each task because validation on another task does not transfer.

    Source: Box 1: Protocol for deploying an… · p. 31

    Current benchmarks may reward early pattern matching while leaving untested whether models perform essential cognitive clinical tasks.

    Source: DISCUSSION · p. 13

Abstract

from arXiv · show

Blinded physician evaluation has been considered by many to be the gold standard for assessing clinical reasoning in large language models (LLMs). This is difficult to scale; thus, prior studies typically rely on small physician panels, often from a single institution or specialty, which both limits the scientific questions investigated and makes it unclear whether findings would be reproduced with a different set of evaluators. To more rigorously and scalably study clinical reasoning in AI models, here we introduce PrecepTron, an LLM fine-tuned for physician-level evaluation of open-ended responses. PrecepTron was trained using low-rank adaptation (LoRA) of a 32-billion-parameter model on a small number of physician examples. We also release GRAND-ROUNDS, a new large-scale physician-annotated benchmark of 9,217 scored responses from 160 clinicians across seven studies. We show that frontier LLMs in typical "LLM-as-a-judge" approaches often disagree with physicians and with each other, but fine-tuning PrecepTron on a small number of cases enables physician-level consistent scoring across tasks. We use PrecepTron to reproduce headline findings from five influential studies assessing LLMs for clinical care in JAMA, Science, and Nature Medicine without new human grading. Using PrecepTron, we then pose new questions about how LLMs reason in medicine that would have been infeasible with human grading alone, including measuring the diagnostic accuracy of frontier LLMs when clinical cases are provided piecemeal, even token by token. Together, PrecepTron and GRAND-ROUNDS provide a foundation for reproducible, large-scale study of how LLMs reason in medicine. All code, data, and labels are made freely available for researchers.

MAIN

The paper introduces PrecepTron and GRAND-ROUNDS to support scalable, reproducible evaluation of open-ended clinical reasoning. It reports that frontier LLM judges often disagree with physicians and one another, while PrecepTron can reach physician-level agreement using few scored cases.

MAIN

  • Expert grading is slow, expensive, and resource-intensive, while LLM judges can be poorly calibrated, disagree with physicians, and raise concerns about sending protected data to external services.The paper also notes uncertainty about which expert panels provide reliable ratings.Source: MAIN · pp. 3–4
  • PrecepTron is a fine-tuned LLM judge for scalable physician adjudication of open-ended clinical reasoning, calibrated to a physician panel with only a handful of scored examples per task.It is built on Qwen3-32B and can run locally on hospital infrastructure with a single 80GB GPU for inference.Source: MAIN · p. 4
  • GRAND-ROUNDS contains 9,217 physician scores across six clinical tasks, contributed by 11 grading physicians from seven studies and covering responses by 160 clinicians and nine AI models..Source: MAIN · p. 4
  • Frontier LLM judges often misalign with physician scores and disagree with one another, while fine-tuning PrecepTron on a few cases reaches physician-level agreement.The paper reports that fine-tuning on as few as two to a few dozen physician-scored cases per task closes the gap.Source: MAIN · p. 4
  • PrecepTron reproduced findings from five high-profile studies without new human grading and was used to investigate questions that physician-only grading would make prohibitive.The studies appeared in JAMA, Nature Medicine, and Science.Source: MAIN · p. 4
  • Across hundreds of cases and thousands of responses, the study tracked diagnostic and management accuracy token by token; frontier LLMs could solve challenging cases using only the first few sentences.The fourth case was described as a borderline judgment.Source: MAIN · pp. 4–5
  • The authors openly release GRAND-ROUNDS and trained PrecepTron models; they say PrecepTron may reduce evaluation costs, improve benchmark reproducibility, and support new scientific questions.Training protocols are also released for researchers who wish to refine the model with their data.Source: MAIN · p. 5

RESULTS

The results assess clinical reasoning in medical AI using PrecepTron, from physician-aligned scoring and reproduction of published findings to piecemeal-case evaluation. They also examine model behavior during piecemeal solving and note how scoring rubrics may capture reasoning quality as well as diagnosis.

Frontier LLMs disagree with physicians and with each other

  • No single base model matched physician agreement across all five tasks; proprietary judges’ mean scores on Landmark Diagnostic Cases differed by up to 17 percentage points, with Gemini at 74%.Source: Frontier LLMs disagree with physicians… · p. 6

Lightweight LoRA fine-tuning aligns PrecepTron with physician scores

  • Trained on 2–46 cases per task, PrecepTron-32B improved over Qwen3-32B by 4–15 percentage points across all five tasks, substantially closing the gap to physicians.It matched or exceeded physician inter-rater agreement on three of five tasks.Source: Lightweight LoRA fine-tuning aligns… · p. 7
  • Without balanced resampling, NEJM Healer quadratic-weighted κ fell from 0.72 to 0.51 despite similar accuracy; removing LLM-generated examples showed no evidence of style-based shortcutting.Source: Lightweight LoRA fine-tuning aligns… · p. 8

Prompting fails to match fine-tuning

  • Prompt-based strategies did not consistently improve the base model: five-shot prompting left some tasks unchanged or lower, including NEJM Healer (76% to 67%); GEPA similarly degraded Grey Matters Management Cases (72% to 63%).Source: Prompting fails to match fine-tuning · p. 8

PrecepTron reproduces five published studies without new human grading

  • PrecepTron reproduced headline findings across five prior studies, including findings on GPT-4 diagnostic performance, chatbot-versus-attending scores, and o1’s performance across emergency-department touchpoints.Scores used each study’s original rubric, and training cases were excluded.Source: PrecepTron reproduces five published… · pp. 8–9
  • With 50 or fewer case-text tokens, GPT-5 solved 42% of NEJM CPC cases by top-10 differential, 14% by top-3 differential, and 12% by top-1 diagnosis.The typical NEJM CPC presentation contains approximately 1,600 tokens.Source: PrecepTron reveals that LLMs solve… · p. 9
  • GPT-5 reached at least 80% of rubric maximum within 50 tokens on four of five held-out Grey Matters questions and all four Landmark diagnostic cases.Three of the five Grey Matters questions reached that threshold within the first 10 tokens.Source: PrecepTron reveals that LLMs solve… · pp. 9–10
  • For Landmark cases, high scores reflect rubric credit for reasoning quality rather than the final diagnosis alone.Source: PrecepTron reveals that LLMs solve… · p. 10
  • Gemma-4-31B showed the same qualitative behavior as GPT-5, solving many cases from short prefixes but requiring more text for the same solve rate.The pattern generalized to a smaller open-source model.Source: PrecepTron reveals that LLMs solve… · p. 10
  • Rapid saturation in management tasks may reflect prompts that embed key clinical details directly.The passage presents this as a hypothesis.Source: PrecepTron reveals that LLMs solve… · p. 10
  • Prior-study rubrics may reward good guesses rather than true clinical reasoning, potentially allowing models to game the rubric.Source: PrecepTron reveals that LLMs solve… · p. 10

DISCUSSION

The discussion presents PrecepTron and GRAND-ROUNDS as a practical framework for scalable, reproducible, physician-aligned evaluation of clinical AI, including reproducing prior findings and testing diagnostic reasoning as cases unfold. It also underscores the need for shared standards to validate and interpret automated judges.

DISCUSSION

  • GRAND-ROUNDS contains 9,217 physician scores across six clinical tasks, and the released protocol and code support calibrating judges from limited labels for larger-scale evaluation.The benchmark can also support studies of judge biases and reuse of physician annotations from influential clinical AI studies.Source: DISCUSSION · pp. 11–13
  • After lightweight fine-tuning on a small number of physician-scored examples, PrecepTron reached physician inter-rater agreement and matched or surpassed larger frontier models.On real emergency-department cases, its concordance with physician evaluators was on par with the physicians themselves.Source: DISCUSSION · p. 11
  • PrecepTron reproduced headline findings from five physician-graded studies and can evaluate new models against prior baselines while preserving the original panels’ scoring preferences.The approach could also swap or study panel-specific preferences, such as comparing generalists with specialists or localizing a national benchmark.Source: DISCUSSION · pp. 11–12
  • Using 93 physician-scored responses spanning 2 cases, calibration improved agreement on the landmark diagnostic task from 46% to 61% (κ from 0.62 to 0.80).Balanced resampling of the training data enabled high agreement with physician panels by preventing the model from collapsing to predicting the majority class.Source: DISCUSSION · pp. 11–12
  • GPT-5 reached at least 80% of rubric maximum on four of five Grey Matters questions within 50 tokens (most within 10), and did the same on all four Landmark Diagnostic Cases within 50 tokens.For CPC cases, a top-10 differential contained the correct diagnosis in 42% of cases from 50 or fewer tokens; these cases were unavailable in pretraining, suggesting the result was not explained by memorization.Source: DISCUSSION · p. 13
  • These piecemeal-case results suggest current benchmarks may reward early pattern matching without testing information integration, revision of impressions, or avoidance of premature closure.The authors call for evaluations that vary information over time and credit appropriate updating.Source: DISCUSSION · p. 13
  • PrecepTron’s 32-billion-parameter size allows local deployment, while the released datasets and methods can support evaluation on protected clinical data and other rubric-based tasks with few physician scores.Local use avoids sending patient or institutional data to external organizations.Source: DISCUSSION · p. 14
  • The judge inherits its source physician panel’s preferences, small training sets may miss rubric edge cases, and physician evaluation remains essential for clinical impact and human–AI interaction studies.Physician agreement was modest on some tasks; reproducing score distributions and group-level orderings did not recreate randomized comparisons or human–AI interaction protocols.Source: DISCUSSION · p. 14
  • A relatively small model improved alignment with a target physician panel, enabling reproduction of prior high-impact findings and evaluation at a scale impractical with human grading alone.Together, PrecepTron and GRAND-ROUNDS provide a practical framework for more rigorous, scalable, and reproducible evaluation of clinical AI.Source: DISCUSSION · pp. 14–15
  • Automated evaluation requires shared expectations for how LLM judges are validated, reported, and interpreted, including how closely they reproduce expert assessments and where they fail.Studies should also report how sensitive their conclusions are to the choice of judge.Source: DISCUSSION · p. 15
  • Consensus on evaluation practices is essential for readers to trust medical AI results, compare findings across studies, and inform clinicians’ use of clinical AI.Source: DISCUSSION · p. 15

METHODS

The methods describe the assembly of GRAND-ROUNDS from physician-scored clinical responses, evaluation of automated judges against physician scores, and task-specific agreement analyses with bootstrap confidence intervals.

Benchmark Construction (GRAND-ROUNDS)

  • GRAND-ROUNDS aggregates seven studies into 9,217 physician scores across 5,250 response–rubric entries, with responses from 160 clinicians and nine AI models and scoring by 11 physician evaluators.The benchmark covers six clinical tasks; five were used in judge experiments.Source: Benchmark Construction (GRAND-ROUNDS) · p. 16
  • The public harmonized dataset newly releases Grey Matters management and landmark diagnostic cases; BIDMC ER cases remain private, and CPC testing-plan scores were excluded from judge experiments because of score skew.The testing-plan task is included in the data release.Source: Benchmark Construction (GRAND-ROUNDS) · pp. 16–17

Task Rubrics

  • Task rubrics span Bond diagnostic scoring, a 19-point diagnostic-reasoning rubric, normalized case-specific management scores, 10-point R-IDEA notes, and 0–2 CPC testing plans.Management scores sum question-level points and normalize by the case maximum; R-IDEA scores four note components.Source: Task Rubrics · p. 17

Base-model LLM-as-a-judge evaluation

  • Eight LLMs—five open-source and three proprietary—were evaluated as zero-shot automated judges on GRAND-ROUNDS; an ensemble of the three proprietary judges averaged scores and rounded them to the nearest valid rubric score.Source: Base-model LLM-as-a-judge evaluation · p. 18

Training PrecepTron-32B

  • PrecepTron was supervised-fine-tuned from Qwen3-32B with separate adapters per rubric; NEJM CPC and BIDMC ER tasks sharing the Bond rubric were pooled, and case-based splits used 20% of cases for training for tasks with at least 10 cases.Source: Training PrecepTron-32B · p. 18

Prompt-based baselines (few-shot and GEPA)

  • Prompt-based baselines aligned judges without weight updates using five physician-scored examples or GEPA prompt optimization on normalized score alignment with held-out case validation.GEPA explored up to 15 candidate prompts and selected the one maximizing validation alignment.Source: Prompt-based baselines (few-shot and… · p. 20

Statistical Analysis

  • Judge accuracy was defined as scoring within one raw point on Bond tasks or within 10% of normalized physician score on the remaining tasks.The reference was the reconciled physician score, or the mean when no reconciled score was available; Grey Matters question scores were summed.Source: Statistical Analysis · p. 20
  • Accuracy was computed at the case level, while all other tasks were scored per entry; quadratic-weighted Cohen’s κ was a secondary metric, calculated per entry after rounding scores to the nearest integer.Source: Statistical Analysis · p. 20
  • Physician inter-rater agreement was computed on entries scored by at least two non-reconciled physicians.Source: Statistical Analysis · p. 20
  • 95% confidence intervals for both metrics used a non-parametric bootstrap over scored units, with 2,000 resamples and a fixed random seed.Scored units were entries, or summed cases for management accuracy.Source: Statistical Analysis · p. 20

CODE AND DATA AVAILABILITY

The paper provides public access to its code, trained adapters, and benchmark data, while excluding some sensitive case materials.

CODE AND DATA AVAILABILITY

Box 1: Protocol for deploying an LLM-as-a-judge on a clinical task

Box 1 recommends validating clinical LLM judges against physician scores and fine-tuning when agreement falls short. The accompanying table and figures describe benchmark comparisons, judge agreement, adaptation strategies, and evaluations using short case prefixes.

Box 1: Protocol for deploying an LLM-as-a-judge on a clinical task

  • The protocol starts with diverse physician-scored examples, selects a judge for data and compute constraints, validates agreement and biases, and reports its calibration and scores.It recommends two or more physician ratings per response and case-level train/test splitting.Source: Box 1: Protocol for deploying an… · p. 31
  • Lightweight LoRA fine-tuning was the only compared strategy that consistently reached physician-level agreement; few-shot prompting and GEPA did not consistently improve performance.Source: Box 1: Protocol for deploying an… · p. 31

Table 1: Overall Performance of All Models on GRAND-ROUNDS

Supplementary Fig. 1: PrecepTron matches physician-level agreement across tasks (Cohen’s Kappa)

  • Supplementary Fig. 1 compares each judge's quadratic-weighted Cohen's κ with physician scores across five tasks, using physician inter-rater κ as the reference.Error bars show 95% bootstrap confidence intervals.Source: Figure 1 · p. 37

Supplementary Fig. 2: LoRA fine-tuning outperforms prompt-based methods

  • Supplementary Fig. 2 compares agreement accuracy for the base model, few-shot prompting, GEPA optimization, and LoRA fine-tuning.Error bars show 95% bootstrap confidence intervals.Source: Figure 2 · p. 38

Supplementary Fig. 3: Retraining PrecepTron without AI examples in Landmark Diagnostic Cases

  • Supplementary Fig. 3 compares held-out Landmark diagnostic scores across physicians, the base judge, and PrecepTron models trained with or without AI-authored examples.The plot shows medians, interquartile ranges, and individual responses.Source: Figure 3 · p. 39

Supplementary Fig. 4: PrecepTron-32B agreement with physicians

  • Supplementary Fig. 4 presents per-task confusion matrices, with physician scores in rows and PrecepTron scores in columns.Source: Figure 4 · p. 40

Supplementary Fig. 5: LLM-judge Model Preferences Across Four Clinical-Reasoning Tasks

  • Supplementary Fig. 5 arranges mean normalized scores by judge and response generator across four tasks, outlining self-judging cells in red.Cells average scores over all cases in each task.Source: Figure 5 · p. 41
Loading 2609.12822v2…