PaperlayerView exact revision on arXiv

Source-linked AI summary

Automating Expert-Level Medical Reasoning Evaluation of Large Language Models

Shuang Zhou, Wenya Xie, Jiaxi Li, Zaifu Zhan, Meijia Song, Han Yang, Cheyenna Espinoza, Lindsay Welton, Xinnie Mai, Yanwei Jin, Zidu Xu, Yuen-Hei Chung, Yiyun Xing, Meng-Han Tsai, Emma Schaffer, Yucheng Shi, Ninghao Liu, Zirui Liu, Rui Zhang

arXiv:2507.07988v1cs.CL

TL;DR

Existing medical-reasoning evaluations lack either reliable expert-level assessment or scalability, and rigorous rationale benchmarks remain limited. The paper introduces MedThink-Bench and LLM-w-Ref, combining expert-annotated reasoning trajectories with reference-based LLM judging. LLM-w-Ref strongly agrees with expert assessments, while smaller models can outperform larger proprietary models on reasoning evaluation.

  • Problem

    Existing medical-reasoning evaluations are limited by weak semantic and logical assessment, poor scalability, and a lack of rigorous benchmarks with expert-validated rationales.

    Source: Introduction · pp. 2–3

  • Method

    MedThink-Bench provides 500 expert-annotated questions across ten medical domains, while LLM-w-Ref evaluates generated rationales against discrete expert reasoning steps using LLM-as-a-Judge.

    Source: Introduction · p. 3, Evaluation framework · p. 14

  • Results

    LLM-w-Ref shows strong concordance with expert assessments, and MedGemma-27B, HuatuoGPT-o1-70B, and DeepSeek-R1 are the top three reasoning performers with scores of 0.769, 0.737, and 0.727, respectively.

    Source: Discussion · pp. 11, 13

  • Takeaways & Limitations

    MedThink-Bench offers a foundational resource for scalable assessment of trustworthy medical reasoning and supports safer, more responsible clinical integration of LLMs.

    Source: Introduction · p. 4, Discussion · p. 13

    The benchmark may contain pretraining-related data leakage and remains relatively small because expert annotation is costly.

    Source: Discussion · p. 12

Abstract

from arXiv · show

As large language models (LLMs) become increasingly integrated into clinical decision-making, ensuring transparent and trustworthy reasoning is essential. However, existing evaluation strategies of LLMs' medical reasoning capability either suffer from unsatisfactory assessment or poor scalability, and a rigorous benchmark remains lacking. To address this, we introduce MedThink-Bench, a benchmark designed for rigorous, explainable, and scalable assessment of LLMs' medical reasoning. MedThink-Bench comprises 500 challenging questions across ten medical domains, each annotated with expert-crafted step-by-step rationales. Building on this, we propose LLM-w-Ref, a novel evaluation framework that leverages fine-grained rationales and LLM-as-a-Judge mechanisms to assess intermediate reasoning with expert-level fidelity while maintaining scalability. Experiments show that LLM-w-Ref exhibits a strong positive correlation with expert judgments. Benchmarking twelve state-of-the-art LLMs, we find that smaller models (e.g., MedGemma-27B) can surpass larger proprietary counterparts (e.g., OpenAI-o3). Overall, MedThink-Bench offers a foundational tool for evaluating LLMs' medical reasoning, advancing their safe and responsible deployment in clinical practice.

Introduction

Medical reasoning evaluation requires methods that are both expert-faithful and scalable, but existing metrics, human assessment, and benchmarks each have important limitations. MedThink-Bench and LLM-w-Ref address these gaps with expert-annotated reasoning trajectories and reference-based automated evaluation.

Introduction

  • Existing evaluation methods trade off fidelity and scalability: similarity metrics miss medical semantics, human assessment is labor-intensive, and LLM judges can hallucinate or show bias.These limitations leave automated evaluation with expert-level factuality unresolved.Source: Introduction · p. 3
  • The benchmark addresses the lack of rigorous rationale-evaluation datasets by using expert annotations rather than potentially flawed LLM-generated rationales.Existing datasets may be narrow or rely on artificial intelligence-generated reasoning whose credibility and alignment with expert judgment are uncertain.Source: Introduction · p. 3
  • MedThink-Bench contains 500 challenging questions across ten medical domains, each with expert-generated fine-grained, step-by-step reasoning trajectories.Ten medical experts collaboratively annotated the questions and reasoning trajectories through consensus.Source: Introduction · p. 3, Figure 1 · p. 4
  • LLM-w-Ref combines expert-curated reasoning trajectories with LLM-as-a-Judge to assess intermediate reasoning for expert-level factual consistency at scale.The framework evaluates whether model-generated rationales support discrete expert reasoning steps.Source: Introduction · p. 3
  • LLM-w-Ref correlates strongly with expert evaluation, while benchmarking twelve models shows that MedGemma-27B and HuatuoGPT-70B can outperform larger proprietary models such as OpenAI-o3 and DeepSeek-R1.The study positions MedThink-Bench as a resource for assessing trustworthiness in medical decision-making and supporting safer clinical integration.Source: Introduction · pp. 3–4

Results

MedThink-Bench evaluates medical reasoning across 500 expert-annotated questions in ten domains, while LLM-w-Ref compares model rationales with fine-grained references. Across evaluations, LLM-w-Ref aligned more closely with experts than alternative automated metrics and supported robust, scalable reasoning assessment.

Dataset

  • 500 complex questions across ten medical scenarios received fine-grained reasoning-step annotations from medical experts.The domains include diagnosis, treatment, pharmacology, pathology, prognosis, and other clinical areas.Source: Dataset · p. 5

Comparison of evaluation metrics on LLM reasoning

  • 0.759 (95% CI: 0.730–0.789) was the highest expert evaluation score, achieved by MedGemma-27B among twelve evaluated LLMs.Expert evaluation scores ranged from 0.453 (95% CI: 0.419–0.485) for Med42-70B to 0.759 for MedGemma-27B.Source: Comparison of evaluation metrics on… · p. 5

Correlation analysis between expert and automated evaluation

  • Pearson correlations with expert scores ranged from 0.68 to 0.87 for LLM-w-Ref, versus -0.17 to 0.45 for text-similarity metrics and 0.01 to 0.27 for LLM-w/o-Ref.LLM-w-Ref scores generally aligned closely with expert scores, while competing automated measures diverged more.Source: Correlation analysis between expert… · p. 8

The impact of the judge model in LLM-w-Ref

  • Smaller judge models produced higher LLM-w-Ref performance than large judge models, whose scores ranged from 0.704 to 0.891.The tested judge models included Llama-3-8B-Instruct, Llama-3.2-3B-Instruct, and Llama-3.2-1B-Instruct.Source: The impact of the judge model in… · p. 8

The prompt sensitivity in LLM-w-Ref

  • 0.538 (95% CI: 0.500–0.571) to 0.567 (95% CI: 0.533–0.599) was the performance range across five semantically similar prompt variations.The predicted rationales came from Llama-3.3-70B and GPT-4o-mini served as the fixed judge model.Source: The prompt sensitivity in LLM-w-Ref · p. 9

Efficiency comparison

  • 9.0 minutes for text-similarity metrics and 310.7 minutes for LLM-w-Ref contrasted with 3708.3 minutes for human evaluation.The LLM-w-Ref timing used HuatuoGPT-o1-70B as judge; text-similarity timing summed all evaluated metrics.Source: Figure 5 · p. 9, Efficiency comparison · pp. 10–11

Case study

  • Llama-3.3-70B produced an incorrect answer while following a partially correct reasoning trajectory, showing why final accuracy alone can miss reasoning quality.Another case showed flawed reasoning leading to a correct answer.Source: Figure 6 · p. 10, Case study · p. 11

Discussion

MedThink-Bench and LLM-w-Ref support scalable, expert-aligned evaluation of medical reasoning while revealing differences between reasoning quality and answer accuracy. The study also identifies robustness and dataset-scale limitations that constrain interpretation and downstream use.

Benchmark findings

  • MedGemma-27B, HuatuoGPT-o1-70B, and DeepSeek-R1 were the top three medical-reasoning performers, while some smaller open-source models surpassed larger commercial counterparts.Their scores were 0.769, 0.737, and 0.727, respectively; MedGemma-27B outperformed OpenAI-o3 and DeepSeek-R1 in the reported comparison.Source: Discussion · p. 11

Evaluation framework

  • LLM-w-Ref aligned strongly with expert evaluations, with Pearson correlations ranging from 0.68 to 0.87, while also improving efficiency over human evaluation.The framework combines LLM-as-a-Judge with fine-grained expert rationale annotations for automated assessment.Source: Discussion · pp. 11, 13

Evaluation comparisons

  • Text-alignment metrics and LLM-w/o-Ref correlated weakly with expert judgments because they miss medical semantics, reasoning-chain structure, or reliable reference rationales.BERTScore remains limited by word-level comparison, whereas LLM-w/o-Ref depends strongly on evaluator-model performance and biases.Source: Discussion · p. 12

Reasoning versus accuracy

  • Reasoning performance did not consistently match MCQ accuracy: OpenAI-o3 had the highest prediction accuracy at 0.692 despite trailing MedGemma-27B and HuatuoGPT-o1-70B in reasoning.Rationale-based evaluation captures partial correctness and flawed reasoning that answer accuracy alone can miss.Source: Discussion · p. 12

Limitations

  • Potential pretraining exposure may inflate prediction accuracy, and the expert-annotated benchmark’s limited size constrains its use for training or fine-tuning.The dataset remains suited to rigorous evaluation, but its annotation cost limits scale relative to AI-generated alternatives.Source: Discussion · p. 12

Methods

The study constructs an expert-annotated medical reasoning benchmark and evaluates generated rationales through step-level human, reference-based LLM, reference-free LLM, and text-similarity methods. It also benchmarks commercial and open-source models under controlled inference settings.

Data curation

  • 500 challenging questions from ten medical datasets are filtered, deduplicated, and restricted to text-based questions requiring multi-step reasoning.The source datasets include MedBullets, MMLU-Pro, MedExQA, MedXpertQA, Humanity’s Last Exam, MedQA-USMLE, PubMedQA, MedMCQA, MMLU-Medicine, and HEAD-QA.Source: Data curation · p. 13

Data annotation

  • Ten medical experts curate the benchmark, with two physicians independently annotating each question and a third resolving disagreements.The dataset spans ten domains, including diagnosis, treatment, prognosis, pharmacology, and policy and ethics.Source: Data annotation · p. 13

Evaluation framework

  • Human evaluation checks whether annotated reasoning steps appear in generated rationales, while prediction accuracy uses exact match and dataset scores average instance-level scores.Text-similarity baselines include BLEU, ROUGE-L, METEOR, BLEURT, and BERTScore, which are used primarily for comparison because they do not explicitly assess clinical reasoning structure.Source: Evaluation framework · pp. 14–15
  • LLM-w-Ref compares generated rationales with expert reasoning steps and scores the proportion of steps the rationale supports.The judge receives the question, generated rationale, and each discrete expert step, enabling fine-grained evaluation of logical correctness and stepwise completeness.Source: Evaluation framework · p. 14
  • Reference-free judging estimates the number of necessary reasoning steps before counting how many are sufficiently reflected in the generated rationale.This baseline tests whether expert-annotated reference rationales are necessary for reliable evaluation.Source: Evaluation framework · p. 14

LLM baselines

  • The benchmark includes commercial and open-source reasoning and non-reasoning LLMs, evaluated with temperature 0, seed 42, and a 4096-token limit.Closed-source models are accessed through APIs, while open-source models are loaded with vLLM.Source: LLM baselines · p. 16

Data Availability

The study’s medical questions come from public datasets, and the expert-curated rationale-annotated data are intended for release upon paper acceptance.

  • The medical questions are sourced from public datasets that may be available for research upon reasonable request.Source: Data Availability · p. 16
  • The listed sources include ten medical QA datasets, while the expert-curated rationale annotations are planned for release upon paper acceptance.Source: Data Availability · p. 16

Related papers

Loading 2507.07988v1…