PaperlayerView exact revision on arXiv

Source-linked AI summary

AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator

Zhihao Fan, Jialong Tang, Wei Chen, Siyuan Wang, Zhongyu Wei, Jun Xi, Fei Huang, Jingren Zhou

arXiv:2402.09742v4cs.CL

TL;DR

LLMs perform strongly on static medical question-answering benchmarks, but their ability to manage complex, multi-turn clinical interactions remains under-assessed. AI Hospital and the MVME benchmark evaluate that interaction and add a dispute-resolution collaboration mechanism. Interactive LLMs nevertheless remain substantially behind one-step GPT-4 on key diagnostic metrics.

  • Problem

    LLMs’ performance on static medical question-answering benchmarks does not establish how well they handle multi-turn clinical diagnosis, where patients may struggle to communicate their condition.

    Source: §1 · p. 1

  • Method

    AI Hospital simulates consultations among Doctor, Patient, Examiner, and Chief Physician agents, while MVME uses Chinese medical records to assess symptom collection, examination recommendations, and diagnoses.

    Source: §1 · pp. 1–2

  • Results

    Interactive LLMs achieve less than 50% of one-step GPT-4’s performance on diagnostic results, rationales, and treatment plans, and dispute resolution improves performance only to a degree.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The findings underscore current LLMs’ difficulty replicating clinical reasoning in multi-turn interactions and the need for further research to bridge the gap with human physicians.

    Source: §1 · p. 3

    The benchmark relies primarily on Chinese medical records, which may limit generalizability to other languages and healthcare systems.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Artificial intelligence has significantly advanced healthcare, particularly through large language models (LLMs) that excel in medical question answering benchmarks. However, their real-world clinical application remains limited due to the complexities of doctor-patient interactions. To address this, we introduce \textbf{AI Hospital}, a multi-agent framework simulating dynamic medical interactions between \emph{Doctor} as player and NPCs including \emph{Patient}, \emph{Examiner}, \emph{Chief Physician}. This setup allows for realistic assessments of LLMs in clinical scenarios. We develop the Multi-View Medical Evaluation (MVME) benchmark, utilizing high-quality Chinese medical records and NPCs to evaluate LLMs' performance in symptom collection, examination recommendations, and diagnoses. Additionally, a dispute resolution collaborative mechanism is proposed to enhance diagnostic accuracy through iterative discussions. Despite improvements, current LLMs exhibit significant performance gaps in multi-turn interactions compared to one-step approaches. Our findings highlight the need for further research to bridge these gaps and improve LLMs' clinical diagnostic capabilities. Our data, code, and experimental results are all open-sourced at \url{https://github.com/LibertFan/AI_Hospital}.

1 Introduction

AI Hospital and the MVME benchmark frame LLM diagnosis as a dynamic, multi-turn clinical interaction rather than a static question-answering task. Experiments reveal a substantial gap between interactive LLMs and one-step GPT-4, despite a collaborative dispute-resolution mechanism.

1 Introduction

  • Static medical question-answering benchmarks do not capture the multi-turn clinical process, where patients may struggle to communicate their conditions fully.The paper identifies limited research assessing LLMs’ ability to participate in this patient-centered diagnostic interaction.Source: §1 · p. 1
  • AI Hospital simulates a medical visit through a Doctor interacting with Patient, Examiner, and Chief Physician agents, supporting realistic assessment of complex clinical scenarios.The Doctor gathers information, recommends examinations, and diagnoses; the Examiner supplies test results and the Chief Physician evaluates performance.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • MVME uses high-quality Chinese medical records to evaluate symptom collection, examination recommendations, and diagnosis, with link-based metrics adding entity-level assessment.GPT-3.5 and GPT-4 simulate the framework’s NPCs.Source: §1 · p. 2
  • The dispute-resolution mechanism has multiple Doctors independently discuss the same record, while a Centre Agent guides discussion toward clearer issues and consensus.The authors propose this collaboration to enhance diagnostic accuracy.Source: §1 · p. 2
  • On diagnostic results, rationales, and treatment plans, interactive LLM performance is less than 50% of one-step GPT-4’s performance when it receives all patient information at once.Dispute resolution improves performance to a degree but remains below the one-step upper bound.Source: §1 · pp. 2–3
  • The authors conclude that current LLMs struggle to ask pertinent questions, elicit crucial symptoms, and recommend appropriate examinations during multi-turn diagnosis.They call for further research to bridge the gap with human clinical reasoning.Source: §1 · p. 3

2 Setup of AI Hospital

AI Hospital reconstructs a medical visit from medical records, assigning different information and responsibilities to agents as the Doctor conducts a variable-length diagnostic dialogue. The Chief Physician then evaluates the Doctor’s final report against the record.

2 Setup of AI Hospital

  • The framework runs a diagnostic phase with Patient, Examiner, and Doctor followed by a Chief Physician evaluation; the number of diagnostic turns depends on the Doctor’s strategy.The Doctor’s diagnostic phase ends at diagnosis or a predefined maximum number of rounds.Source: §2 · p. 3, §2.4 · p. 4

2.1 Agents Setup with Medical Records

  • Medical records separate subjective information, objective test results, and diagnosis and treatment, assigning them to Patient, Examiner, and Chief Physician while the Doctor starts without record access.The Chief Physician has access to all information; the Doctor gathers it through interaction.Source: §2.1 · p. 3

2.2 Agent Behavior Setting for NPCs

  • NPC behavior is tailored to each role: Patients may communicate colloquially and incompletely, Examiners return matching test results, and the Chief Physician compares the Doctor’s report with the record.GPT-3.5 powers Patient and Examiner, while GPT-4 drives the Chief Physician.Source: §2.2 · pp. 3–4

2.3 Agent Behavior Setting for Player

  • The Doctor is prompted to gather symptoms and history, request tests when needed, and combine subjective and objective findings; evaluated LLMs must follow instructions to interact effectively.Source: §2.3 · p. 4

2.4 Dialogue Flow in AI Hospital

  • The Patient opens with a complaint generated from the record, then the Doctor converses with Patient and Examiner until diagnosing or reaching the round limit.Special symbols identify each message’s intended recipient.Source: §2.4 · p. 4

3 MVME: Evaluation of LLMs as Intern Doctors for Clinical Diagnosis

MVME evaluates LLMs as doctor agents using multi-view criteria and a physician-reviewed collection of Chinese medical records. The benchmark assesses information gathering, examination recommendations, diagnoses, rationales, and treatment plans.

3 MVME: Evaluation of LLMs as Intern Doctors for Clinical Diagnosis

  • MVME uses high-quality Chinese medical records to assess LLMs as Doctor agents for clinical diagnosis.The records include 506 cases; expert validation deemed 94% correct.Source: §3 · p. 4, §3.2 · p. 5

3.1 Multi-View Evaluation Criteria

  • The five-part Doctor report covers symptoms, examinations, diagnostic results, rationales, and treatment plans, with GPT-4-driven Chief Physician scoring each part against the medical record.Four discrete scores, from 1 to 4, assess performance; disease-entity overlap metrics additionally measure diagnostic results.Source: §3.1 · p. 5

3.2 MVME Dataset Construction

  • The dataset contains 506 Chinese medical-record cases collected across departments, with physicians reviewing samples and experts judging 94% of records correct.Doctors in corresponding departments assessed diagnosis and treatment sections; a record was considered correct when all three sections were fundamentally accurate.Source: §3.2 · p. 5

4 Collaborative Diagnosis of LLMs Focused on Dispute Resolution

The collaborative diagnosis approach brings multiple LLM Doctors into interactive consultations and uses a Central Agent to moderate disagreements. Iterative, targeted discussion is intended to refine their understanding and diagnostic reports.

4 Collaborative Diagnosis of LLMs Focused on Dispute Resolution

  • Different LLM Doctors consult the same Patient, producing diverse dialogue trajectories and diagnostic reports.A Central Agent, also called the Chief Physician, participates as moderator.Source: §4 · pp. 5–6
  • The Chief Physician consolidates Doctors’ collected data, checks disputed points with the Patient and Examiner, and synthesizes a patient-condition summary.Source: §4 · p. 6
  • Across discussion iterations, the Chief Physician identifies disagreements and steers targeted discussion toward progressively refined understanding and consensus.Source: §4 · p. 6

5 Experiments

Agent-behavior evaluations report scores above 95 across metrics, but interactive diagnostic performance remains substantially below the one-step GPT-4 upper bound. More complete collected information is positively correlated with higher diagnostic quality.

5.1 Agent Behavior Analysis in AI Hospital Framework

  • In 50 randomly selected records, GPT-3.5, GPT-4, Wenxin-4.0, and Qwen-Max generated 50 dialogue trajectories each for manual metric labeling.Source: §5.1 · p. 6
  • All agent-behavior metrics score above 95, indicating reliable Patient, Examiner, and Doctor behavior in the framework.The evaluation covers Patient response relevance and honesty, Examiner understanding and result return, and Doctor report consistency.Source: §5.1 · p. 6, Table 2 · p. 6

5.2 Can LLMs Diagnose Like Doctors?

  • Even GPT-4 achieves less than half the one-step GPT-4 upper-bound performance in interactive diagnosis.Table 3 presents clinical-consultation evaluation with reference; Table 4 reports link-based diagnostic-result evaluation.Source: Table 3 · p. 7, §5.2 · p. 7, Table 4 · p. 7
  • LLMs with fewer parameters tend to show weaker interactive abilities, with Baichuan 13B cited as a lower-performing interactive diagnosis example.Source: §5.2 · p. 8
  • More complete collected symptoms and medical examinations are significantly positively correlated with higher diagnostic quality across results, rationales, and treatment plans.The analysis uses linear regression; the authors also report difficulty eliciting symptoms and recommending correct examinations.Source: §5.2 · p. 8

6 Further Analysis

The collaboration mechanism improves diagnostic performance and helps doctors reach consensus more quickly. GPT-4 errors in reviewed failure cases fall into three categories: omitted examinations, focus on complications, and erroneous judgment.

6.1 Collaboration Mechanism

  • Dispute Resolution increases consensus achieved within the first four discussion rounds by having the Chief Physician highlight controversial issues for focused discussion.Figure 2 compares discussion rounds with and without the mechanism.Source: Figure 2 · p. 8, §6.1 · p. 8
  • Collaborative diagnosis exceeds GPT-4, three Doctors outperform two, and removing Dispute Resolution reduces the two-Doctor system’s effectiveness.The compared settings are 3 Doctors, 2 Doctors, and 2 Doctors without Dispute Resolution.Source: §6.1 · p. 8

6.2 Reasons for Failure Cases

  • Omitted examinations can leave conditions such as gallbladder stones undetected, while complication-focused errors overlook underlying conditions such as diabetes.The examples illustrate failure to recommend an abdominal ultrasound and failure to inquire into broader health status.Source: §6.2 · p. 9
  • Among GPT-4’s one-point misdiagnoses, 99 involved omitted auxiliary examinations, 52 focused exclusively on complications, and 68 involved erroneous judgment.The analysis reviewed 219 cases rated one point by the Chief Physician.Source: §6.2 · p. 9, Table 5 · p. 9
  • GPT-4 also sometimes reaches incorrect conclusions despite complete symptom and examination information.The paper attributes this category to insufficient medical expertise embedded in the model.Source: §6.2 · p. 9

7 Related Works

Prior medical AI evaluation largely centers on noninteractive tasks, while LLM-based agents have expanded capabilities for complex tasks. The paper positions its work as a multi-agent clinical diagnosis environment addressing the challenge of integrating LLM agents into disease diagnosis.

LLM Powered Agents

  • Earlier medical agents often relied on rules or traditional machine learning, while LLM-powered agents use tool learning [20], retrieval augmentation [22] [23], role-playing, and communication [25] [26].The paper identifies integrating LLM agents into disease diagnosis as a continuing challenge [28].Source: §7 · p. 9

Medical Large Language Models

  • Medical LLMs including Med-PaLM [29], DoctorGLM [30], BenTsao [31], and DISC-MedLLM are fine-tuned for medical QA, health inquiries, and doctor-patient dialogue.The passage also lists ChatGLM-Med, Bianque-2, ChatMed-Consult, and MedicalGPT.Source: §7 · p. 9

8 Conclusion

The paper presents AI Hospital and MVME for evaluating LLMs in interactive clinical diagnosis, and reports that its dispute-resolution-centered collaboration approach is effective. It also identifies benchmark breadth, evaluation cost, and agent optimization as future priorities.

8 Conclusion

  • AI Hospital and MVME evaluate LLM clinical diagnosis through interactive consultations, while a medical-director-supervised dispute-resolution mechanism supports intern-doctor collaboration.The paper reports efficacy for the approach and plans broader benchmarks, cost-effective evaluation, and agent optimization.Source: §8 · p. 10

Limitations

The paper identifies limitations in the framework and benchmark’s scope, simulation fidelity, treatment-plan evaluation, and API use.

Limitations

  • Primarily Chinese records and 506 cases may limit generalizability and coverage of rare diseases; simulated interactions may not capture human nuances, and treatment-plan evaluation omits feasible alternatives.Extensive use of OpenAI’s LLM API may also increase environmental burden.Source: Limitations · p. 10

Ethics Consideration

AI Hospital specifies a finite, multi-agent diagnostic dialogue and a separate iterative process for doctors to exchange facts and resolve disagreements; the paper also describes data-protection and bias safeguards.

Ethics Consideration

  • The study reports de-identifying medical records and seeking dataset diversity and representativeness to address privacy and bias concerns.It also says the records will be publicly accessible for transparency and reproducibility.Source: Ethics Consideration · p. 10

A Detailed Description of the Dialogue Flow in AI Hospital

  • The diagnostic dialogue begins with a patient-generated complaint, proceeds through doctor inquiry and examination-result exchanges, and ends at diagnosis or a maximum-round limit.The termination rules apply to the diagnostic phase; evaluation rounds are predetermined.Source: A Detailed Description of the Dialogue… · pp. 13, 15

B Detail of Collaborative Algorithm

  • A Central Agent, also called Chief Physician, coordinates multiple Doctors without access to patient information, guiding their collaboration toward an improved diagnosis.The passage distinguishes this Central Agent from the Chief Physician role in the AI Hospital framework.Source: B Detail of Collaborative Algorithm · p. 15

B.1 Exchange of Factual Information

  • Doctors relay symptoms and test outcomes to the Chief Physician, who checks disputed facts with the Patient and Examiner and synthesizes a shared summary.Source: §B.1 · p. 16

B.2 Discussions on Dispute Resolution

  • Doctors iteratively critique one another’s reports using the Chief Physician’s dispute summaries; the Chief Physician sets unresolved issues as the next agenda or finalizes the report when consensus is reached.Source: §B.2 · p. 16

C Detailed Explanation of Performance in MVME

The section relates information completeness to diagnostic quality and describes limitations in information gathering, along with checks on the medical records and Examiner workflow.

C Detailed Explanation of Performance in MVME

D Expert Verification for Medical Records

  • Doctors at two tertiary Class A hospitals reviewed the medical records, which the authors describe as audited cases for the benchmark.Tencent Questionnaire 3 facilitated the quality inspection.Source: D Expert Verification for Medical… · p. 17

E Attacking on Examiner: Reliability of Examiner’s Workflow

  • In 50 attack samples, the Examiner classified requests without legitimate examination items as invalid with 100% consistency.The tested workflow releases results only when the patient specifies the requested examinations.Source: E Attacking on Examiner: Reliability… · p. 17

F Human Evaluation for Agent Behavior

The appendix reports reviewer agreement measures, notes a prompt-design concern, and describes potential extensions and applications of AI Hospital.

F Human Evaluation for Agent Behavior

  • Agreement rates across the four behavior-questionnaire questions were 99.1%, 95.6%, 99.4%, and 100.0%; a third reviewer resolved disagreements.Three medical students completed the questionnaires.Source: F Human Evaluation for Agent Behavior · p. 19

G Prompts for Different Medical Roles

  • Overly complex Doctor prompts could disrupt dialogue flow, while the final prompts were designed to adapt to most LLMs.The paper gives non-adherence and Doctors outputting patient responses as examples of flow issues.Source: G Prompts for Different Medical Roles · p. 19

H Potential of AI Hospital Framework

Loading 2402.09742v4…