PaperlayerView exact revision on arXiv

Source-linked AI summary

AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments

Samuel Schmidgall, Rojin Ziaei, Carl Harris, Eduardo Reis, Jeffrey Jopling, Michael Moor

arXiv:2405.07960v5cs.HCcs.CL

TL;DR

Static question-answering benchmarks do not capture the sequential, uncertain, resource-limited nature of clinical work. AgentClinic addresses this gap with interactive multimodal clinical simulations and finds substantial differences in how models respond to biases and reasoning tools.

  • Problem

    Existing medical LLM evaluations emphasize static question-answering, whereas clinical work requires sequential decisions under uncertainty, limited information, and finite resources.

  • Method

    AgentClinic is an open-source multimodal benchmark that simulates clinical environments through patient and doctor agents, dialogue-based exams and image readings, diverse specialties and languages, biases, and reasoning tools.

  • Results

    Agents differed substantially in their responses to clinical biases and in their ability to benefit from tools; Llama 3 showed up to 19.7% improvement with tool use.

  • Takeaways & Limitations

    Clinical evaluation of language agents should use interactive, operationalized, dialogue-driven benchmarks that assess sequential decision making in multimodal settings.

  • Takeaways & Limitations

    Simulated patient agents may not fully capture the complexity and variability of real human patients, so their ratings may not perfectly reflect real-world patient perceptions.

Abstract

from arXiv · show

Evaluating large language models (LLM) in clinical scenarios is crucial to assessing their potential clinical utility. Existing benchmarks rely heavily on static question-answering, which does not accurately depict the complex, sequential nature of clinical decision-making. Here, we introduce AgentClinic, a multimodal agent benchmark for evaluating LLMs in simulated clinical environments that include patient interactions, multimodal data collection under incomplete information, and the usage of various tools, resulting in an in-depth evaluation across nine medical specialties and seven languages. We find that solving MedQA problems in the sequential decision-making format of AgentClinic is considerably more challenging, resulting in diagnostic accuracies that can drop to below a tenth of the original accuracy. Overall, we observe that agents sourced from Claude-3.5 outperform other LLM backbones in most settings. Nevertheless, we see stark differences in the LLMs' ability to make use of tools, such as experiential learning, adaptive retrieval, and reflection cycles. Strikingly, Llama-3 shows up to 92% relative improvements with the notebook tool that allows for writing and editing notes that persist across cases. To further scrutinize our clinical simulations, we leverage real-world electronic health records, perform a clinical reader study, perturb agents with biases, and explore novel patient-centric metrics that this interactive environment firstly enables.

1. Introduction

Static medical question-answering benchmarks do not capture the sequential, uncertain, resource-limited nature of clinical work. AgentClinic addresses this gap with an interactive multimodal benchmark that evaluates tools, biases, multiple specialties and languages, and patient-centered outcomes.

  • Static multiple-choice evaluations provide case information upfront, unlike clinical work, which requires sequential decisions under uncertainty and finite resources.
  • AgentClinic introduces an open-source multimodal benchmark that simulates clinical environments through interactions among language agents.
  • The benchmark supports simulated examinations, medical-image readings, 24 clinical biases, nine specialties, seven languages, and patient-centered metrics.
  • AgentClinic evaluates interactive diagnosis, data collection, medical-exam ordering, image understanding, and patient perception rather than static question answering alone.
  • The authors report that current LLMs differ substantially in their response to tools, with some improving accuracy and others declining.

2. AgentClinic: a multimodal agent benchmark for clinical decision making

AgentClinic models clinical decision making with interacting language agents grounded in medical cases, simulated measurements, and multimodal or multilingual scenarios. Its evaluation architecture separates patient, doctor, measurement, and moderator roles while testing biases and comparing performance across settings.

  • Language agents: Four agents—patient, doctor, measurement, and moderator—support the simulation, with the doctor agent evaluated and the others providing the evaluation context.
  • Figure 1 depicts doctor interactions with tools and agents, followed by moderator comparison of the diagnosis with ground truth.
  • Language agent biases: The benchmark introduces cognitive and implicit biases through role-specific instructions that can affect clinical dialogue and treatment decisions.
  • Building agents for AgentClinic: AgentClinic cases are built from USMLE, MIMIC-IV, and NEJM diagnostic materials, with structured case information distributed among agents.
  • Multilingual and Specialist cases: Multilingual cases cover Chinese, Hindi, Korean, Spanish, French, and Persian, with dialogue conducted in the target language.
  • Figure 3 compares model accuracy on MedQA with AgentClinic-MedQA, where MedQA accuracy is only weakly predictive.

3. Results

AgentClinic reveals substantial variation in diagnostic performance across models, specialties, languages, biases, and tools. Interactive evaluation also exposes patient-perception effects and limitations of simulated agents as proxies for real patients.

  • 42.9% was Claude-3.5’s accuracy on AgentClinic-MIMIC-IV, exceeding GPT-4 at 34.0% and most other evaluated models.
  • Reducing AgentClinic-MedQA interactions from N=20 to N=10 decreased accuracy from 52% to 25%, while MedQA accuracy was only weakly predictive of AgentClinic-MedQA accuracy.
  • Bias evaluations: GPT-4’s normalized accuracy under cognitive and implicit biases remained between 92% and 98.6%, whereas Mixtral-8x7B fell as low as 78.4%.Mixtral-8x7B often failed to gather critical information because bias distorted its interpretation of patient cues, whereas GPT-4 sought additional information when hypotheses conflicted with new data.
  • Patient-agent perception: Patient-agent perceptions can diverge from diagnostic accuracy, but simulated confidence, compliance, and consultation ratings may not reflect real-world patient perceptions.Implicit biases strongly affected patient perceptions, while the authors note that language-model patient agents may not capture real human complexity and variability.
  • Agent tools: Claude 3.5 achieved the highest overall tool-based performance at 51.3% average accuracy, while Llama3-70b improved by 9.4% on average across tools.Claude 3.5 peaked at 56.1% with Notebook, whereas GPT-4 reached 42.2% with Reflection CoT and GPT-3.5 declined across all tools.

4. Discussion

AgentClinic broadens medical-AI evaluation through multimodal, interactive clinical simulations that include patient interactions, tools, biases, and multiple specialties and languages. Its results show substantial variation across models, while the simplified environment and possible proprietary-model data leakage constrain interpretation.

  • AgentClinic includes 260 patient cases across 9 medical specialties and 749 cases across 7 multilingual environments.
  • The benchmark simulates patient, doctor, measurement, and moderator agents, with dialogue-based symptom collection and requested medical examinations or image readings.
  • Doctor and patient biases can reduce diagnostic accuracy and decrease patients’ treatment follow-up, confidence in doctors, and willingness to consult again.
  • The benchmark remains a simplified clinical environment, and proprietary-model training data may include evaluation datasets such as MedQA.

B.2. The evaluation of language models in medicine

Medical LLM evaluation commonly uses knowledge-focused, static multiple-choice benchmarks, whereas clinical competence also involves dialogue-driven performance in simulated settings. AgentClinic addresses this gap with multimodal agents, patient interactions, specialist cases, bias simulation, and tool integration.

  • LLMs are typically evaluated with medical-knowledge benchmarks such as USMLE-style exams, despite clinical work requiring dialogue and sequential decision making.
  • MedQA supplies complete case context and asks an LLM to select one answer from 4–5 multiple-choice options.
  • Related medical benchmarks generally retain similar multiple-choice formats, while red-teaming studies have found inappropriate responses in clinically relevant scenarios.
  • Earlier dialogue-based systems such as AMIE evaluate conversational diagnosis but do not provide AgentClinic’s full combination of multimodality, biases, tools, multilingual settings, and specialist cases.
  • AgentClinic evaluates doctor agents in a four-agent setup comprising patient, doctor, measurement, and moderator roles.

D. Statistical analysis

AgentClinic-MIMIC-IV and AgentClinic-MedQA produce different diagnostic accuracies across evaluated language models, with Claude 3.5 highest in both supplied result sets. The confidence intervals quantify uncertainty around each model’s accuracy.

  • The reported 95% confidence intervals range from [37%, 50%] for Claude 3.5 to [5%, 12%] for Llama 3 70B-Instruct on AgentClinic-MIMIC-IV.
  • 42.9% accuracy was achieved by Claude 3.5 on AgentClinic-MIMIC-IV, compared with 8.5% for Llama 3 70B-Instruct.
  • The reported 95% confidence intervals range from [55%, 68%] for Claude 3.5 to [2%, 7%] for Llama 2 70B-chat on AgentClinic-MedQA.
  • 62.1% accuracy was achieved by Claude 3.5 on AgentClinic-MedQA, compared with 4.5% for Llama 2 70B-chat.

D.3. AgentClinic-NEJM

AgentClinic-NEJM evaluates diagnostic performance when models must request medical images rather than receiving them initially. Accuracy is low across the tested models, with Claude 3.5 showing comparatively strong performance in the supplied example.

  • 27.7% accuracy was achieved by GPT-4 when images were provided initially.The 95% confidence interval was [21%, 33%].
  • 25.4% accuracy was achieved by GPT-4 when images had to be requested from the measurement agent.The 95% confidence interval was [20%, 31%].
  • Claude 3.5 achieved 62.1% accuracy on AgentClinic-MedQA, while Llama 2 70B-chat achieved 4.5%.The reported 95% confidence intervals were [55%, 68%] and [2%, 7%], respectively.

E. Bias prompts

The bias-prompt appendix specifies how patient and doctor behavior is altered and how patient perceptions are elicited. It also reports that interaction limits substantially affect diagnostic accuracy, with both insufficient and excessive dialogue reducing performance.

  • Bias prompts: Bias prompts add instructions intended to make patient and doctor agents behave more biasedly.The appendix describes these instructions as part of the patient and doctor prompts.
  • Bias prompts: The patient prompt requires dialogue-only responses of 1-3 sentences while withholding the disease explicitly.The patient may convey symptoms only when asked.
  • Bias prompts: A patient convinced of having cancer despite the doctor’s contrary view exemplifies an anchoring-style bias prompt.The prompt instructs the patient to remain convinced of the diagnosis formed from online symptom research.
  • Bias follow-on questions: Education and gender biases produced low patient ratings despite identical symptoms and diagnosis.The biased agents differed in bias presentation, and the appendix reports their reasons for low ratings.
  • Qualitative bias evaluation prompts: The qualitative evaluation asks patients to rate consultation likelihood, therapy follow-up likelihood, and confidence in the doctor’s assessment from 1 to 10.Each measure uses low, average, and high or good-assessment ranges.
  • F.1. How does limited time affect diagnostic accuracy?: Increasing interactions beyond N=20 also reduced accuracy, from 52% at N=20 to 48% at N=25 and 43% at N=30.The authors suggest that larger inputs become difficult for language models to process.
  • F.1. How does limited time affect diagnostic accuracy?: Real clinical interaction time can be limited because family physicians may ask 3.2 questions and spend less than 2 minutes before concluding.The appendix contrasts this constraint with deployed language agents, which are not necessarily time-limited.

F.2. Does the patient language model affect accuracy?

Diagnostic accuracy depends partly on which language model plays the patient and on how completely the doctor extracts information through interaction. Patient-model differences are accompanied by distinct dialogue behaviors, while coverage is higher for correct diagnoses.

  • 52% accuracy with GPT-4 doctor and patient agents fell to 48% with a GPT-3.5 patient and 46% with a Mixtral patient.The comparison was conducted on AgentClinic-MedQA.
  • GPT-4 patient agents were more likely than GPT-3.5 agents to reveal additional symptomatic information.GPT-3.5 patient agents were more likely to repeat the doctor’s questions.
  • GPT-3.5 doctor accuracy was 38% with a GPT-4 patient and 37% with a GPT-3.5 patient.The near-equal results challenge the expectation that the same-model pairing would perform much worse.
  • Coverage measures the proportion of relevant patient information extracted through dialogue or measurement interactions.The analysis compares dynamically gathered information in AgentClinic-MedQA with information provided upfront in MedQA.
  • 67% was the average information coverage in AgentClinic-MedQA.Coverage was 72% for correct diagnoses and 63% for incorrect diagnoses.

F.4. Multi-Agent Evaluations

Multi-agent diagnostic frameworks improve performance across the reported configurations, with Claude-3.5-Sonnet leading both debate and delegated-task settings. The single-model o1-preview result is higher still but was not evaluated across all benchmarks because of its cost.

  • Multi-Agent Debate: 64.1% ± 3.4 was Claude-3.5-Sonnet’s accuracy in Multi-Agent Debate, versus 51.7% ± 3.0 for GPT-4 and 37.9% ± 3.1 for GPT-4o.Multi-Agent Debate uses multiple doctor agents that debate and converge on a diagnosis.
  • MedAgents: 65.2% ± 3.6 was Claude-3.5-Sonnet’s accuracy in MedAgents, versus 53.1% ± 3.1 for GPT-4 and 40.1% ± 3.3 for GPT-4o.MedAgents uses structured task delegation among agents.
  • The multi-agent frameworks improved performance across all reported configurations compared with single-agent baselines.The authors associate this pattern with task specialization enabling more comprehensive data collection and interpretation.
  • 80.6 ± 5.6 was o1-preview’s accuracy on AgentClinic-MedQA, which the authors report as outperforming all models.It was not included across all benchmarks because inference cost was extraordinarily high, including about 20x higher cost than GPT-4o and Claude-3.5.

H. Constructing datasets

The benchmark constructs AgentClinic cases from MIMIC-IV patients with single diagnoses and represents each case through structured clinical information, examinations, and test results.

  • Dataset selection: 200 MIMIC-IV patients with a single diagnosis were selected from approximately 6,000 eligible patients.The broader MIMIC-IV dataset contains approximately 40,000 patients, most with multiple simultaneous diagnoses.
  • Case representation: Each case records patient demographics, history, symptoms, medical history, and social history for the diagnostic task.The structured fields include age and gender, reported history, primary and secondary symptoms, ongoing treatments, and lifestyle habits.
  • Clinical information: The case representation also includes review of systems, vital signs, cardiovascular and pulmonary examinations, and medical-test findings.Tests may include electrocardiograms and chest X-rays, with each test represented by summarized findings.
  • Diagnostic objective: The doctor is evaluated against a singular correct diagnosis based on the assembled patient information.The case format includes a string indicating the diagnosis based on the available information and a corresponding evaluation objective.
  • Example case: The example patient is a 45-year-old man with sudden chest pain and shortness of breath, hypertension, hyperlipidemia, and a smoking history.His symptoms began while walking, improved somewhat when sitting, and were not accompanied by cough, fever, leg swelling, or palpitations.

Test Results

The example’s initial cardiac, pulmonary, and blood-test results are largely normal, while CT pulmonary angiography identifies an acute pulmonary embolism and establishes the correct diagnosis.

  • Cardiac testing: The electrocardiogram shows normal sinus rhythm without ST elevations, depressions, or T-wave abnormalities.
  • Imaging: The chest X-ray shows no lung infiltrates, a normal cardiac silhouette, and no pneumothorax.
  • Blood testing: Troponin is normal in the blood tests.
  • Definitive finding: Acute segmental pulmonary embolism is found in the right lower lobe on CT pulmonary angiography.
  • Diagnosis: The correct diagnosis for the case is pulmonary embolism.

J.2. Example dialogue from doctor agent ordering tests results MIMIC-IV

The example dialogue illustrates a doctor agent that orders tests, uses reasoning and research tools, and operates within dialogue, language, and bias constraints during diagnosis.

  • Test ordering: The doctor first requests a complete blood count, which returns a panel of hematological measurements.The reported results include white blood cells, hemoglobin, hematocrit, platelets, and differential counts.
  • Test ordering: The doctor then requests a basic metabolic panel, whose returned results include electrolytes, kidney markers, enzymes, and lipase.
  • Reasoning: Chain-of-thought prompts ask the agent to articulate reasoning step by step when solving complex problems.
  • Tool use: The notebook tool lets the doctor agent retain tips from previous patient interactions for use in future cases.The notebook persists across patients and can store diagnostic insights such as the value of symptom timing and onset.
  • Tool use: Adaptive medical retrieval allows the doctor agent to search internet or textbook databases for relevant information.The available sources include PubMed, StatPearls, Wikipedia, and 18 medical textbooks.
  • Agent protocol: The doctor must ask questions within a fixed limit, issue tests through a specified command, and declare a diagnosis when ready.
  • Agent protocol: Patient and measurement agents respond in the target language, while the patient reports symptoms without explicitly revealing the disease.
  • Evaluation: The moderator compares the doctor’s diagnosis with the correct diagnosis using a binary yes-or-no response.
Loading 2405.07960v5…