PaperlayerView exact revision on arXiv

Source-linked AI summary

Capabilities of GPT-4 on Medical Challenge Problems

Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, Eric Horvitz

arXiv:2303.13375v2cs.CLcs.AI

TL;DR

As general-purpose LLMs are considered for specialized tasks, their performance on medical challenge problems is an important evaluation question. The authors compare GPT-4 with GPT-3.5 and Flan-PaLM 540B on USMLE and MultiMedQA, while probing image questions, calibration, memorization, and qualitative student support. Zero-shot GPT-4 scored 86.65% on USMLE Self-Assessment versus GPT-3.5’s 53.61%, significantly outperformed GPT-3.5 on MultiMedQA, and outperformed Flan-PaLM 540B on all but one MultiMedQA dataset.

  • Problem

    As general-purpose LLMs are considered for specialized tasks, their performance on medical challenge problems is an important evaluation question.

    Source: §1 · p. 2

  • Method

    The authors compare GPT-4 with GPT-3.5 and Flan-PaLM 540B on USMLE and MultiMedQA, while probing image questions, calibration, memorization, and qualitative student support.

    Source: §1 · pp. 2–3, §9 · p. 22

  • Results

    Zero-shot GPT-4 scored 86.65% on USMLE Self-Assessment versus GPT-3.5’s 53.61%, significantly outperformed GPT-3.5 on MultiMedQA, and outperformed Flan-PaLM 540B on all but one MultiMedQA dataset.

    Source: §1 · p. 2, §9 · p. 22

  • Takeaways & Limitations

    Benchmark performance and qualitative probes suggest potential for GPT-4 to assist healthcare professionals, but do not establish successful clinical use.

    Source: §8 · p. 19

    Clinical use warrants caution because model recommendations can be inaccurate, and fluent hallucinations may be difficult to detect.

    Source: §8 · p. 20

Abstract

from arXiv · show

Large language models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation across various domains, including medicine. We present a comprehensive evaluation of GPT-4, a state-of-the-art LLM, on medical competency examinations and benchmark datasets. GPT-4 is a general-purpose model that is not specialized for medical problems through training or engineered to solve clinical tasks. Our analysis covers two sets of official practice materials for the USMLE, a three-step examination program used to assess clinical competency and grant licensure in the United States. We also evaluate performance on the MultiMedQA suite of benchmark datasets. Beyond measuring model performance, experiments were conducted to investigate the influence of test questions containing both text and images on model performance, probe for memorization of content during training, and study probability calibration, which is of critical importance in high-stakes applications like medicine. Our results show that GPT-4, without any specialized prompt crafting, exceeds the passing score on USMLE by over 20 points and outperforms earlier general-purpose models (GPT-3.5) as well as models specifically fine-tuned on medical knowledge (Med-PaLM, a prompt-tuned version of Flan-PaLM 540B). In addition, GPT-4 is significantly better calibrated than GPT-3.5, demonstrating a much-improved ability to predict the likelihood that its answers are correct. We also explore the behavior of the model qualitatively through a case study that shows the ability of GPT-4 to explain medical reasoning, personalize explanations to students, and interactively craft new counterfactual scenarios around a medical case. Implications of the findings are discussed for potential uses of GPT-4 in medical education, assessment, and clinical practice, with appropriate attention to challenges of accuracy and safety.

1 Introduction

The paper evaluates text-only GPT-4 on medical challenge problems against earlier GPT models and medical models, emphasizing out-of-the-box performance and examining additional capabilities and limitations.

  • With simple zero-shot or randomly selected 5-shot prompts, GPT-4 achieved best-in-class performance without elaborate prompting or domain-specific fine-tuning.The evaluation compares GPT-4 with GPT-3.5 and reported results from Flan-PaLM 540B.Source: §1 · p. 2
  • Zero-shot GPT-4 scored 86.65% on the USMLE Self-Assessment and 86.7% on the Sample Exam, versus 53.61% and 58.78% for GPT-3.5, respectively.It also significantly outperformed GPT-3.5 and Flan-PaLM 540B on MultiMedQA.Source: §1 · p. 2
  • The study also examines text-centric versus image-reliant questions, answer calibration, possible exam-content memorization, and GPT-4’s interactive medical-reasoning explanations for students.The case study explores generating counterfactual scenarios around a medical case.Source: §1 · p. 3
  • The authors discuss possible uses in medical education and healthcare while stressing accuracy, fairness, benchmark limitations, the need for caution, and further evaluation.They anticipate lower-stakes uses with expert oversight and see longer-term potential for GPT-4 and its successors to benefit medicine.Source: §1 · p. 3

2 Methodology

The methodology evaluates text-only GPT-4 on medical multiple-choice datasets using a simple, standardized prompting approach and compares it with GPT-3.5 and other reported models.

2 Methodology

  • All subsequent references to GPT-4 mean the text-only model without vision capabilities.The methodology focuses on this version even though GPT-4 supports multimodal capabilities.Source: §2 · p. 3

2.1 Datasets

  • The evaluation covers six datasets: two NBME USMLE sources and four public medical benchmarks that form much of MultiMedQA [SAT+22].The public benchmarks are MedQA, PubMedQA, MedMCQA, and MMLU.Source: §2.1 · p. 3

2.2 Prompting

  • To measure baseline performance, the study uses [SAT+22]’s prompt structure, with minor chat-format changes for chat-optimized models.Zero-shot prompts leave the few-shot slot blank, and omit the context slot when a dataset provides no additional context; the structure uses minimal context tokens and one generation token per inference. The study avoids chain-of-thought prompting, retrieval-augmented generation, and ensembling, leaving broader optimization for future work.Source: §2.2 · pp. 4–5, Figure 2.1 · p. 4, Figure 2.2 · p. 4

2.3 Model Comparison

  • GPT-4 and GPT-3.5 are evaluated zero-shot and 5-shot; 5-shot accuracy uses leave-one-out cross-validation with five random exemplars per sample.The study also incorporates literature results for ChatGPT, InstructGPT, Flan-PaLM 540B, and Med-PaLM; unavailable Flan-PaLM and Med-PaLM results come from [SAT+22].Source: §2.3 · p. 5

3 Performance of GPT-4 on Medical Competency Exams

On official NBME USMLE practice exams, GPT-4 outperformed GPT-3.5 by over 30 percentage points and cleared the approximate passing threshold; it also performed well on text-only questions referencing unseen media, while alignment was associated with lower raw benchmark performance.

3 Performance of GPT-4 on Medical Competency Exams

  • The evaluation used officially purchased NBME USMLE Sample Exam and Self Assessment materials, which the authors consider among the most authentic available sources for this study.They contrast these materials with unofficial third-party datasets used to approximate USMLE performance.Source: §3 · p. 5

3.1 Results

  • GPT-4 improved by over 30 percentage points over GPT-3.5 on both official USMLE exams and cleared the approximately 60% passing threshold.GPT-4 also exceeded independently reported ChatGPT scores on the Sample Exam; that comparison is qualified because removed questions with media elements, unlike this study.Source: §3.1 · p. 6, Table 1 · p. 6, Table 2 · p. 6

3.1.1 Language- and Vision-Centric Challenges

  • Without vision, GPT-4 achieved 70–80% accuracy on questions referencing media on both exams, though it performed best on text-only questions.The authors attribute its success despite unseen images to logical reasoning and test-taking strategies.Source: §3.1.1 · p. 7, Table 3 · p. 7

3.2 Influence of Alignment and Safety Tuning

  • GPT-4-base performed 3–5% better than the publicly released GPT-4 across 14 experimental datasets, suggesting alignment can reduce raw medical-benchmark performance.The authors describe safety and instruction-following as potential contributors to this tradeoff and suggest refining tuning or using expert medical knowledge to improve performance without sacrificing those capabilities.Source: §3.2 · p. 9, Table 4 · p. 9

4 Medical Challenge Benchmarks

GPT-4 outperforms GPT-3.5 and Flan-PaLM 540B on nearly all tested multiple-choice MultiMedQA datasets, while performing well on difficult non-English questions. The benchmarks cover four datasets; three long-answer datasets were excluded.

4 Medical Challenge Benchmarks

  • The evaluation covers MedQA, PubMedQA, MedMCQA, and medical MMLU; LiveQA, MedicationQA, and HealthSearchQA were excluded as long-answer datasets requiring extensive expert review.HealthSearchQA also does not appear to be publicly available.Source: §4 · p. 9

4.1 Results

  • GPT-4 outperforms GPT-3.5 and Flan-PaLM 540B on every tested dataset except PubMedQA, using zero-shot direct prompts for GPT-4 and GPT-3.5.Flan-PaLM 540B results use few-shot prompts, with 3–5 prompts depending on the dataset.Source: Table 5 · p. 10, §4.1 · p. 10
  • Flan-PaLM 540B few-shot results are used as the closest comparison to the study setup, while the cited Med-PaLM result is preliminary and used an ensemble of chain-of-thought, few-shot prompts.[SAT+22] reports 67.2% for Med-PaLM.Source: §4.1 · p. 10
  • GPT-4 performs well on difficult non-English questions, and the study reports stratified metrics for MedQA and MMLU subcomponents.MedQA includes mainland China and Taiwan questions in English, simplified Chinese, and traditional Chinese; its English/United States version has five-answer and four-option variants.Source: §4.1 · p. 10, Table 5 · p. 10
  • GPT-4 boosts MedQA and MedMCQA performance with a simpler zero-shot prompt than earlier prompt-heavy approaches, extending the trend of reduced prompting effort across model generations.In [LHW22], Codex with an ensemble of 100 chain-of-thought samples scored 60.2 on the USMLE component of MedQA, 59.7 on the MedMCQA dev set, and 78.2 on PubMedQA.Source: §4.1 · pp. 10–11

5 Calibration

On multiple-choice USMLE questions, GPT-4 was significantly better calibrated than GPT-3.5; the section emphasizes that reliable probabilities matter in high-stakes medicine but calibration is distinct from predictive accuracy.

5 Calibration

  • In high-stakes medicine, well-calibrated probabilities can support trustworthy uncertainty estimates and expected-utility decisions, but calibration is not the same as predictive accuracy.Poorly calibrated predictions can mislead users through overconfidence or underconfidence, even when a predictive model is accurate.Source: §5 · p. 11
  • GPT-4’s assigned average probability of 0.96 corresponded to 93% correctness, while GPT-3.5’s similar-probability answers were correct 55% of the time on the official USMLE datasets.The comparison is shown for both the USMLE Self-Assessment and Sample Exam.Source: Figure 5.1 · p. 11, §5 · p. 12
  • Calibration plots bin predictions by estimated probability and compare each bin’s average probability with its true positivity rate, using the selected answer choice’s probability here.The analysis compares GPT-4 and GPT-3.5 in the multiple-choice setting on both official USMLE datasets.Source: §5 · pp. 11–12
  • The experiment covers multiple-choice questions where models score each option; calibration and probabilities for long-form generation remain open research areas.The multiple-choice results suggest that language-model probability calibration may increase with scale.Source: §5 · p. 12

6 Directions and Limitations

The paper establishes baseline GPT-4 performance without specialized prompting, then discusses prompting extensions and limitations in testing for memorization and generalizing benchmark results to live USMLE exams.

6.1 Richer Prompting Strategies

  • The study set baseline performance as its goal and left exploration of prompting techniques and fine-tuning that might raise scores to future work.The authors note that chain-of-thought, self-consistency, and retrieval methods have improved performance in prior work.Source: §6.1 · p. 12
  • Basic two-stage chain-of-thought prompting did not improve GPT-4’s medical-question performance, though other prompt structures may perform better.The experiment asked GPT-4 to reason step by step, then used that generation to request a final prediction.Source: §6.1 · p. 12
  • Expert-curated few-shot examples produced negligible performance differences from random selection for GPT-4 and GPT-3.5 across tested datasets.The authors suggest expert curation may not be necessary for strong performance in newer LLMs, while noting that deeper investigation remains future work.Source: §6.1 · p. 13, Table 6 · p. 13

6.2 Memorization

  • MELD probes possible memorization by prompting a model to generate near-exact matches and comparing them with benchmark samples; it has high precision but unknown recall.A detected match is likely to indicate training-set presence and memorization, but an undetected match does not establish that the data was excluded.Source: §6.2 · p. 13
  • GPT-4 regenerated SQuAD 2.0 questions with 99% overlap 17% of the time, but MELD found no USMLE matches and no USMLE samples with even 50% overlap.The authors caution that failure to detect a match does not prove the data was absent from training; they consider official exam content unlikely to have been included.Source: §6.2 · pp. 13–14

6.3 Focus on multiple choice

  • Benchmark results may not predict live USMLE performance because evaluations focused on multiple-choice questions, omitted quantitative computer-based case simulations, and lacked recent exam questions and scoring criteria.Step 3 includes 13 interactive computer-based case simulations, which were not quantitatively benchmarked.Source: §6.3 · p. 14

7 Beyond Correct Answers: Probing Capabilities

An illustrative interactive case study shows GPT-4 extending beyond exam answers to explain medical reasoning, personalize teaching, and create counterfactual cases, while emphasizing that experts must review generated information.

7.1 Explanation

  • GPT-4 provides students with rich explanations of errors and can hypothesize why they made a mistake.The section presents these abilities through an illustrative dialogue initiated by a single test question.Source: §7 · p. 14, Figure 7.2 · p. 16

7 Beyond Correct Answers: Probing Capabilities

  • Information generated in these interactions and real-world applications cannot be trusted without expert review and confirmation.The paper returns to accuracy concerns in Section 8.Source: §7 · p. 14

7.2 Metacognition

  • GPT-4 demonstrates a theory-of-mind–like ability to understand a student and personalize an educational explanation.The figure caption identifies this capability as part of the case study.Source: §7 · p. 14, Figure 7.3 · p. 17

7.3 Challenge Problem to Interactive Session

  • GPT-4 creates counterfactual clinical scenarios through minimal case rewrites and can reformulate questions by changing lab values.The case study describes these variants as simulating alternative scenarios.Source: §7 · p. 14, Figure 7.4 · p. 18, Figure 7.5 · p. 19

8 From Benchmarks to Real World

GPT-4’s benchmark and clinical capabilities suggest potential to support healthcare work, but proficiency on examinations does not establish readiness for clinical use. Real-world deployment requires careful attention to reliability, bias, and effects on practitioners and society.

Risks of erroneous generations

  • High-stakes use requires caution because fluent generations can contain factual errors, omissions, misleading recommendations, and hallucinations that may be difficult to detect.Reliability is also hard to characterize in open-ended prompting: outputs can vary with prompt wording and model revisions.Source: §8 · pp. 19–20
  • Safer deployment requires research on output veracity, including retrieval grounding, self-consistency checks, context-specific accuracy evaluation, and better calibration signals.The authors also call for improved human-computer interaction, professional quality-assurance practices, and education about reliability challenges.Source: §8 · p. 20

Risks of Bias

  • Fairness risks remain poorly understood, and healthcare-centric LLM recommendations may reflect biases related to race, socioeconomic background, gender, and other factors.Research is needed to understand and address bias in clinical and research uses of these models.Source: §8 · pp. 20–21

Influences on workflows, tasks, and specialties

  • With expert oversight, GPT-4 could support clinical reasoning, decision support, memory jogging, screening, and administrative work, while freeing clinicians for patient engagement and learning.Potential tasks include revising differential diagnoses, identifying and sequencing tests, and constructing therapy plans; models could also support care and communication in underserved regions.Source: §8 · p. 21

Social and societal issues

  • GPT-4’s capabilities could influence medical career and specialty choices and practitioners’ perceptions of the value of human contributions to healthcare.The authors note that perceptions of AI’s growing role may affect choices at both career-entry and training stages.Source: §8 · p. 21

Implications for the future

  • The leap from GPT-3.5 to GPT-4 suggests that scaling may bring further gains on complex real-world challenges, with implications for medicine and other knowledge-intensive professions.The authors warn that such advances could disrupt the social contract around professional education, practice, and status.Source: §8 · p. 22

9 Conclusion · Appendix

The evaluation found that GPT-4 outperformed GPT-3.5 and Flan-PaLM 540B on USMLE-style questions and generally surpassed them on MultiMedQA. The authors highlight its potential in medical education and healthcare while urging prudence about errors and real-world performance.

9 Conclusion

  • GPT-4 significantly outperformed GPT-3.5 and Flan-PaLM 540B on USMLE-representative questions across Steps 1, 2, and 3.The evaluation included zero-shot performance and compared questions relying solely on text with those referring to visual media.Source: §9 · p. 22
  • GPT-4 significantly surpassed GPT-3.5 on MultiMedQA and outperformed Flan-PaLM 540B on all but one dataset.Source: §9 · p. 22
  • The authors explored calibration of GPT-4’s output probabilities and stressed its importance for medical applications.Source: §9 · p. 22
  • Sample outputs demonstrated GPT-4’s capacity for reasoning about USMLE challenge problems through explanations, counterfactual reasoning, differential diagnosis, and testing strategies.Source: §9 · p. 22
  • The authors view GPT-4’s benchmark performance as an indicator of potential to support medical education and healthcare professionals across healthcare delivery.Source: §9 · p. 22
  • They urge prudence because errors and the difficulty of assessing real-world performance remain concerns, and call for evaluating appropriate uses and mitigating risks.They also call for technical innovations to optimize advantages and mitigate risks associated with applications.Source: §9 · p. 22

A Dataset descriptions

The section describes the USMLE practice materials and MultiMedQA datasets used in the evaluation, including their sources, question formats, test splits, and selected benchmark scope.

USMLE Self Assessments

  • 2,173 NBME Self-Assessment questions comprise 1,197 Step 1, 800 Step 2, and 176 Step 3 questions across the listed exam forms.The materials were purchased from NBME Self-Assessment resources.Source: A Dataset descriptions · p. 28

USMLE Sample Exam

  • The USMLE Sample Exam uses official practice materials; it matches Kung et al.’s dataset, but retains questions containing media.Sample materials cover Steps 1, 2, and 3.Source: A Dataset descriptions · p. 28

MedQA

  • 1,273 US, 3,426 Mainland China, and 1,413 Taiwan test questions make up MedQA; Taiwan has four options, while US and Mainland China have five [JPO+21].The US test split also has a simplified four-option version with the same 1,273 questions, created by removing one incorrect option.Source: A Dataset descriptions · p. 28

PubMedQA

  • PubMedQA evaluates yes/no/maybe answers to biomedical questions using PubMed abstract context; results use the reasoning-required setting [JDL+19].Choices are multiple-choice: A is yes, B is no, and C is maybe, matching Liévin et al. [LHW22].Source: A Dataset descriptions · p. 29

MedMCQA

MMLU

B Memorization analysis · C Additional prompt examples

The memorization analysis describes MELD, a method for estimating near-exact matches between generated text and held-out dataset content. The additional prompt examples show a question-specific template and examples of its use for GPT-4 multiple-choice questions.

B Memorization analysis

  • MELD splits each datapoint in half, prompts the model with the first half, and measures similarity between its completion and the held-out second half.A Levenshtein distance ratio of at least 0.95 defines a near-exact match; the dataset-level match rate can provide empirical evidence that the model saw the data during training.Source: B Memorization analysis · p. 29
  • The MELD algorithm reports the percentage of completions whose overlap with ground truth exceeds a chosen threshold.The algorithm describes this threshold as Y%.Source: B Memorization analysis · p. 30
  • MELD splits content into q1 and q2, tokenizes both halves, sets temperature to zero, and generates as many tokens as the tokenized second half contains.It then calculates a Levenshtein distance-based ratio between the second-half tokens and generated text.Source: B Memorization analysis · p. 30

C Additional prompt examples

  • The prompt template replaces double-braced placeholders with question-specific values and encodes few-shot examples as user and assistant chat messages.Source: Figure C.1 · p. 31
  • The additional materials include a sample question using the chat prompt template and a system message instructing the assistant to answer multiple-choice questions.Source: C Additional prompt examples · p. 32
  • Figure C.2 presents an instantiated example of the earlier prompt template.Source: Figure C.2 · p. 32

D Interactive educational session (continued)

The continued educational session illustrates GPT-4 revising a case scenario by changing lab values, suggesting ways to narrow a differential diagnosis, and advising on patient engagement.

D Interactive educational session (continued)

  • When pressed further, GPT-4 provides a more sophisticated rewrite that focuses on altering lab result values.Figure D.1 continues the case study with this revised scenario.Source: Figure D.1 · p. 33
  • GPT-4 recommends information and tests that could narrow the differential diagnosis.Figure D.2 presents this diagnostic guidance.Source: Figure D.2 · p. 34
  • GPT-4 offers advice on engaging with a patient diagnosed with schistosomiasis.Figure D.3 focuses on patient engagement in this case.Source: Figure D.3 · p. 35

Related papers

Loading 2303.13375v2…