PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Language Models Encode Clinical Knowledge

Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Nathaneal Scharli, Aakanksha Chowdhery, Philip Mansfield, Blaise Aguera y Arcas, Dale Webster, Greg S. Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Rajkomar, Joelle Barral, Christopher Semturs, Alan Karthikesalingam, Vivek Natarajan

arXiv:2212.13138v1cs.CL

TL;DR

Medical AI models remain limited in expressive, interactive clinical use. This paper introduces broad medical QA benchmarks, human evaluation, and instruction prompt tuning; Flan-PaLM led multiple-choice benchmarks, while Med-PaLM improved safety-related evaluations but remained below clinicians.

  • Problem

    Medical AI models are largely single-task systems that lack the expressive and interactive capabilities expected in real-world clinical workflows.

  • Method

    The paper combines diverse medical QA datasets with multidimensional human evaluation and instruction prompt tuning to assess and align LLMs for medicine.

  • Results

    Flan-PaLM achieved state-of-the-art performance across four multiple-choice datasets, exceeding previous MedQA state-of-the-art by over 17%, while Med-PaLM improved human-rated alignment and harm outcomes.

  • Takeaways & Limitations

    Instruction prompt tuning can improve medical LLM accuracy, factuality, consistency, safety, harm, and bias measures, narrowing but not closing the gap with clinicians.

  • Takeaways & Limitations

    The study’s bias evaluation is limited and does not comprehensively assess potential harms, fairness, or equity.

Abstract

from arXiv · show

Large language models (LLMs) have demonstrated impressive capabilities in natural language understanding and generation, but the quality bar for medical and clinical applications is high. Today, attempts to assess models' clinical knowledge typically rely on automated evaluations on limited benchmarks. There is no standard to evaluate model predictions and reasoning across a breadth of tasks. To address this, we present MultiMedQA, a benchmark combining six existing open question answering datasets spanning professional medical exams, research, and consumer queries; and HealthSearchQA, a new free-response dataset of medical questions searched online. We propose a framework for human evaluation of model answers along multiple axes including factuality, precision, possible harm, and bias. In addition, we evaluate PaLM (a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM, on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA, MedMCQA, PubMedQA, MMLU clinical topics), including 67.6% accuracy on MedQA (US Medical License Exam questions), surpassing prior state-of-the-art by over 17%. However, human evaluation reveals key gaps in Flan-PaLM responses. To resolve this we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, recall of knowledge, and medical reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal important limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLM models for clinical applications.

1 Introduction

This work introduces MultiMedQA and HealthSearchQA to evaluate clinical question answering beyond limited automated benchmarks, then assesses PaLM variants using prompting and human evaluation. Flan-PaLM achieves state-of-the-art benchmark performance, while instruction prompt tuning improves alignment but leaves important safety and clinical-use limitations.

  • Motivation: Existing medical question-answering benchmarks largely assess classification accuracy or automated generation metrics, despite medical answers requiring comprehension, knowledge recall, and expert reasoning.The safety-critical domain also requires evaluation of hallucinated misinformation and bias that could exacerbate health disparities.
  • Benchmark: MultiMedQA combines six existing medical question-answering datasets with HealthSearchQA, a new dataset of 3375 commonly searched consumer medical questions.The benchmark spans medical exams, medical research, and consumer medical questions.
  • Limitations: Human evaluation revealed limitations in Flan-PaLM’s scientific grounding, harm, and bias, and further evaluations are needed before LLMs become viable for clinical applications.The paper identifies fairness, equity, and bias as priorities for future evaluation and mitigation.
  • Results: 67.6% accuracy on MedQA, 57.6% on MedMCQA, and 79.0% on PubMedQA were achieved by Flan-PaLM, which attained state-of-the-art performance across the evaluated benchmarks.These results used few-shot, chain-of-thought, and self-consistency prompting; MedQA performance was more than 17% above prior state of the art.

2 Related work

Prior work shows that scaling transformer-based language models improves performance and data efficiency, while in-context prompting enables rapid adaptation and apparent reasoning without parameter updates. Biomedical language models demonstrate domain-specific promise but are generally smaller than broad LLMs, whose hallucination, bias, and reasoning limitations motivate systematic evaluation.

  • Large language models (LLMs): Model performance and data efficiency scale with model size and dataset size, following the success of larger transformer-based training.LLMs owe their success to scaling up training of transformer-based models.
  • Large language models (LLMs): In-context few-shot abilities let LLMs adapt to diverse tasks without gradient-based parameter updates and generalize to unseen tasks with suitable prompting.Appropriate prompting strategies can also elicit apparent reasoning abilities.
  • Large language models (LLMs): LLMs can act as implicit knowledge bases, but they risk hallucinations, amplified social biases, and deficient reasoning, motivating benchmarks such as BIG-bench.BIG-bench was introduced as a community-wide initiative to benchmark tasks and quantify gaps between human and LLM language capabilities.
  • LLMs for science and biomedicine: Curated scientific and biomedical corpora have enabled effective discriminative and generative models, although these systems are typically smaller in scale and scope than GPT-3 and PaLM.Examples include SciBERT, BioNLP, BioMegatron, BioBERT, PubMedBERT, DARE, ScholarBERT, and BioGPT.
  • LLMs for science and biomedicine: Closest precedents include Galactica and a medical question-answering study combining Instruct GPT-3 with chain-of-thought prompting on MedQA, MedMCQA, and PubMedQA.The cited study applied chain-of-thought prompting to improve results on these datasets.

3 Methods

The study evaluates medical question answering through the MultiMedQA benchmark, a human-evaluation framework, and models adapted to medical-domain instructions. MultiMedQA combines heterogeneous datasets, while instruction prompt tuning provides a lightweight way to adapt LLMs to medical instructions.

  • Datasets: MultiMedQA combines multiple-choice and long-form datasets spanning professional exams, medical research, and consumer medical questions.The benchmark includes MedQA, MedMCQA, PubMedQA, LiveQA, MedicationQA, MMLU clinical topics, and HealthSearchQA.
  • Datasets: 3375 commonly searched consumer questions comprise HealthSearchQA, a new open benchmark for consumer medical question answering.The dataset was curated from seed medical conditions and associated symptoms using publicly available search-engine questions.
  • Human evaluation: A pilot framework evaluates long-form model answers to consumer medical questions in LiveQA, MedicationQA, and HealthSearchQA.The framework addresses physical and mental health-related harms among multiple dimensions of harm.
  • Modeling: Self-consistency prompting samples multiple decoding outputs and selects the final answer by majority or plurality vote for multiple-choice benchmarks.The method is motivated by the possibility of multiple reasoning paths in complex medical questions.
  • Modeling: Instruction prompt tuning adapts LLMs to medical instructions through a data-efficient, parameter-efficient, and compute-efficient soft-hard prompt approach.The method combines a soft prompt with a hard prompt and targets instruction following across medical datasets.

4 Results

Flan-PaLM 540B achieved strong results across medical multiple-choice benchmarks, with instruction tuning and scaling improving performance. Human evaluation nevertheless found substantial gaps versus clinicians, although Med-PaLM improved factual retrieval, completeness, safety, helpfulness, and question-intent alignment.

  • Multiple-choice benchmark results: 67.6% MCQ accuracy was achieved by Flan-PaLM 540B on four-option MedQA questions, surpassing DRAGON by 20.1%.On five-option MedQA questions, the model obtained 62.0%.
  • Multiple-choice benchmark results: 57.6% performance was achieved on the MedMCQA dev set, exceeding the previous state-of-the-art result of 52.9%.
  • Multiple-choice benchmark results: 79.0% accuracy on PubMedQA surpassed BioGPT by 0.8%, while single-rater human performance was 78.0%.The result suggests an inherent ceiling for this task.
  • Scaling and prompting: Instruction-tuned Flan-PaLM outperformed baseline PaLM across MedQA, MedMCQA, and PubMedQA at all model sizes.The largest improvement occurred on PubMedQA, where 8B Flan-PaLM exceeded baseline PaLM by over 30%.
  • Scaling and prompting: Approximately 2x improvement followed scaling from 8B to 540B in both PaLM and Flan-PaLM, with gains more pronounced on MedQA and MedMCQA.For Flan-PaLM, 540B outperformed 62B by over 14%.
  • Human evaluation: 61.9% of Flan-PaLM answers agreed with scientific consensus, versus 92.9% for clinicians across 140 evaluated questions.Clinician answers also scored 97.8% for correct medical-knowledge retrieval, compared with 76.3% for Flan-PaLM and 95.4% for Med-PaLM.

5 Discussion

The discussion attributes strong medical question-answering performance to LLM scale combined with instruction prompt tuning, while emphasizing that scale alone does not ensure safe clinical use. The results suggest memorization alone is insufficient to explain scaling gains, and that prompt tuning can improve model alignment.

  • Scaling and instruction prompt tuning: Strong medical question-answering performance may emerge from LLM scale combined with effective instruction prompt tuning.This interpretation is presented as a result of the discussion rather than as a standalone mechanism.
  • Scaling and instruction prompt tuning: Approximately 2x accuracy improvement occurred when scaling PaLM from 8-billion to 540-billion parameters across medical question-answering datasets.PaLM 8-billion performance on MedQA was only slightly better than random, while PaLM 540-billion improved by over 30%; similar improvements occurred on MedMCQA and PubMedQA.
  • Scaling and instruction prompt tuning: Similar performance deltas on contaminated and cleaned test datasets suggest memorization alone does not explain gains from scaling PaLM models.The pre-training corpus may have included substantial high-quality medical content, raising memorization as one possible explanation.
  • Scaling and instruction prompt tuning: Medical answering performance requiring recall, reading comprehension, and reasoning improves with LLM scale, including on the out-of-domain MedQA dataset.The models outperformed BioGPT, PubMedGPT, and Galactica on PubMedQA without finetuning, while scale and instruction fine-tuning benefits were more pronounced on MedQA.
  • Safety and alignment: Human evaluations show scale alone is insufficient because Flan-PaLM can generate inappropriate answers for safety-critical medical use.The discussion states that Med-PaLM demonstrates instruction prompt tuning as a data- and parameter-efficient alignment technique for improving accuracy, factuality, consistency, safety, and harm-related factors.

6 Limitations

The study’s benchmarks, language coverage, model capabilities, and human-evaluation framework have important limitations that constrain clinical interpretation. Future work must develop more realistic, multilingual, rigorous, equitable, and safety-focused evaluation and deployment approaches.

  • Benchmark scope: MultiMedQA is diverse but not exhaustive, motivating expansion to additional medical and scientific domains.The authors specifically identify biology and additional formats as future additions.
  • Benchmark scope: Multiple-choice tasks are easier than real clinical work, which requires eliciting patient information and synthesizing it into assessments and plans.The authors call for benchmark tasks reflecting real-world clinical workflows.
  • Benchmark scope: The study used only English-language datasets, while future benchmarks should support multilingual evaluations.Multilingual response capability is also identified as a needed model capability.
  • Model capabilities: Despite state-of-the-art multiple-choice performance, the models remained below clinician-expert level on many clinically important axes.Needed capabilities include authoritative grounding, effective uncertainty communication, and multilingual responses.
  • Evaluation limitations: The evaluation framework was subjective and incomplete, with limited responses and single-rater assessments constraining generalizability and bias evaluation.The authors note that harm and consensus vary across populations, cultures, locations, lived experiences, and health literacy, while current bias assessment is not comprehensive.
  • Future deployment: Clinical deployment will require substantial research to ensure safety, reliability, efficacy, privacy, and ethical quality assessment across use contexts.The paper emphasizes evaluation frameworks as a critical research agenda, alongside participatory, contextualized, and interdisciplinary approaches to downstream harms and health equity.

7 Conclusion

Foundation AI models and large language models offer an opportunity to make medical AI easier, safer, and more equitable to use, but medicine remains an especially complex application domain. This research highlights both the opportunities and challenges of applying these technologies to medicine and calls for broad collaboration to translate early findings responsibly.

  • Foundation AI models and large language models could make medical AI easier, safer, and more equitable to use, although medicine is an especially complex domain.
  • The study highlights opportunities and challenges in applying these technologies to medicine and encourages collaboration to translate early findings responsibly into improved healthcare.

Appendix · A.1 Hyperparameters and model selection

Med-PaLM was produced by instruction prompt tuning Flan-PaLM 540B with a frozen base model and a 100-token soft prompt, yielding 1.84M trainable parameters. Hyperparameters were grid-searched, training used fixed batch and step settings, and checkpoint selection relied on clinician rankings of held-out examples rather than an automated validation metric.

  • A.1 Hyperparameters and model selection: A 100-token soft prompt was tuned on Flan-PaLM 540B to produce Med-PaLM.
  • A.1 Hyperparameters and model selection: The base model was frozen, leaving 1.84M trainable parameters from an embedding dimension of 18432.
  • A.1 Hyperparameters and model selection: The learnable parameters were randomly initialized uniformly over [-0.5, 0.5].
  • A.1 Hyperparameters and model selection: Learning rates of 0.001, 0.003, and 0.01 were grid-searched with AdamW and weight decay factors of 0.001 and 0.00001.
  • A.1 Hyperparameters and model selection: Training used a batch size of 32 across all runs for 200 steps.
  • A.1 Hyperparameters and model selection: A clinician ranked responses on held-out HealthSearchQA, MedicationQA, and LiveQA examples to select the best-performing checkpoint.These examples were not used for training or human evaluation.
  • A.1 Hyperparameters and model selection: Manual validation was used instead of automated metrics such as negative log-likelihood because such metrics may not correlate with human judgments of generated outputs.

A.2 Variation of results

Repeated temperature-sampled evaluations on MedQA showed low result variance for the best-performing model, indicating high consistency despite stochastic decoding.

  • A.2 Variation of results: 0.078 observed variance across four MedQA evaluations with the best-performing model suggested a high degree of consistency under stochastic decoding.The evaluations used repeated temperature-sampled decodes with self-consistency.

A.3 MMLU ablations

The ablation compares Flan-PaLM 540B using few-shot, chain-of-thought, and self-consistency prompting on MMLU clinical topics. Self-consistency generally performs best, although some topics favor few-shot or chain-of-thought prompting, and Flan-PaLM 540B achieves state-of-the-art performance across the topics.

  • A.3 MMLU ablations: The ablation evaluates Flan-PaLM 540B with few-shot, chain-of-thought, and self-consistency prompting on MMLU clinical topics.The comparison also includes PaLM 540B results with few-shot prompting.
  • A.3 MMLU ablations: Self-consistency prompting produces the best results for most MMLU clinical topics.
  • A.3 MMLU ablations: A few topics perform better with standard few-shot or chain-of-thought prompting, while Flan-PaLM 540B remains state-of-the-art across the topics.

A.4 Scaling plots

Scaling plots compare PaLM and Flan-PaLM across prompting strategies on MedQA and MedMCQA. Performance scales strongly, with steeper gains as model size increases.

  • Scaling plots: Scaling plots compare PaLM and Flan-PaLM using few-shot prompting on MedQA and MedMCQA.These comparisons appear in Figure A.1.
  • Scaling plots: Scaling plots also compare Flan-PaLM few-shot prompting with few-shot plus chain-of-thought and self-consistency prompting on MedQA and MedMCQA.This comparison appears in Figure A.2.
  • Scaling plots: Performance scales strongly, with a steeper increase as the LLM model size grows.The observation applies to the reported scaling plots.

A.5 Model card for Med-PaLM

Med-PaLM uses the same system type and implementation frameworks as Flan-PaLM. The paper presents Med-PaLM-specific portions of its model card in Table A.2.

  • Model card: Med-PaLM uses the same system type and implementation frameworks as Flan-PaLM.The paper presents Med-PaLM-specific portions of its model card in Table A.2.

A.6 Med-PaLM multiple-choice evaluation … A.9 Chain-of-Thought prompt examples

Instruction prompt tuning yielded 67.2% MedQA accuracy in a preliminary multi-dataset experiment, while detailed human evaluations showed substantial improvements from Flan-PaLM to Med-PaLM across consensus, harm, reasoning, bias, and helpfulness. The appendices provide few-shot and chain-of-thought prompt examples, including evidence-focused instructions and medical research examples.

  • A.6 Med-PaLM multiple-choice evaluation: 67.2% accuracy was achieved on MedQA after instruction prompt tuning Flan-PaLM across four clinical multiple-choice datasets, using chain-of-thought and self-consistency.Training used five clinician-written few-shot exemplars per example, with dataset-specific instructions.
  • A.7 Detailed human evaluation results: 92.9% agreement with scientific and clinical consensus was reported for Med-PaLM, compared with 61.9% for Flan-PaLM.The comparison used expert answers as the reference.
  • A.7 Detailed human evaluation results: 6.0% of Med-PaLM responses were judged potentially harmful, compared with 29.6% for Flan-PaLM and 6.5% for clinician-generated answers.These figures measure the possible extent of harm.
  • A.7 Detailed human evaluation results: 95.4% and 93.5% of Med-PaLM answers showed correct retrieval and reasoning, versus 76.3% and 85.7% for Flan-PaLM.Clinician answers scored 97.8% for retrieval and 97.7% for reasoning.
  • A.7 Detailed human evaluation results: 80.1% of Med-PaLM answers were judged helpful, improving on Flan-PaLM’s 59.6% but remaining below clinicians’ 90.8%.Med-PaLM also directly addressed user intent in 94.0% of cases, compared with 90.8% for Flan-PaLM and 95.9% for clinicians.
  • A.9 Chain-of-Thought prompt examples: Chain-of-thought prompt examples are provided for MedQA, MedMCQA, PubMedQA, and MMLU in Tables A.18–A.21.The appendix explicitly identifies these tables as chain-of-thought prompt examples.
  • A.9 Chain-of-Thought prompt examples: The chain-of-thought instructions require step-by-step reasoning, consideration of scientific evidence strength, and a single final option for medical research questions.Examples include interpreting meropenem pharmacokinetics, dementia-caregiver outcomes, and adherence to abnormal Pap-smear follow-up.
Loading 2212.13138v1…