Source-linked AI summary
Large language models in medicine: the potentials and pitfalls
Jesutofunmi A. Omiye, Haiwen Gui, Shawheen J. Rezaei, James Zou, Roxana Daneshjou
TL;DR
Healthcare LLM use is expanding, but practitioners need a grounded understanding of how these systems are developed, applied, and limited. This review synthesizes their training, medical applications, and unresolved challenges, finding broad potential alongside concerns about accuracy, bias, privacy, and ethics.
Problem
Growing healthcare adoption and institutional partnerships make it important for practitioners to understand LLM development, applications, and pitfalls in medicine.
Method
The review surveys LLM training approaches, medical models and applications, proposed clinical tasks, and associated accuracy, bias, privacy, and ethical concerns.
Results
Domain-specific medical pre-training can outperform generalist LLMs on biomedical tasks, while fine-tuning general models can achieve strong medical-task results efficiently; applications include patient education and communication.
Takeaways & Limitations
LLMs show growing potential for medical knowledge, patient communication, administrative work, education, and research, but their use requires attention to unresolved limitations.
Takeaways & Limitations
LLM training data may contain bias, overlap between training and testing sets, outdated information, and material that is difficult to quality-check or update.
Abstract
from arXiv · showhide
Large language models (LLMs) have been applied to tasks in healthcare, ranging from medical exam questions to responding to patient questions. With increasing institutional partnerships between companies producing LLMs and healthcare systems, real world clinical application is coming closer to reality. As these models gain traction, it is essential for healthcare practitioners to understand what LLMs are, their development, their current and potential applications, and the associated pitfalls when utilized in medicine. This review and accompanying tutorial aim to give an overview of these topics to aid healthcare practitioners in understanding the rapidly changing landscape of LLMs as applied to medicine.
2. Architecture of LLMs
LLMs are built through progressively specialized training stages, beginning with broad self-supervised pre-training and continuing through fine-tuning and prompting for task-specific use. Transformer attention supports context-sensitive language processing, while later stages adapt the general base model to controlled outputs and specialized applications.
- Transformer attention helps LLMs learn relationships between relevant words while ignoring irrelevant information in sentences and longer passages.
- Pre-training uses large unlabeled corpora and next-word prediction to create a general language-generating base model with billions of learned parameters.The process is resource-intensive and may include internet text, code, social media posts, books, and proprietary scientific datasets.
- Fine-tuning adapts the base model using narrower datasets, human scoring, comparison-based feedback, or embedded principles for specific tasks and controlled outputs.Examples include medical transcripts for healthcare applications and legal briefs for legal assistants.
- Few-shot and zero-shot prompting provide task guidance at inference, while instruction prompt tuning offers a cost-effective way to improve downstream medical-task performance.These approaches augment the core training processes of fine-tuned models and can improve their alignment with specialized tasks.
- The training process moves from broad unsupervised inputs to narrower datasets, human feedback, and specialized prompting for augmented task performance.
3. Overview of current medical-LLMs
Medical LLMs include general-purpose and domain-specific models, with applications spanning patient education, clinical workflow support, and medical education. Their promise is tempered by concerns about data quality, bias, hallucinations, privacy, ethics, and the need for rigorous evaluation and oversight.
- Model landscape: Transformer-based LLMs expanded beyond earlier language models through larger parameter counts and capabilities such as reasoning and in-context learning.The paper contrasts modern LLMs with statistical and neural predecessors that used fewer parameters and smaller datasets.
- Model development: Domain-specific pre-training can outperform generalist models on biomedical tasks, while fine-tuning general models can achieve strong results with greater computational efficiency.The paper identifies this efficiency as promising because medical training data are limited.
- Model performance: Scaling and medical adaptation have produced models with improved accuracy, calibration, and physician-like responses, although they remain unequal to real clinical expertise.Med-PaLM is presented as evidence of these improvements and of growing potential in healthcare.
- Medical applications: General-purpose and biomedical models support patient education, clinical workflow tasks, medical education, and academic applications.Examples include answering patient questions, supporting tumor boards, drafting clinical documents, answering medical board questions, and assisting research.
- Pitfalls: Medical LLMs remain vulnerable to dataset overlap, outdated information, structural bias, hallucinations, confirmation bias, unpredictable evolution, and privacy risks.Reported privacy concerns include personally identifiable information in training data and inferences about patients from correlational patterns.
- Pitfalls: Ethical risks include emotional harm from difficult disclosures and misuse of generated text for misinformation, plagiarism, impersonation, or false documents.The paper notes that conversational interfaces may expose patients to difficult topics without personalized emotional support.
6. Tutorial with ChatGPT
The tutorial demonstrates ChatGPT’s medical and research uses while showing that outputs require verification, iterative prompting, privacy protection, and careful oversight. It also outlines future developments involving multimodality, institution-specific models, synthetic data, regulation, explainability, and holistic evaluation.
- Tutorial with ChatGPT: ChatGPT can generate clinical authorization letters, patient handouts, medical examination questions, and research-paper outlines from targeted prompts.The tutorial uses these four prompt types to illustrate potential applications across clinical administration, patient education, medical education, and research.
- Tutorial with ChatGPT: Generated medical content may appear accurate while containing hallucinated sources, and repeated prompts can produce different outputs because the models are stochastic.The acid-reflux handout example is described as apparently accurate, but its listed sources do not exist; the text also asks whether repetition yields the same result.
- Tutorial with ChatGPT: Iterative prompting can improve generic research outlines by requesting additional detail, such as a literature review and introduction.The tutorial describes continuing the conversation with the model to add detail and enhance an initially generic outline.
- Future of LLM in medicine: Future LLMs may integrate text, images, audio, and genetics, while longer context and lower resource requirements could improve medical performance and access.The review identifies multimodality, improved architectures, longer context lengths, and reduced compute requirements as emerging directions.
- Future of LLM in medicine: Institution-specific LLMs could reflect local procedures, guidelines, and challenges, with potential to enhance productivity, reduce burnout, and improve patient care.These models are presented as a possible consequence of reduced compute requirements and institution-specific training data.
- Future of LLM in medicine: Synthetic medical data may help address scarce research data and support more diverse, inclusive, and equitable medical research.The review presents synthetic data as a promising approach to expanding training data diversity.
- Future of LLM in medicine: Clinical adoption requires validation frameworks, fairness protections, privacy governance, explainability, and holistic metrics covering accuracy, reasoning, bias, and fairness.The review emphasizes these safeguards because inaccurate outputs can have severe consequences and cause patient harm.
- Future of LLM in medicine: The future role of LLMs is framed as supporting healthcare workers rather than replacing them, potentially helping address healthcare shortages and inefficiencies.This conclusion is stated as a likely direction for future medical use.
8. Conclusion
LLMs offer opportunities across administrative work, clinical knowledge support, education, and research, but unresolved bias, data-quality, unpredictability, privacy, and ethical concerns limit current healthcare adoption.
- Conclusion: LLMs could support administrative tasks, clinician knowledge, medical education, and medical research across several areas of medicine.Examples include summarizing medical notes, translating patient materials, creating examination questions, and generating research ideas.
- Conclusion: Bias, data quality, unpredictable outputs, patient privacy, and ethical concerns remain unresolved barriers to implementation in healthcare.The conclusion states that these limitations make current translation of LLM innovation into healthcare adoption difficult.
- Conclusion: Healthcare professionals must weigh LLM opportunities against these limitations when considering incorporation into practice.The paper places this assessment responsibility on physicians and other healthcare professionals.