Source-linked AI summary
Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health
Shubo Tian, Qiao Jin, Lana Yeganova, Po-Ting Lai, Qingqing Zhu, Xiuying Chen, Yifan Yang, Qingyu Chen, Won Kim, Donald C. Comeau, Rezarta Islamaj, Aadit Kapoor, Xin Gao, Zhiyong Lu
TL;DR
Biomedical and health applications of ChatGPT and other LLMs span information retrieval, question answering, summarization, information extraction, and medical education, but their domain-specific opportunities and challenges require evaluation. This literature survey finds strongest progress in text generation, while performance varies across tasks and hallucinations and high-stakes deployment remain concerns.
Problem
The survey examines whether LLMs can transform biomedical and health applications despite the distinct complexities and risks of this domain.
Method
The authors conducted a literature survey using biomedical LLM and application keywords across PubMed, medRxiv, arXiv, and Google Scholar.
Results
LLMs achieved new state-of-the-art performance on text generation tasks such as medical question answering, but underperformed smaller fine-tuned models for information extraction and varied across downstream tasks.
Takeaways & Limitations
LLMs hold promising prospects for biomedical and health systems, but retrieval augmentation, rigorous evaluation, privacy protection, and regulation are needed to address identified risks.
Takeaways & Limitations
Fabricated information can make ChatGPT-based retrieval dangerous, and no actual deployment of ChatGPT or other LLMs in biomedical practice had been reported.
Abstract
from arXiv · showhide
ChatGPT has drawn considerable attention from both the general public and domain experts with its remarkable text generation capabilities. This has subsequently led to the emergence of diverse applications in the field of biomedicine and health. In this work, we examine the diverse applications of large language models (LLMs), such as ChatGPT, in biomedicine and health. Specifically we explore the areas of biomedical information retrieval, question answering, medical text summarization, information extraction, and medical education, and investigate whether LLMs possess the transformative power to revolutionize these tasks or whether the distinct complexities of biomedical domain presents unique challenges. Following an extensive literature survey, we find that significant advances have been made in the field of text generation tasks, surpassing the previous state-of-the-art methods. For other applications, the advances have been modest. Overall, LLMs have not yet revolutionized biomedicine, but recent rapid progress indicates that such methods hold great potential to provide valuable means for accelerating discovery and improving health. We also find that the use of LLMs, like ChatGPT, in the fields of biomedicine and health entails various risks and challenges, including fabricated information in its generated responses, as well as legal and privacy concerns associated with sensitive patient data. We believe this survey can provide a comprehensive and timely overview to biomedical researchers and healthcare practitioners on the opportunities and challenges associated with using ChatGPT and other LLMs for transforming biomedicine and health.
1. Introduction
This survey examines ChatGPT and other LLMs in biomedical and health applications, covering their potential uses and domain-specific limitations.
- The survey covers biomedical information retrieval, question answering, medical text summarization, information extraction, and medical education.
- It reviews biomedical-specific LLMs developed by training from scratch or fine-tuning existing pretrained models.
- The literature survey focuses on ChatGPT and other generative AI LLMs and searches PubMed, medRxiv, arXiv, and Google Scholar.
- The paper discusses both the capabilities and limitations of LLMs in biomedicine and health.
2. Overview ChatGPT and Domain-specific LLMs
The section introduces LLM architectures, adaptation strategies, and their emerging biomedical applications. It highlights stronger performance on biomedical question answering alongside cost and task-specific adaptation challenges.
- Overview of General LLMs: Encoder-only, decoder-only, and encoder-decoder LMs differ in architecture and training objectives, with transformers providing the foundation for most modern LMs.
- Overview of General LLMs: Decoder-only LMs such as GPT models use autoregressive training and can address unseen tasks through zero-shot, one-shot, or few-shot prompting.
- LLMs for Biomedical and Health Applications: Med-PaLM 2 approached human-expert performance on MedQA in less than half a year, advancing from GPT-3.5’s human-passing level.
- LLMs for Biomedical and Health Applications: Instruction fine-tuning, RLHF fine-tuning, and soft prompt tuning adapt LLMs for specific applications through supervised, reinforcement, or learned-prompt approaches.
- LLMs for Biomedical and Health Applications: Pre-training biomedical models from scratch or existing checkpoints can be costly as model sizes grow, whereas instruction fine-tuning, soft prompt tuning, and prompt engineering can be more accessible.
- LLMs for Biomedical and Health Applications: Biomedical-specific and general-purpose LLMs are being developed and evaluated across diverse biomedical application tasks.
3. Applications of ChatGPT and LLMs in Biomedicine and Health
ChatGPT and other LLMs are being explored across biomedical information retrieval, question answering, summarization, information extraction, and medical education. The survey reports promising but uneven benefits, alongside persistent risks from fabricated or difficult-to-verify outputs and limited evaluation evidence.
- Information Retrieval: LLMs may summarize information returned by traditional retrieval systems, but generated summaries still require careful verification because they are not guaranteed error-free.Pilot studies found effective reading comprehension and fluent, high-fidelity summaries when sufficient context and background information were provided.
- Information Retrieval: ChatGPT can enrich biomedical search queries by generating more specific terms, concepts, or synonyms for improved retrieval.It has been used to formulate and refine Boolean queries for systematic reviews across more than 100 topics.
- Question Answering: Large LLMs achieve strong biomedical question-answering results, while smaller models perform similarly to smaller domain-enhanced state-of-the-art systems.Med-PaLM 2 and GPT-4 lead three common biomedical QA tasks; they are comparable to human experts on MedQA and exceed them on PubMedQA.
- Question Answering: Retrieval augmentation combines LLMs with search systems so answers are generated from retrieved supporting documents, potentially reducing hallucinations but not eliminating errors.The survey also describes augmenting models with database APIs when information is inaccessible through ordinary search.
- Question Answering: Clinical chatbots have been instruction-tuned on biomedical corpora, but their accuracy, generalizability, and practical utility remain unclear because evaluations often use small private datasets.This limits conclusions about their effectiveness for consumer-facing medical information needs.
- Information Extraction: LLMs show promise for biomedical information extraction when labeled data are unavailable, yet they do not surpass fine-tuned models and may generate unverifiable or false entities and relations.Generative paraphrases complicate verification, while selecting effective prompts for NER and RE remains challenging.
4. Limitations and Risks of LLMs
LLMs pose biomedical and health risks involving unreliable outputs, bias, privacy, legal and ethical accountability, and difficult evaluation. These concerns are especially consequential in high-stakes settings, where transparency and expert assessment remain important.
- Hallucination: Hallucinated content can be plausible but incorrect, potentially causing serious harm in medical advice and clinical decision-making.LLMs may also lack transparency about their sources, so healthcare professionals should verify generated information.
- Clinical complexity: Biomedical LLMs may miss clinical nuance, produce erroneous results, and struggle with non-standard medical terminology.Their ability to analyze large datasets and identify patterns does not replicate the judgment and experience of human clinicians.
- Fairness and bias: Bias learned from historical data can be amplified in biomedical applications, potentially perpetuating healthcare inequalities and harming treatment quality.Limited demographic information in medical datasets, partly due to privacy concerns, makes fairness research more difficult.
- Privacy: Training corpora may contain private information, and biomedical or clinical data can expose sensitive patient information.Prior work also indicates that language models may leak personal details or help identify private individuals.
- Legal and ethical concerns: Legal and ethical concerns include accountability, copyright, authorship, and disclosure of LLM use in research and publication.Machine-generated text may lack copyright protection in some jurisdictions, while LLMs cannot take responsibility for harmful or misleading outputs.
- Evaluation: Biomedical LLMs require evaluation of performance, safety, and bias, but expert assessment of free-text outputs is labor-intensive.Multiple-choice medical benchmarks can proxy knowledge and reasoning, yet they do not reproduce real-life questions without answer choices.
5. Discussion and Conclusion
The survey finds that LLM performance varies across biomedical tasks, with strong progress in text generation but weaker information extraction performance than smaller fine-tuned models. Although deployment prospects are promising, no actual ChatGPT or LLM deployment had been reported, and risks, limitations, and closed-source training data require prudence.
- Findings: LLMs achieved new state-of-the-art performance on text-generation tasks such as medical question answering but underperformed smaller fine-tuned models for information extraction.The survey reports that performance varies across biomedical downstream tasks.
- Opportunities: Prospective applications include summarizing medical literature, radiology reports, and clinical notes, while translating medical jargon for patients and personalizing medical education.The paper presents these as promising future uses of LLM-powered systems.
- Deployment: No actual deployment of ChatGPT or other LLMs in biomedical or clinical practice had been reported at the time of the survey.The paper therefore calls for prudence because biomedical and health settings are high stakes and involve model limitations and risks.
- Challenges: Closed-source training data create a transparency challenge that increases the difficulty of evaluating LLMs.This limits visibility into the data underlying ChatGPT and other LLMs.
- Future directions: Future development requires strategies addressing harmful content, bias, privacy, legal use, ethics, and model transparency.Suggested measures include retrieval augmentation, careful prompting, rigorous evaluation, diversified data, bias detection, deidentification, and regulation.