Source-linked AI summary
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics
Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, Erik Cambria
TL;DR
Healthcare LLM research is advancing beyond earlier single-task PLMs, but practical use requires addressing complex clinical data and deployment concerns. This survey synthesizes applications, technologies, training practices, benchmarks, and accountability issues, finding a shift toward generative, data-centered healthcare AI. It identifies fairness, accountability, transparency, and ethics as major constraints, while noting limitations related to medical knowledge and training processes.
Problem
Healthcare applications need systems that can handle complex clinical data, advanced semantic understanding, multimodal inputs, and interactive tasks beyond the limitations of single-task PLMs.
Method
The paper conducts a comprehensive survey of healthcare LLM data, technologies, applications, training methods, benchmarks, open-source resources, and fairness, accountability, transparency, and ethics.
Results
The survey identifies a transition from PLMs to LLMs and from discriminative, model-centered approaches toward generative, data-centered approaches, with Med-PaLM 2 and GPT-4 among the most robust healthcare LLMs.
Takeaways & Limitations
LLMs show their strongest advantages on complex healthcare tasks and increasingly support advanced applications, including multimodal healthcare use and interactive dialogue.
Takeaways & Limitations
Healthcare LLM effectiveness remains limited by the extent of existing medical knowledge, while RLHF is constrained by costly expert feedback and training stability challenges.
Abstract
from arXiv · showhide
The utilization of large language models (LLMs) in the Healthcare domain has generated both excitement and concern due to their ability to effectively respond to freetext queries with certain professional knowledge. This survey outlines the capabilities of the currently developed LLMs for Healthcare and explicates their development process, with the aim of providing an overview of the development roadmap from traditional Pretrained Language Models (PLMs) to LLMs. Specifically, we first explore the potential of LLMs to enhance the efficiency and effectiveness of various Healthcare applications highlighting both the strengths and limitations. Secondly, we conduct a comparison between the previous PLMs and the latest LLMs, as well as comparing various LLMs with each other. Then we summarize related Healthcare training data, training methods, optimization strategies, and usage. Finally, the unique concerns associated with deploying LLMs in Healthcare settings are investigated, particularly regarding fairness, accountability, transparency and ethics. Our survey provide a comprehensive investigation from perspectives of both computer science and Healthcare specialty. Besides the discussion about Healthcare concerns, we supports the computer science community by compiling a collection of open source resources, such as accessible datasets, the latest methodologies, code implementations, and evaluation benchmarks in the Github. Summarily, we contend that a significant paradigm shift is underway, transitioning from PLMs to LLMs. This shift encompasses a move from discriminative AI approaches to generative AI approaches, as well as a shift from model-centered methodologies to data-centered methodologies. Also, we determine that the biggest obstacle of using LLMs in Healthcare are fairness, accountability, transparency and ethics.
1. Introduction
Healthcare LLMs extend earlier language-model approaches by supporting broader data processing, complex reasoning, and more interactive applications. This survey traces that transition while emphasizing that fairness, accountability, transparency, and ethics increasingly constrain practical deployment.
- Healthcare applications: Healthcare LLM applications target difficult needs including rare-disease diagnosis, complex patient narratives, personalized treatment planning, and question answering.The survey describes LLM-based systems for medical QA, dialogue, diagnosis, and tailored treatment support.
- From PLMs to LLMs: LLMs emerged as a major transition from PLMs, expanding Healthcare AI from single-task processing toward broader understanding, generation, and reasoning.PLMs such as BERT and RoBERTa were simpler and more efficient for basic tasks but were limited in dynamic interaction with complex medical data.
- Healthcare capabilities: LLMs can process patient records, clinical notes, research papers, and medical images, enabling analysis of multimodal healthcare information.Healthcare applications increasingly combine text, images, and other data types, while specialized models also support active inquiry and medical reasoning.
- Deployment concerns: As model performance improves, fairness, accountability, transparency, and ethics are presented as increasingly significant impediments to practical healthcare implementation.The paper frames these concerns alongside privacy, legal responsibility, and unequal access to healthcare technology.
- Survey scope: The survey compares healthcare LLMs with PLMs and with one another while compiling data, training, performance, and publicly available resources.Its stated contributions include a development roadmap, public data and task-performance lists, and analysis of fairness, accountability, transparency, and ethics.
2. What LLMs can do for healthcare? from fundamental tasks to advanced applications
LLMs extend healthcare language technologies from fundamental information extraction and classification toward complex question answering, multimodal analysis, dialogue, and advanced applications. Their advantages increase with contextual and multimodal complexity, although PLMs remain competitive for simpler or retrieval-oriented tasks and major healthcare challenges persist.
- Fundamental tasks: Healthcare applications progress from fundamental information extraction toward advanced tasks involving complex data, semantic understanding, and generation.NER and RE support downstream normalization, knowledge-base construction, and dialogue, while the survey organizes applications from fundamental tasks to advanced uses.
- Limitations: Despite their capabilities, LLMs remain comparable to specially trained SOTA PLMs in information extraction involving professional terms and symbols.The survey therefore assigns distinct advantages to PLMs and LLMs rather than treating either model family as universally superior.
- Text classification: LLMs provide a clear advantage over PLMs in healthcare text classification because specialized terminology and context require deeper contextual reasoning.Terms such as “MI” can have different meanings depending on context, making contextual integration important for healthcare classification.
- Task-dependent comparison: For short-text classification, SOTA PLMs and LLMs are comparable, whereas PLMs are more practical than LLMs for efficient information retrieval.LLMs can be overly complex and resource-intensive when used as simple retrievers, while PLMs are generally more lightweight.
- Question answering: LLMs improve healthcare question answering and support prompt-based formulations of NER, RE, and other tasks.Med-PaLM 2 approached or exceeded state-of-the-art performance across MedMCQA, PubMedQA, and MMLU clinical-topic datasets.
- Advanced applications: LLM-based systems enable end-to-end healthcare dialogue and multimodal applications, including VQA and automated medical-report generation.These systems can reduce reliance on extra submodules and incorporate text, images, and other modalities into healthcare workflows.
- Healthcare challenges: Healthcare deployment remains constrained by medical decision-making complexity, privacy and ethical concerns, accountability, unequal access, and limited knowledge in poorly understood diseases.The survey identifies these challenges as persistent obstacles requiring collaboration across healthcare, technology, law, and ethics.
3. From PLMs to LLMs for healthcare
The survey describes a shift from PLMs to LLMs in Healthcare, marked by generative and data-centered approaches, broader training strategies, and expanded evaluation. It also identifies fairness, accountability, transparency, and ethics as continuing concerns.
- Paradigm shifts: Healthcare LLM research shifts from discriminative PLMs toward generative models evaluated on question answering and conversation tasks.PLMs mainly emphasized NLU tasks, whereas LLMs support generation-oriented evaluation.
- Paradigm shifts: The field is also moving from model-centered architecture improvements toward data-centered development emphasizing high-quality, diverse, and matched multimodal healthcare data.Relevant data include EHRs, medical images, and medical sequence signals.
- Limitations and trends: Healthcare LLM development remains constrained by costly training, undisclosed data mixtures, expensive expert feedback, reward instability, and risks of catastrophic forgetting.The survey also notes that most investigated models derive from general LLMs and that smaller models may lose performance.
- Training methods: Healthcare LLMs are commonly adapted through supervised fine-tuning, while RLHF and DPO are increasingly used in multi-stage training pipelines.SFT is widely employed; Zhongjing combines PT, SFT, and RLHF, while Qilin-Med combines PT, SFT, DPO, and RAG.
- Evaluation: Evaluation evidence includes 77.6% on PubMedQA for Galactica, 72.8% accuracy on MMLU-Medical for JMLR, and 42.8% on CMExam for Qilin-Med with RAG.JMLR’s MMLU-Medical result exceeded Meditron-70B’s 68.9%, while Qilin-Med improved from 40.0% after SFT to 42.8% with RAG.
- Evaluation: The survey identifies Med-PaLM 2 and GPT-4 as among the most robust models, while reporting that the performance gap between LLMs and humans has narrowed.GPT-4 is general-purpose, whereas Med-PaLM 2 is designed specifically for Healthcare.
4. Usage and data for healthcare LLM
Healthcare LLM usage spans parameter-free adaptation, reasoning, agentic systems, and data-centered development. The survey emphasizes promising capabilities alongside sensitivity to demonstration quality, computational cost, and data scale.
- In-context Learning: In-context learning combines demonstrations with test inputs, adapting responses to specific healthcare knowledge without updating model parameters.Its success depends on input similarity, label relevance, demonstration format, and input-label pairing.
- Chain-of-Thought: Chain-of-thought prompting adds intermediate reasoning steps that can improve reasoning and planning while making healthcare decisions more interpretable.However, detailed reasoning increases computational complexity and latency, which may limit emergency deployment.
- AI Agents: AI agents combine planning, memory, tool use, and action, with LLMs envisioned as controllers coordinating specialized models for healthcare processes.The survey notes that such agents have not yet been widely adopted in healthcare.
- Healthcare Training Data: Healthcare LLM development increasingly follows a data-centered approach, with larger pre-training datasets treated as key drivers of general capability.The survey organizes healthcare datasets and associated computation costs to support assessment of training requirements.
- Healthcare Training Data: Common healthcare LLM data sources include electronic health records, scientific literature, web data, and public knowledge bases, with QA and dialogue frequent structures.Healthcare’s text, image, and time-series data make multimodal LLMs a promising research direction.
- Usage and Data: Non-parametric methods such as in-context learning, chain-of-thought, and agents can enhance healthcare LLM capabilities without expensive training processes.The survey presents these methods as directions for constructing more complete healthcare AI systems.
5. Improving fairness, accountability, transparency, and ethics
Healthcare LLM deployment raises interconnected concerns about fairness, accountability, transparency, privacy, and ethics because model outputs can affect patient well-being and safety. The survey reviews bias sources, mitigation strategies, interpretability methods, privacy risks, and preventive accountability measures.
- Fairness: Fairness aims to prevent LLM outputs from producing unjust advantages or disadvantages based on attributes such as race, gender, or socioeconomic status.Uneven demographic representation in pre-training data can contribute to healthcare bias.
- Fairness: Chest X-ray models trained on public datasets have shown underdiagnosis tendencies for female, Black, Hispanic, and Medicaid-covered patients.The survey links these disparities to underrepresentation and uneven demographic distributions in training data.
- Fairness: Healthcare fairness interventions include importance weighting, resampling, instruction fine-tuning, and prompt engineering.RLHF is identified as a representative instruction-fine-tuning technique for aligning models with human instructions.
- Accountability: Hallucinations, counterfactual outputs, and vulnerability to perturbations threaten reliability because healthcare outputs can affect people’s health and lives.Fact-based evaluations reported no performance enhancement for ChatGPT over earlier versions, raising concerns about faithfulness.
- Accountability: Accountability requires preventive measures, including user access to human-generated references and ongoing safety evaluation by developers and system safety engineers.The survey contrasts dynamic assurance with static assurance completed only before deployment.
- Transparency: Chain-of-thought prompting provides reasoning steps that can improve interpretability, but it requires costly annotations and raises challenges in evaluating interpretability.For PLMs, quantitative methods can assess explanations without retraining and under distribution shifts.
- Privacy and Ethics: Sensitive EHR content can leak from models despite de-identification, creating risks of recovering patients’ personal health information.Federated learning keeps data localized during training and may support more diverse and less biased models.
- Privacy and Ethics: Ethical deployment requires recurring bias assessments, rigorous privacy safeguards, transparent systems, and collaboration among ethicists, technologists, clinicians, and patients.The survey frames these activities as necessary for harnessing benefits while minimizing risks.
6. Discussion
Healthcare LLMs support decision-making, diagnosis, research, resource access, and multimodal analysis, but practical adoption remains constrained by integration complexity and global disparities. The discussion emphasizes both application breadth and unresolved infrastructure, data, and regulatory challenges.
- Healthcare Applications: Healthcare LLM technologies support six core healthcare issues, combining foundational extraction tasks with increasingly important generative QA and dialogue systems.NER, relation extraction, and text classification are widely used in real-world scenarios.
- Clinical Decision Support: Clinical decision support systems use structured extraction, similar-patient analysis, and LLM-based responses to provide flexible medical decision support.Advanced LLM-based systems can address user-posed health queries but remain rare in practice.
- Early Diagnosis: LLMs can analyze historical health data for early disease signals and individual risk prediction using NER, relation extraction, text classification, QA, and dialogue.The discussion highlights applications to cancer, cardiovascular, and brain diseases.
- Medical Research and Access: In medical research, NER and relation extraction identify literature keywords while semantic textual similarity finds related studies, reducing literature-review time.Researchers can query LLMs for multiple potential answers that inspire further exploration.
- Medical Research and Access: QA and dialogue systems can provide chatbots for common healthcare questions and recommend human experts for complex cases, helping address limited access in remote areas.This use targets uneven healthcare-resource distribution rather than replacing expert care.
- Multimodal Healthcare LLMs: Multimodal healthcare LLMs integrate text, medical images, and time-series data to support more comprehensive analyses of complex medical phenomena.Persistent challenges include data heterogeneity, integration complexity, and the need for large-scale, high-quality datasets.
- Integration with Healthcare Processes: Hospital IT integration is difficult because systems use different formats, standards, interfaces, and complex workflows, while retrieving training data can disrupt operations.Hospital consolidations can further fragment existing IT systems.
- Global Challenges: Global deployment faces a digital divide and divergent legal and ethical frameworks, requiring greater accessibility, local adaptation, and cross-national collaboration.Shared governance mechanisms are presented as part of addressing these differences.
7. Conclusion
The survey describes a transition from PLMs to LLMs, including shifts from discriminative to generative AI and from model-centered to data-centered approaches. It concludes that interpretability, privacy, medical knowledge, workflow integration, and interaction remain major barriers to practical healthcare adoption.
- Conclusion: Healthcare LLM research reflects a shift from PLMs toward generative AI and data-centered approaches, alongside broader examination of data, technologies, applications, and governance.The survey covers fairness, accountability, transparency, and ethics as part of this transition.
- Conclusion: Interpretability, privacy protection, medical knowledge enhancement, healthcare-process integration, and interaction with patients and doctors remain substantial adoption obstacles.These challenges hinder translation of innovative LLMs into practical healthcare use.
CRediT authorship contribution statement
The contribution statement assigns authors roles spanning writing, investigation, formal analysis, methodology, and conceptualization.
- Kai He contributed to writing, investigation, formal analysis, and conceptualization.
- Rui Mao and Qika Lin contributed to writing, methodology, and conceptualization.
- Yucheng Ruan and Xiang Lan contributed to writing, with Xiang Lan also contributing to conceptualization.