Source-linked AI summary
Radiology-GPT: A Large Language Model for Radiology
Zhengliang Liu, Aoxiao Zhong, Yiwei Li, Longtao Yang, Chao Ju, Zihao Wu, Chong Ma, Peng Shu, Cheng Chen, Sekeun Kim, Haixing Dai, Lin Zhao, Lichao Sun, Dajiang Zhu, Jun Liu, Wei Liu, Dinggang Shen, Xiang Li, Quanzheng Li, Tianming Liu
TL;DR
Radiology produces abundant text requiring specialized interpretation, while externally hosted LLMs face privacy and regulatory barriers in clinical settings. Radiology-GPT applies instruction tuning to radiology data, and the authors report superior performance over existing baselines, while noting limits in the diversity of its tuning instructions.
Problem
Radiology produces extensive text requiring specialized interpretation, but commercial LLMs face clinical barriers including privacy concerns, HIPAA, IRB approvals, and external patient-data uploads.
Source: §1 · pp. 1–2Method
Results
Radiology-GPT displays superior performance over existing baselines in the authors’ evaluations, and is reported as comparable to ChatGPT in understandability and slightly better in coherence, but slightly behind in relevance.
Source: §4 · p. 7, §6 · p. 13Takeaways & Limitations
The authors present Radiology-GPT as a promising locally deployable AI solution for radiology and propose qualitative and quantitative metrics for evaluating models in healthcare settings.
Source: §6 · pp. 12–13The current instruction-tuning data primarily uses Findings-to-Impression pairs, and the authors say more diverse, clinically meaningful pairs could enhance the model.
Source: §3.3 · p. 6
Abstract
from arXiv · showhide
We introduce Radiology-GPT, a large language model for radiology. Using an instruction tuning approach on an extensive dataset of radiology domain knowledge, Radiology-GPT demonstrates superior performance compared to general language models such as StableLM, Dolly and LLaMA. It exhibits significant versatility in radiological diagnosis, research, and communication. This work serves as a catalyst for future developments in clinical NLP. The successful implementation of Radiology-GPT is indicative of the potential of localizing generative large language models, specifically tailored for distinctive medical specialties, while ensuring adherence to privacy standards such as HIPAA. The prospect of developing individualized, large-scale language models that cater to specific needs of various hospitals presents a promising direction. The fusion of conversational competence and domain-specific knowledge in these models is set to foster future development in healthcare AI. A demo of Radiology-GPT is available at https://huggingface.co/spaces/allen-eric/radiology-gpt.
1 Introduction
Radiology produces extensive text requiring specialized interpretation, yet externally hosted LLMs face privacy and regulatory barriers in clinical settings. Radiology-GPT is presented as a locally deployable, radiology-focused model that outperforms general instruction-tuned models.
1 Introduction
- Clinical use of large commercial models is constrained by HIPAA, privacy concerns, IRB approvals, and patient-data uploads to external platforms.Radiology generates reports, clinical notes, and imaging annotations that require sophisticated understanding and interpretation.Source: §1 · pp. 1–2
- Generative LLMs are described as reducing reliance on complex fine-tuning and labor-intensive manual annotation, potentially lowering development time and cost.The authors present this as making clinical adoption more feasible and appealing.Source: §1 · p. 2
- Radiology-GPT outperforms instruction-tuned models not specially trained for radiology, including StableLM and Dolly.The model is trained on MIMIC-CXR, providing radiology-specific language and content.Source: §1 · p. 2
- Radiology-GPT is a dedicated, localized model intended to address clinical privacy and regulatory challenges while applying LLMs to radiology.The work identifies the need for a radiology-specific foundational model that can operate within regulatory boundaries.Source: §1 · p. 2
- The model uses Alpaca instruction tuning and a large, richly annotated radiology dataset to handle complex radiological language and tasks.The authors describe generative LLMs as flexible with input structure and output generation, including for reasoning tasks.Source: §1 · p. 2
- The authors identify localized models for specialties such as Radiation Oncology and Cardiology as a future direction, alongside radiology text interpretation for clinicians and researchers.They also describe conversational capability as a potential aid to medical professionals.Source: §1 · p. 2
2 Related work
Related work describes the shift toward capable, accessible LLMs and instruction-tuned open models. It also surveys domain-specific language models and motivates combining their tailored knowledge with LLM strengths for radiology.
2.1 Large language models
- LLMs have expanded language-model capabilities through few-shot and zero-shot in-context learning, while open-source models such as LLaMA and Bloom broaden access and reproducibility.Instruction-tuned models including Alpaca, StableLM, and Dolly are presented as able to interpret and execute complex instructions; Alpaca is described as cost-effective and reproducible.Source: §2.1 · p. 3
2.2 Domain-specific language models
- Domain-specific language models are trained or fine-tuned on domain text to target specialized tasks and language patterns, with examples spanning agriculture, education, and healthcare.ClinicalRadioBERT is described as a radiation-oncology model trained on clinical notes and radiotherapy literature; the text argues radiology needs strengths from both LLMs and domain-specific models.Source: §2.2 · p. 3
3 Methodology
Radiology-GPT is built by preprocessing radiology reports and instruction-tuning an Alpaca-based model to map Findings to Impression. The current tuning uses that single instruction pair, with more diverse clinically meaningful pairs under development.
3.1 Dataset and preprocessing
- MIMIC-CXR reports supplied paired Findings and Impression text; the official split yielded 122,014/957/1,606 train/validation/test reports, while OpenI was prepared as an independent external dataset.Reports without either section, with fewer than 10 Findings words, or with fewer than 2 Impression words were excluded.Source: §3.1 · pp. 4–5
3.2 Experimental setting
- Training used Alpaca-7B with LoRA on four Nvidia A100 80GB GPUs; the small, portable LoRA weights were chosen to ease sharing and deployment.The LoRA rank was 8 and its scaling factor was 16.Source: §3.2 · p. 5
3.3 Instruction tuning
- Instruction tuning trains the model to generate a report’s Impression from its Findings, learning radiology-specific language and knowledge from these paired examples.The training prompt asks the model to derive the impression from findings in the radiology report.Source: §3.3 · pp. 5–6
- The current task is limited to Findings-to-Impression pairs; the authors are working with radiologists to develop more varied clinically meaningful instruction pairs.Source: §3.3 · p. 6
4 Evaluation
The evaluation compared radiology impression generation using five expert-rated dimensions. Radiology-GPT outperformed the tested general models and showed a distinct trade-off against ChatGPT, with comparable understandability, stronger coherence, and slightly lower relevance.
4 Evaluation
- Evaluation used understandability, coherence, relevance, conciseness, and clinical utility rather than string-matching metrics, given radiologists’ varied writing styles and interpretations.Source: §4 · p. 7
- Two radiologists independently rated outputs for 10 randomly selected reports from MIMIC-CXR and OpenI, and their scores were averaged.Each of the five metrics was scored on a 1-to-5 scale.Source: §4 · p. 7
- Against ChatGPT, Radiology-GPT had comparable understandability and slightly better coherence, but slightly lower relevance associated with its shorter responses.Radiology-GPT scored higher in conciseness and clinical utility.Source: §4 · pp. 7, 9, Figure 4 · p. 8
- Radiology-GPT and ChatGPT outperformed StableLM-7B, Dolly-12B, and LLaMA-7B; Dolly’s larger model size did not close the performance gap.LLaMA-7B, which had not been instruction-tuned, struggled most with the instructions and radiology-specific knowledge.Source: Figure 5 · p. 8, §4 · p. 9
5 Discussion
The discussion presents Radiology-GPT as a potential aid for clinical decisions and communication, while outlining future specialty-panel and instruction-data directions. It emphasizes privacy benefits alongside reliability, coordination, and oversight concerns.
5.1 Clinical Decision Support
- Radiology-GPT may advise imaging choices and report preparation, but should augment rather than replace professional judgment; future work may integrate computer-vision models.The authors also call for stronger handling of complex medical terminology to improve relevance and reliability in clinical decision-making.Source: §5.1 · p. 10
5.2 Augmentation of Patient Communication
- Patient communication could benefit from clearer reports, but accurate and trustworthy patient-facing outputs remain restricted without stringent supervision.Future work should reduce inconsistencies, improve domain grounding, and validate communications before delivery.Source: §5.2 · p. 10
5.3 AGI Expert Panel
- A proposed virtual expert panel would combine localized models from different specialties, but coordination, conflicting recommendations, and human oversight remain challenges.The panel is envisioned as support for comprehensive decision-making in complex cases.Source: §5.3 · p. 11
5.4 Privacy and ethics
- Hospital-server deployment can mitigate PHI-leak risks and support HIPAA compliance, while inaccurate advice, training-data bias, and misinformation remain ethical concerns requiring oversight.The discussion calls for auditing, fairness, accountability, and transparency in model development and deployment.Source: §5.4 · pp. 11–12
5.5 Expanding Instruction Tuning Data in Radiology
- Current tuning centers on Findings–Impression pairs; radiologist-guided expansion to patient explanations, differential diagnoses, research summaries, and protocol choices could broaden workflow coverage.The authors propose engaging practicing radiologists to identify real-world tasks aligned with clinical needs.Source: §5.5 · p. 12
6 Conclusion
Radiology-GPT is presented as a locally deployable, radiology-specific model with superior performance over evaluated baselines. The conclusion describes its potential as a foundation for multimodal and specialty-specific healthcare AI applications.
6 Conclusion
- Radiology-GPT addresses the need for a locally deployable AI solution tailored to radiology.The conclusion characterizes it as combining large-language-model capabilities with radiological context.Source: §6 · p. 12
- Radiology-GPT displayed superior performance over existing baselines in the authors’ evaluations, which used proposed qualitative and quantitative metrics.The authors describe those metrics as a framework for assessing healthcare models.Source: §6 · p. 13
- The model provides a foundation for incorporating radiological images and extending localized, privacy-respecting language models to other medical specialties.The conclusion presents these as avenues for future application.Source: §6 · p. 13