Source-linked AI summary
Deep learning in radiology: an overview of the concepts and a survey of the state of the art
Maciej A. Mazurowski, Mateusz Buda, Ashirbani Saha, Mustafa R. Bashir
TL;DR
Radiology’s digital, image-centered, and structured work creates opportunities for deep learning, but limited medical imaging data and implementation challenges remain. The paper reviews radiology practice, explains deep learning concepts, surveys task-organized research and methods, and concludes that deep learning is likely to play a significant role in radiology while responsibility and broader integration questions remain unresolved.
Problem
Radiology has substantial opportunities for deep learning, but medical imaging datasets are limited and incorporating these algorithms into clinical practice raises unresolved challenges.
Method
The paper reviews radiology practice, introduces deep learning concepts, and surveys radiology studies organized by task and algorithmic approach.
Results
The review covers classification, detection, transfer learning, off-the-shelf features, and other deep learning applications across radiology tasks.
Takeaways & Limitations
Deep learning is expected to play a significant role in radiology, with future progress depending on collaboration between machine learning scientists and radiologists.
Takeaways & Limitations
Deep learning results in radiology are not yet sufficient to replace a radiologist across the entire scope of diagnostic work, and responsibility for computer mistakes remains an ethical and legal challenge.
Abstract
from arXiv · showhide
Deep learning is a branch of artificial intelligence where networks of simple interconnected units are used to extract patterns from data in order to solve complex problems. Deep learning algorithms have shown groundbreaking performance in a variety of sophisticated tasks, especially those related to images. They have often matched or exceeded human performance. Since the medical field of radiology mostly relies on extracting useful information from images, it is a very natural application area for deep learning, and research in this area has rapidly grown in recent years. In this article, we review the clinical reality of radiology and discuss the opportunities for application of deep learning algorithms. We also introduce basic concepts of deep learning including convolutional neural networks. Then, we present a survey of the research in deep learning applied to radiology. We organize the studies by the types of specific tasks that they attempt to solve and review the broad range of utilized deep learning algorithms. Finally, we briefly discuss opportunities and challenges for incorporating deep learning in the radiology practice of the future.
1 Introduction
Deep learning has advanced rapidly in image-related tasks, supported by data, computing power, and algorithmic development. Radiology is a natural application area, and this paper reviews its clinical opportunities, concepts, research, and future challenges.
- Deep learning uses interconnected units that learn patterns from data to perform complex tasks, including image localization and classification.Unlike pre-programmed instructions, these methods learn from large amounts of data.
- 0.153 error in 2012 versus 0.258 in 2011 marked a dramatic ImageNet improvement for deep learning-based image classification.ImageNet contains more than 14,000,000 annotated natural images across 1,000 predefined categories.
- Data availability, increased processing power, and rapid algorithmic development are identified as interconnected reasons for deep learning’s recent success.Affordable graphical processing units and easier algorithm use broadened participation in developing algorithms and datasets.
- Radiology became a natural application area because its primary strength is image analysis, although medical datasets and specialized expertise were initially limited.The article states that radiology research productivity increased remarkably during the preceding 2–3 years.
- The paper surveys radiology opportunities, introduces deep learning concepts, organizes prior research by task, and discusses future implementation challenges.Its structure moves from radiology practice and deep learning fundamentals to recent work and future remarks.
2 The practice of radiology
Radiology centers on interpreting digital medical images for diagnosis and intervention, making its structured workflow attractive for artificial intelligence. The section identifies detection, diagnosis, workflow support, and emerging interpretation tasks as opportunities for deep learning.
- Radiology uses medical images to detect, diagnose, and characterize disease and to guide procedural interventions.
- The radiologist’s primary diagnostic duty is to interpret images and produce reports, while limited interaction and structured workflow make the work attractive for artificial intelligence.
- Deep learning could analyze radiology’s predominantly digital primary data and support the practice’s diagnostic and operational tasks.
- Disease Detection: Rapidly distinguishing abnormalities from normal anatomy is difficult because images contain many findings, most of which are ultimately normal.In screening mammography, approximately 80% of examinations should be negative under the ACR BI-RADS guideline.
- Diagnosis and Management: Diagnosis and management require integrating features such as size, location, signal, borders, heterogeneity, and temporal change under sometimes complex guidelines.BI-RADS classification combines mass shape, margin, and density to guide follow-up or biopsy decisions.
- Deep learning could assess many features rapidly and reproducibly, correlate imaging data with outcomes, and reduce laborious human interpretation.
- Workflow Support: Growing image volume and complexity make workflow support important, including linking current and prior examinations across disease sites, measurements, and treatment histories.PACS systems may fail to present these data reliably because labeling, positioning, anatomy, and modality vary between examinations.
- Emerging Applications: Radiology algorithms could also address interpretation tasks not routinely performed by radiologists, such as finding relationships between tumor imaging features and genomic characteristics.
3 An introduction to deep learning
Deep learning uses layered networks of interconnected units to learn increasingly high-level representations from data. Convolutional neural networks extract image features through convolutions and pooling, while training strategies address limited medical-imaging data.
- Terminology: Deep learning is a machine-learning sub-discipline whose layered units generate increasingly high-level representations of inputs such as images.
- Convolutional Neural Networks: CNNs typically contain 10–30 layers, shared weights that perform convolutions, pooling for small-shift invariance, and specialized activation functions.
- Convolutional Neural Networks: Convolutional layers generate features ranging from edges to complex shapes, while fully connected layers use them to make classification decisions.
- The learning process: Training changes network weights using batches of examples and error backpropagation or stochastic gradient descent to reduce prediction error.
- The learning process: Training from scratch can overfit medical-imaging data because networks may contain more than 10 million parameters and datasets are limited.
- The learning process: Transfer learning fine-tunes a network pretrained on another dataset, whereas off-the-shelf features feed intermediate outputs into traditional classifiers.
- Deep learning vs traditional machine learning: Traditional machine learning requires manually designed features, whereas deep learning jointly learns feature extraction and decision making but requires larger datasets.
4 Deep learning in radiology: state of the art
Deep learning research in radiology spans classification, segmentation, and detection, with methods adapted to limited medical-imaging data and task-specific constraints. CNNs, transfer learning, fully convolutional networks, three-dimensional models, and recurrent networks are surveyed across applications.
- The survey organizes radiology applications by task and reviews methods and their evolution over recent years.
- Classification: Radiological classification includes abnormality detection, benign–malignant classification, prognostication, genomic characterization, and data organization.
- Classification: CNN classifiers commonly use convolutional layers followed by fully connected layers, while transfer learning and off-the-shelf features address limited radiological data.
- Classification: Training from scratch is used when sufficient data are available, with medical-imaging architectures often adapting AlexNet or VGG using fewer layers and weights.
- Segmentation: Segmentation divides images into regions for organs, substructures, or lesions, often as preprocessing for feature extraction and classification.
- Segmentation: Patch-based pixel classification is computationally inefficient and may miss global context because it repeatedly processes overlapping local windows.
- Segmentation: Fully convolutional networks process whole images or large portions and output segmentation maps; common architectures include U-Net and Fully Convolutional DenseNet.
- Segmentation: Three-dimensional data are often processed as two-dimensional slices because 3D networks require substantially more parameters and data, although successful 3D applications exist.
5 Future of deep learning in radiology
Deep learning is expected to shape radiology, but broad replacement of radiologists remains unsupported and implementation raises legal, ethical, patient-acceptance, regulatory, and workflow challenges.
- Predictions range from automating mundane tasks to collaboration with radiologists or replacing their image-interpretation role.
- Current radiology results comparable to expert humans apply only to a very small minority of radiologists’ tasks.
- Legal and ethical implementation questions include responsibility for mistakes made by computer systems.
- Patients may accept or reject image interpretation without radiologist involvement, while regulatory issues also remain.
- A practical challenge is incorporating deep learning into workflow so that it improves rather than disrupts radiology practice.
6 Conclusion
The paper reviews deep learning principles, radiology practice, current research, workflow incorporation, and implementation challenges. It concludes that deep learning will significantly affect radiology, with continued collaboration shaping the field’s next five years.
- The paper discusses deep learning principles and current radiology practice to clarify incorporation of these algorithms into radiology workflow.
- It reviews progress and the state of the art, then addresses implementation challenges and questions in medicine.
- The authors conclude that deep learning will play a significant role in radiology.
- Collaboration between machine learning scientists and radiologists may address questions raised in the article over the next five years.