PaperlayerView exact revision on arXiv

Source-linked AI summary

Radiological images and machine learning: trends, perspectives, and prospects

Zhenwei Zhang, Ervin Sejdic

arXiv:1903.11726v1eess.IVcs.LG

TL;DR

Radiological image analysis needs methods that can handle complex information across multiple imaging modalities and clinical applications. This review surveys machine-learning methods and their radiology uses, finding that systems can achieve accuracy comparable to radiologists while remaining limited by dataset size, feature selection, and interpretability. The authors discuss future development toward more accurate and clinically useful tools.

  • Problem

    Radiological imaging produces complex data across modalities and applications, motivating machine-learning methods for image analysis and clinical decision support.

  • Method

    The paper reviews machine-learning concepts, recent techniques, radiological imaging applications, current challenges, and future research directions.

  • Results

    Machine-learning systems have shown accuracy comparable to radiologists across reported applications, while the field remains under development.

  • Takeaways & Limitations

    Machine-learning techniques are expected to become essential components of clinical tools for assessing patients’ health.

  • Takeaways & Limitations

    Clinical application is constrained by limited datasets, difficult feature selection, and limited interpretability of deep neural networks.

Abstract

from arXiv · show

The application of machine learning to radiological images is an increasingly active research area that is expected to grow in the next five to ten years. Recent advances in machine learning have the potential to recognize and classify complex patterns from different radiological imaging modalities such as x-rays, computed tomography, magnetic resonance imaging and positron emission tomography imaging. In many applications, machine learning based systems have shown comparable performance to human decision-making. The applications of machine learning are the key ingredients of future clinical decision making and monitoring systems. This review covers the fundamental concepts behind various machine learning techniques and their applications in several radiological imaging areas, such as medical image segmentation, brain function studies and neurological disease diagnosis, as well as computer-aided systems, image registration, and content-based image retrieval systems. Synchronistically, we will briefly discuss current challenges and future directions regarding the application of machine learning in radiological imaging. By giving insight on how take advantage of machine learning powered applications, we expect that clinicians can prevent and diagnose diseases more accurately and efficiently.

1 Introduction

This review introduces radiological imaging modalities and surveys recent machine-learning research for analyzing radiological images. It focuses on contributions published mainly from mid-2014 to mid-2017 and outlines current limitations and future research directions.

  • Imaging technologies: Radiology uses X-ray, CT, MRI, PET, and ultrasound imaging to detect, diagnose, and treat disease.These modalities differ in the information they capture and their practical limitations, including radiation exposure and poor ultrasound penetration through bone or air.
  • Review scope: The paper reviews machine-learning techniques used in radiological image analysis and applications across multiple radiology problems.It covers general machine-learning concepts, recent methods, applications, technological limitations, and future research areas.
  • Review scope: The review emphasizes recent contributions published between the middle of 2014 and the middle of 2017.The literature search used databases including Google Scholar, PubMed, IEEE Xplore, and SpringerLink, with keywords spanning methods, applications, and imaging modalities.
  • Imaging technologies: MRI provides better soft-tissue resolution, whereas ultrasound is more convenient but cannot capture information well through bone.

2 Machine Learning in Radiology

Machine learning in radiology includes supervised, unsupervised, and semi-supervised learning methods applied to image analysis, classification, segmentation, retrieval, and clinical decision support. Common models include linear methods, SVMs, decision trees, ensembles, neural networks, and deep learning.

  • Applications: Machine-learning algorithms extract information from medical images and support radiologists in interpreting them.Applications include segmentation, computer-aided systems, brain studies, neurological diagnosis, image retrieval, and registration.
  • Types of learning: Supervised learning maps labeled inputs to outputs, unsupervised learning identifies structure without labels, and semi-supervised learning expands a small labeled set with unlabeled data.
  • Imaging modalities: CT and PET provide complementary disease information: CT offers better resolution, while PET may reveal intense FDG uptake absent from CT abnormalities.
  • Feature representation: Feature extraction is central because high-dimensional radiological images provide more information but complicate optimization and classification.Reliable feature selection is important for improving classification performance.
  • Machine-learning methods: Linear models, SVMs, decision trees, ensemble classifiers, neural networks, and deep learning are among the commonly used methods in radiology.SVMs use kernels for non-linearly separable data, while decision trees split feature space and are often ensembled as random forests.
  • Machine-learning methods: Deep learning can select features directly from data for classification and detection, avoiding manually designed features.

2.4 Evaluating Machine Learning Techniques

The review describes cross-validation and classification metrics used to evaluate machine-learning systems, covering overlap, discrimination, and sensitivity-specificity performance.

  • Cross-validation repeatedly partitions data into k subsets so each instance serves in training and testing, reducing variance in accuracy estimates.
  • The Dice similarity coefficient measures spatial overlap between two segmented regions, ranging from 0 for no overlap to 1 for complete agreement.
  • Sensitivity measures correct disease detection, whereas specificity measures correct identification of healthy subjects.
  • ROC curves describe the relationship between sensitivity and specificity across classification thresholds.
  • AUC summarizes the entire ROC curve and represents the probability that a model distinguishes between classes.

3.1 Segmentation

Segmentation is central to radiological diagnosis but is time-consuming and dependent on clinician experience. The reviewed studies apply tree-based, neural, regression, dictionary-learning, and multimodal methods, while image quality remains a constraint.

  • Image segmentation supports diagnosis and treatment by exposing structural and spatial anatomical information, but manual analysis is time-consuming and experience-dependent.
  • Tree-based methods segment brain lesions and other structures using learned features from MRI and spatially informed voxel representations.
  • Neural networks classify brain MRI pixels into lesioned and healthy tissue, while lower tissue contrast makes early-brain segmentation more difficult.
  • Multimodal MRI segmentation combines T1, T1-IR, and T2-FLAIR information with ground-truth labels to produce segmentation results.
  • Regression, dictionary learning, and sparse coding have been used for intervertebral-disc and multi-organ segmentation.
  • Low contrast, limited resolution, and partial-volume effects make radiological boundaries difficult to define, while acquiring multiple modalities can be inconvenient.

3.2 Computer Aided Diagnosis

Machine-learning CAD systems support radiologists by detecting and assessing potential pathologies across several imaging tasks. Reported benefits include accurate, consistent, and rapid analysis, but current systems remain limited in clinical scope.

  • CAD systems detect, mark, and assess potential pathologies to support radiologists facing data overload and limited human resources.
  • CAD applications use linear discriminant, Bayesian, neural-network, and SVM classifiers for detecting lesions, masses, fractures, degenerative disease, and cancer.
  • Feature choice substantially affects model accuracy in diagnosis tasks, including cervical-cancer prediction and lung-lesion detection from MRI and CT.
  • Deep belief and convolutional neural networks have produced promising results for chest pathology, cancer, and lung-disease diagnosis, including with unlabeled data.
  • CAD systems have reported over 99% accuracy for oral-cancer detection and reviewed breast-cancer charts 30 times faster than humans.
  • Current systems often target one disease and one measurement, while datasets mainly cover common diseases rather than rare cases.

3.3 Functional Brain Studies and Neurological Diseases

Machine learning is used to extract brain features, study functional connectivity, and classify neurological disease. The reviewed approaches include SVM-based feature selection and hierarchical ensemble classification, with performance varying across diagnostic tasks.

  • Brain studies use cortical thickness, brain-structure volume, and voxel tissue-probability maps as features related to pathological changes.
  • SVM-based feature selection reduces imaging features for brain-metastasis and radiation-necrosis classification from contrast-enhanced T1-weighted images.
  • 50 preterm-born and 50 term-born infants were compared with SVM, revealing stronger inter- and intra-hemispheric functional connections in full-term infants.
  • Hierarchical Alzheimer’s classification integrates patch-level low-level classifiers, coarse-scale high-level classifiers, and final ensemble voting across imaging features.
  • Mild cognitive impairment converted to Alzheimer’s disease was classified with 57.4% accuracy, only slightly above majority classification.
  • Multimodal and surface-based approaches combine imaging features with regression, bagging, or discriminant classification for neurological disease studies.

3.4 Image Retrieval

Image retrieval applies machine learning to search medical-image archives by visual content rather than keywords, using supervised, semi-supervised, and unsupervised approaches. The reviewed work emphasizes feature extraction, similarity measurement, and the need for accurate, efficient systems that scale beyond small datasets and limited disease categories.

  • Content-based retrieval: Content-based retrieval searches medical images by rich visual content and similarity rather than keywords or image tags.It is intended to support diagnostic, educational, and research use of large medical-image databases.
  • Machine-learning approaches: Recent retrieval studies use online dictionary learning, ensemble learning, principal component analysis, and convolutional neural-network fusion strategies.One X-ray classification approach combined domain-transferred convolutional neural networks with sparse spatial-pyramid classification and outperformed either single method.
  • Retrieval performance: Local wavelet pattern features with similarity measurement retrieved top-10 images from the same category with 100% precision in the example.The figure presents a query image followed by the top 10 retrieved images.
  • Machine-learning approaches: Unsupervised K-SVD retrieval alternates clustering similar images with dictionary generation until the clusters converge, without requiring classification training data.This approach is not restricted to a specific context.
  • Limitations and requirements: Medical image retrieval requires both accuracy and performance time, while many contributions remain limited to small datasets and few disease cases.Practical models and relevant image-feature extraction are needed for better results as hospital image databases expand.

3.5 Image Prediction

Image prediction estimates radiological images in another modality or at higher quality to provide more detailed functional information for diagnosis. The reviewed methods predict standard-dose PET from low-dose PET with MRI or T1 information, while multimodal inputs improve estimated-image results, although this research remains limited and concentrated on brain data.

  • Image prediction: Image prediction estimates radiological images in different modalities or at higher resolution to provide detailed functional information for assessment and diagnosis.PET image quality depends on injected dose and imaging time, making low-dose PET prediction a focus of research.
  • Prediction methods: Deep auto-context convolutional neural networks estimate standard-dose PET from low-dose PET, with better results when low-dose PET and T1 images are both included.The comparison evaluates low-dose PET alone against the combination of low-dose PET and T1 images.
  • Prediction methods: Regression forests and mapping-based sparse representation methods predict standard-dose PET from low-dose PET combined with multimodal MRI information.The methods use local-intensity patches, graph-based distribution mapping, and patch-selection dictionary learning.
  • Limitations and challenges: Compared with classification and segmentation, radiological image-prediction research remains limited, with current contributions focused mainly on brain data.Hybrid scanners create integrated images, but matching correspondences among modalities remains a primary challenge.

4 Current Challenges

The review identifies clinical translation challenges involving dataset scale and sharing, feature quality, class imbalance, model updating, regulatory deployment, and interpretability. These constraints leave open questions about robustness, independent clinical performance, and safe use of machine-learning systems in radiology.

  • Data and evaluation: Radiology studies often use small datasets because patient access and diagnostic follow-up are limited, weakening confidence in models trained on only a few cases.Some cited studies included only around twenty patients.
  • Data and evaluation: Hundreds or more image sets will be needed to improve robustness and accuracy, supported by large databases and sharing centers such as ADNI, NIH repositories, and TCIA.The review also calls for systematic evaluation across diseases and public datasets.
  • Features and imbalance: Data quality and feature selection remain critical because low-level features may miss disease appearance, while redundant high-dimensional features can cause overfitting, underfitting, or misclassification.Selecting informative features is presented as necessary for robust models.
  • Model development: Transfer learning addresses limited datasets through directions targeting equipment and protocol bias, abnormalities within one source, and feature representations across domains.These directions support applying learned information across radiological-imaging settings.
  • Features and imbalance: Class imbalance can bias prediction toward normal data because abnormal cases are a minority, and most studies have not properly addressed this issue.How to use imbalanced data to improve algorithm accuracy remains open.
  • Future directions: Online learning could update diagnostic models from streaming hospital images, making greater use of radiological data that currently remains unused for training.The proposal responds to the large annual production of hospital radiological images and resource constraints in training.
  • Clinical deployment: Few radiological machine-learning models are commercialized because deployment must comply with government requirements and clinical-trial protocols.The review notes that fixed-form requirements may conflict with models that require updating.
  • Interpretability: Deep neural networks remain difficult to interpret because their many parameters obscure how diagnostic decisions are made between input and output.This opacity can prevent medical experts from verifying model conclusions and may be dangerous when conclusions are wrong.

5 Conclusion and Future Work

Machine learning is being applied across five radiological-image applications and has produced results comparable to radiologists, but broader clinical use remains constrained by data, accuracy, validation, and trust requirements. Future progress depends on better models, representative data, and evidence that improves patient outcomes.

  • The review covers image segmentation, computer-aided detection and diagnosis, functional brain and neurological disease studies, image classification and retrieval, and image registration.
  • Machine learning-based systems have shown results comparable to those of radiologists themselves.
  • System accuracy still needs improvement beyond radiologist performance, or widespread application may remain limited.
  • Deep-learning accuracy and performance are highly limited by data size, while traditional methods remain relevant when labeled data are scarce.
  • Translation into clinical practice remains constrained by required clinical trials and difficulty justifying algorithmic decisions across all cases.
  • The field is developing algorithms for higher accuracy and lower computational complexity, with the expectation that machine learning will become essential in future clinical tools.
Loading 1903.11726v1…