Source-linked AI summary
Highly accurate model for prediction of lung nodule malignancy with CT scans
Jason Causey, Junyu Zhang, Shiqian Ma, Bo Jiang, Jake Qualls, David G. Politte, Fred Prior, Shuzhong Zhang, Xiuzhen Huang
TL;DR
Computational approaches for predicting lung nodule malignancy from CT scans have struggled to match experienced radiologists. NoduleX combines deep-learning CNN and radiomic features, achieving approximately 0.99 AUC on independent validation.
Problem
Computational approaches for predicting lung nodule malignancy from CT scans have struggled to achieve performance comparable to experienced radiologists.
Method
NoduleX systematically compares CNN, radiomic quantitative image feature, and combined models using separate training and validation datasets.
Results
~0.99 AUC was achieved on the independent validation test for lung nodule malignancy prediction.
Takeaways & Limitations
NoduleX provides an effective framework for highly accurate lung nodule malignancy prediction from CT scans.
Takeaways & Limitations
The model may not directly apply to datasets with substantially different CT image quality or ground-truth definitions.
Abstract
from arXiv · showhide
Computed tomography (CT) examinations are commonly used to predict lung nodule malignancy in patients, which are shown to improve noninvasive early diagnosis of lung cancer. It remains challenging for computational approaches to achieve performance comparable to experienced radiologists. Here we present NoduleX, a systematic approach to predict lung nodule malignancy from CT data, based on deep learning convolutional neural networks (CNN). For training and validation, we analyze >1000 lung nodules in images from the LIDC/IDRI cohort. All nodules were identified and classified by four experienced thoracic radiologists who participated in the LIDC project. NoduleX achieves high accuracy for nodule malignancy classification, with an AUC of ~0.99. This is commensurate with the analysis of the dataset by experienced radiologists. Our approach, NoduleX, provides an effective framework for highly accurate nodule malignancy prediction with the model trained on a large patient population. Our results are replicable with software available at http://bioinformatics.astate.edu/NoduleX.
RESULTS
NoduleX uses deep-learning CNN features and radiomic quantitative image features to classify lung nodule malignancy from clinical CT scans. On LIDC/IDRI data, it achieved high validation performance, reaching an AUC of 0.993 for S1 versus S45 classification.
- NoduleX: an approach for nodule malignancy classification: NoduleX combines deep-learning convolutional neural network features with radiomic quantitative image features to classify lung nodule malignancy from clinical CT scans.The framework is illustrated in Figure 1, while radiomic features were designed to capture visual cues identified by radiologists and used with radiologist segmentations.
- NoduleX has consistent performance with high accuracy: 0.993 AUC, 95.2% accuracy, 94.2% sensitivity, and 96.2% specificity were achieved by the S1-versus-S45 model combining CNN predictions with radiomic quantitative image features.The CNN-only model achieved AUC 0.974, accuracy 91.3%, sensitivity 88.5%, and specificity 94.2%; the S12-versus-S45 model achieved AUC 0.938 with 87.9% accuracy, sensitivity, and specificity.
- Radiomics approach for nodule malignancy classification of the LIDC/IDRI cohort: Random Forest radiomics baselines averaged mean AUC 0.91 and mean accuracy 81% for S1 versus S45, and mean AUC 0.86 and mean accuracy 75% for S12 versus S45.Each baseline used one positive and one negative training nodule, with the remaining nodules tested across 200 repetitions.
DISCUSSION
NoduleX achieves ~0.99 AUC for lung nodule malignancy prediction on an independent LIDC/IDRI validation test, commensurate with experienced radiologists’ reviews. The discussion emphasizes evaluation pitfalls, dataset limitations, and the framework’s potential for retraining on larger CT datasets.
- Evaluation considerations: NoduleX evaluation must use independent validation, patient-level separation, size-independent testing, and complete information when reporting classification results.The discussion identifies these as common pitfalls that can bias or weaken lung nodule malignancy model evaluation.
- Limitations: Direct absolute comparison with prior models is difficult because studies often use proprietary datasets or different subsets of public data.Shewaye and Mekonnen [2016] is cited for the resulting difficulty of cross-validating computational approaches on completely different datasets.
- Limitations: The LIDC/IDRI cohort is too small for training very sophisticated CNN models, and training and validation rely on the same cohort.These constraints introduce challenges in interpreting NoduleX’s performance and generalizing beyond the evaluated data.
- Interpretation: Radiomics features can distinguish nodules assigned to very different classes by experts, suggesting that malignancy prediction with radiologist ratings as ground truth is relatively easy.The discussion frames this finding as an important property of the available LIDC/IDRI classification problem.
- Future directions: NoduleX can be retrained on larger CT datasets and updated as new scans and radiologist knowledge become available.The authors anticipate that the framework could retain similarly high accuracy on other datasets.
- Main result: ~0.99 AUC was achieved on an independent validation test using >1000 nodules from 1018 LIDC/IDRI patients classified by four experienced thoracic radiologists.The result is described as commensurate with experienced radiologists’ reviews and as a substantial step toward matched computational and clinical performance.
METHODS
NoduleX predicts lung nodule malignancy from clinical CT scans by combining deep CNN representations with radiological quantitative image features. The method uses consensus radiologist annotations from the LIDC/IDRI cohort, centroid-centered 3-D CNN inputs, and alternative CNN, QIF, and combined classifiers.
- Nodule Selection: Consensus radiologist annotations determine nodule selection, malignancy ratings, segmentations, and centroid locations used to extract nonoverlapping study nodules and CNN inputs.The consensus malignancy rating is the rounded average of annotating radiologists’ ratings, and the CNN region is centered on the average segmentation centroid.
- CNN Input Volume Extraction: CNN inputs are small centroid-centered 3-D CT volumes whose dimensions vary by architecture, including 47×47×5 and 21×21×5 regions.The study also tested 21×21×3 and 31×31×3 input sizes.
- NoduleX Classification: NoduleX evaluates CNN21 and CNN47 models with and without 50 QIF features, concatenating QIF and CNN vectors for combined classification.Classification uses either the CNN softmax output or a classifier applied to the concatenated feature vector.
- Description of the datasets: The experiments use LIDC/IDRI CT scans reviewed by four experienced thoracic radiologists and evaluate malignancy-label datasets including 520-nodule S1-versus-S45 and 664-nodule S12-versus-S45 configurations.The cohort contains scans from 1018 patients/cases, while the reported class configurations define negative and positive classes from radiologist consensus ratings.
- Description of the models of NoduleX: The method trains and validates separate QIF, CNN, and combined models for predicting malignant lung nodules from CT scans.CNN models use deep convolutional features, whereas QIF models use radiological quantitative image features.