Source-linked AI summary
Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18F-FDG PET/CT images
Hongkai Wang, Zongwei Zhou, Yingci Li, Zhonghua Chen, Peiou Lu, Wenzhi Wang, Wanyu Liu, Lijuan Yu
TL;DR
The study examines how machine-learning methods classify mediastinal lymph-node metastasis of NSCLC from PET/CT images, addressing limitations of conventional feature thresholding and debated texture features. It compares classical feature-based models, CNN, and human doctors, finding that CNN performs similarly to the best classical methods and doctors while avoiding tumor segmentation and feature calculation. Diagnostic features remain more discriminative than texture features for small lymph nodes, motivating their incorporation into CNNs.
Problem
Prior PET/CT assessment relied on limited thresholded features, while texture-feature effectiveness remained debated and reported median sensitivity was 62%.
Method
The study compared four classical models using diagnostic and texture features with a CNN using lymph-node-centered PET/CT image patches, and evaluated them against human doctors.
Results
CNN performance was not significantly different from the best classical methods or human doctors for mediastinal lymph-node metastasis classification.
Takeaways & Limitations
CNN avoids tumor segmentation and hand-crafted feature calculation, making the process more convenient and less prone to user bias while avoiding debated texture features.
Takeaways & Limitations
Texture features performed worse because small lymph nodes provided too few voxels for meaningful heterogeneity measurement, while CNN omitted important diagnostic features such as SUV and tumor size.
Abstract
from arXiv · showhide
The present study shows that the performance of CNN is not significantly different from the best classical methods and human doctors for classifying mediastinal lymph node metastasis of NSCLC from PET/CT images. Because CNN does not need tumor segmentation or feature calculation, it is more convenient and more objective than the classical methods. However, CNN does not make use of the import diagnostic features, which have been proved more discriminative than the texture features for classifying small-sized lymph nodes. Therefore, incorporating the diagnostic features into CNN is a promising direction for future research.
Background
The study addresses limited diagnostic power in PET/CT-based mediastinal lymph-node metastasis assessment by comparing classical feature-based methods with deep learning and clinical judgment. It focuses on whether deep learning can analyze PET/CT images without segmentation or hand-crafted feature calculation.
- 18F-FDG PET/CT studies of mediastinal lymph-node metastasis commonly relied on thresholding limited image features such as size, SUVmax, and SUVmean.
- The median sensitivity reported for this diagnostic task over the preceding decade was 62%.
- PET/CT texture features capture uptake heterogeneity, but their effectiveness remains debated because imaging protocols, lesion size, and processing affect their values.
- Deep learning uses deep neural networks to learn appearance patterns of target objects from large training datasets and has been increasingly applied to medical-image analysis.
- The study compared classical feature-based methods, diagnostic features used by doctors, a state-of-the-art deep-learning approach, and human doctors for this classification task.
Methods
The study used retrospective PET/CT data, clinician-established lymph-node labels, classical machine-learning models based on engineered features, and a CNN based on image patches. Classical inputs comprised diagnostic and texture feature sets, while CNN operated directly on cropped multimodal patches with augmentation and cross-validation.
- PET/CT images from 168 patients were retrospectively retrieved from a hospital database covering June 2009 through September 2014.
- Four institute doctors reached consensus on benign or malignant lymph-node status using size, SUV, CT values, and visual contrast among other diagnostic factors.
- The evaluated models were RF, SVM, AdaBoost, BP-ANN, and CNN, with 10 times 10-fold cross-validation used for machine-learning evaluation.
- CNN classified lymph-node-centered image patches rather than feature vectors, producing benign and malignant scores from learned appearance patterns.
- CNN inputs comprised six axial PET and CT patches, each resampled to 51 × 51 pixels at 1.0-mm size and taken at three axial positions.
- Classical methods used 13 diagnostic features and 82 texture features, with diagnostic, texture, combined, and selected feature sets compared.
Results
Diagnostic features were the strongest feature set for classical methods, while CNN and ensemble methods performed comparably with human doctors overall. BP-ANN consistently underperformed, and SVM offered the fastest training.
- Feature comparison: D13 was the optimal feature set for every classical method, whereas T82 was the worst.
- Feature comparison: For most comparisons, D13, A95, and S6 showed no evident performance differences, but adding texture features markedly reduced BP-ANN performance.For BP-ANN, A95 performed as poorly as T82.
- Method comparison: AdaBoost achieved the highest AUC, CNN achieved the highest ACC, and BP-ANN achieved the lowest AUC and ACC.
- Method comparison: CNN, RF, and AdaBoost had higher ACC than human doctors, but these differences were not significant after Bonferroni and FDR corrections.Human doctors had no AUC value, so comparisons with doctors used ACC only.
- Method comparison: At the doctors’ sensitivity level, CNN, RF, and AdaBoost had better specificity, while SVM had close specificity and BP-ANN’s ROC curve remained below the others.When higher sensitivity or specificity was considered, SVM quickly caught up with CNN, RF, and AdaBoost.
- Method comparison: SVM was the fastest to train at approximately 3 s per fold, compared with approximately 6 s for RF, 40 s for AdaBoost, and 10 min for CNN.All methods required less than 1 s for testing a single cross-validation.
Discussion
Diagnostic features outperformed texture features, while CNN performed comparably to the strongest classical methods and human doctors. The study identifies training-data and architecture constraints that may limit performance.
- Feature comparison: Diagnostic features outperformed texture features, whereas texture-feature performance was compromised by the predominance of small lymph nodes.Small lymph nodes provide too few voxels for meaningful heterogeneity measurement; 80.3% had short-axis diameters below 4 mm.
- Method comparison: AdaBoost, random forest, and SVM outperformed ANN on both AUC and ACC.The authors relate the stronger performance of ensemble tree methods to complementary feature use across decision trees.
- Comparison with human doctors: All five machine learning methods had higher sensitivity but lower specificity than the human doctors.Using ACC as a more balanced criterion, RF, AdaBoost, and CNN were better than doctors, but differences were not significant after Bonferroni and FDR corrections.
- CNN comparison: CNN was not significantly better than RF, AdaBoost, or SVM, despite learning image appearance patterns without using SUV after normalization.The learned appearance patterns include local contrast, nearby tissues, and boundary sharpness, which the authors describe as complementary to diagnostic features.
- CNN limitations: The CNN used a reduced-layer AlexNet architecture because the cross-validation training data remained limited despite 729-fold augmentation.The authors therefore did not use deeper architectures designed for much larger training sets.
- CNN limitations: Mixing PET and CT patches in one network may limit CNN performance because the modalities contain different diagnostic information.The authors suggest separate subnetworks with output-layer fusion as a future architecture.
Conclusions
The study concludes that diagnostic features are more discriminative than texture features for mediastinal lymph-node classification, while CNN offers comparable performance with a more convenient workflow. Future work will incorporate diagnostic features and address dual-modality architecture and multicenter evaluation.
- Conclusions: Diagnostic features were more discriminative than texture features, mainly because texture calculations are unreliable for small lymph nodes.The conclusion links the limitation to small lymph-node size.
- Conclusions: CNN performance was not significantly different from the best classical methods despite excluding important diagnostic features such as SUV and tumor size.CNN avoids hand-crafted features, tumor segmentation, and feature selection, making the process more convenient and less prone to user bias.
- Future directions: Future work will incorporate diagnostic features into CNN and design dedicated network structures for dual-modality PET/CT images.The study also plans multicenter data collection for more generalizable evaluation.
Authors’ contributions
The listed contributors divided responsibilities across study conception, method development, data preparation, CNN and texture-feature work, parameter tuning, diagnosis, and pathology analysis. The authors report no competing interests and describe ethics oversight for the study.
- Authors’ contributions: LY, WL, and HW developed the study concept, while HW programmed cross-validation and diagnostic-feature calculation and wrote the manuscript.
- Authors’ contributions: WL prepared data samples, ZZ conducted CNN research and texture-feature calculations, and ZC fine-tuned the classical ANN parameters.
- Competing interests: The authors report no competing interests.
- Ethics: The study followed institutional and national ethical standards, including the 1964 Helsinki Declaration and later amendments.The text states that formal consent was not required for this type of study.
Submit your manuscript to a journal and benefi t from:
The journal advertises convenient online submission, rigorous peer review, immediate publication upon acceptance, open access, high visibility, and copyright retention.
- Submit your manuscript to a journal and benefi t from:: The journal offers convenient online submission and rigorous peer review.
- Submit your manuscript to a journal and benefi t from:: Accepted articles are described as receiving immediate publication and open-access availability.
- Submit your manuscript to a journal and benefi t from:: The journal promotes high visibility while allowing authors to retain copyright.