Source-linked AI summary
Interpreting Chest X-rays via CNNs that Exploit Hierarchical Disease Dependencies and Uncertainty Labels
Hieu H. Pham, Tung T. Le, Dat T. Ngo, Dat Q. Tran, Ha Q. Nguyen
TL;DR
Detecting multiple thoracic diseases from chest X-rays remains challenging. The paper uses CNNs with hierarchical dependency-aware conditional training and label smoothing for uncertain samples, achieving 0.940 mean AUC on validation and specialist-level performance on an independent test.
Problem
Accurately detecting multiple thoracic diseases from chest X-rays is challenging, motivating multi-label classification for 14 common observations.
Method
The method trains CNN classifiers with disease-hierarchy dependencies, conditional training, and label smoothing for uncertain labels on CheXpert data.
Results
The six-model ensemble achieved 0.940 mean AUC on validation and 0.930 mean AUC on the hidden test, outperforming 2.6 out of 3 radiologists.
Takeaways & Limitations
The proposed method outperformed previous state-of-the-art results on CheXpert and performed on par with specialists in an independent test.
Abstract
from arXiv · showhide
The chest X-rays (CXRs) is one of the views most commonly ordered by radiologists (NHS),which is critical for diagnosis of many different thoracic diseases. Accurately detecting thepresence of multiple diseases from CXRs is still a challenging task. We present a multi-labelclassification framework based on deep convolutional neural networks (CNNs) for diagnos-ing the presence of 14 common thoracic diseases and observations. Specifically, we trained astrong set of CNNs that exploit dependencies among abnormality labels and used the labelsmoothing regularization (LSR) for a better handling of uncertain samples. Our deep net-works were trained on over 200,000 CXRs of the recently released CheXpert dataset (Irvinandal., 2019) and the final model, which was an ensemble of the best performing networks,achieved a mean area under the curve (AUC) of 0.940 in predicting 5 selected pathologiesfrom the validation set. To the best of our knowledge, this is the highest AUC score yetreported to date. More importantly, the proposed method was also evaluated on an inde-pendent test set of the CheXpert competition, containing 500 CXR studies annotated by apanel of 5 experienced radiologists. The reported performance was on average better than2.6 out of 3 other individual radiologists with a mean AUC of 0.930, which had led to thecurrent state-of-the-art performance on the CheXpert test set.
2. Proposed approach
The proposed approach incorporates hierarchical disease dependencies and uncertainty labels into CNN-based multi-label classification of chest X-rays. It uses conditional training, label smoothing, and advanced CNN architectures on CheXpert data.
- Dataset and settings: CheXpert contains 224,316 scans from 65,240 patients, covering 14 observations labeled positive, negative, or uncertain.The data are split into 223,414 training studies, 200 validation studies, and 500 hidden test studies.
- Exploiting disease dependencies: Conditional training first uses samples with positive parent labels to predict child labels, then finetunes CNNs on the whole dataset.This procedure adapts conditional learning to the CheXpert lung disease hierarchy.
- Dealing with uncertainty labels: Uncertain labels are handled with label smoothing regularization and, in the U-zeros+LSR approach, replaced by random values near zero.The random value u is sampled uniformly between approach-specific hyper-parameters.
- Deep learning model and training procedure: The study evaluates DenseNet, Inception-ResNetv2, Xception, and NASNetLarge CNNs with a 14-dimensional sigmoid output for pathology probabilities.Lung localization removes irrelevant image regions before standard-size images are fed into the networks.
3. Experiments and results
Ablation studies found benefits from both conditional training and label smoothing, while an ensemble achieved strong validation and hidden-test performance against prior methods and radiologists.
- Ablation studies: 0.894 AUC was achieved by the baseline with U-Ones+CT+LSR on the validation set, versus 0.860 mean AUC with U-Ones.The reported improvement was 4%.
- Validation results: 0.940 mean AUC was achieved by the six-model ensemble on the validation set, outperforming previous state-of-the-art results.The ensemble was evaluated on five selected diseases.
- Hidden-test results: 0.930 mean AUC was achieved on the hidden test set, where the model outperformed 2.6 out of 3 radiologists when averaged over five diseases.The comparison used radiologists’ operating points relative to the model’s ROC curves.
4. Conclusion
The paper presents a CNN-based computer-aided diagnosis approach that integrates disease dependencies and uncertainty labels for thoracic disease classification. Experiments report performance above prior state of the art and on par with specialists on an independent test.
- Conclusion: The approach targets high-precision classification of common thoracic diseases from chest X-rays using deep learning.It is framed as a computer-aided diagnosis system.
- Conclusion: The training procedure integrates dependencies among diseases and uncertainty labels into advanced CNN training.This combines the paper’s hierarchical-label and uncertain-sample strategies.
- Conclusion: The proposed method outperforms previous state-of-the-art methods on the CheXpert dataset and performs on par with specialists in an independent test.The conclusion characterizes these findings as the outcome of extensive experiments.