Source-linked AI summary
Comparing Different Deep Learning Architectures for Classification of Chest Radiographs
Keno K. Bressem, Lisa Adams, Christoph Erxleben, Bernd Hamm, Stefan Niehues, Janis Vahldiek
TL;DR
Chest-radiograph classifiers commonly reuse deep architectures despite radiographs being grayscale and containing fewer classes than typical image datasets. This study compares neural-network architectures on CheXpert and finds that smaller models can match or surpass deeper networks while requiring fewer computational resources.
Problem
It was unclear whether very deep neural networks are necessary for classifying comparatively simple grayscale chest radiographs.
Method
The study systematically compared fifteen neural-network architectures on the CheXpert chest-radiograph dataset using AUROC, AUPRC, and training-time analyses.
Results
Smaller networks such as ResNet-34, AlexNet, and VGG-16 achieved comparable or superior classification performance to deeper models, with VGG-16 reaching an AUPRC of 0.709.
Takeaways & Limitations
Smaller networks can reduce hardware requirements and training time without loss of performance in chest-radiograph classification.
Takeaways & Limitations
The work compared architectures rather than completely optimizing one specific network.
Abstract
from arXiv · showhide
Chest radiographs are among the most frequently acquired images in radiology and are often the subject of computer vision research. However, most of the models used to classify chest radiographs are derived from openly available deep neural networks, trained on large image-datasets. These datasets routinely differ from chest radiographs in that they are mostly color images and contain several possible image classes, while radiographs are greyscale images and often only contain fewer image classes. Therefore, very deep neural networks, which can represent more complex relationships in image-features, might not be required for the comparatively simpler task of classifying grayscale chest radiographs. We compared fifteen different architectures of artificial neural networks regarding training-time and performance on the openly available CheXpert dataset to identify the most suitable models for deep learning tasks on chest radiographs. We could show, that smaller networks such as ResNet-34, AlexNet or VGG-16 have the potential to classify chest radiographs as precisely as deeper neural networks such as DenseNet-201 or ResNet-151, while being less computationally demanding.
Introduction
Chest radiograph classification remains an active research area despite widespread use of established neural-network architectures. This study tests whether shallower networks can match or exceed deeper models with fewer resources.
- Because radiographs contain fewer relevant pathologies than many other imaging tasks, shallow networks may be sufficient for classification.
- The study systematically examines fifteen openly available neural-network architectures for basic chest-radiograph classification.
- The central hypothesis is that shallow networks might outperform deeper networks while requiring fewer resources.
Methods
The study compares fourteen convolutional neural networks using selected CheXpert labels and standardized training and evaluation procedures. Images were restricted to frontal radiographs and uncertainty-labeled cases were excluded.
- Training used five CheXpert findings: cardiomegaly, edema, consolidation, atelectasis, and pleural effusion.
- Only frontal radiographs without uncertainty labels were used, leaving 135,494 images from 53,388 patients for training.
- Models were trained with 320 x 320 pixel images, batch sizes of 16 or 32, and staged classification-head then whole-network training.
- The analysis trained fourteen CNNs spanning ResNet, DenseNet, VGG, SqueezeNet, and AlexNet architectures.
- Performance was assessed with ROC and precision-recall curves using AUROC and AUPRC.
Results
Performance depended on the metric: deeper networks generally led pooled AUROC, while shallower models achieved the highest pooled AUPRC. Training time increased substantially with network depth.
- AUROC: 0.882 was the highest pooled AUROC, achieved by ResNet-152, followed by DenseNet-161 and ResNet-50 at 0.881.
- AUROC: The best individual AUROC values exceeded the CheXpert baseline for cardiomegaly and pleural effusion.
- AUPRC: 0.709 was the highest pooled AUPRC, achieved by VGG-16, followed by AlexNet at 0.701 and ResNet-34 at 0.688.
- Training time: AlexNet required 20 minutes for batch size 32, whereas DenseNet-201 required 5:11 hours with batch size 16.
- Training time: Increasing batch size from 16 to 32 accelerated training by 29.9% ± 9.34% on average.
Discussion
The study finds that deeper networks do not necessarily perform better on chest radiographs. Smaller models can achieve comparable or superior classification while reducing computational demands.
- More complex networks do not necessarily perform better than shallow networks for chest-radiograph classification.
- AlexNet, ResNet-34, and VGG-16 may surpass deeper models such as ResNet-150 and DenseNet-201.
- Lower resource demands may enable higher image resolutions and facilitate human-in-the-loop annotation workflows.
- The work focused on architecture comparison rather than complete optimization of one specific network.
- Smaller networks require less hardware and training time without loss of performance.
Tables
The tables compare models using AUROC, AUPRC, and training duration. Deeper models generally performed better on AUROC, shallower models on AUPRC, while training time increased with network depth.
- Table 1 reports AUROC for each architecture and finding, including pooled model performance.
- ResNet-152, DenseNet-161, and ResNet-50 achieved the strongest pooled AUROC results.
- Table 2 reports AUPRC, where ResNet-34, AlexNet, and VGG-16 achieved the best results despite being shallower networks.
- Table 3 summarizes average training duration per epoch and overall training time for each network.The reported times are averages of five training runs rounded to the nearest minute.
Receiver Operating Characteristic Curves
Figures 1–3 present ROC curves for all models, distinguishing individual trainings from pooled performance across five trainings.
- Figures 1, 2, and 3 display ROC curves for all models.
- Colored lines represent individual training runs.
- Black lines represent pooled performance over five trainings.
Precision Recall Curves
Figures 1–3 also display precision-recall curves for all models, separating individual training runs from pooled five-training performance.
- Figures 1, 2, and 3 display precision-recall curves for all models.
- Colored lines represent individual training runs.
- Black lines represent pooled performance over five trainings.