Parallel Training Using a CNN-DNN Architecture for Accelerated Development of Diagnostic Models

• AI means artificial intelligence, and ML means machine learning.Source: Abbreviations · p. 2
• CNN and DNN denote convolutional and dense feedforward neural networks; ResNet and DenseNet denote residual and densely connected networks.Source: Abbreviations · p. 2
• COVID-19 is coronavirus disease 2019, and nCP is non-COVID-19 pneumonia.Source: Abbreviations · p. 2
• ROC is the receiver operating characteristic curve, and AUC is the area under the curve.Source: Abbreviations · p. 2

1. Introduction

• Medical images support non-invasive disease assessment, while AI algorithms can streamline radiology workflows and enhance diagnostic sensitivity.Source: §1 · p. 2
• Prompt diagnosis and isolation during new-disease emergencies make efficient imaging-model training important for pandemic preparedness.Source: §1 · p. 3
• Large medical-image models can have millions or billions of parameters and long training times, motivating parallelization methods [19] and.Source: §1 · p. 3
• The study tests the model-parallel method from [21] on clinical lung CT data, extending it to DenseNet [22] with advanced preprocessing and augmentation.Source: §1 · p. 3

2. Materials and methods

• The study compares global and model-parallel CNN-based models for binary classification of chest CT scans with and without COVID-19-related pneumonia.Source: §2 · p. 4, §2.4 · p. 7

2.1. Dataset

• The retrospective dataset contains 300 scans, evenly split between COVID-19 and non-COVID-19 pneumonia, collected at three German university hospitals.COVID-19 cases required pulmonary infiltration and a positive RT-PCR within 48 hours; the non-COVID-19 pneumonia group required inflammatory infiltration and a negative RT-PCR after January 2020.Source: §2.1 · pp. 4–5

2.2. Data preprocessing

• Preprocessing resampled, normalized, intensity-clipped, and body-cropped scans; training augmentation included rotation, crop-or-padding, and flipping.Figure 2 illustrates automatic body cropping applied to all scans.Source: §2.2 · p. 5, Figure 2 · p. 5

2.3. Neural network architectures

• The evaluated architectures were DenseNet121, 20-layer ResNet20, and a 3D CNN; DenseNet121 and ResNet20 use axial slices as input channels.The 3D CNN performs three-dimensional convolutions, while the 2D models aggregate feature maps with global average pooling before binary classification.Source: §2.3 · p. 6, Table 1 · p. 6

2.4. Model parallel training method

• CNN-DNN decomposes images into subimages, trains proportionally smaller local CNNs independently and in parallel, then uses a trained DNN to combine their class probabilities.The local CNNs match the global model type and produce separate probability distributions for their respective subimages.Source: §2.4 · pp. 6–7

2.5. Implementation details and evaluation

• Experiments used an eight-GPU NVIDIA V100 cluster, an 80/10/10 train-validation-test split, and test-set accuracy, ROC curves, AUC, and GPU training time for evaluation.Training used Adam, adaptive learning-rate scaling on plateaus, and early stopping; model-parallel local CNNs were distributed across the GPUs.Source: §2.5 · p. 8

3.1. Performance of the global models

• Global DenseNet121 reached approximately 77% training accuracy and 71% validation and test accuracy; global ResNet20 reached approximately 70% training and 68% validation and test accuracy.The authors suggest ResNet20 may not be complex enough for this dataset, which contains 150 samples per class.Source: §3.1 · p. 8, Table 2 · p. 10, Table 3 · p. 10
• Global 3D CNN validation and test accuracy was only slightly lower than DenseNet121, making it a possible alternative when high classification accuracy is sought.The authors note that efficiently training a 3D CNN with many trainable parameters may require a larger dataset.Source: §3.1 · p. 9, Table 4 · p. 11

3.2. Performance of the model parallel approach

• DenseNet121 parallelization enhanced training, validation, and test accuracy across all tested decompositions; training time fell by factors of approximately 3.6 and 6.6 for 2 × 2 × 1 and 4 × 4 × 2 subimages, respectively.The table compares global DenseNet121 with CNN-DNN configurations and reports accuracy, training time, and evaluation time.Source: §3.2 · pp. 9–10, Table 2 · p. 10
• ResNet20 CNN-DNN accuracy was 2-3% lower than for the global model, while training time fell by approximately 4.5 to 18.8 across tested decompositions.The ResNet20 parallel models also had lower accuracy than parallel DenseNet121 models across the three split datasets.Source: §3.2 · p. 10, Table 3 · p. 10
• Training time fell by a factor of 17 to 31 for the parallel 3D CNN, which also improved classification accuracy, particularly on validation and test data.The reported time reduction uses the tested model-parallel decompositions.Source: §3.2 · p. 11, Table 4 · p. 11

Discussion and conclusion

• The 3D CNN showed the highest potential for reducing training time among the tested models.Source: §4 · pp. 11–12
• Validation and test accuracy improved for DenseNet121 and 3D CNN, which also achieved the best overall validation and test accuracy.Source: §4 · p. 12
• The authors frame fast retraining as relevant to future pandemics with changing imaging patterns from evolving virus subclones.Source: §4 · p. 13
• The approach has so far been tested only with CNN-based architectures and binary classification; validation across more classes, centers, and diseases is warranted before clinical implementation.Source: §4 · p. 13
• Training and inference time fell consistently across architectures, while diagnostic accuracy often improved with parallel CNN-DNN training.Source: §4 · p. 13
• Figure 4 shows correct and incorrect CT classifications for three models, while Figure 5 presents validation and test ROC curves across global and subdomain models.Source: Figure 4 · p. 14, Figure 5 · p. 15

Authors’ contributions

• JWH contributed methodology, implementation, visualization, formal analysis, and original-draft and review writing.Source: Authors’ contributions · p. 16
• AJ, PF, DM, AG, RF, AMB, RS, and JK contributed data curation; AJ, RS, and JK also contributed to writing.Source: Authors’ contributions · p. 16
• TP contributed conceptualization and review writing, while AK contributed conceptualization, methodology, supervision, and review writing.Source: Authors’ contributions · p. 16

Appendix A. Supplementary material

• Exemplary chest CT slices in the appendix are scaled to 224 × 224 × 64 voxels.Source: Figure A.6 · p. 22