Deep Learning for Automatic Pneumonia Detection

1. Introduction

• Chest X-ray interpretation is complicated by other lung conditions and by changes in patient positioning and inspiration depth.Source: §1 · p. 1
• The approach automatically localizes lung opacities and achieved one of the best performances in the RSNA Pneumonia Detection Challenge.Source: §1 · p. 1
• The proposed system combines a single-shot detector, squeeze-and-excitation convolutional networks, augmentations, and multi-task learning.Source: §1 · p. 1
• Radiologists are known to vary in their interpretations of chest radiographs.Source: §1 · p. 1

2. Dataset

• The publicly provided dataset contains frontal-view X-rays from 26,684 unique patients.Source: §2 · p. 2
• Images are labeled Normal, No Lung Opacity / Not Normal, or Lung Opacity, using radiological reports.Source: §2 · p. 2
• Lung Opacity images have bounding boxes, but some visible opacity regions are not diagnosed as pneumonia.Source: §2 · p. 2
• The dataset is described as well-balanced across classes.Source: §2 · p. 2

3. Evaluation

• Average precision for an image is calculated from precision values at IoU thresholds from 0.4 to 0.75 in steps of 0.05.Source: §3 · p. 2
• A predicted object counts as a hit when its intersection over union with a ground-truth object exceeds 0.4.Source: §3 · p. 2
• The competition mAP is the mean of individual image average precisions on the test dataset.Source: §3 · p. 2

4. Model

• The proposed single model uses RetinaNet with an SE-ResNext101 encoder and ensembles predictions across checkpoints and four folds.The approach is motivated by the memory footprint and inference time of large, diverse ensembles at test time.Source: §4 · p. 2

4.1. Base model

• Base RetinaNet was extended to train on empty-box images, add small-anchor detection, and predict global class labels with dropout regularization.The global classification output was not used directly to classify images.Source: §4.1 · p. 2

4.2. Model training

• Training used 25,684 patients and combined whole-image classification, box classification, and anchor-regression losses with weights.The dataset was considered reasonably balanced, so no extra balancing techniques were used.Source: §4.2 · pp. 2–3

4.3. Model encoders

• Among tested encoders, SE-ResNext architectures showed optimal performance on this dataset with a good accuracy-and-complexity trade-off.The tested set included Xception, NASNet-A-Mobile, ResNet, SE-ResNext, DualPathNet, Inception-ResNet-v2, and PNASNet architectures.Source: §4.3 · p. 3, Figure 2 · p. 3

4.4. Multi-task learning

• With fixed augmentations, adding global image classification improved results over training the model only for regression.The auxiliary classification output was not used directly to classify images.Source: §4.4 · p. 3
• ImageNet-pretrained RetinaNet with an SE-ResNext-101 encoder and dropout probabilities of 0.5 or 0.75 showed the best test metrics on this dataset.Without pretraining, models took longer to converge; dropout also helped align the epochs of optimal classification and regression results.Source: §4.4 · p. 4

5. Images preprocessing and augmentations

• Images were resized to 512 × 512px; 256px degraded results, while full-resolution images were impractical with heavier models.The dataset’s limited size motivated augmentations to reduce overfitting.Source: §5 · p. 4
• The preprocessing included mild rotations up to 6 degrees, occasional blur, noise, and gamma changes, with brightness and gamma augmentation kept limited.Figure 5 shows an example scan with heavy augmentations.Source: §5 · p. 4, Figure 5 · p. 5

5.1. Ablation study

• The ablation compared no augmentation, light augmentation, heavy augmentation, heavy augmentation without rotation, and heavy augmentation with custom rotation using fixed model hyperparameters.Only the augmentation set changed across training sessions.Source: §5.1 · p. 4
• Heavy augmentations without rotation gave the best validation metrics; custom rotation improved results over usual rotation, while insufficient augmentation showed overfitting.Table 3 reports validation mAP for the augmentation sets.Source: §5.1 · pp. 4–5, Table 3 · p. 5
• Custom rotation rotated two points at one-third and two-thirds of each box edge, then formed the new box from the rotated points’ minima and maxima.This approach was tested against usual rotation and no rotation.Source: §5.1 · p. 5

6. Postprocessing

• Training boxes came from a single expert, whereas test ground truth used the intersection of three radiologists’ labels, yielding smaller boxes, especially in complicated cases.The authors used outputs from folds or checkpoints and the 20th percentile of anchor sizes, further reduced according to the spread between the 20th and 80th percentiles.Source: §6 · p. 6
• NMS thresholds had a dramatic impact on mAP, and validation-optimal thresholds varied from 0.45 to 1 across epochs.The train and test sets also had different optimal NMS thresholds because their labeling processes differed.Source: §6 · p. 6, Figure 8 · p. 6
• An alternative postprocessing approach scaled predicted test boxes to 87.5% of their original size to reflect the train–test labeling difference.The coefficient was chosen to approximately match the sizes from the percentile-based approach.Source: §6 · p. 6

7. Results

• Predictions from four cross-validation folds and several checkpoints of the same model were combined before applying NMS and optimizing thresholds.The ensemble is shown in Figure 9.Source: §7 · p. 6, Figure 9 · p. 6
• The method placed second in the challenge.Table 4 presents the final leaderboard results using mAP on the private test set.Source: Table 4 · p. 6, §7 · p. 6

8. Discussion

• Other winning solutions also used ensembles of RetinaNet models with varied inputs and encoders [23].Source: §8 · p. 6
• All three leading teams found that lowering the test-prediction NMS threshold relative to validation improved test-set scores.Other winners also systematically reduced predicted box sizes.Source: §8 · p. 7
• The teams linked these train–test differences to annotation procedures: one expert labeled training data, while test ground truth used three radiologists’ label intersection.Source: §8 · p. 7

• The method achieved one of the best results in the RSNA Pneumonia Detection Challenge.Source: §9 · p. 7
• The localization model uses a RetinaNet single-shot detector with ImageNet-pretrained Se-ResNext101 encoders.Source: §9 · p. 7
• A global classification output, heavy data augmentation, and an ensemble of four folds and several checkpoints were used to improve accuracy and generalization.Source: §9 · p. 7
• The method omits test-time augmentation and is reported to offer a good trade-off between accuracy and resources.Source: §9 · p. 7
• Ablation studies showed accuracy improvements from the proposed approaches.Source: §9 · p. 7