Source-linked AI summary
Impact of lung segmentation on the diagnosis and explanation of COVID-19 in chest X-ray images
Lucas O. Teixeira, Rodolfo M. Pereira, Diego Bertolini, Luiz S. Oliveira, Loris Nanni, George D. C. Cavalcanti, Yandre M. G. Costa
TL;DR
The paper examines whether lung segmentation changes COVID-19 identification from chest X-rays and what image content informs predictions. It compares segmented and non-segmented classification with explainability methods, finding that segmentation focuses models on lung information despite lower F1 scores, while cross-source database bias remains.
Problem
The authors identify limited critical analysis of which image content supports COVID-19 classification and whether results from combined image databases are biased.
Source: §1 · p. 2Method
The study compares segmented and non-segmented chest X-ray classification across sources, using U-Net, three CNN classifiers, and explainability methods.
Source: §1 · p. 2Results
F1-Score was 0.88 for segmented multi-class classification and 0.83 for segmented COVID-19 identification, versus 0.92 and 0.94, respectively, without segmentation; segmentation focused models on lung information.
Source: §6 · p. 19Takeaways & Limitations
The authors conclude that segmentation can improve prediction quality and reliability by restricting model focus to lung information, even when it does not improve classification performance.
Source: §6 · p. 19Even after lung segmentation, strong database bias means the multi-class results are not entirely reliable.
Source: §5.3 · p. 18
Abstract
from arXiv · showhide
COVID-19 frequently provokes pneumonia, which can be diagnosed using imaging exams. Chest X-ray (CXR) is often useful because it is cheap, fast, widespread, and uses less radiation. Here, we demonstrate the impact of lung segmentation in COVID-19 identification using CXR images and evaluate which contents of the image influenced the most. Semantic segmentation was performed using a U-Net CNN architecture, and the classification using three CNN architectures (VGG, ResNet, and Inception). Explainable Artificial Intelligence techniques were employed to estimate the impact of segmentation. A three-classes database was composed: lung opacity (pneumonia), COVID-19, and normal. We assessed the impact of creating a CXR image database from different sources, and the COVID-19 generalization from one source to another. The segmentation achieved a Jaccard distance of 0.034 and a Dice coefficient of 0.982. The classification using segmented images achieved an F1-Score of 0.88 for the multi-class setup, and 0.83 for COVID-19 identification. In the cross-dataset scenario, we obtained an F1-Score of 0.74 and an area under the ROC curve of 0.9 for COVID-19 identification using segmented images. Experiments support the conclusion that even after segmentation, there is a strong bias introduced by underlying factors from different sources.
1 Introduction
The introduction motivates examining whether lung segmentation changes COVID-19 classification and what image regions models use. It outlines a multi-source, three-class study using CNNs and explainability methods.
1 Introduction
- The study asks whether lung segmentation affects COVID-19 identification and whether it can mitigate bias from combining image databases.The authors argue that analysis of image content contributing to model results has received insufficient critical attention.Source: §1 · p. 2
- The authors compare segmented and whole-image classification across sources, using lung opacity, COVID-19, and normal as the three classes.They use U-Net for segmentation and VGG16, ResNet50V2, and InceptionV3 for classification.Source: §1 · p. 2
- LIME identifies image superpixels that support a prediction, while Grad-CAM visualizes activations associated with a CNN prediction.The authors use both methods to assess the impact of lung segmentation on COVID-19 identification.Source: §1 · p. 2
- The authors report that whole-image models may rely on features outside the lungs, making strong classification performance potentially unreliable.They propose that segmentation may focus classification on lung information and make performance more realistic and better reasoned.Source: §1 · p. 2
2 Related Works
Related work spans explainability, segmentation, and COVID-19 image classification, while raising concerns about database bias and evaluation protocols. The authors distinguish their study by combining lung segmentation, classification, and explainability in one protocol.
2 Related Works
- Related studies applied explainability to COVID-19 imaging, including CAM, LIME, and SHAP for CT classification and LRP for chest-X-ray classification.One study also reported differences between internal and external validation.Source: §2 · pp. 3–4
- Other approaches combined disease classification with regional explanations or segmentation, including a three-step chest-X-ray pipeline and joint lesion segmentation and classification for CT.The CT study explored the correlation between 3D lesion segmentation and disease classification.Source: §2 · p. 3
- Reports of high COVID-19 image-classification rates have been accompanied by concerns that database combinations and testing protocols can produce biased or unrealistic results.Prior work also found undesired shortcut use in high-performing chest-radiograph systems.Source: §2 · p. 4
- The authors state that, to their knowledge at publication, prior work had not used their combined protocol of region segmentation, classification, and XAI.They note that rapidly emerging research made a complete account of the state of the art infeasible.Source: §2 · pp. 3–4
3 Material and Methods
The method isolates lung segmentation’s impact by comparing image-classification paths with and without segmentation, followed by explainability analysis.
3 Material and Methods
- The proposed method evaluates lung segmentation’s impact on COVID-19 identification using chest X-ray images.The study focuses on reliable identification among pneumonia caused by other microorganisms.Source: §3 · p. 4
- The proposed classification approach is organized into three phases: lung segmentation in Phase 1, classification in Phase 2, and XAI in Phase 3.Source: §3 · p. 4
- Skipping segmentation for non-segmented images isolates that stage as an ablation; the authors also made their code and database available for reproduction.Source: §3 · p. 5
3.1 Lung Segmentation (Phase 1)
The first phase produces lung-only regions of interest, aiming to remove background that may add noise. U-Net predicts binary lung masks, which were built and refined through a semiautomated process.
3.1 Lung Segmentation (Phase 1)
- Lung segmentation removes background while retaining the lung area; U-Net maps each CXR to a binary region-of-interest mask, with predicted masks manually reviewed and corrected when needed.The process used additional masked datasets and repeated prediction and review until the segmentation was judged satisfactory.Source: §3.1 · pp. 5–6
3.1.1 Lung Segmentation Database
- The segmentation dataset contains 1,645 CXR images with a 95/5 train/test split and a separate validation set drawn from 5% of training data.The random holdout validation split was chosen because the task predicts lung masks irrespective of image class.Source: §3.1.1 · p. 6
3.1.2 U-Net
- U-Net uses contracting and expanding paths; the adapted architecture adds dropout and batch normalization in each block to improve training time and reduce overfitting.All images were resized to 400px × 400px, and experiments with smaller and larger dimensions brought no significant improvement.Source: §3.1.2 · p. 6
- After mask prediction, morphological opening removes small bright spots and dilation smooths the mask boundary before images are cropped to the region of interest and resized to 300px × 300px.Data augmentation was also used during segmentation training.Source: §3.1.2 · pp. 6–7
3.2 Classification (Phase 2)
The classification phase compares full and segmented CXRs using three CNN architectures and evaluates cross-source COVID-19 generalization and database-source bias. The database uses three classes and includes images from multiple sources.
3.2 Classification (Phase 2)
- VGG16, ResNet50V2, and InceptionV3 classify full and segmented CXRs independently, with ImageNet-pretrained convolutional layers and added fully connected layers.The evaluation also tests COVID-19 recognition across sources and whether segmentation reduces source classification.Source: §3.2 · p. 7
3.2.1 COVID-19 Database (RYDLS-20-v2)
- RYDLS-20-v2 contains 2,678 CXRs across three classes—lung opacity, COVID-19, and normal—and draws images from eight sources.The split keeps images from the same patient together, distributes sources across train and test, and balances classes as far as those constraints allow.Source: §3.2.1 · p. 8
3.2.2 COVID-19 Generalization
- COVID-19 generalization uses segmented CXRs in a two-fold binary setup to test whether training on one source can identify COVID-19 in another.The folds include Cohen and RSNA Kaggle images, with RSNA appearing in both folds, so the experiment is not completely bias-free.Source: §3.2.2 · p. 9
3.2.3 Database Bias
- The database-bias evaluation tests whether a classifier can identify image source, and whether that ability is reduced when using segmented rather than full CXRs.The source-classification setup uses Cohen, RSNA, and a combined Other class; it contains 2,678 images with an 80/20 train/test split.Source: §3.2.3 · pp. 9–10
3.2.4 Data Augmentation
- Training used data augmentation to virtually increase the sample size, applying transformations with their default 50% probability.The transformations and their parameters are listed in Table 9.Source: §3.2.4 · p. 10
3.3 XAI (Phase 3)
The XAI phase examines which image regions support predictions, addressing whether models trained on full CXRs may rely on areas outside the lungs. LIME and Grad-CAM heatmaps show that segmented-image models focus primarily on lungs, unlike many full-image models.
3.3 XAI (Phase 3)
- Because performance metrics alone cannot establish whether a model attends to the correct image regions, the study uses XAI to inspect that focus.The stated concern is that information outside the lung area can limit application even when accuracy is high.Source: §3.3 · p. 10
- LIME and Grad-CAM identify important regions supporting predicted classes; the study summarizes test-image regions into one heatmap per classifier, class, and XAI approach.LIME identifies supportive superpixels, while Grad-CAM uses gradients flowing into the last convolutional layer.Source: §3.3 · pp. 10–11
- Models trained using segmented lungs focus primarily on the lung area, while models trained using full CXR images frequently focus elsewhere.For the multi-class scenario, full-image Lung Opacity and Normal heatmaps for InceptionV3 and VGG16 almost did not focus on the lungs.Source: §3.3 · p. 11
4 Results
The results assess lung segmentation, classification, cross-database COVID-19 identification, source bias, and model explanations. Segmented images produced more lung-focused heatmaps, although classification F1-Score was often lower than with full images.
4.1 Lung Segmentation Results
- The manually created lung masks underperformed other sources, while Cohen v7labs also performed somewhat worse; the authors link these differences to mask quality and lung-heart overlap.The remaining segmentation databases performed outstandingly.Source: §4.1 · p. 13, Table 11 · p. 13
4.2 Multi-class Classification
- Full-image models outperformed segmented-image models for COVID-19 and lung opacity F1-Score, while the settings were on par for the normal class.Across models, segmented images had lower F1-Score, though the authors considered them potentially more realistic.Source: Table 12 · p. 13, §4.2 · p. 13
4.3 COVID-19 Generalization
- The cross-database COVID-19 folds achieved F1-Scores of 0.77 and 0.7; the second fold had fewer COVID-19 training examples.The authors describe the results as far from random and present a ROC curve for this scenario.Source: §4.3 · p. 13
4.4 Database Bias
- Segmentation reduced differences between database sources, but sources remained identifiable; source overlap with disease classes and the mixed-source Other class constrained evaluation.The macro-averaged F1-Score excludes Other because it combines six sources and is not directly comparable with Cohen or RSNA.Source: §4.4 · pp. 13–14, Table 14 · p. 14
4.5 XAI Results
- Segmented-image heatmaps focused primarily on lungs, whereas full-image heatmaps were more chaotic and often missed the lung area; full-image models also used outside-lung information.VGG16's segmented Lung Opacity heatmap was a small exception, also focusing substantially on other regions.Source: §4.5 · pp. 14, 16, Figure 10 · p. 15, Figure 11 · p. 15
5 Discussions
Discussion of the classification results finds lower F1-Scores with segmentation, while explanations indicate that full-image models used burned-in annotations. Segmentation reduced database-source classification, but residual bias remained.
5.1 Multi-class Classification
- p = 0.019 in a Wilcoxon test and a Bayes Factor of 2.1 supported lower F1-Score for segmented than non-segmented images.Figure 12 presents the classification results by segmentation as a boxplot.Source: Figure 12 · p. 16, §5.1 · p. 16
- COVID-19 F1-Score fell from 0.94 with full images to 0.83 after segmentation, while lung-opacity F1-Score fell from 0.91 to 0.89.The authors conjecture that portable-X-ray annotations contributed to the impact; Normal was minimally affected.Source: §5.1 · p. 16
- Grad-CAM and LIME explanations showed full-image models focusing on burned-in annotations rather than lung opacities in some classes.The cited Grad-CAM example shows a large gradient on CXR annotations.Source: §5.1 · p. 16, Figure 13 · p. 17
5.2 XAI
- LIME is model-agnostic and Grad-CAM model-specific; the authors applied both to test images and aggregated explanations into broader heatmaps rather than relying on handpicked examples.The authors caution that these techniques are not definitive and can complement one another.Source: §5.2 · p. 18
5.3 COVID-19 Generalization and Database Bias
- Database-source F1-Score fell from 0.93 for full images to 0.78 for segmented images, yet RSNA remained identifiable at 0.91, leaving multi-class results not entirely reliable.Tests indicated a statistically significant reduction in macro-averaged F1-Score with segmentation (p = 0.024; Bayes Factor 4.6).Source: §5.3 · p. 18
5.4 Concluding Remarks
- The authors conclude lung segmentation is essential for CXR COVID-19 identification, while residual database bias and other image and model aspects still warrant further evaluation.They note that their previous pneumonia-identification results might have been biased by image patterns unrelated to the lungs.Source: §5.4 · p. 18
6 Conclusion
The conclusion finds that lung segmentation shifts attention toward lung information and can improve prediction quality despite lower classification F1 scores. Results also highlight cross-source bias and the lack of a definitive COVID-19 CXR benchmark.
6 Conclusion
- Segmentation removes background information, reduces the chance of data leak, and forces models to focus on important image areas, which may improve reliability even without better classification performance.The conclusion notes that very few prior works used lung segmentation and that they focused only on classification performance.Source: §6 · p. 19
- 0.88 and 0.83 F1-Scores were achieved with segmented images for multi-class classification and COVID-19 identification, versus 0.92 and 0.94 with non-segmented images; cross-source COVID-19 identification achieved macro-averaged F1-Score 0.74.The cross-source experiment predicts COVID-19 in one source using COVID-19 images from another source.Source: §6 · p. 19
- 0.034 Jaccard distance and 0.982 Dice coefficient were achieved by the general lung-segmentation model, built for binary masks across multiple sources, including manually created masks.The authors state the approach was on par with current state-of-the-art lung segmentation in CXR images.Source: §6 · p. 19
- LIME and Grad-CAM showed that segmented-image models focused primarily on lung-area information, and the authors say this improves prediction quality despite lower F1-Score.The authors did not claim state-of-the-art classification results, citing the absence of a reliable definitive benchmark, clinical subtlety of small performance differences, and CXR's non-gold-standard status.Source: §6 · p. 19
- The authors report the best COVID-19 identification rate among other pneumonia types using segmented CXR images, while cautioning that direct comparisons across studies are unfair and no definitive benchmark exists.The reported best-rate claim is qualified as being to the authors' knowledge and in a less biased configuration.Source: §6 · p. 19
- Future work will expand the database and use more CNN architectures, explore more sophisticated segmentation to isolate COVID-19 opacities, and evaluate predictions with methods such as SHAP.Source: §6 · p. 19