PaperlayerView exact revision on arXiv

Source-linked AI summary

COVID-19 identification in chest X-ray images on flat and hierarchical classification scenarios

Rodolfo M. Pereira, Diego Bertolini, Lucas O. Teixeira, Carlos N. Silla, Yandre M. G. Costa

arXiv:2004.05835v3cs.LGstat.ML

TL;DR

The paper asks whether COVID-19 pneumonia can be identified from widely available CXR images despite pathogen diversity and strong class imbalance. It builds RYDLS-20 and evaluates flat and hierarchical classifiers using resampling, multiple feature representations, and fusion, obtaining the strongest COVID-19 result with hierarchical classification and the highest overall macro-avg F1-Score with multi-class classification.

  • Problem

    The study addresses identification of COVID-19 pneumonia among other pathogen-specific pneumonias and healthy lungs using CXR images in an imbalanced setting.

  • Method

    The paper constructs RYDLS-20 and evaluates flat and hierarchical classification using resampling, texture and pre-trained CNN features, and early or late fusion.

  • Results

    The best COVID-19 identification result was achieved by hierarchical classification, while multi-class learners achieved the best macro-avg F1-Score of approximately 0.65.

  • Takeaways & Limitations

    The proposed hierarchical classification approach provides the paper’s best COVID-19 recognition result in this multi-pathogen, imbalanced CXR setting.

  • Takeaways & Limitations

    The experiments use a 70/30 holdout split because the small, highly imbalanced database would leave very few examples per fold under cross-validation.

Abstract

from arXiv · show

The COVID-19 can cause severe pneumonia and is estimated to have a high impact on the healthcare system. The standard image diagnosis tests for pneumonia are chest X-ray (CXR) and computed tomography (CT) scan. CXR are useful in because it is cheaper, faster and more widespread than CT. This study aims to identify pneumonia caused by COVID-19 from other types and also healthy lungs using only CXR images. In order to achieve the objectives, we have proposed a classification schema considering the multi-class and hierarchical perspectives, since pneumonia can be structured as a hierarchy. Given the natural data imbalance in this domain, we also proposed the use of resampling algorithms in order to re-balance the classes distribution. Our classification schema extract features using some well-known texture descriptors and also using a pre-trained CNN model. We also explored early and late fusion techniques in order to leverage the strength of multiple texture descriptors and base classifiers at once. To evaluate the approach, we composed a database, named RYDLS-20, containing CXR images of pneumonia caused by different pathogens as well as CXR images of healthy lungs. The classes distribution follows a real-world scenario in which some pathogens are more common than others. The proposed approach achieved a macro-avg F1-Score of 0.65 using a multi-class approach and a F1-Score of 0.89 for the COVID-19 identification in the hierarchical classification scenario. As far as we know, we achieved the best nominal rate obtained for COVID-19 identification in an unbalanced environment with more than three classes. We must also highlight the novel proposed hierarchical classification approach for this task, which considers the types of pneumonia caused by the different pathogens and lead us to the best COVID-19 recognition rate obtained here.

1 Introduction

The study targets COVID-19 pneumonia identification from CXR images across flat and hierarchical classification settings. It addresses real-world class imbalance using resampling, engineered and CNN features, and feature-fusion strategies.

  • Study aim: CXR was selected because it is cheaper, faster, less radiation-intensive, and more widely available than CT, despite lower precision.The paper notes that CXR interpretation can be difficult because inflammatory white patches may resemble other conditions.
  • Study aim: CXR images are used to identify COVID-19 pneumonia alongside pneumonia caused by other pathogens and healthy lungs.The study focuses on viruses, bacteria, and fungi, with COVID-19 as the principal target.
  • Database: RYDLS-20 combines CXR images from public repositories and healthy cases, with class frequencies designed to reflect a real-world distribution.Healthy cases are the majority, followed by viral pneumonia, bacterial pneumonia, and fungal pneumonia.
  • Classification schema: The experiments compare flat multi-class classification with hierarchical classification that organizes pneumonia by pathogen type.The flat setting assigns one label per image, while the hierarchical setting uses pathogen-based structure.
  • Feature representation: Features come from texture descriptors and a pre-trained CNN, with early and late fusion used to combine descriptor strengths and classifier predictions.Early fusion combines features before training; late fusion combines predictions after individually training feature sets.

2 Theoretical Background

The background frames pneumonia as a pathogen-diverse disease diagnosed with radiologic imaging, while distinguishing flat from hierarchical classification. It also explains why imbalance requires resampling and why CXR remains practically relevant despite CT’s greater precision.

  • COVID-19 Pandemic and Pneumonia Disease: Pneumonia is a group of infections caused by different organisms, including viruses, bacteria, and fungi, rather than a single disease.COVID-19 pneumonia is presented as one complication within this broader disease group.
  • COVID-19 Pandemic and Pneumonia Disease: CXR and CT can capture pneumonia-related inflammation, but CT is more precise while CXR is cheaper, faster, and more available.The paper describes CT as the gold standard but notes its cost, acquisition time, and limited availability.
  • Flat and Hierarchical Classification: Flat multi-class classification assigns one output label per sample, whereas hierarchical classification predicts within a class taxonomy organized by an IS-A relation.The paper’s pneumonia hierarchy groups pathogens and their associated pneumonia types.
  • Flat and Hierarchical Classification: The pneumonia hierarchy contains fourteen labels and seven leaf nodes, including COVID-19, SARS, MERS, Varicella, Streptococcus, and Pneumocystis categories.Coronavirus is subdivided into COVID-19, SARS, and MERS within the hierarchy.
  • Imbalanceness Data and Resampling: Real-world pneumonia CXR datasets are naturally imbalanced because healthy lungs and some pathogen classes occur more often than others.The paper identifies rare pathogen classes as minority patterns that may be important to classify correctly.
  • Imbalanceness Data and Resampling: Resampling rebalances class distributions through undersampling majority instances or oversampling minority instances, including synthetic creation.These methods preprocess the dataset before model training and can also be applied to multi-class problems.

3 Related Works

The reviewed literature covers texture-based medical-image analysis and AI methods for pneumonia and COVID-19 detection, using both CXR and CT images. Prior studies report strong results, while the authors note that rapidly emerging COVID-19 research makes the current state of the art difficult to characterize.

  • The related-work review covers texture descriptors, pneumonia detection in CXR images, and AI-based COVID-19 detection using CXR or CT scans.
  • CT-based studies exploit greater pulmonary detail, whereas CXR-based approaches offer lower cost and faster, more widely available imaging.
  • 0.96 AUROC was reported for COVID-19 and 0.95 for other viral pneumonia by the CT-based COVNet model.
  • 92.4% accuracy was reported for COVID-Net on the COVIDx test dataset as a whole.

4 Proposed Method

The proposed method analyzes pneumonia and COVID-19 in CXR images through multiple feature-extraction and classification phases. It supports flat and hierarchical classification, with optional resampling before prediction generation.

  • The method focuses on CXR-based classification of pneumonia types and COVID-19 among pneumonia caused by other microorganisms.
  • Figure 4 represents the proposed classification schema for COVID-19 identification in CXR images.
  • The schema includes feature extraction, early fusion, optional data resampling, and classification for multi-class and hierarchical scenarios.
  • The method does not automatically standardize image sizes before feature extraction, although manual cropping excludes undesirable patterns.

4.1 Feature Extraction (Phase 1)

Feature extraction combines handcrafted texture descriptors with automatically learned Inception-V3 features. The study evaluates these representations separately and in combination to exploit their potential complementarity.

  • The study evaluates handcrafted and non-handcrafted features separately and in combined experimental setups.
  • Combining descriptor categories may exploit complementarity because they do not necessarily make the same classification errors.
  • Texture descriptors were selected because of reported performance in general or medical-image applications.
  • Handcrafted Texture Descriptors: LBP encodes each pixel neighborhood as a binary pattern by comparing neighboring gray intensities with the central pixel, then represents the image using a pattern histogram.
  • Handcrafted Texture Descriptors: EQP extends LBP and LTP by using five-valued quinary encoding instead of binary or ternary encoding.
  • Automatically Learned Features with Inception-V3: Inception-V3 supplies automatically learned features, with 2,048 extracted weight values used after training.

4.2 Early Fusion (Phase 2)

Early fusion combines features from different texture descriptors before model training and testing. Using eight features, the authors evaluated 2×2 and 3×3 combinations, producing 84 feature sets.

  • Early fusion groups features from different descriptors into one feature set before training and testing.
  • Using eight different features, the study tested 2×2 and 3×3 combinations, yielding 84 feature sets.

4.3 Resampling (Phase 3)

The approach uses resampling to address severe class imbalance, while avoiding image augmentation because it may preserve inflammation patterns and increase overfitting risk.

  • Resampling is central to the classification schema because healthy-lung cases substantially outnumber pneumonia cases.
  • Image augmentation was avoided because rotations, translations, and distortions do not alter pneumonia inflammation spots and may increase overfitting.
  • Binary resampling algorithms were adapted with One-Against-All for multiclass datasets and applied to hierarchical leaf-node label paths.
  • Resampling was optional, allowing classification without balancing to support performance analysis and prediction combination.

4.4 Classification Approaches and Late Fusion (Phase 4)

The study evaluates flat and hierarchical classification alongside early and late fusion, combining complementary feature and classifier outputs through several rules and selection criteria.

  • Flat and hierarchical classification are performed within the proposed COVID-19 identification schema.
  • Late fusion uses sum, product, or voting rules to combine classifier predictions for each label.
  • Late fusion can exploit complementary classifier outputs when learners make different misclassifications.
  • Classifier selection tests Top-N, Best-per-Feature, and Best-per-Classifier criteria, including N=5 for Top-N.
  • Figure 5 illustrates how late-fusion strategies combine predictions across M samples and L labels.

5 Experimental Setup

The experiments use the RYDLS-20 chest X-ray database, multiple classifiers and resampling methods, and macro-averaged F1-Score to evaluate imbalanced flat and hierarchical classification.

  • RYDLS-20 contains 1,144 chest X-ray images, seven labels, fourteen hierarchical label paths, and a 70/30 train-test split.
  • Images came from the Cohen GitHub repository, Radiopaedia, and the NIH Chest X-ray14 dataset.
  • The dataset lacks information about the imaging machine, image origins, patient ages, and ethnicities.
  • The multi-class distribution commonly has seven–nine training samples and three test samples per pneumonia label, complicating learning.
  • The database reflects real-world imbalance, with healthy cases most frequent and fungal pneumonia least frequent among listed categories.
  • Cross-validation was deferred because half the labels contain only 10–12 samples, making five- or ten-fold splits potentially misleading.
  • The study evaluates kNN, SVM, MLP, decision trees, Random Forests, and Clus-HMC across 16 tested resampling methods.
  • Macro-averaged F1-Score was selected because averaging by class better reflects imbalanced-class performance than accuracy.

6 Experimental Results

Results are reported for flat and hierarchical scenarios using COVID-19-specific and macro-averaged F1-Score perspectives. Hierarchical classification produced the strongest COVID-19 result, while multiclass classification produced the strongest overall macro-average.

  • The experiments report both overall macro-averaged F1-Score and COVID-19-specific F1-Score for each classification scenario.
  • Multi-Class Classification Results: ≈0.83 was the best COVID-19 F1-Score in the multi-class scenario, achieved by individual predictions and late fusion.
  • Multi-Class Classification Results: ≈0.65 was the best multi-class macro-avg F1-Score, achieved by both individual and late fusion schemas.
  • Multi-Class Classification Results: The multi-class best case used MLP with LBP and RENN, yielding COVID-19 F1-Score ≈0.76, SARS 1.0, and Normal 0.98.
  • Hierarchical Classification Results: ≈0.89 was the best COVID-19 F1-Score in the hierarchical scenario, obtained with early fusion of BSIF, EQP, and LPQ using SMOTE or TL.
  • Hierarchical Classification Results: The hierarchical scenario achieved its best macro-avg F1-Score with early fusion, while SMOTE appeared across all classification schemas.
  • Results Summary: Hierarchical classification outperformed multi-class classification for COVID-19 identification in individual and early-fusion schemas.
  • Results Summary: ≈0.65 was the best overall macro-avg F1-Score, achieved by multi-class learners with individual and late fusion.

7 Discussions

The discussion compares feature representations, classifiers, resampling methods, and fusion strategies across flat and hierarchical classification scenarios. Results show that the strongest choices vary by scenario, with early fusion helping hierarchical classification but not multi-class classification.

  • Feature representation: LPQ ranked first for COVID-19 identification in the multi-class scenario, while BSIF ranked first in the hierarchical scenario.For macro-avg results, LBP ranked first in multi-class classification and LETRIST ranked first hierarchically.
  • Base classifier: MLP was the best-ranked base classifier across the multi-class classification contexts, with an overall average ranking of 1.17.
  • Resampling: ENN most improved multi-class COVID-19 identification, while AllKNN and RENN were strongest for multi-class macro-avg F1-Score.
  • Resampling: SMOTE most improved hierarchical COVID-19 identification, while SMOTE and SMOTE Borderline-1 were strongest for hierarchical macro-avg F1-Score.
  • Resampling: Tomek Link had the best overall average resampling ranking at 3.83 because it removes differently labeled, similar instances that can confuse learners.This similarity is relevant because some CXR images are visually alike despite representing different pneumonia types.
  • Fusion strategies: Early fusion improved hierarchical COVID-19 identification and macro-avg F1-Score, whereas late fusion did not improve results.In the multi-class scenario, neither early nor late fusion was effective for either outcome.

7.5 Which kinds of labels are easier/harder to predict?

Normal and COVID-19 were among the easier labels to predict, while several pathogen-specific classes were more difficult. The hierarchical representation improved COVID-19 identification by using pathogen relationships, although CXR variability contributed to recognition errors.

  • Multi-class classification: Normal and COVID-19 achieved better performance than other labels in the best multi-class prediction.These classes were also the two largest in the database.
  • Multi-class classification: SARS also performed comparatively well, possibly because its images had a visually distinctive, less-dark appearance.The passage attributes this outcome to the MLP classifier extracting peculiar visual content.
  • Hierarchical classification: Normal, Pneumonia, Acellular, Viral, Coronavirus and COVID-19 showed excellent or good performance in the hierarchical scenario.The hierarchical setting included internal nodes in addition to leaf labels.
  • Misclassification patterns: Six COVID-19 samples were misclassified as Normal, while Streptococcus, Varicella and Pneumocystis samples were misclassified as COVID-19 in the multi-class confusion matrix.The confusion matrix also showed samples from different labels mixed up with Normal.
  • Misclassification patterns: Normal CXR appearances varied because of patient characteristics, scan machines and radiologist protocols, making similarity-based recognition errors understandable.The database included pneumonia-free images with substantially different visual characteristics.

8 Concluding Remarks and Future Works

The study addresses imbalanced multi-pathogen pneumonia identification from CXR images by proposing flat and hierarchical classification with the RYDLS-20 database. It achieved stronger COVID-19 identification hierarchically, while emphasizing screening utility rather than definitive diagnosis and calling for larger, more robust evaluations.

  • Concluding remarks: The proposed schema targets pneumonia classification across pathogens in realistic imbalanced settings.The motivation reflects the greater prevalence of healthy cases and differing pathogen frequencies.
  • Concluding remarks: RYDLS-20 contains 1,144 CXR images across seven classes, including 1,000 Normal images and 90 COVID-19 images.The remaining images were distributed almost equally among the other pathogens.
  • Results: 0.65 macro-avg F1-Score was achieved with multi-class MLP using LBP and ENN, while COVID-19 identification reached 0.89 F1-Score hierarchically using BSIF, EQP, LPQ and SMOTE+TL.These are the reported best configurations for the two scenarios.
  • Results: The authors report 0.8889 F1-Score as the best nominal rate obtained for COVID-19 identification in an unbalanced environment with more than three classes.They caution that direct comparisons with literature are not fair when datasets and evaluation conditions differ.
  • Scope and implications: The proposal is intended to support emergency-service screening rather than provide a definitive COVID-19 diagnosis.The authors describe the approach as a promising direction requiring further investigation.
  • Future work: Future work includes larger databases, deeper learning techniques, cross-validation, alternative hierarchical classifiers and additional feature extraction.The stated goal is a more robust evaluation of the proposal.
Loading 2004.05835v3…