PaperlayerView exact revision on arXiv

Source-linked AI summary

Viral Pneumonia Screening on Chest X-ray Images Using Confidence-Aware Anomaly Detection

Jianpeng Zhang, Yutong Xie, Guansong Pang, Zhibin Liao, Johan Verjans, Wenxin Li, Zongji Sun, Jian He, Yi Li, Chunhua Shen, Yong Xia

arXiv:2003.12338v4eess.IVcs.CV

TL;DR

Viral pneumonia screening must address diverse appearances, novel viruses, and limited access to rapid diagnostic testing, while classification models face dataset shift. The paper proposes CAAD, a one-class anomaly detector with confidence prediction, and reports stronger screening results than binary classifiers, including on unseen COVID-19 data. Its X-VIRAL evaluation includes 5,977 viral pneumonia, 18,619 non-viral pneumonia, and 18,774 healthy-control cases.

  • Problem

    Viral pneumonia has diverse appearances and evolving viruses create dataset shift that limits classification approaches, motivating screening against non-viral pneumonia and healthy controls.

  • Method

    CAAD reformulates viral pneumonia screening as one-class anomaly detection with a shared feature extractor, anomaly detection module, and confidence prediction module.

  • Results

    83.61% AUC and 71.70% sensitivity were achieved on unseen X-COVID cases without COVID-19 training data.

  • Takeaways & Limitations

    Anomaly detection outperformed binary classification for viral pneumonia screening, while confidence learning reduced false negatives and supported failure prediction.

Abstract

from arXiv · show

Cluster of viral pneumonia occurrences during a short period of time may be a harbinger of an outbreak or pandemic, like SARS, MERS, and recent COVID-19. Rapid and accurate detection of viral pneumonia using chest X-ray can be significantly useful in large-scale screening and epidemic prevention, particularly when other chest imaging modalities are less available. Viral pneumonia often have diverse causes and exhibit notably different visual appearances on X-ray images. The evolution of viruses and the emergence of novel mutated viruses further result in substantial dataset shift, which greatly limits the performance of classification approaches. In this paper, we formulate the task of differentiating viral pneumonia from non-viral pneumonia and healthy controls into an one-class classification-based anomaly detection problem, and thus propose the confidence-aware anomaly detection (CAAD) model, which consists of a shared feature extractor, an anomaly detection module, and a confidence prediction module. If the anomaly score produced by the anomaly detection module is large enough or the confidence score estimated by the confidence prediction module is small enough, we accept the input as an anomaly case (i.e., viral pneumonia). The major advantage of our approach over binary classification is that we avoid modeling individual viral pneumonia classes explicitly and treat all known viral pneumonia cases as anomalies to reinforce the one-class model. The proposed model outperforms binary classification models on the clinical X-VIRAL dataset that contains 5,977 viral pneumonia (no COVID-19) cases, 18,619 non-viral pneumonia cases, and 18,774 healthy controls.

II. RELATED WORK

The paper situates viral pneumonia screening within deep-learning chest X-ray analysis and deep anomaly detection. It distinguishes its goal from prior work by targeting viral pneumonia against all non-viral pneumonia and healthy controls.

  • Deep learning has advanced chest X-ray diagnosis, staging, delineation, and differentiation of viral from bacterial pneumonia.
  • Deep-learning models can make mistakes in real-world applications, motivating methods for predicting failure cases.
  • Traditional anomaly detection uses one-class methods such as OC-SVM and SVDD, but suffers from poor computational scalability and the curse of dimensionality.

III. METHODS

The method combines a shared feature extractor with anomaly scoring to model viral pneumonia as an anomaly relative to non-viral pneumonia and healthy controls. Its one-class formulation uses reference scores and contrastive learning, and is designed to handle limited positive data.

  • The CAAD architecture shares a feature extractor between anomaly detection and confidence prediction, with anomaly scores ranking abnormal images above normal ones.The confidence network outputs scores from 0 to 1, while the anomaly function maps an image to a real-valued score.
  • EfficientNet-B0 pretrained on ImageNet serves as the feature extractor because it balances performance and complexity.
  • The anomaly detection module is a multilayer perceptron that generates one anomaly score for each input X-ray.It contains three 100-neuron hidden layers and one output neuron.
  • The training loss compares anomaly scores with a Gaussian reference distribution using randomly sampled values and a contrastive objective.The reference distribution uses µ = 0, σ = 1, and l = 5,000; the margin parameter is set to 5.
  • The one-class anomaly detector learns primarily from abundant negative data, avoiding explicit modeling of the limited positive class under class imbalance.

C. Confidence prediction network

The confidence prediction network estimates whether anomaly-detection decisions are reliable. It converts anomaly scores into a probability-like abnormality measure that supports failure prediction.

  • The confidence prediction network learns a confidence score for each input to identify failures in anomaly detection.
  • The anomaly score represents abnormality degree, from confirmed viral pneumonia to confirmed non-viral pneumonia or healthy cases.Because the score is not directly probabilistic, the method uses a probability density function for normalization.

1) Confidence criterion for anomaly detection:

The method normalizes anomaly scores with a probability density function and uses the resulting anomaly probability to train confidence estimates. The confidence criterion is high for correct predictions and low for errors.

  • The normalized probability density function maps anomaly-detection scores into the range [0, 1].This produces a probability-like quantity analogous to sigmoid, hyperbolic tangent, or softmax outputs.
  • The anomaly probability is used to learn confidence for distinguishing successful predictions from erroneous ones.
  • 1) Confidence criterion for anomaly detection:: The confidence criterion g approaches 0 for failed predictions and 1 when the true label is successfully predicted.
  • 1) Confidence criterion for anomaly detection:: The confidence prediction module is built on the shared feature extractor and contains four hidden layers with 100 neurons each.

2) Confidence prediction network:

The confidence prediction module treats confidence estimation as regression, using the anomaly detector’s confidence score and standard L2 loss.

  • The confidence score ι is predicted through a regression task because it lies in [0, 1].
  • The module is optimized with the standard l2 loss.

D. Training and inference

CAAD is trained in three stages: anomaly detection, confidence prediction, and joint end-to-end optimization. During inference, anomaly and confidence thresholds jointly determine whether an image is flagged for further examination.

  • Training: Training begins by optimizing the anomaly detection network, combining the shared feature extractor and anomaly detection module.The feature extractor is initialized with ImageNet-pretrained weights, while the other parameters are randomly initialized.
  • Training: The confidence prediction network is then trained while the feature extractor and anomaly detection module remain fixed.Its loss uses confidence scores and generated anomaly probabilities.
  • Training: Finally, the feature extractor, anomaly detection module, and confidence prediction module are jointly optimized end to end.
  • Inference: At inference, the model produces an anomaly score ν and confidence score ι for each test image.
  • Inference: The anomaly threshold is Tano = 1.18, and an image is flagged when ν ≥ Tano or ι < Tconf.Flagged cases receive a positive diagnosis and are recommended for radiologist examination.

E. Performance metrics

The study evaluates diagnostic performance using AUC, sensitivity, specificity, and accuracy, covering overall ranking and classification of positive and negative cases.

  • AUC measures the probability that a model ranks a randomly selected positive instance above a randomly selected negative instance.It is described as a common metric for overall classification performance.
  • Sensitivity and specificity measure the proportions of positives and negatives that are correctly identified.
  • Accuracy is also used to quantitatively assess diagnostic performance.

IV. DATASETS

The evaluation uses the clinical X-VIRAL dataset and compares anomaly detection with binary classification. The reported comparison highlights a sensitivity increase from 79.09% to 85.51%.

  • Datasets: X-VIRAL contains 5,977 viral pneumonia cases, 18,619 non-viral pneumonia cases, and 18,774 healthy controls.The images were collected from 390 township hospitals during 2019.
  • Comparison: The models are compared on X-VIRAL using five-fold cross-validation with ResNet-18 or EfficientNet-B0 feature extractors.Both feature extractors were pretrained on ImageNet, and binary classifiers used cross-entropy loss.
  • Results: 85.51% sensitivity is reported versus 79.09% for the comparison baseline.
  • Results: The results describe anomaly detection as advantageous over binary classification for distinguishing viral pneumonia from non-viral pneumonia and healthy controls.The stated advantage is especially associated with positive-case sensitivity.

B. Confidence learning for failure prediction

CAAD augments anomaly detection with confidence prediction to identify likely failures and improve viral-pneumonia screening sensitivity. On X-VIRAL, increasing the confidence threshold trades accuracy and specificity for sensitivity.

  • Prediction-probability distributions overlap for correct and false predictions, limiting confidence learning from prediction probability alone.
  • CAAD combines AnoDet with confidence prediction, using a shared anomaly threshold and varying confidence threshold Tconf on X-VIRAL.Table III fixes Tano = 1.18 for both models.
  • 13 of 16 illustrated X-VIRAL validation cases were correctly diagnosed by CAAD, while three were misdiagnosed.Correct cases are marked green and incorrect cases red.
  • At Tconf = 0.9, CAAD increased sensitivity from 85.51% to 93.01% while achieving 67.72% specificity.
  • Raising Tconf substantially improves sensitivity but increasingly deteriorates accuracy and specificity.At Tconf = 0.5, CAAD performs comparably to AnoDet.
  • The confidence module is designed to distinguish high-confidence successful predictions from low-confidence failures rather than improve anomaly detection itself.

D. Visualizing region of diagnosis

Grad-CAM visualizations show CAAD diagnoses alongside highlighted image regions, while cross-dataset experiments evaluate generalization and fine-tuning across classification and anomaly-detection models.

  • D. Visualizing region of diagnosis: Grad-CAM overlays indicate that CAAD focuses on suspected lesions when diagnosing X-VIRAL validation images.Eight cases in the top two rows are positive and the remaining cases are negative.
  • Generalization comparison: On unseen X-COVID data without fine-tuning, anomaly-detection models outperform binary classifiers, especially in AUC.All compared models were trained without COVID-19 cases.
  • Generalization comparison: Confidence prediction improves failure prediction in both classifiers and anomaly detectors, producing a substantial sensitivity improvement.
  • Fine-tuning comparison: Fine-tuning improves all four models on X-COVID and narrows the gap between classification-based and anomaly-detection models.
  • Fine-tuning comparison: Despite fine-tuning, classification-based models show poorer generalization ability than anomaly-detection models.

F. Generalize to unseen Open-COVID dataset

CAAD generalizes to the unseen Open-COVID dataset without fine-tuning and achieves stronger reported detection accuracy than the competing methods. Additional experiments examine mixed datasets and the effect of limited positive data.

  • F. Generalize to unseen Open-COVID dataset: 94.93% COVID-19 accuracy and 100% SARS and MERS accuracy were achieved by CAAD on Open-COVID without fine-tuning.These results exceeded those of the three competing methods.
  • F. Generalize to unseen Open-COVID dataset: The mixed evaluation dataset contains 599 positive COVID-19 cases and 2,107 negative cases.It combines selected X-VIRAL negatives with X-COVID healthy controls and COVID-19 cases from X-COVID and Open-COVID.
  • F. Generalize to unseen Open-COVID dataset: Reducing positive training data degrades both anomaly detection and binary classification, but anomaly detection degrades less.This indicates an advantage for anomaly detection under class imbalance.

VI. DISCUSSION

The discussion frames COVID-19 as a novel viral-pneumonia anomaly and reports CAAD’s unseen-dataset performance as comparable to radiologists. It also identifies early disease without visible lung pathology as a sensitivity boundary.

  • VI. DISCUSSION: The approach treats COVID-19 alongside other viral pneumonias and separates them from non-viral pneumonia and healthy controls.
  • VI. DISCUSSION: CAAD achieved 83.61% AUC and 71.70% sensitivity on unseen X-COVID without COVID-19 training cases.
  • VI. DISCUSSION: The reported COVID-19 screening sensitivity was comparable to radiologists’ reported 69% sensitivity.
  • VI. DISCUSSION: The authors attribute limited sensitivity partly to subjects lacking radiographically visible lung pathology at early viral-pneumonia stages.
  • VI. DISCUSSION: The conclusion reports anomaly detection as superior to binary classification for viral-pneumonia screening and confidence learning as useful for predicting failures and reducing false negatives.
Loading 2003.12338v4…