PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning for Chest X-ray Analysis: A Survey

Ecem Sogancioglu, Erdi Çallı, Bram van Ginneken, Kicky G. van Leeuwen, Keelin Murphy

arXiv:2103.08700v1eess.IVcs.CV

TL;DR

Chest radiograph interpretation is challenging, and deep learning research has expanded alongside large public datasets. This survey reviews 295 studies across major CXR tasks, datasets, commercial systems, and future directions, finding that research has largely followed available data and labels rather than clinical needs.

  • Problem

    CXR interpretation is difficult because anatomical superimposition can obscure subtle abnormalities and pathological patterns.

  • Method

    The paper surveys deep learning studies on chest radiographs, categorizing applications and reviewing datasets, commercial software, trends, gaps, and future directions.

  • Results

    295 papers were retained in the final literature review after searching and screening 767 listed papers.

  • Takeaways & Limitations

    Research should focus more directly on clinical needs, with better data, labels, and public challenges for clinically relevant benchmarking.

  • Takeaways & Limitations

    Public CXR datasets often use NLP-generated labels that can be inaccurate, and some conditions cannot be predicted from imaging alone.

Abstract

from arXiv · show

Recent advances in deep learning have led to a promising performance in many medical image analysis tasks. As the most commonly performed radiological exam, chest radiographs are a particularly important modality for which a variety of applications have been researched. The release of multiple, large, publicly available chest X-ray datasets in recent years has encouraged research interest and boosted the number of publications. In this paper, we review all studies using deep learning on chest radiographs, categorizing works by task: image-level prediction (classification and regression), segmentation, localization, image generation and domain adaptation. Commercially available applications are detailed, and a comprehensive discussion of the current state of the art and potential future directions are provided.

1. Introduction

Chest radiography is widely used but difficult to interpret, motivating automated analysis and a comprehensive survey of deep learning research, datasets, trends, and clinical translation.

  • Chest radiography remains the world’s most commonly performed radiological exam because it is cost-effective, uses low radiation, and offers reasonable sensitivity.
  • Superimposed anatomy, subtle abnormalities, and overlapping pathological patterns make CXR interpretation challenging and contribute to high inter-observer variability.
  • Automated CXR analysis has been studied since the 1960s, with potential gains including increased sensitivity and prioritization of time-sensitive cases.
  • Large labeled public datasets, beginning with 112,000 NIH images in 2017, have supported the data demands of deep learning and increased research activity.
  • The survey reviews 295 papers from 2015–2021, catalogs public datasets and caveats, discusses commercial software, and identifies trends, gaps, and future directions.

2. Overview of Deep Learning Methods

The survey introduces deep learning architectures and organizes CXR applications into image-level prediction, segmentation, localization, image generation, and domain adaptation.

  • Deep learning uses neural networks with many hidden layers, while an architecture denotes the network’s overall construction and layer arrangement.
  • Convolutional layers preserve spatial information and share weights by applying local receptive fields across image regions.
  • Transfer learning initializes a target-task network with weights learned on a different source task, enabling training with relatively smaller target datasets.
  • Image-level prediction: Image-level prediction analyzes an entire CXR to predict a category label or continuous value, typically using related classification or regression architectures.
  • Segmentation: Fully convolutional networks produce likelihood maps for larger inputs, while U-Net combines contracting and expanding paths with skip connections to restore input resolution.
  • Localization: Localization identifies image regions with bounding boxes or points; RCNN-family methods and YOLO provide established object-localization architectures.
  • Image generation: Image generation creates realistic images for interpretability, data augmentation, domain emulation, or improving abnormality detection and segmentation.
  • Domain adaptation: Domain adaptation commonly uses adversarial training to encourage domain-invariant features, either by transforming source images or learning invariant representations.

3. Datasets

Large public CXR datasets have enabled deep-learning research, but their labels, image quality, and clinical meaning require careful interpretation. The review catalogs these datasets and distinguishes improved test-set labeling as gold standard data.

  • Dataset construction: Public CXR datasets are constructed using digitized images and labeling methods including automated radiology-report analysis.Natural language processing enables reports to be analyzed automatically to extract labels for individual images.
  • Dataset catalog: The review details each public dataset and assigns acronyms used throughout its literature tables.Table 1 summarizes dataset characteristics, labeling methods, annotation types, and gold standard data.
  • Dataset diversity: Public datasets span large collections such as ChestX-ray14, CheXpert, PadChest, and COVID-focused datasets, with differing formats, populations, and annotations.ChestX-ray14 contains 112,120 CXRs, while CheXpert contains 224,316; other datasets include COVID-specific images and clinical information.
  • Research growth: The number of deep-learning CXR publications increased as publicly available CXR images expanded.Figure 2 compares yearly reviewed publications with the cumulative number of publicly available CXR images.
  • Dataset caution: NLP-derived labels can be inaccurate because reports omit findings, algorithms misinterpret statements, and subtle abnormalities generate expert disagreement.Accurate evaluation therefore benefits from radiologist annotations, laboratory tests, or CT verification rather than relying only on report parsing.
  • Dataset caution: Dataset labels may overlap or require clinical information beyond the CXR, as with consolidation and pneumonia.Pneumonia is rarely diagnosed from imaging alone and may require other clinical signs or symptoms.
  • Dataset caution: Public CXR images may have lower quality than clinical radiographs because of reduced resolution, bit-depth, or screenshot-based collection.Clinical DICOM images typically use 12 bits per pixel and dimensions of approximately 2 to 4 thousand pixels per axis.

4. Deep Learning for Chest Radiography

The survey organizes deep-learning chest-radiography research into major application tasks, covering 295 studies and 187 image-level prediction papers. Across these tasks, studies emphasize image-level labels, anatomical and foreign-object segmentation, localization, image generation, and domain adaptation concerns.

  • The review categorizes 295 studies by image-level prediction, segmentation, localization, image generation, domain adaptation, and other tasks.Studies may address at most two tasks, and some works with equal focus on two tasks appear in both tables.
  • Image-level Prediction: Image-level prediction comprises 187 studies, usually using off-the-shelf deep-learning models to predict pathology, metadata, or dataset-provided labels.Most studies use frontal chest radiographs, while a smaller set evaluates lateral or multiple-view models.
  • Image-level Prediction: 31 studies predict ChestX-ray14 labels, followed by 26 pneumonia studies, 18 pneumothorax studies, 18 tuberculosis studies, and 17 COVID-19 studies.Applications include triaging urgent cases, detecting potentially critical patients, assessing severity or progression, and using additional clinical information.
  • Segmentation: Segmentation includes 58 papers, most commonly targeting anatomy such as the heart, lungs, clavicles, or ribs for computer-aided detection pipelines.Foreign-object segmentation addresses catheters, tubes, and lines, whereas relatively few studies segment abnormalities precisely.
  • Localization: Precise abnormality segmentation remains less common than localization, likely because projection images make precise annotation difficult and segmentation annotation costly.Localization commonly uses bounding-box-type annotations, and weakly supervised learning addresses the cost of collecting large annotated datasets.
  • Segmentation: An LSTM-decoder recurrent architecture achieved a Dice score of 0.97 for lung segmentation on the Montgomery dataset.The approach used recurrent decoding for multi-scale feature integration, while other recurrent networks addressed catheter and tube segmentation.
  • Image Generation: GANs were used in over 50% of the reviewed image-generation studies, including SkrGAN, which adds a sketch-prior constraint through two generator modules.The modules generate a sketched structural representation and the CXR image separately to improve generated-image quality.
  • Domain Adaptation: Most surveyed studies train and test within the same domain, while several studies report significant performance drops on datasets outside the training domain.This cross-domain concern was observed across a variety of chest-radiograph applications.

5. Commercial Products

Commercial CXR analysis has existed since 2008, but translation into clinical practice remains relatively slow. The survey identifies 21 CXR products spanning multiple abnormalities and commonly offering localization.

  • Clinical adoption of commercial CXR analysis remains relatively slow despite longstanding research and a product available since 2008.
  • The survey lists 21 commercial products specifically applicable to chest X-ray after excluding one non-diagnostic texture-analysis product.
  • Pneumothorax identification is the most common commercial task, followed by pleural effusion, nodules, and tuberculosis.
  • 17 of 21 commercial products claim localization of one or more abnormalities, contrasting with literature dominated by image-level prediction.

6. Discussion

The survey finds that public CXR datasets have accelerated research, but label quality, dataset construction, architecture comparisons, and external validation constrain reliable progress toward clinical use. It recommends focusing future data, benchmarking, and AI development more directly on clinical needs and workflows.

  • Datasets and labels: 209 of 295 reviewed publications used one or more public datasets, but NLP-extracted labels require caution and gold-standard test labels are recommended for accurate evaluation.Gold-standard labels may come from expert radiological interpretation, associated CT scans, laboratory results, or other appropriate measurements.
  • Study quality: 142 papers were excluded for scientifically unsound content, including 112 with concerning dataset construction and 61 combining adult COVID-19 subjects with pediatric subjects.A common concern was that labels corresponded to different data sources, allowing images to be differentiated by factors other than the target label.
  • Architectures: There is little evidence that one off-the-shelf architecture outperforms another for a specific task because reported differences are typically small and evaluations often lack optimization or statistical analysis.Inaccurate NLP-extracted evaluation labels further complicate architecture comparisons.
  • Architectures: Ensembles typically perform better than individual models, and most top-10 submissions in several public challenges used network ensembles.The survey also identifies self-adapting frameworks such as nnU-Net as promising for selecting task-specific preprocessing, optimization, and architecture choices.
  • Clinical translation: Most reviewed studies used training, validation, and test samples from the same dataset, while models generally perform more weakly on external-domain datasets.The survey discusses domain adaptation, active learning, and federated learning as possible responses to generalization challenges.
  • Clinical translation: Research direction has been largely shaped by available data and labels rather than clinician needs, while interval-change assessment and preliminary reporting remain relatively underexplored.The survey suggests clinically relevant public challenges with appropriately annotated data could enable more accurate algorithm comparison and benchmarking.
Loading 2103.08700v1…