Source-linked AI summary
SCAN: Structure Correcting Adversarial Network for Organ Segmentation in Chest X-rays
Wei Dai, Joseph Doyle, Xiaodan Liang, Hao Zhang, Nanqing Dong, Yuan Li, Eric P. Xing
TL;DR
CXR organ segmentation is important for computer-aided detection, but limited pixel-level data and substantial population, pathology, and imaging variation make realistic masks difficult. SCAN jointly trains a segmentation network and critic to impose higher-order physiological structure, achieving human-competitive accuracy and stronger generalization with limited data.
Problem
CXR organ segmentation matters for computer-aided detection, but pixel-level annotations are scarce and images vary across populations, pathology, and imaging technology.
Method
SCAN jointly trains a convolutional segmentation network and critic that distinguishes ground-truth annotations from predicted masks and transfers structural regularities to the segmenter.
Results
94.7% lung-field IoU and 86.6% heart IoU were achieved, with 1.8% absolute improvement and performance competitive with human experts; unseen-dataset performance exceeded the vanilla model by 4.3%.
Takeaways & Limitations
SCAN produces accurate, realistic segmentation with very limited training data, without relying on existing trained models or data from other domains, and generalizes to an unseen dataset.
Takeaways & Limitations
The approach uses a parsimonious model for a 247-image grayscale dataset, and GAN training is known to be unstable and can collapse.
Abstract
from arXiv · showhide
Chest X-ray (CXR) is one of the most commonly prescribed medical imaging procedures, often with over 2-10x more scans than other imaging modalities such as MRI, CT scan, and PET scans. These voluminous CXR scans place significant workloads on radiologists and medical practitioners. Organ segmentation is a crucial step to obtain effective computer-aided detection on CXR. In this work, we propose Structure Correcting Adversarial Network (SCAN) to segment lung fields and the heart in CXR images. SCAN incorporates a critic network to impose on the convolutional segmentation network the structural regularities emerging from human physiology. During training, the critic network learns to discriminate between the ground truth organ annotations from the masks synthesized by the segmentation network. Through this adversarial process the critic network learns the higher order structures and guides the segmentation model to achieve realistic segmentation outcomes. Extensive experiments show that our method produces highly accurate and natural segmentation. Using only very limited training data available, our model reaches human-level performance without relying on any existing trained model or dataset. Our method also generalizes well to CXR images from a different patient population and disease profiles, surpassing the current state-of-the-art.
1. Introduction
CXR segmentation supports computer-aided detection but must handle scarce annotations, variable imaging conditions, pathology, and medically meaningful global structure. SCAN addresses these challenges with adversarial structural guidance and reports accurate, human-competitive results with limited data.
- Motivation: 22.5 million X-ray images were requested in the UK public medical sector in 2015/16, creating substantial diagnostic workloads.The total represented over 55% of medical images, including 8 million CXRs.
- Motivation: CXR’s low resolution and 2-D projection make diagnosis challenging, motivating computer-aided detection support for clinicians.
- Motivation: Lung and heart masks provide structural information for assessing conditions such as cardiomegaly, pneumothorax, pleural effusion, and emphysema.Explicit lung masks can also improve interpretability of computer-aided detection.
- Challenges: Limited pixel-level training data and variation across populations, pathology, and imaging technology complicate CXR segmentation.Existing hand-crafted approaches may be brittle, while realistic outputs require balancing local information with global structure.
- SCAN: SCAN uses a critic network to distinguish ground-truth annotations from synthesized masks and transfer higher-order physiological regularities to the segmentation model.The segmentation and critic networks are trained end-to-end through an adversarial process.
- Results: 94.7% lung-field IoU and 86.6% heart IoU were achieved, with 1.8% absolute improvement and performance competitive with human experts.The reported human values were 94.6% for lungs and 87.8% for the heart.
2. Related Work
Prior CXR organ-segmentation methods include heuristic, pixel-classification, deformable, and registration-based approaches, while convolutional networks offer an end-to-end alternative. SCAN applies adversarial training to improve global consistency in segmentation outcomes.
- Lung Field Segmentation: Existing lung-field segmentation methods include rule-based systems, pixel classification, and deformable-model approaches.
- Lung Field Segmentation: The registration-based state-of-the-art method matches test patients to database profiles and linearly deforms lung shapes using key points.Its performance can be brittle when populations or key-point matches differ.
- Convolutional Segmentation: Fully convolutional networks assign semantic classes to pixels, while prior adversarial semantic-segmentation work reported some improvement.
- Convolutional Segmentation: U-net and related convolutional architectures are established for biomedical segmentation, and SCAN extends them with adversarial training for global consistency.
- CXR Models: End-to-end CXR models can directly predict clinical targets but generally require many images and lack well-defined intermediate outputs for interpretability.
3. Problem Definition
The task is to segment the left lung, right lung, and heart in posteroanterior CXR images. Because projection causes organ overlap, the adopted definitions specify which anatomical structures belong inside or outside each mask.
- Task: The segmentation targets are the left lung field, right lung field, and heart in posteroanterior CXR images.PA imaging passes radiation through the patient from back to front.
- Mask Definitions: Lung fields exclude the heart, mediastinum, region below the diaphragm, aorta, and visible superior vena cava.They include pixels where radiation passes through the lung but not those excluded structures.
- Mask Definitions: The heart boundary is visible on two sides, while its top and bottom borders must be inferred because the mediastinum occludes them.
- Mask Definitions: The definitions aim to capture common lung-field and heart anatomy while retaining regions relevant to clinical CXR reading.
4. Structure Correcting Adversarial Network
SCAN jointly trains a fully convolutional segmentation network and a critic to impose higher-order anatomical structure on CXR organ masks. Its compact, from-scratch architecture is designed for gray-scale images and very limited training data.
- Adversarial framework: The critic transfers higher-order structural information to the segmentation network, encouraging masks consistent with global physiological regularities.This regularization targets natural contours that pixel-level training alone may miss.
- Adversarial framework: SCAN jointly trains a segmentation network that predicts pixel-level classes and a critic that distinguishes ground-truth masks from predicted masks.The two networks are optimized through an alternating minimax scheme.
- Segmentation network: The segmentation model is a fully convolutional network whose down-sampling path extracts semantic features and whose up-sampling path produces pixel-level class scores.The architecture is adapted for dense prediction in CXR images.
- Segmentation network: The CXR-specific design uses a parsimonious architecture because the images are gray-scale, the task has three foreground classes, and only 247 labeled images are available.The authors argue that large RGB-pretrained models are poorly matched to this setting.
- Segmentation network: 271k parameters make the segmentation network approximately 500x smaller than the VGG-based down-sampling path used for comparison.The model uses few initial feature maps, deeper convolutional processing, bottleneck-style convolutions, and residual blocks.
- Critic network: The critic mirrors the segmentation network and uses mask predictions as input, with the original CXR image optionally available as an additional channel.Preliminary experiments found that including the original image did not improve performance, so the final critic uses only masks.
5. Experiments
SCAN is evaluated on lung-field and heart segmentation across JSRT and Montgomery datasets, including cross-population generalization, comparisons with FCN and existing methods, and qualitative contour analysis. The experiments show accurate segmentation, strong robustness, and efficient test-time inference.
- Experimental setup: SCAN is evaluated for lung-field and heart segmentation using publicly available JSRT and Montgomery datasets.The datasets represent different countries, diseases, imaging conditions, and patient populations.
- Quantitative comparison: Adversarial training improves both-lung IoU from 92.9% to 94.7% on the JSRT evaluation set.Across λ = 0.1, 0.01, and 0.001, both-lung IoU remains between 94.4% ± 0.4% and 94.7% ± 0.4%.
- Quantitative comparison: SCAN surpasses the registration-based state-of-the-art method and is competitive with human performance on JSRT.The comparison includes lung fields and the heart, while evaluation schemes differ slightly for most non-registration methods.
- Cross-dataset generalization: On Montgomery, FCN-only both-lung IoU falls to 87.1%, whereas SCAN substantially improves performance over this cross-dataset baseline.The model is trained on JSRT and tested on Montgomery, whose population and imaging conditions differ substantially.
- Cross-dataset generalization: Training on both development sets raises SCAN both-lung IoU to 95.1% ± 0.43% on JSRT and 93.0% ± 1.4% on Montgomery without further tuning.These results significantly improve over training on the JSRT development set alone.
- Qualitative comparison: Qualitatively, SCAN corrects FCN failures caused by contrast and internal structures, producing more natural outlines and sharper costophrenic angles.The costophrenic angle is clinically relevant to diagnosing pleural effusion and lung hyperexpansion.
- Efficiency: At test time, SCAN requires only a forward pass through the segmentation network, avoiding critic inference and the registration method’s training-data search.This supports fast inference in settings such as tuberculosis screening.