Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images

Deng-Ping Fan; Tao Zhou; Ge-Peng Ji; Yi Zhou; Geng Chen; Huazhu Fu; Jianbing Shen; Ling Shao

Source-linked AI summary

Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images

Deng-Ping Fan, Tao Zhou, Ge-Peng Ji, Yi Zhou, Geng Chen, Huazhu Fu, Jianbing Shen, Ling Shao

arXiv:2004.14133v4Apr 22, 2020eess.IVcs.CVcs.LG

TL;DR

COVID-19 infection segmentation from CT is challenged by variable infection appearance, low contrast, and limited labeled data. The paper proposes Inf-Net with coarse-to-fine area and boundary modeling, plus Semi-Inf-Net for unlabeled-data utilization; experiments report improved segmentation performance, while multi-class labeling remains partly two-step and the system focuses on segmentation rather than integrated diagnosis.

Problem
COVID-19 CT infection segmentation is difficult because infection characteristics vary, contrast with normal tissue is low, and collecting many labeled images is impractical.
Method
Inf-Net aggregates high-level features for coarse areas, refines boundaries with reverse and edge attention, and extends to semi-supervised learning with unlabeled data.
Results
Inf-Net and Semi-Inf-Net outperform most cutting-edge segmentation models on COVID-SemiSeg and real CT volumes, with semi-supervised learning improving Dice by 5.7%.
Takeaways & Limitations
The system has potential for assessing COVID-19 diagnosis through infection quantification, longitudinal monitoring, and mass screening.
Takeaways & Limitations
The multi-class infection-labeling framework uses a two-step strategy that could lead to sub-optimal learning performance.

Abstract

from arXiv · show

Coronavirus Disease 2019 (COVID-19) spread globally in early 2020, causing the world to face an existential health crisis. Automated detection of lung infections from computed tomography (CT) images offers a great potential to augment the traditional healthcare strategy for tackling COVID-19. However, segmenting infected regions from CT slices faces several challenges, including high variation in infection characteristics, and low intensity contrast between infections and normal tissues. Further, collecting a large amount of data is impractical within a short time period, inhibiting the training of a deep model. To address these challenges, a novel COVID-19 Lung Infection Segmentation Deep Network (Inf-Net) is proposed to automatically identify infected regions from chest CT slices. In our Inf-Net, a parallel partial decoder is used to aggregate the high-level features and generate a global map. Then, the implicit reverse attention and explicit edge-attention are utilized to model the boundaries and enhance the representations. Moreover, to alleviate the shortage of labeled data, we present a semi-supervised segmentation framework based on a randomly selected propagation strategy, which only requires a few labeled images and leverages primarily unlabeled data. Our semi-supervised framework can improve the learning ability and achieve a higher performance. Extensive experiments on our COVID-SemiSeg and real CT volumes demonstrate that the proposed Inf-Net outperforms most cutting-edge segmentation models and advances the state-of-the-art performance.

I. INTRODUCTION

The paper motivates automated COVID-19 infection segmentation from CT images because screening and manual annotation are constrained, then proposes Inf-Net and a semi-supervised extension to address these challenges.

CT imaging can support COVID-19 screening, but RT-PCR has equipment, environment, and false-negative limitations.
Manual infection delineation is time-consuming, subjective, and affected by radiologists’ individual bias and clinical experience.
Inf-Net first estimates coarse infection areas, then models boundaries through reverse attention and edge-attention guidance.
The proposed semi-supervised system uses randomly selected propagation to leverage unlabeled CT images when labeled infection data are scarce.
COVID-SemiSeg contains 100 labeled CT slices and 1600 unlabeled images for semi-supervised infection-segmentation experiments.

II. RELATED WORKS

Related work covers chest-CT segmentation, semi-supervised learning for limited annotations, and artificial intelligence applications for COVID-19 imaging and diagnosis.

A. Segmentation in Chest CT: Chest-CT research has used classical classifiers and deep learning methods to segment nodules, organs, and lesions despite visual similarity between abnormalities and background.
B. Annotation-Efficient Deep Learning: For COVID-19 infection segmentation, semi-supervised models can distinguish target regions from other anomalies using few labeled examples, while pixel-level annotations remain expensive.
B. Annotation-Efficient Deep Learning: Semi-supervised learning combines limited labeled data with abundant unlabeled data to improve model performance, including in medical segmentation.
C. Artificial Intelligence for COVID-19: Prior COVID-19 AI applications include radiological diagnosis, infection-probability estimation, anomaly detection, and classification from CT or chest radiography.

III. PROPOSED METHOD

Inf-Net combines a parallel partial decoder for coarse global localization with reverse-attention and edge-attention mechanisms for progressively refining infection boundaries. Its training uses edge-aware segmentation losses and deep supervision across multiple outputs.

A. Lung Infection Segmentation Network (Inf-Net): The edge-attention module uses low-level features to explicitly learn representations of objective-region boundaries.The predicted edge map is supervised against an edge ground truth derived from the segmentation ground truth.
A. Lung Infection Segmentation Network (Inf-Net): Inf-Net aggregates high-level features with a parallel partial decoder to generate a global infection map.The decoder avoids aggregating low-level features and supplies rough localization for subsequent refinement.
A. Lung Infection Segmentation Network (Inf-Net): Three reverse-attention modules progressively refine coarse predictions by erasing already detected regions and emphasizing remaining infection features.The reverse-attention weights are formed using upsampling, sigmoid activation, and reversal operations.
A. Lung Infection Segmentation Network (Inf-Net): The segmentation loss combines weighted IoU and weighted binary cross-entropy terms to provide global and pixel-level supervision.Both weighted losses emphasize difficult pixels, and λ is set to 1 in the experiments.
A. Lung Infection Segmentation Network (Inf-Net): Deep supervision is applied to three side outputs and the global map during training.The supervised outputs are S3, S4, S5, and Sg.

B. Semi-Supervised Inf-Net

Semi-Inf-Net addresses the scarcity of annotated CT images by progressively enlarging the training set with pseudo-labeled images selected from unlabeled data. The strategy is described as simple, threshold-free, and more robust than other semi-supervised methods.

B. Semi-Supervised Inf-Net: Semi-Inf-Net augments limited labeled training data by randomly selecting unlabeled CT images, generating pseudo labels, and adding them to the training set.The model is repeatedly fine-tuned after pseudo-labeled images are added.
B. Semi-Supervised Inf-Net: The framework requires no predicted-label assessment measures and is threshold-free.These properties are presented as advantages of the training and selection strategy.
B. Semi-Supervised Inf-Net: The authors report that the strategy provides more robust performance than other semi-supervised learning methods and prevents over-fitting.The passage attributes this conclusion to recently released studies.

C. Extension to Multi-Class Infection Labeling

The paper extends Semi-Inf-Net from overall infection segmentation to multi-class labeling of infection types such as GGO and consolidation. It guides this labeling with infection segmentation results alongside the original CT images.

C. Extension to Multi-Class Infection Labeling: The extension targets quantitative evaluation of different lung infection types, including GGO and consolidation.This addresses the additional information clinicians may seek beyond overall infection evaluation.
C. Extension to Multi-Class Infection Labeling: The multi-class framework can use FCN8s or a multi-class U-Net as the labeling backbone.The figure caption describes both architectures as alternatives for improving multi-class infection-labeling accuracy.
C. Extension to Multi-Class Infection Labeling: The infection-region-guided framework feeds Semi-Inf-Net segmentation results and corresponding CT images into a multi-class labeling network.The approach is intended to provide richer information for further diagnosis and treatment of COVID-19.

D. Implementation Details

The implementation uses PyTorch and GPU acceleration, with specified preprocessing, optimization, and staged training procedures. Pseudo-label generation for the unlabeled set uses five randomly selected images per iteration.

D. Implementation Details: The model is implemented in PyTorch and accelerated with an NVIDIA TITAN RTX GPU.These details describe the reported implementation environment.
D. Implementation Details: Pseudo-label generation sets K = 5 randomly selected CT images per iteration and requires 320 iterations for 1600 unlabeled images with batch size 16.The complete pseudo-label generation procedure takes about 50 hours.
D. Implementation Details: Semi-Inf-Net inputs are resized to 352×352 and trained with multi-scale resampling ratios of 0.75, 1, and 1.25.The reported optimizer is Adam with learning rate 1e−4.
D. Implementation Details: The multi-class segmentation network uses 512 × 512 inputs, Xavier initialization, and SGD optimization.The reported learning rate is 1e −10, with weight decay 5e −4 and momentum 0.99.
D. Implementation Details: The multi-class training procedure takes about 45 minutes to complete.This duration is reported separately from the pseudo-label generation procedure.

IV. EXPERIMENTS

Experiments use the limited COVID-19 CT Segmentation data, a newly collected semi-supervised dataset, established segmentation baselines, and multiple evaluation metrics.

Dataset: The COVID-19 CT Segmentation dataset contains 100 axial CT images, highlighting the limited labeled data available for lung infection segmentation.The images were collected by the Italian Society of Medical and Interventional Radiology and labeled by a radiologist.
Dataset: COVID-SemiSeg uses 45 labeled CT images for training, 5 for validation, 50 for testing, and unlabeled CT images from the COVID-19 CT Collection.The dataset was designed to leverage large-scale unlabeled CT images for augmenting training data.
Baselines: Infection-region experiments compare Inf-Net and Semi-Inf-Net with U-Net, U-Net++, Attention-UNet, Gated-UNet, and Dense-UNet.Multi-class experiments additionally compare against DeepLabV3+, FCN8s, and multi-class U-Net.
Evaluation Metrics: Evaluation uses Dice, Sensitivity, Specificity, Precision, Structure Measure, Enhanced-alignment Measure, and Mean Absolute Error.The final prediction map is compared with object-level segmentation ground truth.

1) Structure Measure (Sα):

Structure Measure evaluates similarity between a prediction map and ground-truth mask using object-aware and region-aware components.

Structure Measure: Structure Measure is designed to assess structural similarity between a prediction map and ground-truth mask in a way aligned with human visual perception.Its formulation combines object-aware and region-aware similarity.
Structure Measure: Sα balances object-aware similarity So and region-aware similarity Sr, with experiments using α = 0.5.The default setting is taken from the original Structure Measure paper.
Enhanced-alignment Measure: Enhanced-alignment Measure evaluates local and global similarity between two binary maps.It is described as a recently proposed metric for this purpose.
Mean Absolute Error: Mean Absolute Error measures pixelwise error between the prediction Sp and ground-truth G.The metric averages absolute differences over image pixels.

1) Quantitative Results:

Inf-Net and Semi-Inf-Net outperform the compared baselines on infection-region and multi-class segmentation, with semi-supervision improving quantitative and boundary results.

Quantitative Results: Inf-Net outperforms U-Net and U-Net++ on Dice, Sα, Emeanφ, and MAE, while semi-supervised learning adds a 5.7% Dice improvement.The authors attribute the improvement to reverse-attention and edge-attention modeling, together with semi-supervised learning.
Quantitative Results: On multi-class segmentation, Semi-Inf-Net & MC achieves competitive GGO results and the best consolidation results across the evaluated methods.The pipeline performs better overall on consolidation and average segmentation results in Dice and Sα.
Quantitative Results: 12%: Semi-Inf-Net & MC outperforms multi-class U-Net by this average Dice margin on consolidation segmentation.The reported comparison concerns the average segmentation result for the more challenging consolidation class.
Qualitative Results: Visual results show Inf-Net and Semi-Inf-Net producing masks closer to ground truth with less mis-segmented tissue than the baseline methods.U-Net is described as producing many mis-segmented tissues, while U-Net++ improves but remains unsatisfactory.
Qualitative Results: Semi-Inf-Net produces more accurate boundaries than Inf-Net, especially where Inf-Net has fuzzy boundaries in subtle infection regions.The qualitative comparison identifies boundary accuracy as an advantage of the semi-supervised model.
Qualitative Results: Semi-Inf-Net & MC consistently performs best in the multi-class visual comparison, accurately segmenting both GGO and consolidation infections.DeepLabV3+ variants and FCNs are described as unable to accurately segment either infection type.

D. Ablation Study

Ablation experiments assess PPD, reverse attention, and edge-attention components, identifying PPD and reverse attention as central contributors to performance.

Ablation Setup: The ablation study evaluates the contributions of the PPD, RA, and EA modules in Semi-Inf-Net.These experiments are intended to validate each key component.
PPD: Adding PPD to the backbone boosts performance, indicating that the parallel partial decoder is necessary for improvement.The comparison uses backbone-only and backbone-plus-PPD baselines.
Reverse Attention: Adding RA to the backbone improves major metrics including Dice, Sensitivity, and MAE.The authors state that RA helps the model distinguish true infected areas.
Component Interactions: PPD and RA are identified as the two central components responsible for Inf-Net's good performance.The relevant configuration performs better than the other tested settings on most metrics.

4) Effectiveness of EA:

The explicit edge-attention module improves Inf-Net’s segmentation performance. The method is also evaluated on real CT volumes, while its current scope and performance have practical boundaries.

Effectiveness of EA: EA effectively improves Inf-Net’s segmentation performance.The comparison is reported across the indicated ablation results in Table IV.
Real CT-volume evaluation: The real-volume evaluation uses 638 slices from nine CT volumes, including 285 non-infected and 353 infected slices.These data are described as the test set for evaluating model performance.
Limitations and future work: Inf-Net currently focuses on segmenting lung infections rather than integrating COVID-19 classification, segmentation, and infection quantification.The authors identify an integrated automatic diagnosis system as future work.
Limitations and future work: The method may lose some accuracy on non-infected slices, motivating an additional slice-wise classifier to select infected slices.The proposed solution is an infected-versus-non-infected classifier before segmentation.