Source-linked AI summary
Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields
Patrick Ferdinand Christ, Mohamed Ezzeldin A. Elshaer, Florian Ettlinger, Sunil Tatavarty, Marc Bickel, Patrick Bilic, Markus Rempfler, Marco Armbruster, Felix Hofmann, Melvin D'Anastasi, Wieland H. Sommer, Seyed-Ahmad Ahmadi, Bjoern H. Menze
TL;DR
Liver and lesion segmentation is important for quantitative biomarkers, but heterogeneous low-contrast CT makes combined automatic segmentation difficult. The paper cascades FCNs for liver and lesion prediction and refines them with a dense 3D CRF. On 3DIRCADb, the method reports improved segmentation metrics, including 94.3% mean liver Dice after CRF, with 0.8s per-slice runtime using CRF.
Problem
Combined automatic liver and lesion segmentation remains challenging on heterogeneous, low-contrast CT data, although it is important for quantitative biomarkers and clinical decision support.
Method
The method cascades two FCNs, first segmenting the liver and then lesions within its predicted ROI, and refines the volume using a dense 3D CRF.
Results
94.3% mean liver Dice is reported after 3D dense CRF refinement, with 0.8s runtime per slice using CRF.
Takeaways & Limitations
The framework provides combined automatic liver and lesion segmentation and competes with state-of-the-art methods.
Takeaways & Limitations
3D convolutional FCN implementation is constrained by GPU hardware, available VRAM, and anisotropic medical-volume resolution.
Abstract
from arXiv · showhide
Automatic segmentation of the liver and its lesion is an important step towards deriving quantitative biomarkers for accurate clinical diagnosis and computer-aided decision support systems. This paper presents a method to automatically segment liver and lesions in CT abdomen images using cascaded fully convolutional neural networks (CFCNs) and dense 3D conditional random fields (CRFs). We train and cascade two FCNs for a combined segmentation of the liver and its lesions. In the first step, we train a FCN to segment the liver as ROI input for a second FCN. The second FCN solely segments lesions from the predicted liver ROIs of step 1. We refine the segmentations of the CFCN using a dense 3D CRF that accounts for both spatial coherence and appearance. CFCN models were trained in a 2-fold cross-validation on the abdominal CT dataset 3DIRCAD comprising 15 hepatic tumor volumes. Our results show that CFCN-based semantic liver and lesion segmentation achieves Dice scores over 94% for liver with computation times below 100s per volume. We experimentally demonstrate the robustness of the proposed method as a decision support system with a high accuracy and speed for usage in daily clinical routine.
1 Introduction
Automatic liver and lesion segmentation is clinically relevant but remains difficult on heterogeneous, low-contrast CT data. The paper proposes cascaded FCNs and dense 3D CRFs to address combined automatic segmentation.
- Manual and semi-manual liver and lesion segmentation is subjective, operator-dependent, and time-consuming.
- Combined automatic segmentation is challenged by low liver-lesion contrast, variable tumor intensity, tissue abnormalities, and lesion size and number.
- Existing statistical, image-processing, graph-based, and machine-learning methods were not widely applied clinically because of speed and robustness limitations on heterogeneous CT data.
- CNNs are motivated for this task because they have shown robustness to varying image appearance.
- The paper contributes cascaded FCNs for sequential liver and lesion segmentation and a dense 3D CRF post-processing step for higher accuracy.
2 Methods
The method cascades two slice-wise FCNs to segment liver and then lesions within the predicted liver ROI, followed by dense 3D CRF refinement. It is evaluated on heterogeneous CT volumes using cross-validation and class-balanced training.
- The label set contains background, liver, and lesion classes, represented as 0, 1, and 2, respectively.
- The study uses 15 tumor-containing 3DIRCADb volumes with 2-fold cross-validation, spanning different scanners, contrast levels, lesion sizes, and lesion counts.
- The first UNet segments the liver, whose cropped and resampled prediction becomes the input to a second UNet that segments lesions.
- Cascading is intended to learn hierarchical, task-specific representations while restricting lesion segmentation to the liver region.
- Class weighting is introduced because severe pixel imbalance makes lesion segmentation difficult without balancing.
- The pipeline preprocesses and augments CT slices, trains cascaded FCNs, then refines the predicted volume with a dense 3D CRF.
- Dense 3D CRF inference combines FCN probabilities with spatial coherence and image appearance across all slices.
- The CRF uses a complete voxel graph and mean-field approximation to estimate the minimum-energy labeling.
3 Results and Discussion
The cascaded FCN improves combined liver-lesion segmentation over a single FCN, while dense 3D CRF refinement further improves reported metrics. The approach combines accuracy gains with measured per-slice runtime.
- 94.3% mean Dice for liver versus 93.1% after applying the 3D dense CRF.
- The CFCN achieves higher quantitative segmentation scores than the single FCN architecture.
- 0.4s per slice without CRF and 0.8s with CRF are the reported CFCN runtimes.
- The framework performs combined liver and lesion segmentation in comparison with cited state-of-the-art methods.
4 Conclusion
Cascaded FCNs combined with dense 3D CRFs support automatic localization and volumetric segmentation of liver lesions in heterogeneous CT volumes. The approach segments these volumes in under 100s each and is presented as promising for clinical routine.
- Under 100s each, heterogeneous CT volumes from different scanners and protocols can be segmented with the proposed approach.
- The authors provide trained models under an open-source license for fine-tuning in other medical CT applications.
- Multiple cascaded FCNs could generalize the method beyond liver and lesions to segment multiple organs in medical data.