Source-linked AI summary
Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks
Patrick Ferdinand Christ, Florian Ettlinger, Felix Grün, Mohamed Ezzeldin A. Elshaera, Jana Lipkova, Sebastian Schlecht, Freba Ahmaddy, Sunil Tatavarty, Marc Bickel, Patrick Bilic, Markus Rempfler, Felix Hofmann, Melvin D Anastasi, Seyed-Ahmad Ahmadi, Georgios Kaissis, Julian Holch, Wieland Sommer, Rickmer Braren, Volker Heinemann, Bjoern Menze
TL;DR
Manual liver-and-lesion segmentation is time-consuming and subjective, while robust automation remains difficult in heterogeneous, low-contrast CT and MRI volumes. This paper uses cascaded FCNs, with 3D CRF post-processing, and reports strong liver segmentation across datasets alongside lesion segmentation.
Problem
Robust automated combined segmentation remains an open problem because liver-lesion contrast, lesion appearance, tissue abnormalities, size, and lesion count vary substantially.
Method
The method cascades FCNs that sequentially segment the liver and lesions, using predicted liver regions for lesion segmentation and dense 3D CRFs to incorporate across-slice information.
Results
Across public, clinical CT, and clinical MRI datasets, liver Dice reached 94.3% after 3D CRF, 91% on one dataset, and 87% in MR-DWI, while lesion Dice reached 69.7% in MRI.
Takeaways & Limitations
Cascaded FCNs with dense 3D CRFs support automatic localization and combined volumetric liver-lesion segmentation, with models released for CT fine-tuning and potential multi-organ extension.
Takeaways & Limitations
3D CRF hyperparameter tuning is time-consuming and task-dependent, and random search may not generalize well to heterogeneous HCC lesions.
Abstract
from arXiv · showhide
Automatic segmentation of the liver and hepatic lesions is an important step towards deriving quantitative biomarkers for accurate clinical diagnosis and computer-aided decision support systems. This paper presents a method to automatically segment liver and lesions in CT and MRI abdomen images using cascaded fully convolutional neural networks (CFCNs) enabling the segmentation of a large-scale medical trial or quantitative image analysis. We train and cascade two FCNs for a combined segmentation of the liver and its lesions. In the first step, we train a FCN to segment the liver as ROI input for a second FCN. The second FCN solely segments lesions within the predicted liver ROIs of step 1. CFCN models were trained on an abdominal CT dataset comprising 100 hepatic tumor volumes. Validations on further datasets show that CFCN-based semantic liver and lesion segmentation achieves Dice scores over 94% for liver with computation times below 100s per volume. We further experimentally demonstrate the robustness of the proposed method on an 38 MRI liver tumor volumes and the public 3DIRCAD dataset.
1. Introduction
Liver and lesion segmentation supports analysis of hepatic disease, but automated combined segmentation remains difficult because CT and MRI volumes contain heterogeneous, low-contrast lesions. The paper proposes cascaded FCNs and evaluates their generalization across datasets and modalities.
- Manual and semi-manual segmentation is subjective, operator-dependent, and time-consuming.
- Automated combined liver-and-lesion segmentation remains open because lesions differ in contrast, tissue abnormality, size, and number.
- Liver and lesion appearance varies in shape, size, and contrast, with substantial intensity overlap producing low overall contrast.
- The paper applies cascaded fully convolutional networks to combined liver and lesion segmentation in low-contrast heterogeneous medical volumes.
- The method is evaluated for generalization and scalability on public, clinical CT, and clinical MRI datasets.
2.1. Overview of our Proposed Segmentation Workflow
The proposed workflow preprocesses medical volumes, performs sequential liver and lesion segmentation with two cascaded FCNs, and refines the resulting probabilities using a dense 3D CRF.
- The workflow has three stages: data preparation, cascaded FCN segmentation, and dense 3D CRF refinement.
- The first FCN segments the liver, and the second segments lesions within the predicted liver region of interest.
- The dense 3D CRF refines CFCN probabilities to produce the final segmentation volume.
2.2. Data Preparation
CT data are prepared slice-wise through intensity windowing and contrast enhancement, while augmentation expands training variation to teach invariance to noise and deformations.
- CT Hounsfield values are windowed to [−100, 400] to exclude irrelevant organs and objects.
- Histogram equalization increases contrast within the liver to improve differentiation of abnormal tissue.
- Elastic deformation, translation, rotation, and Gaussian noise augmentation increase training data and teach desired invariance properties.
2.3. Cascaded Fully Convolutional Neural Networks
The method models background, liver, and lesion labels with cascaded U-Nets, using the liver prediction to restrict lesion segmentation. Class balancing addresses severe lesion-pixel imbalance, while transfer learning and slice-wise processing with 3D CRF support training and volumetric refinement.
- The FCN uses L = {0, 1, 2} for background, liver, and lesion, respectively, and models voxel label probabilities from the image volume.
- A 53% test Dice score was obtained for lesion segmentation with the U-Net on the 3DIRCAD dataset.
- Two U-Nets specialize sequentially: the first segments liver from abdominal CT, and the second segments lesions from the liver region or mask.
- Class balancing weights the cross-entropy loss because lesion pixels typically comprise less than 1% of the data.
- The approach uses pretrained-network transfer learning as an initialization strategy for training on medical image segmentation.
2.4. 3D Conditional Random Field
The method formulates FCN soft predictions as MAP inference in a dense CRF, combining spatial coherence and image appearance to refine voxel labels. Pairwise influence is tuned through learned weights and kernel widths, and mean-field approximation estimates the optimal labelling.
- GPU and VRAM constraints, together with anisotropic medical-volume resolution, limit volumetric 3D FCN training at full resolution.The authors instead exploit locality information across slices within the dataset.
- Source code and trained models are publicly available through the project repository.
- Dense CRF MAP inference refines FCN probability maps by considering spatial coherence and appearance.The CRF is defined on a complete voxel graph, with unary terms from FCN outputs and pairwise terms between voxel labels.
- Pairwise potentials use label disagreement, voxel spatial distance, and original-image intensity difference.Weights wpos and wbil and widths σpos, σbil, and σint control the influence and effective range of these terms.
- Mean-field approximation estimates the minimum-energy labelling, while random search selects CRF weights and kernel settings on the training set.
2.5. Quality Measures
The evaluation uses challenge-established quality measures for binary liver and lesion volumes, with Dice as the main metric and four complementary overlap, volume, and surface-distance measures.
- Dice is the main metric, supplemented by VOE, RVD, ASD, and MSD for binary segmentation volumes.For lesion evaluation, lesions are foreground and all other voxels are background; ground truth and prediction are denoted A and B.
- Dice Score (DICE): Dice measures segmentation overlap and ranges from 0 to 1, with 1 representing a perfect segmentation.
- Volume Overlap Error (VOE): VOE is the complement of the Jaccard coefficient, providing an overlap-error measure.
- Relative Volume Difference (RVD): RVD is an asymmetric metric for comparing the relative volumes of the ground-truth and predicted objects.
- Surface-Distance Measures: ASD evaluates average symmetric distance between object surfaces, while MSD uses the maximum symmetric surface distance instead.MSD is also known as the symmetric Hausdorff distance.
3. Experiments and Results
The experiments evaluate robustness and generalization across public and clinical CT datasets, using class balancing and optional 3D CRF refinement. The method achieves strong liver and lesion segmentation across heterogeneous data.
- Datasets: The evaluation spans a public 3DIRCADb dataset, a heterogeneous clinical CT dataset, and comparisons involving different scanners and centers.The clinical CT dataset contains 100 scans from different patients, while 3DIRCADb includes 20 volumes from various European hospitals and scanners.
- Training: Class balancing improved Dice scores for both liver and lesion segmentation while reducing overfitting.Lesions occupy approximately 0.25% of CT abdomen voxels, compared with about 7% for liver voxels.
- 3DIRCADb results: 93.1% to 94.3% mean liver Dice was achieved after applying the 3D CRF on 3DIRCADb.The improvement was statistically significant, with a reported two-sided paired t-test p-value below 4 · 10^-19.
- 3DIRCADb results: 56% Dice was achieved for lesions on 3DIRCADb using 2-fold cross-validation.The lesion score had a standard deviation of 26%.
- Clinical CT results: 88% and 91% Dice scores were reached by Cascaded FCN and Cascaded FCN + 3D CRF on the clinical CT dataset.The validation-set lesion segmentation achieved 61% ± 25% Dice, compared with a 95% inter-rater Dice overlap from five training cases.
3.3. Clinical Dataset MRI
The method was evaluated on clinical DW-MRI volumes to assess generalization beyond CT. CFCN correctly segmented lesions but undersegmented them in the illustrated cases.
- Dataset: The clinical DW-MRI dataset included 31 patients assessed for primary HCC diagnosis using a 1.5 T MRI scanner and multiple MRI sequences.The protocol included T2-weighted, T1-weighted, contrast-enhanced, and diffusion-weighted imaging.
- Preprocessing and training: The DW-MRI data were N4Bias-corrected before CT-like preprocessing, and the CFCN used the same hardware and training setup.An Adam optimizer with ϵ = 0.1 was used for this experiment.
- Results: 87% liver Dice and 69.7% mean lesion Dice were achieved on MR-DWI with the Cascaded U-Net.In both illustrated cases, lesion undersegmentation produced a Dice score of 85%.
- Application: Automatic liver and lesion segmentation can provide inputs for radiomics algorithms such as the SurvivalNet predictor.The paper presents this as a possible application of the segmentation outputs in medical imaging.
4. Discussion
The discussion emphasizes combined liver-lesion segmentation, clinical scalability, and generalization across modalities. It also identifies 3D CRF tuning as a practical limitation for heterogeneous lesions.
- Combined Segmentation and Clinical Relevance: The framework combines liver and lesion segmentation and generalizes across different medical imaging modalities and multiple organs.Cascading FCNs over smaller subregions is presented as a way to increase segmentation accuracy.
- Architecture: Restricting lesion segmentation to liver pixels significantly boosts the region of interest for lesion prediction.The discussion connects this restriction to cascaded FCN designs and possible future 3D CNN architectures.
- 3D Conditional Random Field: 3D CRF hyperparameter tuning is time-consuming and task-dependent, and random search struggles to find settings that generalize to heterogeneous HCC lesions.The authors point to learned CRF hyperparameters and additional pairwise terms as possible improvements.
5. Conclusion
The conclusion presents cascaded FCNs with dense 3D CRFs as suitable for automatic combined liver-lesion segmentation. The approach is reported to handle heterogeneous CT and DW-MRI volumes in under 100 seconds each.
- Conclusion: Cascaded FCNs and dense 3D CRFs trained on CT volumes are suitable for automatic localization and combined volumetric segmentation of liver and lesions.The authors state that the method competes with state-of-the-art approaches.
- Conclusion: Heterogeneous CT and DW-MRI volumes from different scanners and protocols can be segmented in under 100s each.The conclusion frames CFCNs as promising tools for clinical routine and large-scale clinical trials.
- Conclusion: The trained models are provided under an open-source license for fine-tuning in other CT medical applications.The paper also proposes cascading additional FCNs on lesion regions of interest for future malignancy classification.