Source-linked AI summary
Automatic 3D liver location and segmentation via convolutional neural networks and graph cut
Fang Lu, Fa Wu, Peijun Hu, Zhiyi Peng, Dexing Kong
TL;DR
Liver segmentation from CT is important for clinical interventions but manual delineation is time-consuming and poorly reproducible. The paper combines 3D CNN-based detection and probabilistic segmentation with graph-cut refinement, achieving reported accuracy across two public datasets and high agreement with manual liver volumes.
Problem
Manual CT liver segmentation is subjective, poorly reproducible, and time consuming despite its role in transplant planning, radiotherapy, and volume measurement.
Method
The method learns a liver likelihood map with generative 3D CNNs, thresholds it for initialization, and incorporates it into graph cut for segmentation refinement.
Results
5.9% VOE, 2.7% RVD, 0.91% ASD, 1.88 mm RMSD, and 18.94 mm MSD were obtained on the MICCAI-Sliver07 test set.
Takeaways & Limitations
The fully automatic approach requires no user interaction for initialization and is reported as applicable to automatic hepatic volume evaluation.
Takeaways & Limitations
Performance is limited by large surface distances near liver-vessel connections, rotationally non-invariant CNNs, and difficult left-lobe boundaries with similar surrounding intensities.
Abstract
from arXiv · showhide
Purpose Segmentation of the liver from abdominal computed tomography (CT) image is an essential step in some computer assisted clinical interventions, such as surgery planning for living donor liver transplant (LDLT), radiotherapy and volume measurement. In this work, we develop a deep learning algorithm with graph cut refinement to automatically segment liver in CT scans. Methods The proposed method consists of two main steps: (i) simultaneously liver detection and probabilistic segmentation using 3D convolutional neural networks (CNNs); (ii) accuracy refinement of initial segmentation with graph cut and the previously learned probability map. Results The proposed approach was validated on forty CT volumes taken from two public databases MICCAI-Sliver07 and 3Dircadb. For the MICCAI-Sliver07 test set, the calculated mean ratios of volumetric overlap error (VOE), relative volume difference (RVD), average symmetric surface distance (ASD), root mean square symmetric surface distance (RMSD) and maximum symmetric surface distance (MSD) are 5.9%, 2.7%, 0.91%, 1.88 mm, and 18.94 mm, respectively. In the case of 20 3Dircadb data, the calculated mean ratios of VOE, RVD, ASD, RMSD and MSD are 9.36%, 0.97%, 1.89%, 4.15 mm and 33.14 mm, respectively. Conclusion The proposed method is fully automatic without any user interaction. Quantitative results reveal that the proposed approach is efficient and accurate for hepatic volume estimation in a clinical setup. The high correlation between the automatic and manual references shows that the proposed method can be good enough to replace the time-consuming and non-reproducible manual segmentation method.
Introduction
Automatic liver segmentation from CT is clinically useful but difficult because manual delineation is time-consuming and variable, while liver appearance overlaps surrounding organs and varies across patients. The paper proposes a fully automatic framework combining 3D CNN-based detection with graph-cut refinement.
- Manual liver delineation is subjective, poorly reproducible, and time consuming for clinical applications including transplant planning, radiotherapy, and volume measurement.
- Fully automatic methods are pursued because interactive and semi-automatic approaches require user guidance or extensive interaction.
- Similar intensity distributions with the heart, right kidney, and spleen complicate automatic liver detection.
- Large variation in liver shape and appearance, plus tumors or abnormalities causing intensity inhomogeneity, creates challenging segmentation cases.
- The proposed framework uses 3D CNNs to learn a liver likelihood map and graph cut to refine the initial segmentation without user interaction.
Datasets
The study evaluates the proposed method on 40 contrast-enhanced CT volumes drawn from MICCAI-Sliver07 and 3Dircabd, with varied acquisition dimensions and spacings.
- The broader collected dataset contains 78 contrast-enhanced CT volumes with ground truth in the transversal direction.
- Among the collected volumes, 26 are abnormal and 52 are normal, with pixel spacing of 0.55–0.81 mm and inter-slice distances of 0.7–3 mm.
- 40 contrast-enhanced CT volumes comprise the test datasets, including 10 MICCAI-Sliver07 training cases, 10 MICCAI-Sliver07 test cases, and 20 3Dircabd cases.
- The test volumes have 512×512 in-plane resolution, pixel sizes of 0.54–0.86 mm, slice thicknesses of 0.7–5 mm, and 64–502 slices.
- The proposed method is organized into 3D CNN-based liver detection followed by 3D graph-cut segmentation refinement.
Introduction of CNNs
The CNN section describes convolutional networks as hierarchical feature learners whose 3D architecture processes CT blocks and outputs voxelwise liver probabilities. Training uses logistic regression, regularization, and gradient-based optimization.
- CNNs learn a hierarchy of features by stacking convolutional and subsampling layers, with each convolutional layer extracting feature maps using filters.
- Shared convolutional weights reduce memory requirements and improve performance, while pooling reduces computational complexity and introduces invariance properties.
- The CNN parameters are trained by minimizing logistic-regression cost over labeled samples using gradient-based optimization and backpropagation, with weight decay for regularization.
- The proposed 3D CNN architecture processes 249×249×279 CT blocks through eleven convolutional layers with rectified linear units, pooling, and softmax classification.
- The final log-regression output ranges from 0 to 1 and represents the probability that each voxel in a 248×248×256 block belongs to a class.
- The likelihood maps become more discriminative during training, separating the heart and spleen from the liver after the 13th iteration.
Graph cut based segmentation refinement
The method refines a CNN-generated liver region using graph cut, combining learned likelihood, thresholding, and local appearance information in an energy-minimization framework.
- The proposed method combines a CNN liver likelihood map with graph cut to segment liver from surrounding tissue.
- The graph-cut model assigns binary voxel labels and minimizes an energy balancing data fitting against boundary penalties.The labels represent background and liver regions, while neighboring-voxel discontinuities contribute to the boundary term.
- The boundary term encodes discontinuity between neighboring voxels, while graph construction uses n-links between voxels and t-links connecting voxels to source or sink terminals.
- The data term integrates the CNN probability map, a thresholding map, and a local appearance map weighted by a positive trade-off coefficient.The local appearance representation uses intensity, modified local binary pattern, and local intensity variance features.
- Unlike the user-initialized RAP model, this approach automatically generates a large initial region and incorporates 3D CNN information into the data penalty.The authors describe this integration as addressing RAP’s lack of global information and difficulty capturing complex texture features.
- Figure 5 compares proposed segmentation contours with ground truth in axial and coronal views and displays surface-distance error in millimeters.
Segmentation Procedures
The segmentation procedure comprises preprocessing, initial liver-region localization, and refinement, with visualizations showing initial and final results across challenging cases.
- Segmentation Procedures: The proposed segmentation process contains three stages: preprocessing, initial liver-region localization, and segmentation refinement.
- Segmentation Procedures: Figure 7 displays axial, sagittal, and coronal results for four challenging cases, distinguishing ground truth, CNN initialization, and refined segmentation by color.Ground truth is blue, the initial CNN region is yellow, and the final refined result is red.
- Segmentation Procedures: Figure 8 presents 3D visualizations of the same cases, including surface-distance errors for 3D CNN and proposed segmentations against ground truth.
Preprocessing
Preprocessing applies minimal volume standardization, intensity normalization, and anisotropic diffusion filtering before training and testing.
- Preprocessing: The method relies on CNNs learning useful features from scratch and therefore uses only minimal preprocessing.
- Preprocessing: All volumes are resampled to 256×256×286 after slices without liver are appended or deleted to reduce computational complexity.
- Preprocessing: Intensity values are normalized to [-128,128] by adjusting window width and window level.
- Preprocessing: A 3D anisotropic diffusion filter reduces noise, and all preprocessing steps are applied to both training and test datasets.
Location of the initial liver region
The trained 3D CNN iteratively learns a liver probability map, which is thresholded to locate an initial liver shape before refinement.
- Location of the initial liver region: The CNN is trained for 53 iterations to generate the liver likelihood map.
- Location of the initial liver region: Around the 13th iteration, the network differentiates the heart and spleen from the liver, and validation results converge at around the 40th iteration.
- Location of the initial liver region: Training the network takes approximately 20 hours using four GTX980 GPUs.
- Location of the initial liver region: After training, the probability map is iteratively learned for a test volume and thresholded to obtain the initial liver shape L0.
Segmentation refinement
The method uses a learned liver probability map to initialize and guide graph-cut refinement, with reported implementation settings and runtimes.
- The liver probability map initializes graph cut and enters its energy function to refine the initial segmentation.
- The initial liver shape estimates the intensity range as [m − 3σ, m + 3.5σ].Here, m and σ are the intensity mean and variance over the initial liver shape.
- Graph-cut settings include λ = 70, β = 0.2, a 9×9×5 local window, and LBP parameters τ = 1.5, P = 6, r = 1.
- For 512 × 512 × n test volumes, 3D CNN likelihood-map generation takes about 4 seconds, while graph-cut segmentation takes 20–180 seconds.
Experiments and discussion
The study evaluates segmentation with five geometric accuracy measures and assesses liver-volume agreement between automatic and manual delineations.
- Five metrics quantify accuracy: VOE, RVD, ASD, RMSD, and MSD.RVD is signed to indicate under-segmentation or over-segmentation.
- A perfect result scores zero on all five metrics, while average-quality non-expert manual segmentation is represented by 6.4%, 4.7%, 1 mm, 1.8 mm, and 19 mm.The study treats this manual performance as approximately equivalent to human performance.
- Liver volumes are compared through least-squares correlation and Bland–Altman analysis between automatic and manual segmentations.The analysis also computes the correlation coefficient R and coefficient of variation for observer variability.
Results and discussion
Across typical and challenging cases, graph-cut refinement improves agreement with expert delineations, while benchmark results show strong segmentation and liver-volume agreement alongside documented failure cases.
- Results and discussion: Refinement using the learned liver likelihood map achieves better agreement with ground truth than graph cut without it or 3D CNNs alone.
- Results and discussion: In four challenging cases, proposed-model MSD errors are 17.0 mm, 11.2 mm, 22.1 mm, and 15.3 mm, versus 3D-CNN errors of 22.1 mm, 12.6 mm, 62.6 mm, and 74.5 mm.
- MICCAI-Sliver07: 5.9%, 2.7%, 0.91%, 1.88 mm, and 18.94 mm are the MICCAI-Sliver07 mean VOE, RVD, ASD, RMSD, and MSD results.
- MICCAI-Sliver07: 77.8 is the proposed method’s MICCAI-Sliver07 mean score, exceeding Kainmüller 77.3, Wimmer 76.8, Linguraru 76.2, Heimann 67.6, and Kinda 64.1.
- 3Dircadb: On 3Dircadb, the proposed method performs much better than Chung’s method except for MSD and is slightly better than Kirschner’s and Erdt’s on most measures.Li’s method reports slightly better results than the proposed method.
- Liver-volume analysis: The automatic and manual liver volumes have correlation 0.968, mean difference 57.2 mL, confidence interval −266.4 mL to 380.7 mL, and CV 2.89.
- Limitations: Large surface distances occur occasionally at liver–vessel connections, and failure cases include strong rotation and difficult left-lobe intensity similarity with surrounding organs.The rotation case is attributed to CNNs not being rotationally invariant; the left-lobe case is under-segmented.
Conclusion
The paper presents an automatic 3D-CNN and graph-cut framework for liver segmentation without user initialization and reports clinical applicability for hepatic volume estimation.
- A generative 3D CNN detects the liver and produces a probability map for initial segmentation, which graph cut then refines.
- The framework requires no user interaction for initialization, allowing the method to be performed by non-experts.
- Evaluation on MICCAI-Sliver07 and 3Dircadb showed superior segmentation accuracy compared with state-of-the-art automatic methods.
- High correlation with manual references indicates clinical applicability for hepatic volume estimation.