Automatic 3D liver location and segmentation via convolutional neural networks and graph cut

• Manual liver delineation is subjective, poorly reproducible, and time consuming for clinical applications including transplant planning, radiotherapy, and volume measurement.Source: Introduction · p. 1
• Fully automatic methods are pursued because interactive and semi-automatic approaches require user guidance or extensive interaction.Source: Introduction · pp. 1–2
• Similar intensity distributions with the heart, right kidney, and spleen complicate automatic liver detection.Source: Introduction · p. 2, Figure 1 · p. 2
• Large variation in liver shape and appearance, plus tumors or abnormalities causing intensity inhomogeneity, creates challenging segmentation cases.Source: Introduction · p. 2
• The proposed framework uses 3D CNNs to learn a liver likelihood map and graph cut to refine the initial segmentation without user interaction.Source: Introduction · p. 3, Figure 2 · p. 3

• The broader collected dataset contains 78 contrast-enhanced CT volumes with ground truth in the transversal direction.Source: Training dataset · p. 3
• Among the collected volumes, 26 are abnormal and 52 are normal, with pixel spacing of 0.55–0.81 mm and inter-slice distances of 0.7–3 mm.Source: Training dataset · p. 3
• 40 contrast-enhanced CT volumes comprise the test datasets, including 10 MICCAI-Sliver07 training cases, 10 MICCAI-Sliver07 test cases, and 20 3Dircabd cases.Source: Test dataset · p. 3
• The test volumes have 512×512 in-plane resolution, pixel sizes of 0.54–0.86 mm, slice thicknesses of 0.7–5 mm, and 64–502 slices.Source: Test dataset · p. 3
• The proposed method is organized into 3D CNN-based liver detection followed by 3D graph-cut segmentation refinement.Source: Method · p. 3

• CNNs learn a hierarchy of features by stacking convolutional and subsampling layers, with each convolutional layer extracting feature maps using filters.Source: Introduction of CNNs · p. 3
• Shared convolutional weights reduce memory requirements and improve performance, while pooling reduces computational complexity and introduces invariance properties.Source: Introduction of CNNs · p. 4
• The CNN parameters are trained by minimizing logistic-regression cost over labeled samples using gradient-based optimization and backpropagation, with weight decay for regularization.Source: Introduction of CNNs · p. 4
• The proposed 3D CNN architecture processes 249×249×279 CT blocks through eleven convolutional layers with rectified linear units, pooling, and softmax classification.Source: Architecture of the proposed 3D CNNs · p. 4
• The final log-regression output ranges from 0 to 1 and represents the probability that each voxel in a 248×248×256 block belongs to a class.Source: Architecture of the proposed 3D CNNs · p. 5
• The likelihood maps become more discriminative during training, separating the heart and spleen from the liver after the 13th iteration.Source: Figure 3 · p. 5

• The proposed method combines a CNN liver likelihood map with graph cut to segment liver from surrounding tissue.Source: Graph cut based segmentation refinement · pp. 5–6
• The graph-cut model assigns binary voxel labels and minimizes an energy balancing data fitting against boundary penalties.The labels represent background and liver regions, while neighboring-voxel discontinuities contribute to the boundary term.Source: Graph cut based segmentation refinement · p. 5
• The boundary term encodes discontinuity between neighboring voxels, while graph construction uses n-links between voxels and t-links connecting voxels to source or sink terminals.Source: Graph cut based segmentation refinement · pp. 5–6
• The data term integrates the CNN probability map, a thresholding map, and a local appearance map weighted by a positive trade-off coefficient.The local appearance representation uses intensity, modified local binary pattern, and local intensity variance features.Source: Graph cut based segmentation refinement · pp. 5–6
• Unlike the user-initialized RAP model, this approach automatically generates a large initial region and incorporates 3D CNN information into the data penalty.The authors describe this integration as addressing RAP’s lack of global information and difficulty capturing complex texture features.Source: Graph cut based segmentation refinement · p. 6
• Figure 5 compares proposed segmentation contours with ground truth in axial and coronal views and displays surface-distance error in millimeters.Source: Figure 5 · p. 6, Figure 6 · p. 6

Segmentation Procedures

• The proposed segmentation process contains three stages: preprocessing, initial liver-region localization, and segmentation refinement.Source: Segmentation Procedures · pp. 6–7
• Figure 7 displays axial, sagittal, and coronal results for four challenging cases, distinguishing ground truth, CNN initialization, and refined segmentation by color.Ground truth is blue, the initial CNN region is yellow, and the final refined result is red.Source: Figure 7 · p. 7
• Figure 8 presents 3D visualizations of the same cases, including surface-distance errors for 3D CNN and proposed segmentations against ground truth.Source: Figure 8 · p. 7

Preprocessing

• The method relies on CNNs learning useful features from scratch and therefore uses only minimal preprocessing.Source: Preprocessing · p. 7
• All volumes are resampled to 256×256×286 after slices without liver are appended or deleted to reduce computational complexity.Source: Preprocessing · p. 8
• Intensity values are normalized to [-128,128] by adjusting window width and window level.Source: Preprocessing · p. 8
• A 3D anisotropic diffusion filter reduces noise, and all preprocessing steps are applied to both training and test datasets.Source: Preprocessing · p. 8

Location of the initial liver region

• The CNN is trained for 53 iterations to generate the liver likelihood map.Source: Location of the initial liver region · p. 8
• Around the 13th iteration, the network differentiates the heart and spleen from the liver, and validation results converge at around the 40th iteration.Source: Location of the initial liver region · p. 8
• Training the network takes approximately 20 hours using four GTX980 GPUs.Source: Location of the initial liver region · p. 8
• After training, the probability map is iteratively learned for a test volume and thresholded to obtain the initial liver shape L0.Source: Location of the initial liver region · p. 8

• The liver probability map initializes graph cut and enters its energy function to refine the initial segmentation.Source: Segmentation refinement · p. 8
• The initial liver shape estimates the intensity range as [m − 3σ, m + 3.5σ].Here, m and σ are the intensity mean and variance over the initial liver shape.Source: Segmentation refinement · p. 8
• Graph-cut settings include λ = 70, β = 0.2, a 9×9×5 local window, and LBP parameters τ = 1.5, P = 6, r = 1.Source: Segmentation refinement · p. 8
• For 512 × 512 × n test volumes, 3D CNN likelihood-map generation takes about 4 seconds, while graph-cut segmentation takes 20–180 seconds.Source: Segmentation refinement · p. 9

• Five metrics quantify accuracy: VOE, RVD, ASD, RMSD, and MSD.RVD is signed to indicate under-segmentation or over-segmentation.Source: Evaluation metrics · p. 9
• A perfect result scores zero on all five metrics, while average-quality non-expert manual segmentation is represented by 6.4%, 4.7%, 1 mm, 1.8 mm, and 19 mm.The study treats this manual performance as approximately equivalent to human performance.Source: Evaluation metrics · p. 9
• Liver volumes are compared through least-squares correlation and Bland–Altman analysis between automatic and manual segmentations.The analysis also computes the correlation coefficient R and coefficient of variation for observer variability.Source: Evaluation metrics · p. 9

Results and discussion

• Refinement using the learned liver likelihood map achieves better agreement with ground truth than graph cut without it or 3D CNNs alone.Source: Results and discussion · p. 9
• In four challenging cases, proposed-model MSD errors are 17.0 mm, 11.2 mm, 22.1 mm, and 15.3 mm, versus 3D-CNN errors of 22.1 mm, 12.6 mm, 62.6 mm, and 74.5 mm.Source: Results and discussion · p. 9

MICCAI-Sliver07

• 5.9%, 2.7%, 0.91%, 1.88 mm, and 18.94 mm are the MICCAI-Sliver07 mean VOE, RVD, ASD, RMSD, and MSD results.Source: Results and discussion · p. 9
• 77.8 is the proposed method’s MICCAI-Sliver07 mean score, exceeding Kainmüller 77.3, Wimmer 76.8, Linguraru 76.2, Heimann 67.6, and Kinda 64.1.Source: Results and discussion · p. 9

3Dircadb

• On 3Dircadb, the proposed method performs much better than Chung’s method except for MSD and is slightly better than Kirschner’s and Erdt’s on most measures.Li’s method reports slightly better results than the proposed method.Source: Results and discussion · p. 10

Liver-volume analysis

• The automatic and manual liver volumes have correlation 0.968, mean difference 57.2 mL, confidence interval −266.4 mL to 380.7 mL, and CV 2.89.Source: Results and discussion · p. 10

Limitations

• Large surface distances occur occasionally at liver–vessel connections, and failure cases include strong rotation and difficult left-lobe intensity similarity with surrounding organs.The rotation case is attributed to CNNs not being rotationally invariant; the left-lobe case is under-segmented.Source: Results and discussion · pp. 10–11

• A generative 3D CNN detects the liver and produces a probability map for initial segmentation, which graph cut then refines.Source: Conclusion · p. 11
• The framework requires no user interaction for initialization, allowing the method to be performed by non-experts.Source: Conclusion · p. 11
• Evaluation on MICCAI-Sliver07 and 3Dircadb showed superior segmentation accuracy compared with state-of-the-art automatic methods.Source: Conclusion · p. 11
• High correlation with manual references indicates clinical applicability for hepatic volume estimation.Source: Conclusion · p. 11