Source-linked AI summary
Modified U-Net (mU-Net) with Incorporation of Object-Dependent High Level Features for Improved Liver and Liver-Tumor Segmentation in CT Images
Hyunseok Seo, Charles Huang, Maxime Bassenne, Ruoxiu Xiao, Lei Xing
TL;DR
Liver and liver-tumor segmentation in CT is difficult because conventional U-Net designs duplicate low-resolution information and inadequately capture boundaries and small objects. The proposed mU-Net adds residual-path and object-dependent feature extraction mechanisms, achieving accurate segmentation, particularly for fuzzy boundaries and small targets.
Problem
Conventional U-Net segmentation can duplicate low-resolution information and insufficiently extract high-resolution edge and small-object features.
Method
The mU-Net adds a residual path with object-dependent upsampling and convolution layers in skip connections to extract edge and global features.
Results
The mU-Net provides accurate liver and liver-tumor segmentation, with particularly strong results for thick-slice images with unclear boundaries.
Takeaways & Limitations
The network is particularly beneficial for segmenting objects with fuzzy boundaries and small targets, without requiring preprocessing.
Takeaways & Limitations
The approach is computationally more intensive and may have limited generalizability; its MSE loss may not adequately capture structure-similarity information.
Abstract
from arXiv · showhide
Segmentation of livers and liver tumors is one of the most important steps in radiation therapy of hepatocellular carcinoma. The segmentation task is often done manually, making it tedious, labor intensive, and subject to intra-/inter- operator variations. While various algorithms for delineating organ-at-risks (OARs) and tumor targets have been proposed, automatic segmentation of livers and liver tumors remains intractable due to their low tissue contrast with respect to the surrounding organs and their deformable shape in CT images. The U-Net has gained increasing popularity recently for image analysis tasks and has shown promising results. Conventional U-Net architectures, however, suffer from three major drawbacks. To cope with these problems, we added a residual path with deconvolution and activation operations to the skip connection of the U-Net to avoid duplication of low resolution information of features. In the case of small object inputs, features in the skip connection are not incorporated with features in the residual path. Furthermore, the proposed architecture has additional convolution layers in the skip connection in order to extract high level global features of small object inputs as well as high level features of high resolution edge information of large object inputs. Efficacy of the modified U-Net (mU-Net) was demonstrated using the public dataset of Liver tumor segmentation (LiTS) challenge 2017. The proposed mU-Net outperformed existing state-of-art networks.
A. Backgrounds with mathematical preliminaries
The paper reviews CNN and U-Net fundamentals, including convolution, activation, pooling, decoding, skip connections, and loss minimization. It introduces the mU-Net schematic as a response to information loss and duplication in conventional U-Net processing.
- Mathematical preliminaries: The feed-forward network maps input I through encoding and decoding layers parameterized by θ to produce the output C(I; θ).The decoding stage uses convolutional layers and up pooling or up sampling to recover matrix size.
- Mathematical preliminaries: Training iteratively updates θ by backpropagation and gradient descent to minimize a predefined loss comparing estimated and labeled outputs.The labeled signal is denoted O, and G measures the error between prediction and desired output.
- U-Net architecture: U-Net preserves high-resolution information by connecting encoder features to corresponding decoder layers through skip connections.These connections address spatial-information loss caused by pooling.
- mU-Net architecture: The mU-Net schematic uses object-dependent residual-path processing to preserve small-object information, restrict large-object skip features to edges, and adaptively combine paths.Additional convolution layers extract higher-level global features for small inputs and higher-level edge features for large inputs.
B. Limitation of the skip connection in the U-Net and pooling
The conventional U-Net skip connection can duplicate low-resolution information while inadequately processing high-resolution edges. Pooling can also remove spatial information from small objects, creating a mismatch between feature extraction and object scale.
- Skip-connection limitations: Conventional U-Net skip connections transfer full feature information, duplicating low-resolution content already propagated through deeper pooled layers.The duplicated low-resolution information may smooth object boundaries, especially when boundaries are fuzzy.
- Pooling limitations: Pooling can erase small-object spatial information, while the number of pooling operations suitable for large objects may be excessive for small objects.The paper illustrates this scale dependence using stages defined by feature-matrix size.
- Motivation for mU-Net: The proposed mU-Net addresses these issues by using an object-dependent residual path and additional skip-connection convolutions.This design controls how features are combined according to object size.
- Skip-connection limitations: High-resolution edge information bypasses sufficient convolutional processing, so high-level features are extracted disproportionately from low-resolution information.The imbalance limits the network’s use of detailed boundary information.
C. Proposed mU-Net architecture
The mU-Net redesigns U-Net skip connections with object-dependent residual processing and additional convolutions. It preserves small-object features while extracting edge information for large objects and higher-level features overall.
- The residual path adaptively filters and combines features with the skip connection to avoid duplicating low-resolution information.
- The architecture uses object-dependent upsampling, with pooling and deconvolution behavior adjusted to preserve small-object information.
- A small object is defined as smaller than 65 pixels, or 28×28 mm2, at stage 4.
- Additional convolution layers extract higher-level global features for small objects and higher-level edge features for large objects.
- The network additionally applies batch normalization, dropout, weight decay, and mean square error loss for multiclass segmentation.
D. Image dataset and data preparation
The study uses the public LiTS-ISBI2017 liver and liver-tumor segmentation dataset based on contrast-enhanced abdominal CT scans.
- The dataset contains 130 abdomen contrast computed tomography scans from the LiTS-ISBI2017 challenge.
- Images have 512×512 input size, in-plane resolution from 0.98×0.98 mm2 to 0.45×0.45 mm2, and slice thickness from 5 mm to 0.45 mm.
- Forty patients were used for training, with 22,500 total images, while five patients were used for testing.
E. Learning parameters and training details
Training used the proposed full-image network with Adam optimization, scheduled learning-rate reduction, and regularization through batch normalization, dropout, and weight decay.
- Parameters were initialized from a truncated normal distribution and updated with Adam using an initial learning rate of 0.0001.
- The learning rate was scaled by 0.9 every 5,000 epochs.
- The network used five pooling operations and full images rather than patches as input.
- Regularization settings included weight decay of 0.003 and dropout probability of 0.8, with batch-normalization moving-average decay set to 0.9.
F. Performance evaluation
Performance evaluation compares predicted and labeled segmentation masks using overlap, volume, and surface-distance metrics, alongside state-of-the-art CNN and U-Net methods.
- Segmentation performance was evaluated with five objective metrics using labeled data as ground truth.
- DSC measures agreement between binary segmentation masks, while RVD measures the relative difference between two object volumes.
- VOE and RVD equal 0% when the predicted and reference segmentation regions have the same volume or region, as defined for the metrics.
- ASSD and MSSD were calculated to quantify distance error between segmentation surfaces.
- The proposed network was compared with four state-of-the-art CNN- and U-Net-based networks, including comparisons involving network parameter counts.
III. RESULTS
The mU-Net produced accurate liver and liver-tumor segmentations across thin and thick CT slices, with quantitative scores consistent with visual comparisons and results on public datasets.
- Visual segmentation results: The proposed network produced the most accurate segmentation and contouring for thin slices, although competing methods also achieved high accuracy.For thick slices, its liver and liver-tumor results were described as obviously better than those of the comparison methods.
- Visual segmentation results: The proposed network showed obviously excellent liver and liver-tumor segmentation and contouring on thick slices compared with other methods.The thick-slice target was 5 mm, whereas the thin-slice target was 0.8 mm.
- Error analysis: Absolute difference maps compare each method with ground truth for both thin and thick slices, with yellow indicating differences.The maps include Qin et al., Han et al., Men et al., Li et al., and the proposed network.
- Quantitative evaluation: Quantitative scores from all test cases were consistent with the segmentation results shown in Figures 6–10.The mU-Net also attained higher scores than listed for other public databases such as 3Dircadb.
- Computational cost: The fastest network was Qin’s, while Li’s networks had the longest training time per epoch and evaluation exceeding 3 seconds.The timing comparison attributes Qin’s speed to its simpler CNN design without skip-connection concatenation or upsampling.
IV. DISCUSSION
The discussion attributes the mU-Net’s performance to object-dependent feature processing that addresses U-Net limitations, while recognizing greater computational cost and several scope limitations.
- Proposed network: The mU-Net was proposed for fully automatic liver and liver-tumor segmentation.The discussion frames it as a deep-learning network designed to improve segmentation in both liver and tumor regions.
- U-Net limitations: Conventional U-Net limitations include duplicated low-resolution information, insufficient high-resolution edge features, and unsuitable pooling for small objects.These limitations can hinder use of boundary and small-object information.
- Feature extraction: The mU-Net adds convolutional processing in the skip connection to extract higher-level edge features for large objects and global features for small objects.The design is intended to use high-resolution edge and small-object information more efficiently.
- Thick-slice segmentation: Severe partial-volume effects blur thick-slice boundaries, making other networks susceptible to unclear liver and tumor boundaries.The proposed network was reported to provide the most accurate tumor segmentation despite uncertain boundaries and poor contrast.
- Performance and cost: The proposed network is computationally more intensive but offers accurate prediction, particularly for fuzzy-boundary objects and small targets.The paper notes that direct comparisons depend on architecture, training data, kernels, and related design factors.
- Limitations: The current approach uses MSE, has limited generalizability, and trains slice-by-slice because of memory limitations.The authors suggest soft-Dice loss and volume-by-volume training as possible improvements.
V. CONCLUSION
The paper concludes that mU-Net can improve liver and tumor segmentation in difficult cases by combining residual-path processing with object-dependent upsampling.
- Conclusion: The mU-Net could provide more accurate liver and tumor segmentation where boundaries are unclear and target objects are small.The conclusion attributes this to residual-path integration and object-dependent upsampling.
- Conclusion: The method avoids duplicated low-resolution information and extracts higher-level edge or global features according to object size.The paper states that no preprocessing is required and suggests applicability to other organs and imaging modalities.