Source-linked AI summary
Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function
Jinzheng Cai, Le Lu, Yuanpu Xie, Fuyong Xing, Lin Yang
TL;DR
Independent 2D slice processing does not explicitly enforce spatial consistency in pancreas segmentation. The paper combines a CNN with recurrent contextual learning and a Jaccard loss, and reports better segmentation than prior state-of-the-art approaches on CT and MRI.
Problem
Prior pancreas-segmentation methods process 2D slices independently without explicitly enforcing spatial consistency between successive slices.
Source: §1 · p. 2Method
The approach adds a convolutional LSTM after CNN training to use neighboring-slice predictions, fine-tunes the integrated network end to end, and trains with Jaccard loss.
Source: §1 · p. 2, §2.2 · p. 4Results
Takeaways & Limitations
Contextual regularization permits explicit enforcement of spatial smoothness, while Jaccard loss produces threshold-free segmentation results.
Source: §4 · p. 7Contextual learning was tested only on pancreas segmentation, although the authors state the approach is directly generalizable to other 3D organ segmentation tasks.
Source: §1 · p. 3
Abstract
from arXiv · showhide
Deep neural networks have demonstrated very promising performance on accurate segmentation of challenging organs (e.g., pancreas) in abdominal CT and MRI scans. The current deep learning approaches conduct pancreas segmentation by processing sequences of 2D image slices independently through deep, dense per-pixel masking for each image, without explicitly enforcing spatial consistency constraint on segmentation of successive slices. We propose a new convolutional/recurrent neural network architecture to address the contextual learning and segmentation consistency problem. A deep convolutional sub-network is first designed and pre-trained from scratch. The output layer of this network module is then connected to recurrent layers and can be fine-tuned for contextual learning, in an end-to-end manner. Our recurrent sub-network is a type of Long short-term memory (LSTM) network that performs segmentation on an image by integrating its neighboring slice segmentation predictions, in the form of a dependent sequence processing. Additionally, a novel segmentation-direct loss function (named Jaccard Loss) is proposed and deep networks are trained to optimize Jaccard Index (JI) directly. Extensive experiments are conducted to validate our proposed deep models, on quantitative pancreas segmentation using both CT and MRI scans. Our method outperforms the state-of-the-art work on CT [11] and MRI pancreas segmentation [1], respectively.
1 Introduction
Pancreas segmentation is challenging, and prior deep methods process 2D slices independently without enforcing consistency across slices. The paper adds recurrent contextual learning and a segmentation-direct loss, and reports improved results on CT and MRI.
1 Introduction
- Manual pancreas segmentation is laborious, tedious, and sometimes prone to inter-observer variability, while detecting volume changes and abnormal growths is a critical diagnostic task.Source: §1 · p. 1
- On NIH-CT-82, deep CNN results rose from 71.8±10.7% to 78.0±8.2% and 81.3±6.3% DSC, while MRI work reported 76.1±8.7% DSC.Deep CNN approaches appear more accurate and numerically more stable than multiatlas registration and label fusion methods; are not completely end-to-end because of segmentation post-processing.Source: §1 · p. 2
- Earlier deep methods segmented 2D slices independently, so the proposed CLSTM uses neighboring-slice predictions to constrain contextual segmentation smoothness end to end.The CLSTM is added after CNN training and fine-tuned with the integrated network.Source: §1 · p. 2
- The proposed Jaccard loss trains segmentation directly against the Jaccard index, addressing foreground-background imbalance and avoiding probability-threshold calibration.Source: §1 · p. 2
- Prior 3D subvolume approaches are computationally expensive and prone to overfitting, while hybrid CNN-RNN methods process sliced scans in sequence [2,14].The passage notes that very high accuracy for complexly shaped organs was not reported for the cited 3D approaches.Source: §1 · p. 2
- The complete model outperforms previous state-of-the-arts by 4% of DSC, evaluated on CT and MRI datasets.Evaluation used a CT dataset of 82 patients and an MRI dataset of 79 patients under 4-fold cross-validation.Source: §1 · p. 3
2 Method
The method combines a pancreas-tailored CNN with recurrent contextual learning across image slices, and trains segmentation with a Jaccard-based loss. The CNN is designed for this task, while the loss directly targets a segmentation metric.
2 Method
- The tailored CNN uses CBR blocks and Scale blocks with auxiliary loss branches, and is trained from scratch for pancreas segmentation.The authors report that ImageNet pretraining did not noticeably improve performance for this application.Source: §2 · p. 3
2.1 Contextual Regularization
- A convolutional LSTM processes successive 2D slices to learn contextual constraints while preserving the CNN's spatial segmentation layout.The method applies the recurrent network to a 2D image sequence formed from a 3D CT or MRI volume.Source: §2.1 · p. 4, Figure 2 · p. 4
2.2 Jaccard Loss
- JACLoss directly optimizes the Jaccard Index using ground-truth and predicted masks, with foreground and background pixel sets defined for the loss.The method relaxes predicted pixel values to probabilities in [0, 1] to approximate the loss.Source: §2.2 · pp. 4–5
- For small objects such as pancreas, JACLoss empirically works better than cross-entropy and class-balanced cross-entropy, assigning larger gradients to foreground pixels.The authors state that this intrinsically balances the foreground and background classes.Source: §2.2 · p. 5
3 Experimental Results and Analysis
Experiments evaluate the models on CT and MRI datasets using four-fold cross-validation. Contextual learning and JACLoss are assessed alongside comparisons with prior methods and segmentation examples.
Datasets
- The evaluation uses NIH-CT-82 with 82 scans and UFL-MRI-79 with 79 T1-weighted scans, under 4-fold cross-validation.The experiments report DSC and JI; no sophisticated post-processing is employed.Source: §3 · p. 5
Analysis on Contextual Regularization
- Contextual learning improves mean DSC by 2.0% and 0.9% for RNN-enhanced models over JAC-64 and JAC-128 on NIH-CT-82.On UFL-MRI-79, RNN-64 gains 1.8% mean DSC over JAC-64, while RNN-128 and JAC-128 perform comparably.Source: §3 · p. 6
Analysis on Jaccard Loss
- JACLoss achieves the highest mean DSC across segmentation thresholds, and its performance is stable from 0.05 to 0.95.The figure compares thresholded results for models trained with different loss functions.Source: §3 · p. 6, Figure 4 · p. 6
Analysis on Contextual Regularization
- Across 80 cases sorted by no-context DSC, contextual learning especially improves cases with low DSC scores.Small fluctuations among good cases are attributed to model updating.Source: §3 · p. 6, Figure 3 · p. 6
Comparison with the State-of-the-art Methods
- RNN-128 performs best on CT-82, while JAC-128 achieves the best result on MRI-79 under 4-fold cross-validation.Table 1 reports DSC and JI as mean ± standard deviation [worst, best].Source: Table 1 · p. 7, §3 · p. 7
- Three CT segmentation examples have DSC values of 90%, 75%, and 60% from left to right.The visualization overlays computerized segmentation in green and human annotation in gold.Source: Figure 5 · p. 7
4 Conclusion
The paper combines a tailor-made CNN with convolutional LSTM contextual regularization and JACLoss for pancreas segmentation. It reports improved results over previous state-of-the-art approaches on both CT and MRI datasets.
4 Conclusion
- A tailor-made CNN followed by convolutional LSTM regularizes slice-level pancreas segmentation through contextual learning.Source: §4 · p. 7
- JACLoss is combined with contextual regularization to produce threshold-free segmentation results.Source: §4 · p. 7
- The quantitative results improve on previous state-of-the-art approaches on both CT and MRI datasets.Source: §4 · p. 7