Source-linked AI summary
Hierarchical 3D fully convolutional networks for multi-organ segmentation
Holger R. Roth, Hirohisa Oda, Yuichiro Hayashi, Masahiro Oda, Natsuki Shimizu, Michitaka Fujiwara, Kazunari Misawa, Kensaku Mori
TL;DR
Foreground-background imbalance can limit boundary accuracy for smaller organs and vessels in volumetric CT segmentation. The paper uses a coarse-to-fine two-stage 3D FCN to narrow the voxel search space, reporting improved Dice scores, particularly for smaller, thinner structures.
Problem
Foreground-background voxel imbalance can lead whole-scan FCNs to focus on separating foreground from background, leaving small organs and vessels with boundary inaccuracies.
Source: §1 · p. 2Method
A coarse-to-fine 3D FCN uses a body mask and first-stage predictions to define a candidate region for a finer second-stage model.
Source: §1 · p. 2, §2.1 · p. 5Results
Takeaways & Limitations
Abstract
from arXiv · showhide
Recent advances in 3D fully convolutional networks (FCN) have made it feasible to produce dense voxel-wise predictions of full volumetric images. In this work, we show that a multi-class 3D FCN trained on manually labeled CT scans of seven abdominal structures (artery, vein, liver, spleen, stomach, gallbladder, and pancreas) can achieve competitive segmentation results, while avoiding the need for handcrafting features or training organ-specific models. To this end, we propose a two-stage, coarse-to-fine approach that trains an FCN model to roughly delineate the organs of interest in the first stage (seeing $\sim$40% of the voxels within a simple, automatically generated binary mask of the patient's body). We then use these predictions of the first-stage FCN to define a candidate region that will be used to train a second FCN. This step reduces the number of voxels the FCN has to classify to $\sim$10% while maintaining a recall high of $>$99%. This second-stage FCN can now focus on more detailed segmentation of the organs. We respectively utilize training and validation sets consisting of 281 and 50 clinical CT images. Our hierarchical approach provides an improved Dice score of 7.5 percentage points per organ on average in our validation set. We furthermore test our models on a completely unseen data collection acquired at a different hospital that includes 150 CT scans with three anatomical labels (liver, spleen, and pancreas). In such challenging organs as the pancreas, our hierarchical approach improves the mean Dice score from 68.5 to 82.2%, achieving the highest reported average score on this dataset.
1 Introduction
3D fully convolutional networks make end-to-end volumetric segmentation feasible, but foreground-background imbalance can leave small organs and vessels inaccurately bounded. The paper proposes a coarse-to-fine, two-stage FCN that narrows the voxel search space while using one model across diverse abdominal structures.
1 Introduction
- A 3D FCN can produce pixel-wise predictions end to end, and 3D convolution makes these methods feasible for volumetric medical images.The paper applies this approach to multi-organ segmentation in clinical CT scans.Source: §1 · p. 2
- Foreground-background imbalance can leave smaller organs and vessels, including the pancreas and gallbladder, with boundary inaccuracies.Whole-scan FCNs may learn mainly to distinguish foreground from background because of the voxel imbalance.Source: §1 · p. 2
- A hierarchical coarse-to-fine approach fine-tunes a second-stage FCN from a first-stage FCN to focus on boundary regions; stage one sees around 40% of voxels using a thresholded body mask.In the second stage, the amount of image voxels is reduced by around 10%, narrowing and simplifying the FCN’s search space.Source: §1 · p. 2
- The first-stage FCN generates the candidate region used to train the second-stage FCN for finer prediction.Figure 1 depicts the two stages and labels their candidate regions C1 and C2.Source: §1 · p. 2, Figure 1 · p. 3
2 Methods
The methods pair a 3D U-Net and class-weighted voxel loss with coarse-to-fine candidate regions, using first-stage predictions to focus second-stage training.
2 Methods
- A 3D U-Net classifies each voxel into background or seven manually labeled structures, using analysis and synthesis paths with skip connections for dense predictions.Source: §2 · pp. 3–4
- Weighted voxel-wise cross-entropy uses softmax probabilities and class weights to balance common background against smaller organs.Source: §2 · p. 4
2.1 Coarse-to-fine prediction
- GPU-constrained training samples 132×132×116 subvolumes, uses 3D elastic deformations, and tiled inference averages probabilities where tiles overlap.Source: §2.1 · pp. 4–5
- A body mask restricts first-stage input regions to about 40%; dilated first-stage predictions define second-stage candidates, with r=3 yielding >99% recall.Source: §2.1 · pp. 4–5
3 Experiments & Results
The hierarchical FCN improved validation Dice scores across organs, with the largest gains in smaller, thinner structures. The experiments also evaluated candidate-region selection and processing on a separate unseen dataset.
Data and training
- Training used 281 contrast-enhanced portal-venous-phase CT images, with 50 cases reserved for validation.Volumes contained 460–1177 slices of 512×512 pixels.Source: §3 · p. 5
Validation set
- 7.5% average Dice improvement per organ was achieved on the 50-case validation set with the hierarchical approach.The reported gains were especially pronounced for small, thin organs.Source: §3 · p. 5, Table 1 · p. 6
Candidate-region selection
- At dilation radius r=3, first-stage predictions achieved >99% average sensitivity and ∼10% average false-positive rate.Figure 2 compares sensitivity and false-positive rate against dilation radius for training and validation.Source: Figure 2 · p. 6
Validation set
- Artery Dice improved from 59.0 to 79.6%, while pancreas Dice improved from 54.8 to 63.1%.The effect was less pronounced for the liver, spleen, and stomach.Source: §3 · p. 6
Qualitative results
- Figure 3 depicts ground truth, non-overlapping tiling of the second-stage candidate region, and the resulting segmentation.Each 44×44×28 output tile is based on a 132×132×116 network input.Source: Figure 3 · p. 7
Unseen test set
- The unseen test cohort came from a different hospital and included 150 scans, with liver, spleen, and pancreas labels.The dataset had slightly higher image quality and a more constrained upper-abdomen field of view than the validation set.Source: §3 · p. 7, Table 2 · p. 8
4 Discussion & Conclusion
The hierarchical coarse-to-fine approach produced competitive multi-organ segmentation without post-processing, with especially visible gains for smaller, thinner organs. Testing on data from a different hospital and scanner supported generalizability across the evaluated settings.
Discussion
- Improved performance was especially visible for smaller, thinner organs, including arteries, veins, and pancreas.No post-processing was applied to the FCN output.Source: §4 · pp. 8–9
Generalizability
- The method was evaluated across separate training/validation and test datasets from different hospitals and scanners, illustrating generalizability to those settings.The authors describe the experiment as illustrating robustness to differences in image quality and populations.Source: §4 · p. 9
Efficiency
- Half-resolution processing improved efficiency and performance compared with original-resolution subvolumes, which would drastically reduce the FCN field of view.The proposed approach is described as deployable on a single GPU.Source: §4 · p. 9
Testing strategy
- During testing, an overlapping-tiles approach produced better results with only moderate additional computational cost.Source: §4 · p. 9