Source-linked AI summary
Abdominal multi-organ segmentation with organ-attention networks and statistical fusion
Yan Wang, Yuyin Zhou, Wei Shen, Seyoun Park, Elliot K. Fishman, Alan L. Yuille
TL;DR
Abdominal multi-organ CT segmentation is important but challenged by weak boundaries, complex backgrounds, and variable organ sizes. The paper combines 2D organ-attention networks with reverse connections and three-view statistical fusion based on structural similarity. On 236 normal cases with 13 annotated structures, the approach outperformed 2D- and 3D-patch methods and was reported as computationally practical.
Problem
Abdominal multi-organ CT segmentation is clinically important but difficult because of weak boundaries, complex backgrounds, and variable organ sizes.
Method
The framework applies two-stage OAN-RCs to axial, sagittal, and coronal views, then statistically fuses their outputs using local structural similarity.
Results
The method outperformed 2D- and 3D-patch-based state-of-the-art methods for 13 structures, with fusion improving DSC and average surface distances.
Takeaways & Limitations
The framework produced robust abdominal CT segmentation and segmented 13 organs in similar to or less than 1 minute, supporting practical clinical use.
Takeaways & Limitations
Evaluation and training may be affected by inter- and intra-observer variation, while different scanning parameters and artifacts may affect performance on other datasets.
Abstract
from arXiv · showhide
Accurate and robust segmentation of abdominal organs on CT is essential for many clinical applications such as computer-aided diagnosis and computer-aided surgery. But this task is challenging due to the weak boundaries of organs, the complexity of the background, and the variable sizes of different organs. To address these challenges, we introduce a novel framework for multi-organ segmentation by using organ-attention networks with reverse connections (OAN-RCs) which are applied to 2D views, of the 3D CT volume, and output estimates which are combined by statistical fusion exploiting structural similarity. OAN is a two-stage deep convolutional network, where deep network features from the first stage are combined with the original image, in a second stage, to reduce the complex background and enhance the discriminative information for the target organs. RCs are added to the first stage to give the lower layers semantic information thereby enabling them to adapt to the sizes of different organs. Our networks are trained on 2D views enabling us to use holistic information and allowing efficient computation. To compensate for the limited cross-sectional information of the original 3D volumetric CT, multi-sectional images are reconstructed from the three different 2D view directions. Then we combine the segmentation results from the different views using statistical fusion, with a novel term relating the structural similarity of the 2D views to the original 3D structure. To train the network and evaluate results, 13 structures were manually annotated by four human raters and confirmed by a senior expert on 236 normal cases. We tested our algorithm and computed Dice-Sorensen similarity coefficients and surface distances for evaluating our estimates of the 13 structures. Our experiments show that the proposed approach outperforms 2D- and 3D-patch based state-of-the-art methods.
1 Introduction
Abdominal multi-organ CT segmentation remains difficult because organs have weak boundaries, complex anatomy and background, and widely varying sizes. The paper introduces OAN-RCs with multi-view statistical fusion to address these challenges efficiently.
- Abdominal organ segmentation supports clinical applications including computer-aided diagnosis, surgery, and radiation therapy.
- Weak boundaries, complex anatomical structures, low tissue contrast, and morphological variation make abdominal CT segmentation challenging.
- Existing atlas-based methods depend on atlas selection, deformable registration, and label fusion, which are difficult under large abdominal inter-subject variation.
- Large size differences between organs can hurt small-structure segmentation because lower network layers typically lack semantic information.
- The proposed OAN-RC uses two-stage organ attention and reverse connections to reduce background complexity, enhance target information, and adapt features across organ scales.
- Three-view 2D segmentations are combined through statistical fusion using direction-dependent local structural similarity, improving performance while retaining computational efficiency.
2 Organ-Attention Networks with Reverse Connections
OAN-RC uses jointly optimized organ-attention stages and reverse connections to improve multi-organ segmentation across differently sized structures. It processes three 2D views separately, with stage-I attention guiding stage-II and fused reverse-connection outputs supporting prediction.
- 2.1 Two-stage Organ Attention Network: OAN comprises two jointly optimized segmentation stages, with stage-I converting an organ probability map into spatial attention for stage-II.The attention-guided second stage combines the learned attention with the original input to emphasize organ regions.
- 2.1 Two-stage Organ Attention Network: Three direction-specific OANs process reconstructed axial, sagittal, and coronal slices from the 3D CT volume.An individual OAN is trained for each view direction.
- 2.2 Reverse Connections: Reverse connections pass high-level semantic information to lower layers, helping represent multi-scale organs, especially smaller structures.These connections enrich fine-scale layers with abstract information from coarse-scale layers.
- 2.2 Reverse Connections: Reverse-connection side outputs and the backbone output are supervised and fused to produce the stage-I activation and probability maps.Side outputs are generated from reverse-connection maps, then progressively combined from high layers toward low layers.
- 2.1 Two-stage Organ Attention Network: At inference, stage-I and stage-II probability maps are fused, and each pixel receives the label with the minimum predicted probability.The final label map is obtained by the stated arg min rule over labels.
3 Statistical Label Fusion Based on Local Structural Similarity
The framework fuses OAN-RC segmentations from axial, sagittal, and coronal views using voxelwise structural similarity and EM-estimated segmentation performance. Local similarity complements global performance levels to produce the final segmentation.
- Multi-view segmentation: OAN-RC produces three segmentation sources from axial, sagittal, and coronal sectional images for fusion.The views provide direction-dependent information from the 2D networks.
- Motivation for fusion: The three-view fusion addresses nonidentical segmentations and missing connectivity that can arise when neighboring slices are processed independently.Boolean operations and majority voting instead assume fixed global weighting of the view-specific results.
- Statistical fusion: The fusion model combines direction-dependent local similarity with global performance levels for the deep-network segmentations.This allows confidence to vary with local and neighboring voxel structures rather than remaining purely global.
- Local structural similarity: Local structural similarity compares 2D patches from each viewing direction with corresponding isotropic 3D patches centered at each voxel.The similarity uses patch means, standard deviations, covariance, and user-defined constants.
- Expectation-maximization: Expectation-maximization alternates voxelwise label-probability estimation with updates of segmentation performance parameters.The E-step estimates latent voxel labels, while the M-step maximizes the expected log likelihood.
4 Experimental Results
The method was evaluated on 236 normal abdominal CT cases with 13 manually annotated structures and compared against 2D- and 3D-based approaches. OAN-RC with statistical fusion improved overall performance and reduced variability across organs.
- Dataset and annotations: 236 normal abdominal CT cases were evaluated using 13 structures annotated by four raters and confirmed by an independent senior expert.The scans were contrast-enhanced portal-venous-phase CT images.
- Comparisons and metrics: The evaluation compared OAN-RC with local structural similarity-based statistical fusion against 3D U-Net, hierarchical 3D FCN, 2D FCN, OAN, and majority-voting variants.Performance was measured using DSC and surface distances against manual annotations.
- Quantitative results: The basic OAN-RC outperformed other state-of-the-art approaches, while local structural similarity-based fusion improved the results further.The reported comparisons use Dice-Sørensen similarity coefficients across the 13 structures.
- Boundary evaluation: Surface-distance evaluation indicated effective performance for both whole-organ volumes and organ boundaries.The study notes that DSC alone does not quantify boundary smoothness or boundary noise.
- Quantitative results: OAN-RC with statistical fusion improved overall mean performance and significantly reduced standard deviations across the 13 organs.Tables 1 and 2 report means and standard deviations for DSC and average surface distances.
- Computation: Training took 40 hours for 120,000 iterations, while average testing and 13-organ fusion required 76.73 seconds and 6.87 seconds, respectively.Training used 177 training cases.
5 Discussion
The OAN-RC models performed similarly to or better than prior state-of-the-art methods, while local structural-similarity fusion improved accuracy and boundary quality. The approach also offers efficient whole-volume segmentation, but evaluation and deployment remain sensitive to annotation variability, image quality, and prior modeling.
- Performance: OAN-RCs alone achieved similar or better performance than summarized state-of-the-art approaches, with significant gains for relatively small, weakly bounded organs such as the gallbladder and pancreas.Direct comparison is limited because the datasets used different CT images and manual annotations.
- Visualization: 3D photo-realistic renderings showed results that were almost visually indistinguishable from manual segmentations across the rendered abdominal structures.The visualization included the aorta, duodenum, IVC, liver, kidneys, pancreas, spleen, and stomach.
- Statistical fusion: Local structural-similarity fusion improved overall DSC and average surface distances, particularly minimum values, supporting more robust segmentation.The fusion combines information from different views to produce cleaner, smoother boundaries and reduce noise and bias.
- Comparison with 3D methods: The method outperformed 3D patch-based approaches while avoiding their larger-parameter, training-data, augmentation, and GPU-memory demands for larger patches.Limited patch sizes can also restrict the receptive field available to 3D patch-based networks.
- Limitations: Evaluation may be affected by inter- and intra-observer variability in manual annotations, despite four observers, expert checking, and randomized 4-fold cross-validation.The authors identify further dedicated experiments as necessary to explore this issue.
- Limitations: Performance on other datasets may change with spatial resolution, artifacts, reconstruction kernels, and scanning parameters because the study used a controlled portal-venous-phase protocol.The dataset was collected at one institute between 2005 and 2009 with almost isotropic resolution.
- Limitations: The fusion prior P(T) can affect final segmentation, and alternative prior models may be needed when view-specific predictions systematically over- or underestimate structures.The authors regard the mean as generally acceptable when independently obtained segmentations do not show clear directional bias.
- Efficiency: 13-organ whole-volume segmentation took similar to or less than 1 minute, indicating practical computational efficiency for clinical environments.The paper reports better performance than state-of-the-art methods alongside this computation time.
6 Conclusion
The paper proposes OAN-RCs with structural-similarity statistical fusion for challenging abdominal CT multi-organ segmentation. The framework improves DSC and surface-distance performance over prior state-of-the-art methods while remaining efficient enough for clinical applications.
- Framework: OAN-RCs combine organ attention, reverse connections, and statistical fusion to address weak boundaries, complex context, and different organ scales in abdominal CT.Reverse connections provide lower layers with abstract semantic information, while fusion exploits local structural similarity.
- Results: The framework improves segmentation performance in terms of DSC and surface distances and performs better than previous state-of-the-art algorithms.The conclusion describes the results as high quality and robust for challenging abdominal CT.
- Clinical relevance: Efficient computation makes the framework practical for clinical settings such as CAD, CAS, and radiation therapy.The stated scope is abdominal CT, although the framework is not specific to one body region.