Source-linked AI summary
Bridging 2D and 3D Segmentation Networks for Computation Efficient Volumetric Medical Image Segmentation: An Empirical Study of 2.5D Solutions
Yichi Zhang, Qingcheng Liao, Le Ding, Jicong Zhang
TL;DR
The paper addresses limited comparisons among 2.5D segmentation methods by reviewing them and conducting a large-scale empirical comparison across three representative tasks. It finds that 2.5D methods can improve 2D baselines, while 3D CNNs are not consistently superior and multi-view fusion can be comparable to 3D CNNs.
Problem
Comparisons among 2.5D methods were limited, leaving uncertainty about which method to use in practical applications.
Method
The paper reviews 2.5D methods and conducts a large-scale empirical comparison on three representative medical image segmentation tasks.
Results
2.5D methods can improve 2D baselines, while 3D CNNs offer no obvious performance advantage and can sometimes perform worse than 2.5D methods.
Takeaways & Limitations
Multi-view fusion of 2D results can achieve segmentation results comparable to 3D CNNs, although it requires training three or four networks.
Takeaways & Limitations
The study tunes each method for fair comparison but evaluates only 2D/3D U-Net architectures; directly applying 3D CNNs to highly anisotropic images may be problematic.
Abstract
from arXiv · showhide
Recently, deep convolutional neural networks have achieved great success for medical image segmentation. However, unlike segmentation of natural images, most medical images such as MRI and CT are volumetric data. In order to make full use of volumetric information, 3D CNNs are widely used. However, 3D CNNs suffer from higher inference time and computation cost, which hinders their further clinical applications. Additionally, with the increased number of parameters, the risk of overfitting is higher, especially for medical images where data and annotations are expensive to acquire. To issue this problem, many 2.5D segmentation methods have been proposed to make use of volumetric spatial information with less computation cost. Despite these works lead to improvements on a variety of segmentation tasks, to the best of our knowledge, there has not previously been a large-scale empirical comparison of these methods. In this paper, we aim to present a review of the latest developments of 2.5D methods for volumetric medical image segmentation. Additionally, to compare the performance and effectiveness of these methods, we provide an empirical study of these methods on three representative segmentation tasks involving different modalities and targets. Our experimental results highlight that 3D CNNs may not always be the best choice. Despite all these 2.5D methods can bring performance gains to 2D baseline, not all the methods hold the benefits on different datasets. We hope the results and conclusions of our study will prove useful for the community on exploring and developing efficient volumetric medical image segmentation methods.
1. Introduction
Volumetric medical image segmentation must balance 2D CNN efficiency against 3D CNN access to spatial context and substantial computational demands. This paper reviews 2.5D methods and compares them across three representative tasks to clarify their performance and practical trade-offs.
- Automatic segmentation supports biomedical and clinical applications, while manual segmentation is labor-intensive and subject to observer variation.
- 2D CNNs offer lighter computation and faster inference but neglect adjacent-slice information, limiting accuracy and causing discontinuities in 3D results.
- 3D CNNs capture volumetric context but incur higher computation, inference latency, overfitting risk, and GPU requirements, especially on small datasets.
- 2.5D methods bridge 2D and 3D CNNs by designing architectures or fusing volumetric information into 2D networks.
- Prior 2.5D studies used different datasets and tasks, leaving cross-method performance, method selection, and quantitative time savings insufficiently established.
- The paper reviews 2.5D methods in three categories and empirically compares them with 2D and 3D CNNs across three datasets spanning CT, MRI, and cardiac, prostate, and abdominal targets.
2. Related Work
The paper reviews computation-efficient 2.5D approaches that add volumetric information to 2D CNNs without the full burden of pure 3D processing. It groups these approaches into multi-view fusion, inter-slice information, and 2D/3D feature fusion.
- 3D CNNs capture volumetric context but require higher memory and time, motivating 2.5D methods that reduce computation while retaining spatial information.
- Multi-view fusion: Anisotropic volumes can make long-axis slices context-poor because of low z-axis resolution, so training 2D CNNs on those planes may be inadvisable.
- Multi-view fusion: Multi-view fusion trains 2D CNNs on sagittal, coronal, and axial planes, then combines their segmentation outputs.
- The reviewed methods differ in fusion stage and mechanism, including majority voting, shallow fusion networks, recurrent models, attention modules, and feature fusion.
- Incorporating inter-slice information: Inter-slice methods provide neighboring slices alongside the target slice, producing segmentation for the middle slice while avoiding heavy 3D computation.
- Fusing 2D/3D features: Other methods fuse features from 2D and 3D CNNs, reducing computation relative to pure 3D CNNs while retaining 3D convolutions for spatial information.
3. Experiments
The experiments systematically compare 2.5D methods under shared settings on three representative datasets spanning MRI and CT modalities and cardiac, prostate, and abdominal targets. The evaluation uses common U-Net-based backbones and complementary region and boundary metrics.
- The study addresses the lack of verified comparisons by evaluating 2.5D methods with the same backbone structure and settings on three datasets.
- Datasets: The datasets cover Cardiac MRI, Spleen CT, and Prostate multimodal MRI segmentation tasks.
- Implementation: U-Net and 3D U-Net provide the principal 2D and 3D comparison backbones under the same experimental environment and settings.
- Evaluation: Dice coefficient measures region mismatch, while 95% Hausdorff Distance evaluates boundary errors.
- Experimental design: The experiments compare multi-view fusion, varying numbers of input slices, inter-slice methods, and 2D/3D feature-fusion stages and mechanisms.
4. Results
Across three tasks, the experiments show that 2.5D methods generally improve on 2D baselines, but their relative effectiveness depends on the dataset and segmentation setting. In particular, 3D CNNs are not consistently the best choice for anisotropic volumes.
- Multi-view fusion: 3.62% and 0.90% Dice improvements over the best 2D view were obtained by majority voting on Cardiac MRI and Spleen CT, respectively.
- Multi-view fusion: 1.48% and 2.06% additional Dice gains over majority voting were achieved by VFN on Cardiac MRI and Spleen CT, respectively, at extra computation cost.
- Multi-view fusion: Weighted voting outperformed majority voting overall, although its improvement over majority voting was not obvious on Cardiac MRI and Spleen CT.
- Inter-slice information: Neighboring-slice inputs generally improved results over single-slice 2D input, but additional slices stopped improving performance when the input reached three slices.
- Inter-slice information: All three evaluated inter-slice methods outperformed 3D U-Net on all metrics, yet their performance varied by dataset and none dominated every task.
- Task dependence: For anisotropic Prostate MRI, 3D CNNs may not be best because downsampling can further lose scarce inter-slice information.
- 2D/3D feature fusion: All 2D/3D feature-fusion methods improved over 2D CNNs, but no method was superior across Cardiac MRI, Spleen CT, and Prostate MRI.
- 2D/3D feature fusion: Encoding-stage fusion can provide useful information with affordable computation cost, while output-stage fusion performed better for Cardiac MRI.
5. Conclusion and Discussion
The study reviews and compares 2.5D strategies for efficient volumetric medical image segmentation across three representative tasks. Results show that 2.5D methods reduce computation relative to 3D CNNs, but their benefits vary by dataset and strategy.
- Study scope: The study reviews 2.5D methods and compares them empirically across three representative volumetric medical image segmentation tasks.The tasks involve different modalities and targets.
- Main findings: 3D CNNs may not always be the best choice, particularly for anisotropic volumes where their performance advantages are not obvious and can be worse than some 2.5D methods.Directly applying 3D CNNs to high-discontinuity slices may involve irrelevant features and hinder learning.
- Multi-view fusion: Multi-view fusion can produce results comparable to 3D CNNs, but requires training three networks, or four with VFN, to obtain final segmentations.Its improvement for anisotropic volumes is not significant.
- Efficiency trade-offs: Inter-slice methods improve performance while training one network with only a slight computational increase, whereas 2D/3D feature fusion improves segmentation efficiency over 2D CNNs.All 2.5D methods have fewer parameters and lower computation cost than 3D CNNs.
- Limitations: The study is limited by incomplete reproduction of all methods and by using only 2D/3D U-Net backbones, which may not be optimal for every segmentation task.The authors nevertheless tuned each method to support a fair comparison.