Source-linked AI summary
An Exploration of 2D and 3D Deep Learning Techniques for Cardiac MR Image Segmentation
Christian F. Baumgartner, Lisa M. Koch, Marc Pollefeys, Ender Konukoglu
TL;DR
This paper addresses automated cardiac-structure segmentation for large-scale cardiac-function analysis. It evaluates 2D and 3D CNNs on short-axis MR images, finding 2D approaches consistently outperform 3D networks while architecture has a minor accuracy effect.
Problem
Automated cardiac-structure segmentation is needed to analyze normal and pathological cardiac function at large scale and support diagnosis and treatment development.
Method
The paper evaluates state-of-the-art 2D and 3D CNN architectures, modifications, loss functions, and training techniques for segmenting three cardiac structures.
Results
2D approaches consistently outperformed modified 3D U-Net, while the exact architecture played a minor role; mean Dice coefficients were 0.950 (LV), 0.893 (RV), and 0.899 (Myo).
Takeaways & Limitations
Slice-by-slice 2D processing is beneficial for these short-axis MR images, while batch normalisation and cost-function choice substantially affect performance.
Takeaways & Limitations
Many ACDC images exclude the apex and basal heart regions, which are particularly challenging to segment.
Abstract
from arXiv · showhide
Accurate segmentation of the heart is an important step towards evaluating cardiac function. In this paper, we present a fully automated framework for segmentation of the left (LV) and right (RV) ventricular cavities and the myocardium (Myo) on short-axis cardiac MR images. We investigate various 2D and 3D convolutional neural network architectures for this task. We investigate the suitability of various state-of-the art 2D and 3D convolutional neural network architectures, as well as slight modifications thereof, for this task. Experiments were performed on the ACDC 2017 challenge training dataset comprising cardiac MR images of 100 patients, where manual reference segmentations were made available for end-diastolic (ED) and end-systolic (ES) frames. We find that processing the images in a slice-by-slice fashion using 2D networks is beneficial due to a relatively large slice thickness. However, the exact network architecture only plays a minor role. We report mean Dice coefficients of $0.950$ (LV), $0.893$ (RV), and $0.899$ (Myo), respectively with an average evaluation time of 1.1 seconds per volume on a modern GPU.
1 Introduction
The paper addresses automated cardiac-structure segmentation on short-axis MR images, where low through-plane resolution motivates comparing 2D and 3D CNN approaches. It evaluates state-of-the-art architectures and training techniques for segmenting three cardiac structures.
- Automated cardiac-structure segmentation supports large-scale analysis of normal and pathological cardiac function and may contribute to diagnosis and treatment methods.
- Deep CNN-based segmentation is emerging as the state-of-the-art as more data and machine-learning and computing resources become available.
- Short-axis MR images are stacks acquired over multiple cardiac cycles that may be misaligned and typically have 5−10 mm through-plane resolution.
- The study compares state-of-the-art 2D and 3D CNNs for three cardiac structures, focusing on whether 3D context helps despite low through-plane resolution.
- The framework achieved accurate segmentation with an inference time of 1.1 s per 3D image.
2 Method
The method standardizes cardiac MR data, compares four CNN architectures, and investigates optimization and post-processing choices for automated segmentation. It combines 2D and modified 3D designs with batch normalization, alternative losses, interpolation, and connected-component filtering.
- Pre-processing: Images and segmentations were resampled to common resolutions, without through-plane resampling for 2D networks, while the 3D network used 2.5×2.5×5 mm resolution.
- Network Architectures: Four architectures were investigated: FCN-8, 2D U-Net, modified 2D U-Net, and modified 3D U-Net.
- Network Architectures: The modified 2D U-Net reduced feature maps in its upsampling transposed convolutions to the number of classes.
- Network Architectures: The modified 3D U-Net used only one through-plane max-pooling and upsampling step because repeated operations otherwise lost through-plane spatial information before the third pooling step.
- Optimisation: Batch normalization was applied after every convolutional and transposed convolutional layer and improved convergence and results across architectures.
- Optimisation: The study compared standard cross entropy, weighted cross entropy, and direct Dice optimization, using softmax outputs for the Dice loss.
- Post-processing: Predictions were resampled using linear interpolation on continuous softmax outputs before selecting the highest-scoring label at each voxel.
- Post-processing: Keeping only the largest connected component for each structure reduced outliers and substantially lowered Hausdorff distance, while increasing average Dice by approximately 0.0003.
3 Experiments and Results
Experiments on 100 ACDC patient scans compared cardiac segmentation networks, losses, and processing choices. Slice-wise 2D approaches performed best overall, while architecture had a minor effect and preprocessing choices materially affected accuracy.
- 3.1 Data: 100 patient scans were split into 80 training and 20 validation subjects, with ED and ES reference masks for LV, RV, and Myo.The dataset contained short-axis cine-MRI acquired at 1.5T and 3T, with 5–10 mm through-plane resolution.
- 3.3 Experiment 1: Comparison of Loss Functions: Cross entropy outperformed direct Dice optimisation, while weighted and unweighted cross entropy performed similarly.The authors continued with unweighted cross entropy because it produced slightly better results and class imbalance did not appear problematic.
- 3.4 Experiment 2: Comparison of Network Architectures: 2D U-Net variants outperformed FCN-8 and modified 3D U-Net, with the modified 2D U-Net slightly better than the original.This supports slice-by-slice processing for these short-axis MR images.
- 3.5 Discussion and Conclusion: 1.1 seconds per volume was the inference time for 2D networks, compared with 2.2 seconds for 3D networks on an Nvidia Titan Xp GPU.Clinical measures were also calculated for the best-performing modified 2D U-Net.
- 3.5 Discussion and Conclusion: 0.005 was the consistent average Dice improvement from linearly interpolating softmax outputs instead of discrete masks during resampling.The authors also report that resampling predictions to the original resolution was a significant source of errors.
- 3.5 Discussion and Conclusion: 0.899 Dice was achieved for myocardium by the modified 2D U-Net, while direct comparisons with other studies remained complicated by differing datasets.The authors note that many ACDC images omit the apex and basal heart regions, which are particularly challenging to segment.