Source-linked AI summary
A Fully Convolutional Neural Network for Cardiac Segmentation in Short-Axis MRI
Phi Vu Tran
TL;DR
Automated LV and RV segmentation from cine MRI is needed because manual contouring is time-consuming and variable, while prior methods face generalization and clinical-use limitations. The paper applies a fully convolutional neural network trained end-to-end for pixel-wise labeling, reporting state-of-the-art accuracy across multiple datasets and metrics, with fast GPU inference.
Problem
Automated cardiac segmentation is needed to replace time-consuming, variable manual ventricular delineation and address technical and generalization challenges in prior methods.
Method
A fully convolutional neural network is trained end-to-end from whole-image MRI inputs and ground truths to produce dense pixel-wise ventricular labels.
Results
The FCN achieves state-of-the-art segmentation accuracy across multiple metrics and benchmark MRI datasets with variability across sites, scanners, populations, and heart conditions.
Takeaways & Limitations
The model is fast and runs on commodity GPU resources, supporting cardiac segmentation at massive scales.
Takeaways & Limitations
Accuracy remains limited at apical and basal slices with ambiguous or imperceptible object boundaries, motivating collection of more labeled examples there.
Abstract
from arXiv · showhide
Automated cardiac segmentation from magnetic resonance imaging datasets is an essential step in the timely diagnosis and management of cardiac pathologies. We propose to tackle the problem of automated left and right ventricle segmentation through the application of a deep fully convolutional neural network architecture. Our model is efficiently trained end-to-end in a single learning stage from whole-image inputs and ground truths to make inference at every pixel. To our knowledge, this is the first application of a fully convolutional neural network architecture for pixel-wise labeling in cardiac magnetic resonance imaging. Numerical experiments demonstrate that our model is robust to outperform previous fully automated methods across multiple evaluation measures on a range of cardiac datasets. Moreover, our model is fast and can leverage commodity compute resources such as the graphics processing unit to enable state-of-the-art cardiac segmentation at massive scales. The models and code are available at https://github.com/vuptran/cardiac-segmentation
1 Introduction
Short-axis cine MRI segmentation supports cardiac functional assessment, but manual delineation is laborious and variable, while automated methods face substantial technical and generalization challenges. This paper proposes a fully convolutional approach that learns pixel-wise ventricular labels without hand-engineered priors.
- Motivation: Accurate LV and RV contours are needed to derive ventricular volume, stroke volume, ejection fraction, and myocardial mass from cine MRI.Manual delineation is time-consuming and prone to high intra- and inter-observer variability.
- Technical challenges: Automated LV and RV segmentation must handle overlapping intensities, contour variability, severe class imbalance, fuzzy boundaries, cross-site variability, and cine-MRI noise.These challenges span both image appearance and anatomical variation.
- Limitations of prior methods: Previous automated evaluations often used small benchmarks that may not represent variability across sites, institutions, scanners, and populations.Prior methods also required a priori ventricular knowledge, which could contribute to overfitting.
- Proposed approach: The proposed FCN learns features from whole-image inputs and ground truths, trains end-to-end in one stage, and infers a class label at every pixel.The architecture avoids feature engineering and hard-coded a priori knowledge.
- Contribution: The paper presents what it identifies as the first CNN application for pixel-wise labeling in cardiac MRI.The contribution targets semantic segmentation of the left and right ventricles.
2 Previous Work
Earlier ventricular-segmentation research established challenge benchmarks and explored semi-automated, automated, and model-based approaches. CNNs offered end-to-end feature learning, while fully convolutional networks extended this paradigm to dense pixel-wise prediction.
- LV segmentation: LV segmentation research was advanced by the MICCAI 2009 and STACOM 2011 challenges, which aimed to improve automated segmentation.These challenges focused on delineating LV endocardial and epicardial contours throughout the cardiac cycle.
- LV segmentation: Earlier LV methods included semi-automated restricted-Boltzmann-machine and level-set models, automated 3D+time frameworks, and machine-learning approaches.The semi-automated approach required user input, whereas other methods targeted automated contour generation.
- RV segmentation: RV segmentation shared LV goals and technical difficulties but received less attention partly because no common database with expert ground-truth contours was available.This limited the shared basis for RV algorithm development and evaluation.
- Limitations of prior methods: Previous statistical-shape, feature-engineering, and multi-atlas methods tended to overfit one dataset and generalized poorly to others.These methods were also not amenable to transfer learning.
- CNNs: CNNs learn feature hierarchies end-to-end without complicated preprocessing, postprocessing, or feature-engineering schemes.This learning strategy underlies their strong performance across visual-recognition benchmarks.
- Fully convolutional networks: Fully convolutional networks remove fully connected layers and learn nonlinear upsampling filters that map coarse outputs to dense pixel space.This adapts classification architectures for per-pixel labeling from whole-image inputs and ground truths.
3 Experimental Framework
The framework evaluates one fully convolutional network across multiple cardiac MRI benchmarks for automated LV and RV segmentation. It combines multi-scale preparation, augmentation, dense pixel prediction, regularized training, and transfer learning for limited-data settings.
- Datasets: The study evaluates an FCN model for automated LV and RV segmentation across the Sunnybrook, LVSC, and RVSC cardiac MRI datasets.Sunnybrook and RVSC provide expert manual contours for selected splits, while LVSC uses expert-guided semi-automated contours for training and consensus references for validation.
- Data Preparation: Multi-scale center cropping creates multiple ventricular views, reduces background pixels, and accelerates computation before FCN training.The ROI wholly contains the ventricles, while augmentation supplies multiple resolutions and mitigates class imbalance.
- Data Preparation: Mean-variance normalization standardizes each MRI image to zero mean and unit variance, while affine transformations and cropping inflate training data 12-fold.The preparation targets intensity variation across scanners and sites and seeks to reduce overfitting and improve generalization.
- FCN Architecture: The FCN uses 15 convolution layers, three strided max-pooling layers, and skip connections to combine coarse semantic and fine appearance information for dense pixel classification.The model has roughly 11 million parameters and produces a class-membership heatmap for every input pixel.
- Training Protocol: Training minimizes per-pixel multinomial logistic loss with momentum-based stochastic gradient descent, dropout, L2 regularization, and Xavier initialization.The protocol uses momentum 0.9, dropout ratio 0.5, and L2 weight decay of 0.0005; transfer learning instead initializes selected layers from an LV source model and fine-tunes on target tasks.
4 Empirical Evaluation
The evaluation compares FCN cardiac segmentation with prior methods across multiple datasets and metrics, while also assessing transfer learning, runtime, scalability, and difficult-slice performance.
- Evaluation metrics: The evaluation uses overlap, contour-distance, classification, and good-contour metrics to compare automated segmentations against manual ground truth.Metrics include Dice, Jaccard, APD, Hausdorff distance, sensitivity, specificity, PPV, NPV, and percentage of good contours.
- LV segmentation: On the Sunnybrook dataset, the FCN achieves comparable Dice to Avendi et al. for automated LV endocardium segmentation and the best scores on all other evaluation measures.The comparison covers LV endocardium and epicardium segmentation on the combined validation and online sets.
- Runtime and scalability: The model segments 830 Sunnybrook images in less than 25 seconds, 1,028 RVSC images in less than a minute, and 29,859 LVSC images in under 19 minutes.These runtimes demonstrate fast inference across validation, online, test, and large-scale datasets.
- LV segmentation: For LV myocardium segmentation on the LVSC validation set, the FCN achieves the best scores in three of five metrics: Jaccard index, specificity, and negative predictive value.Results are compared with previous fully automated methods using the CS* consensus.
- RV segmentation: For RV endocardium and epicardium segmentation, the FCN outperforms previous fully automated and semi-automated methods across all evaluation metrics.Fine-tuning also produces a significant accuracy boost over random initialization.
- Limitations: The main limitation is failure to segment cardiac objects in difficult slices, especially apical slices with ambiguous or imperceptible boundaries.Such errors have minor impact on volume computation but can limit applications such as fiber-structure analysis.
5 Conclusion
The paper demonstrates that a fully convolutional neural network can segment both ventricles in cardiac MRI with state-of-the-art accuracy across multiple datasets and metrics. The model is also fast enough to run on commodity GPU resources, while difficult apical and basal slices remain a target for improvement.
- A single FCN model learns features for segmenting both the left and right ventricle in cardiac MRI.
- The FCN achieves state-of-the-art segmentation accuracy across multiple metrics and benchmark MRI datasets with real-world variability.
- The model is fast and can run on commodity GPU resources for cardiac segmentation at massive scales.
- Improving segmentation at ambiguous apical and basal locations would require more labeled examples from these difficult regions.