Source-linked AI summary
Recurrent Fully Convolutional Neural Networks for Multi-slice MRI Cardiac Segmentation
Rudra P K Poudel, Pablo Lamata, Giovanni Montana
TL;DR
Fully automatic LV segmentation is clinically useful but difficult because cardiac shape and boundary quality vary across MR slices and subjects. The paper proposes an end-to-end RFCN that jointly detects and segments the LV from full slice stacks using inter-slice dependence, and reports improved contour delineation near the apex across two independent evaluations.
Problem
LV segmentation is challenging because cardiac shape varies across slices, phases, patients, and scanners, while boundaries can be weakened by blood flow, papillary muscles, and trabeculations.
Method
RFCN jointly detects and segments the LV from an entire cardiac MR slice sequence while recurrent features model inter-slice spatial dependence.
Results
The experiments show that propagating information from adjacent slices improves segmentation quality, with notable gains in delineating contours around the apex.
Takeaways & Limitations
RFCN combines LV detection and segmentation in one architecture with reported computational benefits and potential for real-time application.
Takeaways & Limitations
RFCN achieved a lower Dice index in the most apical PRETERM slice than in MICCAI, 0.76 versus 0.85, warranting further investigation.
Abstract
from arXiv · showhide
In cardiac magnetic resonance imaging, fully-automatic segmentation of the heart enables precise structural and functional measurements to be taken, e.g. from short-axis MR images of the left-ventricle. In this work we propose a recurrent fully-convolutional network (RFCN) that learns image representations from the full stack of 2D slices and has the ability to leverage inter-slice spatial dependences through internal memory units. RFCN combines anatomical detection and segmentation into a single architecture that is trained end-to-end thus significantly reducing computational time, simplifying the segmentation pipeline, and potentially enabling real-time applications. We report on an investigation of RFCN using two datasets, including the publicly available MICCAI 2009 Challenge dataset. Comparisons have been carried out between fully convolutional networks and deep restricted Boltzmann machines, including a recurrent version that leverages inter-slice spatial correlation. Our studies suggest that RFCN produces state-of-the-art results and can substantially improve the delineation of contours near the apex of the heart.
1 Introduction
LV segmentation is clinically valuable but difficult because cardiac anatomy and image quality vary across slices, phases, patients, and scanners. The paper proposes RFCN to jointly detect and segment the LV from entire short-axis image stacks while exploiting inter-slice spatial dependence.
- Motivation: Automatic LV segmentation supports structural and functional cardiac measurements used in diagnosis and disease management.Examples include volume estimation and ejection fraction.
- Motivation: LV segmentation is challenging because shape varies across slices, cardiac phases, patients, and scanning machines, while boundaries can be weak.Blood flow, papillary muscles, and trabeculations contribute to weak boundaries.
- Prior approaches: Traditional active-contour methods require energy functions that accommodate variability and often depend on the ROI used for initialization.Machine-learning methods address some issues but require large training datasets.
- Prior approaches: Deep neural-network pipelines commonly localize, segment, and refine the LV in three sequential stages.The paper contrasts this staged design with a jointly trained architecture.
- Contribution: RFCN extends fully convolutional networks to jointly detect and segment the LV from the entire stack of short-axis MR images.The approach is trained end-to-end and targets inter-slice spatial dependences.
- Contribution: RFCN is motivated by learning global anatomical structure across slices, particularly to improve segmentation where cardiac boundaries are weak.The paper specifically highlights difficult regions such as the apex.
2 Datasets
The study evaluates LV segmentation on two independent short-axis cardiac MR datasets with expert endocardial annotations. These include the 45-case MICCAI 2009 challenge dataset and a larger 234-subject PRETERM cohort with standardized acquisition and resampling details.
- Datasets: The experiments use two independent short-axis cardiac MR datasets with expert manual endocardial segmentation for each axial slice.Further dataset and acquisition details are provided for both cohorts.
- MICCAI 2009 dataset: The MICCAI 2009 dataset contains 45 CINE MRI exams spanning heart failure, hypertrophy, and healthy-subject categories.Its training, validation, and online testing subsets each contain 15 cases.
- PRETERM dataset: The PRETERM dataset contains 234 subjects aged 20 to 39 years, including 102 followed prospectively since preterm birth and 132 born at term.The preterm group has reported mean gestational age and birth weight measurements.
- Stack organization: The image stacks represent progression from the heart base toward the apex, where LV regions become smaller and boundaries more blurred.This stack structure motivates using spatial correlations across slices.
- Acquisition: The cardiac MR images were acquired on a 1.5-T Siemens Sonata scanner with 7 mm slices, 3 mm inter-slice gaps, and 2 mm resampled in-plane resolution.The resampled slice pixel size was 212 × 212.
3 Recurrent fully-convolutional networks
RFCN combines convolutional feature extraction, recurrent global-feature modeling, and upsampling to segment every slice in a cardiac MR stack. Its recurrent component processes inter-slice information while the expanding path restores spatial resolution.
- Architecture: RFCN extends a fully convolutional encoder–decoder architecture with contracting and expanding paths for dense pixel-wise prediction.Its expanding path uses many feature channels to propagate context to higher-resolution layers, similarly to U-net.
- Architecture diagram: The architecture diagram distinguishes feature maps, copied maps, convolution, pooling, upsampling, 1 × 1 convolution, and GRU recurrence by box and arrow types.The recurrent connection specifically handles inter-slice dependences learned through GRU.
- Architecture: The network receives the entire sequence of S slices at one cardiac phase and outputs equal-sized corresponding LV masks.The formulation is designed to leverage inter-slice spatial dependences.
- Architecture: RFCN comprises feature-extraction, global-feature, and upsampling components.The feature-extraction path is applied independently to each image before recurrent global modeling.
- Recurrent global features: A recurrent mapping h_s = φ(h_s−1, e_s) captures spatial changes across slices while preserving the feature-map size.The global-feature block uses compressed feature maps with dimensions 256 × 30 × 30.
- Upsampling: The upsampling path restores resolution through fractional-stride convolution, feature concatenation, and convolutional layers with ReLU.It combines upsampled features with aligned high-resolution features in a U-net-like design.
4 Other architectures and model training
The study compares FCN and RFCN with DBN and recurrent DBN alternatives. Convolutional models use cross-entropy-based optimization, whereas DBN variants use contrastive divergence and dropout.
- Other architectures: DBN-based LV segmentation uses a DBN for LV region detection followed by distance-regularized level sets informed by a separate DBN.These models segment each slice independently of the others.
- Other architectures: The recurrent DBN, or RDBN, stacks recurrent restricted Boltzmann machines to process the full slice sequence through additional bias units.It was assessed to test whether modeling cross-slice dependence improves performance.
- Training: FCN and RFCN were trained by minimizing cross-entropy, using stochastic gradient descent with momentum for FCN and RMSProp for RFCN.Back-propagation also optimized the GRU parameters in RFCN.
- Training: RFCN was initialized from FCN weights, and reported results were the best of five experiments with random parameter initialization.Batch normalization was added after each convolutional layer.
5 Experimental results
RFCN was evaluated against FCN, DBN, and RDBN on MICCAI and PRETERM cardiac MRI datasets using contour and overlap metrics. It generally matched or exceeded competing methods, with particularly strong performance in central and apical LV regions.
- The evaluation used good contours, Dice index, and average perpendicular distance (APD) to assess LV endocardium segmentations.
- 2.05 mm APD was achieved by RFCN on MICCAI, outperforming FCN while obtaining a comparable Dice index and higher good-contour performance.
- RFCN achieved the best Dice index and APD on PRETERM, with APD as small as 1.56 mm, despite DBN and RDBN receiving LV-focused regions of interest.
- RDBN generally improved on DBN across cardiac locations, supporting benefits from modeling inter-slice dependencies when the LV was already localized.
- RFCN outperformed FCN around the central slices and apex in both datasets, although FCN performed better near the MICCAI base.
6 Conclusions
The paper concludes that RFCN can jointly detect and segment the LV from full short-axis MRI stacks by recurrently propagating information from base to apex. Across two applications, this improved contextual representation and segmentation quality, especially near difficult apical contours, while leaving some cohort-dependent behavior for further study.
- RFCN performs LV localization sequentially from the heart base to apex through a recurrent modification of fully convolutional networks.
- Information propagation from adjacent slices improved context information and had a positive effect on segmentation quality across two independent applications.
- RDBNs generally outperformed DBNs on segmentation when the LV had already been localized, providing additional evidence for modeling inter-slice dependencies.
- RFCN showed notable improvements in delineating apical cardiac contours, which are more difficult to identify.
- RFCN's most-apical-slice Dice index was 0.85 on MICCAI versus 0.76 on PRETERM, an unexpected cohort difference requiring further investigation.
- RFCN combines LV detection and segmentation in one architecture, with reported computational benefits and potential for real-time application.
- Future extensions include 3D convolutions, bidirectional base-to-apex and apex-to-base processing, and temporal modeling across cardiac phases.