PaperlayerView exact revision on arXiv

Source-linked AI summary

A Combined Deep-Learning and Deformable-Model Approach to Fully Automatic Segmentation of the Left Ventricle in Cardiac MRI

M. R. Avendi, A. Kheradvar, H. Jafarkhani

arXiv:1512.07951v1cs.CV

TL;DR

LV segmentation from cardiac MRI is needed to calculate clinical indices such as ventricular volume and ejection fraction. The paper develops a fully automatic method combining deep learning with deformable models and reports strong agreement with reference contours, outperforming state-of-the-art methods.

  • Problem

    LV segmentation from cardiac MRI is an essential step for calculating clinical indices including ventricular volume, ejection fraction, mass, wall thickness, and wall-motion abnormalities.

  • Method

    The method uses convolutional networks for LV localization, stacked autoencoders for shape inference, and deformable models for final segmentation.

  • Results

    94% overall DM was achieved by the integrated deformable model, with DM and conformity improving by 4% and 0.08 over the best reported values, respectively.

  • Takeaways & Limitations

    The proposed approach provides fully automatic LV segmentation with improved robustness and accuracy, particularly for challenging basal and apical slices.

  • Takeaways & Limitations

    Training data were artificially enlarged but highly correlated, which could limit performance, and network hyperparameters were selected empirically.

Abstract

from arXiv · show

Segmentation of the left ventricle (LV) from cardiac magnetic resonance imaging (MRI) datasets is an essential step for calculation of clinical indices such as ventricular volume and ejection fraction. In this work, we employ deep learning algorithms combined with deformable models to develop and evaluate a fully automatic segmentation tool for the LV from short-axis cardiac MRI datasets. The method employs deep learning algorithms to learn the segmentation task from the ground true data. Convolutional networks are employed to automatically detect the LV chamber in MRI dataset. Stacked autoencoders are utilized to infer the shape of the LV. The inferred shape is incorporated into deformable models to improve the accuracy and robustness of the segmentation. We validated our method using 45 cardiac MR datasets taken from the MICCAI 2009 LV segmentation challenge and showed that it outperforms the state-of-the art methods. Excellent agreement with the ground truth was achieved. Validation metrics, percentage of good contours, Dice metric, average perpendicular distance and conformity, were computed as 96.69%, 0.94, 1.81mm and 0.86, versus those of 79.2%-95.62%, 0.87-0.9, 1.76-2.97mm and 0.67-0.78, obtained by other methods, respectively.

1. Introduction

LV segmentation from cardiac MRI supports clinical assessment but remains manually performed, making it tedious, time-consuming, and variable. The paper addresses these limitations with a combined deep-learning and deformable-model approach for fully automatic LV segmentation.

  • Cardiac MRI LV segmentation enables calculation of ventricular volume, ejection fraction, mass, wall thickness, and wall-motion abnormalities.
  • Manual expert delineation is the clinical standard but is tedious, time-consuming, and prone to intra- and inter-observer variability.
  • Existing automatic methods include pixel classification, image-based, deformable, active appearance and shape models, and atlas models.
  • Pixel classification, image-based, and deformable methods are described as having low robustness and accuracy while requiring extensive user interaction.
  • The proposed method combines deep learning with deformable models to develop and validate fully automated, accurate, and robust LV segmentation.
  • The authors report successful deep-learning training with limited data using artificial data enlargement, pre-training, and careful design, while addressing deformable-model shrinkage, leakage, and initialization sensitivity.

2. Materials and Methods

The study uses MICCAI 2009 cardiac MRI data to train and evaluate a three-stage LV segmentation pipeline. The method localizes the LV with convolutional networks, then processes cropped regions for subsequent shape and segmentation stages.

  • Datasets: 45 MRI datasets from the MICCAI 2009 LV segmentation challenge were used, with training data for learning and validation and online datasets for evaluation.
  • Overall method: The proposed pipeline processes a short-axis cardiac MR image stack through LV localization, shape inference, and segmentation stages.
  • Automatic detection: The first stage locates the LV and computes a region of interest to reduce computational complexity and improve accuracy.
  • Automatic detection: 256 × 256 images are down-sampled to 64 × 64 before convolutional-network processing for automatic LV detection.
  • Automatic detection: Convolutional filtering produces 100 feature maps from the 64 × 64 input, with each resulting feature map sized 54 × 54.
  • Automatic detection: Average pooling over non-overlapping 6 × 6 neighborhoods reduces each feature map to 9 × 9 resolution.
  • Automatic detection: Pooled features are unrolled into an 8100-dimensional vector, mapped to a 32 × 32 ROI mask, up-sampled, and used to crop a 100 × 100 ROI.

Training Convolutional Network

The convolutional-network training uses sparse autoencoder pre-training to initialize filters, then learns ROI masks from manually centered binary labels. The broader pipeline also trains a stacked autoencoder to infer LV shape from sub-images and output boundary masks.

  • Filter initialization: Sparse autoencoder pre-training initializes convolutional filters instead of random initialization because labeled training data are limited.The learned filters are transferred to the convolutional network for subsequent training.
  • Filter initialization: Approximately 10^4 randomly selected 11 × 11 patches from raw training images train an autoencoder with 121 input/output units and 100 hidden units.Each patch is unrolled into a 121-dimensional vector before entering the autoencoder.
  • Filter initialization: The autoencoder reconstructs each input patch from hidden representations, so input values serve as labels and no manually labeled data are required.Its objective combines reconstruction error, weight decay, and a sparsity constraint on hidden activations.
  • ROI detection: Binary masks centered on manually delineated LV contours provide labels for training the automatic detection network.The masks contain a white ROI foreground on a black background and are down-sampled before optimization.
  • Shape inference: The stacked autoencoder receives sub-images and produces binary masks representing the LV boundary to infer LV shape.Its two hidden layers build abstract representations before the output layer generates the mask.

Training stacked-AE

The stacked autoencoder is pretrained layer by layer and then fine-tuned to infer LV shape, which initializes and constrains a deformable segmentation model. Quadratic fitting estimates and corrects slice-center misalignment for aligned 3D contours.

  • Training stacked-AE: Layer-wise pretraining uses sparse autoencoders before supervised training of the final layer and whole-network fine-tuning.Pretraining uses unlabeled data, while the final layer uses expert-derived binary masks; fine-tuning minimizes a supervised criterion.
  • Segmentation and Alignment: The deformable model combines length, region, and prior-shape energies to evolve a contour toward the object boundary.The energy weights are α1 = 1, α2 = 0.5, and α3 = 0.25.
  • Segmentation and Alignment: The inferred shape initializes the level-set contour and is incorporated into the energy function to limit shrinkage and leakage around papillary muscles and low-contrast borders.Gradient descent iteratively updates the contour until its length becomes stationary.
  • Segmentation and Alignment: Quadratic polynomial fitting models slice-center curvature and estimates actual centers from misaligned contour centers affected by motion artifacts.The contours are affine-registered using the estimated centers to produce an aligned stack for 3D reconstruction.

3. Implementation Details

Implementation uses grouped training images, overfitting controls, and a MATLAB-based workstation setup. The workflow includes shape inference, deformable-model processing, and reconstructed 2D/3D segmentation outputs.

  • Training data: Training images and contours are divided into large-contour and small-contour groups representing basal or middle slices and apical slices.The groups contain around 135 and 125 images, respectively.
  • Training controls: Overfitting is addressed using layer-wise pretraining, l2 regularization, sparsity constraints, cross-validation, early stopping, and data enlargement.The network depth was limited to three hidden layers and hidden-layer units were kept small.
  • Outputs: Implementation outputs include aligned contour centers, integrated-model outcomes, and automatic LV segmentations in 2D and 3D.The figures include examples spanning the base, mid-ventricle, and apex.
  • Computational setup: The method was developed in MATLAB 2014a and trained on the MICCAI training dataset before testing on online and validation datasets.Experiments used a Dell Precision T7610 workstation with an Intel Xeon 2.6 GHz CPU and 32 GB RAM.

4. Validation Process

Validation compares automated LV contours with expert manual annotations using contour metrics and clinical-volume analyses. Performance is summarized slice by slice with dataset-level means and standard deviations.

  • Reference standard: Automated segmentation is evaluated against expert manual annotations as the gold standard.The evaluation code is from the MICCAI 2009 challenge.
  • Segmentation metrics: The validation metrics are average perpendicular distance, Dice metric, Hausdorff distance, percentage of good contours, and conformity coefficient.A contour is classified as good when APD is less than 5mm.
  • Clinical indices: Clinical validation computes EDV, ESV, and EF from automatic and manual segmentations for correlation and Bland-Altman analyses.Pearson tests provide slopes, intercepts, and correlation values; coefficient of variation assesses observer variability.
  • Aggregation: Each metric is computed slice by slice, then summarized by a mean value and standard deviation for every dataset.This produces dataset-level summaries for the validation assessment.

5. Results

Results show progressive improvement from unconstrained deformation to shape inference and the integrated method, with qualitative agreement across challenging slices and quantitative agreement for clinical indices.

  • Ablation and qualitative results: The integrated deep-learning and deformable-model pipeline is compared with an unconstrained deformable model and deep-learning shape inference.The comparison uses two typical images to examine the role of each processing step.
  • Qualitative results: Automatic and manual contours are illustrated across base, mid-ventricular, and apical slices, including cases with papillary muscles and varied cardiac conditions.The examples include ischemic and non-ischemic heart failure, LV hypertrophy, and normal cases.
  • Quantitative results: 0.99 correlation with ground truth was measured for EDV, ESV, and EF on the validation dataset.Bland-Altman analyses reported mean differences of -13 cm3 for EDV, -3.5 cm3 for ESV, and -2.4% for EF.
  • Computational performance: 63.3 seconds, 3.4 hours, and 34.25 minutes were the reported training times for autoencoder filters, the convolutional network, and the stacked autoencoder.After training, segmentation of a typical MR image required 0.25 seconds for ROI detection, 0.002 seconds for shape inference, and 0.2 seconds for deformable segmentation.

6. Discussion

The method combines deep learning with deformable models to improve fully automatic LV segmentation, especially in challenging slices. It achieved strong agreement with reference contours and rapid per-image testing, while remaining constrained by limited data and 2D processing.

  • Method: Deep learning localizes the LV and infers its shape, which is integrated with deformable models to improve segmentation accuracy and robustness.Convolutional networks extract the region of interest, while stacked autoencoders infer shape; deformable models refine the result.
  • Robustness: Integrated shape-constrained contours prevented leakage and inward shrinkage in cases affected by low contrast, papillary muscles, or apical resolution limits.The inferred shape improved results over unconstrained deformable models in basal and apical images.
  • Validation: 94% overall DM was obtained for final contours, compared with 90% for initial inferred-shape contours.Table 2 also reported improvements of 4% in DM and 0.08 in conformity over the best cited competing results.
  • Clinical indices: High correlation was found for three clinical cardiac indices, while Bland–Altman agreement exceeded Queiros et al. but was slightly below two semi-automated frameworks.The comparison is limited by the different degrees of automation across methods.
  • Efficiency: Less than 0.5 seconds was required on average to segment a typical image during testing.Training was performed offline, and runtime comparisons across studies were considered unreliable because computing platforms differed.
  • Limitations: The study used 2D processing because large inter-slice gaps and motion artifacts complicate direct 3D analysis.The authors also note that artificially enlarged training data were highly correlated and that hyperparameters were selected empirically.

7. Conclusion

The paper presents a fully automatic LV segmentation method that combines learned localization and shape inference with deformable models. On the MICCAI 2009 database, it showed accurate, robust agreement with reference contours, while larger clinical-data testing remains future work.

  • Conclusion: The method uses deep learning to detect and infer LV shape, then incorporates that shape into deformable models for fully automatic segmentation.The approach was evaluated against the MICCAI 2009 database gold standard.
  • Conclusion: The integrated approach improved robustness and accuracy, particularly for challenging basal and apical slices.The paper reports excellent agreement and high correlation with reference contours.
  • Future work: Testing the method on a larger set of clinical data is identified as future research.
Loading 1512.07951v1…