PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep learning for cardiac image segmentation: A review

Chen Chen, Chen Qin, Huaqi Qiu, Giacomo Tarroni, Jinming Duan, Wenjia Bai, Daniel Rueckert

arXiv:1911.03723v1eess.IVcs.CVcs.LGq-bio.QM

TL;DR

Cardiac image segmentation lacks a systematic review focused specifically on deep-learning applications across major modalities and anatomical targets. The paper surveys over 100 studies, summarizes public datasets and code, and synthesizes progress alongside barriers to clinical deployment. It concludes that deep learning has matched or surpassed prior state of the art in various cardiac segmentation applications, while scarcity of labels, limited generalization, privacy, and other limitations remain.

  • Problem

    Existing reviews covered general medical or cardiovascular image analysis, but none provided a systematic overview focused on cardiac segmentation applications.

  • Method

    The paper reviews deep-learning cardiac segmentation methods across MRI, CT, and US, covering cardiac structures, public datasets, and public code repositories.

  • Results

    Deep learning methods have matched or surpassed previous state of the art in various cardiac segmentation applications, supported by larger public datasets, advanced architectures, and powerful hardware.

  • Takeaways & Limitations

    The review provides a reading basis for newcomers and supports transparent benchmarking, code sharing, and continued progress in cardiac image segmentation.

  • Takeaways & Limitations

    Deep-learning methods remain limited by data privacy concerns and weak generalization to unseen scanners, modalities, and pathologies.

Abstract

from arXiv · show

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research.

1 INTRODUCTION

Cardiac image segmentation supports quantitative assessment across MRI, CT, and US, while deep learning has become increasingly prominent because it learns features automatically and benefits from growing data and computing resources. This review systematically surveys cardiac segmentation methods, datasets, code, and remaining barriers to clinical deployment.

  • Cardiac image segmentation partitions images into anatomically meaningful regions from which clinical measures such as ventricular volumes and ejection fraction can be extracted.
  • MRI, CT, and US are the principal modalities covered, with segmentation targets including the ventricles, atria, and coronary arteries.
  • Deep learning learns intricate image features automatically in an end-to-end procedure, reducing reliance on the feature engineering and prior knowledge required by traditional methods.
  • The number of deep-learning cardiac segmentation papers increased strongly in recent years, while MR segmentation publications exceeded those in CT and US, especially in 2017.
  • The review covers influential work through 1 August 2019, organizes methods across cardiac tasks and modalities, and summarizes public datasets, code, challenges, and future directions.

2 FUNDAMENTALS OF DEEP LEARNING

This section introduces neural-network architectures and building blocks used for cardiac image segmentation, emphasizing how networks transform images into pixel-wise predictions. It covers CNNs, FCNs, recurrent networks, autoencoders, GANs, and related architectural enhancements.

  • Convolutional Neural Networks (CNNs): CNNs transform images through convolution, pooling, and fully connected layers into task-specific outputs such as class probabilities, coordinates, labels, or regression values.Pooling reduces redundant features, while fully connected layers reduce feature dimensionality for inference.
  • Convolutional Neural Networks (CNNs): Patch-based CNN segmentation predicts the center-pixel class for repeatedly sampled patches, but overlapping-patch inference is computationally inefficient.FCNs are therefore more commonly used for efficient end-to-end pixel-wise segmentation.
  • Fully Convolutional Neural Networks (FCNs): FCNs use encoder-decoder structures to convert whole images into same-size pixel-wise segmentations, while U-net skip connections recover spatial context lost during downsampling.FCNs remove the need for patch selection, and U-net or its 3D variants have achieved promising accuracy across cardiac segmentation tasks.
  • Recurrent Neural Networks (RNNs): RNNs model sequential or volumetric cardiac images by retaining information from earlier inputs, supporting contextual propagation between adjacent slices and improved inter-slice coherence.The approach is suited to cine MRI, ultrasound movies, and volumetric data.
  • Autoencoders: Autoencoders learn compact latent representations without supervision and can provide semantic or shape features to guide cardiac image segmentation.Their encoder-decoder structure reconstructs the input from a low-dimensional representation.
  • Generative Adversarial Networks (GANs): GAN-based adversarial training uses a discriminator to distinguish predicted segmentation maps from ground truth, encouraging anatomically plausible outputs.GANs otherwise use generator-discriminator competition to model real-data distributions and create new image examples.

2.2 Training Neural Networks

Training neural networks requires paired images and labels, an optimizer, and a loss function that guides parameter updates through backpropagation. Common losses include MSE for regression, cross-entropy and soft-Dice for segmentation, with weighted variants addressing class imbalance; overfitting is a central challenge.

  • Training requires paired images and labels, an optimizer, and a loss function that supplies update signals through backpropagation.The training goal is to reduce prediction error while updating model parameters iteratively.
  • Common loss functions: MSE is used for regression tasks such as heart localization, calcium scoring, landmark detection, and image reconstruction.The loss compares target and predicted values across data samples.
  • Common loss functions: Cross-entropy summarizes pixel-wise probability errors between predicted segmentation probabilities and target maps across classes.It is described as the most common loss for image classification and segmentation.
  • Common loss functions: Soft-Dice loss penalizes pixel-level mismatch between predicted and target segmentation maps.It is specifically designed for object segmentation.
  • Common loss functions: Weighted cross-entropy and weighted soft-Dice losses address class imbalance by weighting rare classes or small objects.These variants modify the loss term to account for underrepresented targets.
  • Overfitting: Limited training images relative to the number of learnable parameters make overfitting the biggest training challenge in medical image analysis.Proposed countermeasures include regularization, dropout, ensemble learning, data augmentation, and transfer learning.

2.3 Evaluation Metrics

Cardiac segmentation algorithms are commonly evaluated with volume-based, surface-distance-based, and clinical performance metrics. The review mainly reports Dice scores because they facilitate comparison, with Dice measuring overlap between automatic and manual segmentations.

  • Evaluation metrics fall into volume-based, surface-distance-based, and clinical performance categories.Examples include Dice and Jaccard scores, contour and Hausdorff distances, and ventricular volume and mass.
  • The review mainly reports Dice scores to make comparisons between segmentation methods easier.Dice measures the overlap ratio between two results, such as automatic and manual segmentations, on a scale from 0 to 1.

3 DEEP LEARNING FOR CARDIAC IMAGE SEGMENTATION

Deep learning-based cardiac segmentation spans MRI, CT, and ultrasound and targets structures including ventricles, atria, vessels, and scars. These methods support quantitative follow-up of cardiovascular structure and function, with ventricle segmentation especially common in MR and ultrasound.

  • The review covers deep learning applications across MRI, CT, and ultrasound for segmenting cardiac organs and tissues.Targets include the left ventricle, coronary vessels, and scars.
  • Deep learning segmentation facilitates follow-up quantitative analysis of cardiovascular structure and function.The passage identifies organ and tissue segmentation as an efficient and effective route to these analyses.
  • Ventricle segmentation constitutes a large portion of the reviewed work, especially in MR and ultrasound domains.

3.1 Cardiac MR Image Segmentation

Cardiac MR segmentation primarily addresses chambers, while abnormal tissue segmentation remains less developed because relevant public datasets are limited and the task is difficult. Reviewed approaches include FCNs, temporal and anatomical modeling, multi-task and multi-stage networks, and hybrid methods, with strong reported results for atrial and selected scar segmentation.

  • Cardiac MR image segmentation: Cardiac MRI visualizes cardiac structures and pathological tissues without ionising radiation, enabling quantitative assessment of anatomy, function, and scars.Different imaging sequences support cine anatomy and function assessment and LGE scar imaging.
  • Cardiac MR image segmentation: Most cardiac MR segmentation studies target chambers such as the LV, RV, and LA, whereas myocardial scars and atrial fibrosis receive fewer studies.The review attributes this difference to limited relevant public datasets and task difficulty.
  • Ventricle segmentation: Temporal information, anatomical constraints, auxiliary tasks, multi-stage pipelines, and hybrid classical methods are used to improve cardiac MR segmentation.Examples include motion estimation, cardiac-function estimation, ROI localization, orientation normalization, level sets, deformable models, atlases, and graph cuts.
  • Ventricle segmentation: End-to-end FCNs became the dominant state-of-the-art approach for bi-ventricle segmentation, simultaneously segmenting the LV and RV.The benchmark study cited by the review identifies FCNs as the basis of most state-of-the-art methods.
  • Atrial segmentation: A two-stage 3D U-net ranked first in LASC’18, achieving a mean Dice score of 0.93 on 54 test cases for left atrium segmentation.The first network localizes the atrial center from down-sampled images, and the second segments cropped full-resolution regions.
  • Scar segmentation: Traditional scar-segmentation methods are sensitive to local intensity changes and often require manual ROI segmentation, limiting suitability for large-scale studies or clinical deployment.
  • Scar segmentation: Fully automated scar segmentation remains challenging, with reported mean Dice of 0.58 for one U-net method and mean Dice above 0.68 for semi-automated alternatives.An RNN using motion patterns achieved an overall Dice score of 0.90 against manual LGE annotations without contrast agents.

3.2 Cardiac CT Image Segmentation

Cardiac CT segmentation covers substructures, coronary arteries, calcium, and plaques, with deep learning methods ranging from staged pipelines to end-to-end CNNs. The reviewed approaches target clinically relevant measurements and disease assessment, but coronary and plaque segmentation remain challenging or limited areas.

  • Cardiac substructures and clinical targets: Cardiac CT segmentation targets the LV, RV, LA, RA, myocardium, aorta, pulmonary artery, coronary arteries, calcium, and coronary plaques.These structures support cardiac function analysis, disease diagnosis, stenosis grading, blood-flow simulation, surgical planning, and risk scoring.
  • Two-step segmentation: Two-step methods first localize or extract a region of interest before CNN-based classification or segmentation.This strategy appears in cardiac substructure segmentation and calcium detection, where staged processing narrows the subsequent prediction problem.
  • Coronary arteries: Coronary centerline extraction is challenging because nearby structures, coronary veins, and motion artifacts can interfere with the task.Deep learning is therefore used either before or after traditional centerline-extraction methods, including classifiers for vessel orientation or leak removal.
  • End-to-end CNNs: End-to-end CNNs predict dense vessel segmentation maps, with multi-task learning, multi-scale supervision, and shape priors used to improve predictions.Multi-task networks can perform segmentation across tasks and modalities, while 3D U-Net supervision supports multi-scale feature learning and voxel-level predictions.
  • Coronary calcium: Direct CAC segmentation with U-Net and DenseNet combinations achieved sensitivity over 90% in one reported study.These approaches segment CAC directly from non-contrast cardiac or chest CT for precise quantification.
  • Coronary plaques: Only a limited number of studies investigate deep learning for non-calcified and mixed-calcified coronary plaque segmentation and quantification.The clinical motivation is that these plaques can potentially rupture and obstruct an artery, causing ischemic events and severe consequences.

3.3 Cardiac Ultrasound Image Segmentation

Deep learning-based cardiac ultrasound segmentation addresses difficult image quality conditions through staged, deformable, temporal, and direct CNN-based approaches. Reviewed studies emphasize LV and multi-chamber segmentation, while 3D ultrasound remains computationally and technically challenging.

  • Imaging challenges: Ultrasound segmentation is challenged by low signal-to-noise ratio, speckle noise, low contrast, edge dropout, and acoustic shadows.These imaging characteristics make voxel-wise tissue classification difficult, motivating combinations of deep learning with deformable models.
  • Task focus: Most reviewed ultrasound methods focus on LV segmentation, while relatively few address the aortic valve and left atrium.The review summarizes these approaches in its overview of deep learning methods for ultrasound segmentation.
  • Pipeline designs: Two-stage pipelines localize a target region using a rigidly transformed bounding box before segmenting the structure within that region.The reduced search region increases robustness of the overall segmentation framework; one-step methods reduce computational complexity instead.
  • Temporal information: Temporal-coherence methods use relationships between nearby frames to improve the accuracy and robustness of LV segmentation.These methods exploit the fact that cardiac ultrasound is recorded as a temporal image sequence.
  • Direct CNN segmentation: Larger annotated datasets significantly benefit U-Net LV segmentation in 2D ultrasound, while CNNs can perform real-time view classification and segmentation.The reviewed evidence links direct CNN segmentation to the collection and labeling of large-scale datasets.
  • 3D ultrasound: 3D ultrasound offers potentially more accurate volume-related indices but has lower temporal resolution, lower image quality, and a larger neural-network parameter space than 2D ultrasound.These properties create computational challenges; proposed responses include 2D-to-3D pipelines, transfer learning, shape constraints, and adversarial training for domain shift.
  • Clinical and public-dataset validation: Automatic segmentation produced clinical metrics comparable or superior to manual segmentation on a large clinical dataset, and encoder-decoder CNNs had lower error than inter-observer error on a public 500-patient dataset.The public dataset contains apical four-chamber images annotated for LV and LA segmentation with uneven imaging quality and varying patient conditions.

3.4 Discussion

Across cardiac modalities, state-of-the-art segmentation is dominated by CNN-based FCN and U-Net architectures augmented for feature learning, class imbalance, robustness, and anatomical plausibility. The review also emphasizes that public data, transparent benchmarking, and shared code support continued progress and reproducibility, while imaging dimensionality influences network design.

  • Common architectures and techniques: Current state-of-the-art cardiac segmentation methods mainly use CNNs based on FCN or U-Net architectures.Common enhancements address feature learning, class imbalance, generalization, robustness, and anatomically plausible outputs.
  • Data and performance: Publicly available training subjects are associated with improved reported performance across the literature.Public challenge datasets provide data for exploiting deep learning and platforms for transparent evaluation and comparison.
  • Reproducibility: Transparent benchmarking and code sharing are described as essential for continued progress in cardiac image segmentation.The review summarizes public datasets and code repositories as resources for reproducible research.
  • Network dimensionality: The target image type affects the choice between 2D, 3D, and 2D+ network structures.For 3D acquisitions, 2D+ networks incorporate multi-planar, multi-slice, or recurrent cross-slice context while retaining 2D-based designs.

4 CHALLENGES AND FUTURE WORK

The review identifies limited annotation, weak generalization to unseen data, interpretability, image quality, and privacy as major barriers to clinical deployment. It outlines strategies including alternative supervision, active learning, domain adaptation, augmentation, smart imaging, and privacy-preserving learning.

  • Scarcity of Labels: Annotated cardiac images are scarce because labeling is time consuming and requires substantial expertise.Most reviewed studies use fully supervised training, which requires many annotated images.
  • Scarcity of Labels: Transfer, weakly supervised, semi-supervised, self-supervised, and unsupervised learning are proposed to reduce dependence on paired annotations.Active learning is also presented as a way to select representative images and reduce redundant labeling workload and training cost.
  • Model Generalization: Deep learning models often generalize poorly to unseen scanners, modalities, abnormal cases, and pathological cases because they are biased by their training datasets.Large heterogeneous training sets and unsupervised domain adaptation are discussed as responses, although collecting sufficiently broad labeled data may not scale.
  • Model Generalization: Data augmentation and normalization can make training distributions more similar to real-world test distributions, including when training uses a single scanner.The review describes geometric and photometric transformations as common augmentation mechanisms.
  • Interpretability and Robustness: Deep learning systems remain difficult to interpret and can be vulnerable to adversarial perturbations, undermining verification and trustworthiness.The review notes that medical systems may produce wrong diagnoses after adversarial noise or small rotations.
  • Smart Imaging: Low-quality or artifact-heavy acquisitions can cause segmentation failures, motivating image-quality checks, failure prediction, and feedback during image acquisition.The review argues that improving imaging quality can improve imaging-based diagnostic accuracy.
  • Data Privacy: Privacy-preserving deep learning remains an early-stage challenge requiring combinations of cryptography, distributed computing, federated learning, and differential privacy.The desired systems must balance security, runtime, and generalizability.

5 CONCLUSION

The review surveys deep learning techniques for cardiac segmentation across MRI, CT, and US, covering cardiac ventricles, atria, and vessels. It also highlights unresolved challenges that require further research.

  • 5 CONCLUSION: The review covers mostly CNN-based deep learning approaches for segmenting cardiac ventricles, atria, and vessels in MRI, CT, and US.It presents recent progress across the three common imaging modalities.
  • 5 CONCLUSION: The review outlines future potential and limitations that may hinder widespread clinical deployment of deep learning-based cardiac segmentation methods.The conclusion emphasizes increasing awareness of challenges that call for future contributions.
  • 5 CONCLUSION: The paper includes terminology for imaging, cardiac structures and indexes, machine learning, and cardiac image segmentation datasets.The listed dataset abbreviations include ACDC, CETUS, MM-WHS, LASC, LVSC, and RVSC.

1. York: http://www.cse.yorku.ca/˜mridataset/

The cited LASC’13 dataset website is listed as a source for datasets summarized in the review.

  • Dataset Resources: LASC’13 is listed at the Cardiac Atlas challenge webpage.The URL is provided as www.cardiacatlas.org/challenges/left-atrium-segmentation-challenge/.
Loading 1911.03723v1…