PaperlayerView exact revision on arXiv

Source-linked AI summary

3D Consistent & Robust Segmentation of Cardiac Images by Deep Learning with Spatial Propagation

Qiao Zheng, Hervé Delingette, Nicolas Duchateau, Nicholas Ayache

arXiv:1804.09400v1cs.CVcs.AIcs.LGstat.ML

TL;DR

Cardiac MRI segmentation remains difficult because anisotropic, potentially misaligned slices challenge accurate and spatially consistent labeling. The paper introduces propagation-aware networks that segment stacks iteratively using the previous slice’s prediction as context, achieving comparable or better state-of-the-art distance performance while emphasizing spatial consistency and generalization.

  • Problem

    Cardiac MRI is difficult to segment accurately because anisotropic, potentially misaligned slices challenge automated methods, while existing 2D and 3D approaches have complementary limitations.

  • Method

    The method uses U-net variants to segment slices iteratively from base toward apex, conditioning each prediction on the adjacent slice and its previously predicted mask.

  • Results

    The method achieves performance comparable to or better than the state of the art in distance measures across unseen cases and cohorts.

  • Takeaways & Limitations

    Explicit propagation provides spatially consistent segmentation, including improved robustness on difficult apical slices relevant to downstream 3D mesh construction.

Abstract

from arXiv · show

We propose a method based on deep learning to perform cardiac segmentation on short axis MRI image stacks iteratively from the top slice (around the base) to the bottom slice (around the apex). At each iteration, a novel variant of U-net is applied to propagate the segmentation of a slice to the adjacent slice below it. In other words, the prediction of a segmentation of a slice is dependent upon the already existing segmentation of an adjacent slice. 3D-consistency is hence explicitly enforced. The method is trained on a large database of 3078 cases from UK Biobank. It is then tested on 756 different cases from UK Biobank and three other state-of-the-art cohorts (ACDC with 100 cases, Sunnybrook with 30 cases, RVSC with 16 cases). Results comparable or even better than the state-of-the-art in terms of distance measures are achieved. They also emphasize the assets of our method, namely enhanced spatial consistency (currently neither considered nor achieved by the state-of-the-art), and the generalization ability to unseen cases even from other databases.

I. INTRODUCTION

Cardiac MRI segmentation is difficult because manual labeling is costly and anisotropic, potentially misaligned slices complicate automated analysis. Existing 2D and 3D approaches each have important limitations, motivating propagation-based 3D consistency.

  • Manual cardiac-image segmentation is tedious and time-consuming, while MRI slices may be anisotropic and misaligned.
  • 2D methods are lightweight but may fail on difficult apical slices and cannot explicitly maintain consistency across slices.
  • 3D methods reduce training-image numbers, require more GPU memory, and often need downsampling that loses information.
  • The proposed method predicts each current slice using a previously predicted adjacent-slice segmentation, without assuming a unique correct segmentation.
  • The method aims to enforce spatial consistency while retaining computational advantages associated with 2D methods.

II. DATA

The paper addresses dataset-dependent basal-slice conventions by adapting UK Biobank ground truth and defining procedures for identifying the base slice. The adapted definition closely agrees with visually estimated standard basal slices.

  • Segmentation conventions are roughly consistent below the base but can vary substantially above it within and across datasets.
  • UK Biobank ground truth is adapted by setting above-base slices to background, removing basal-slice RVC labels, and retaining below-base labels.
  • The basal slice is identified using anatomical mask relationships and substantial changes in RVC area between adjacent slices.
  • 59 of 100 standard-versus-ground-truth basal-slice pairs matched exactly, 40 differed by one slice, and 1 differed by two slices.

III. METHODS

The method first determines a cardiac region of interest and then segments cropped images using propagation-aware networks. ROI-net produces heart/background probabilities, while LVRV-net or LV-net performs multiclass cardiac segmentation.

  • The pipeline consists of ROI determination followed by segmentation with propagation using either LVRV-net or LV-net.
  • A. Region of Interest (ROI) Determination: ROI-net: ROI-net is a U-net variant that predicts pixel-wise heart/background probabilities from MRI images using a sigmoid output.
  • A. Region of Interest (ROI) Determination: ROI-net: For centered Sunnybrook and RVSC images, cropping is used instead of dataset-specific ROI prediction.
  • A. Region of Interest (ROI) Determination: ROI-net: The predicted heart is defined as the union of LVC, LVM, and RVC, and the ROI is determined from the ED stack.
  • B. Segmentation with Propagation: LVRV-net and LV-net: LVRV-net and LV-net generate four- or three-class pixel probabilities for cardiac segmentation on the determined ROIs.

1) Training:

Training uses cropped cardiac images and contextual inputs from adjacent slices and their masks. The networks are trained on specified UK Biobank sub-stacks, with null context at unavailable stack boundaries.

  • ROI-net training uses ED slices from the 40% of slices immediately below the base to avoid basal and apical regions unsuitable for ROI determination.
  • The ROI determined on an ED stack is reused for the corresponding ED and ES stacks before cropping.
  • LVRV-net and LV-net segment S[i] using the adjacent slice S[i-1] and its mask M[i-1] as contextual input.
  • The contextual mask is represented with four one-hot channels for BG, LVC, LVM, and RVC, alongside the adjacent image channel.
  • Unavailable adjacent images or masks are replaced with null inputs at stack boundaries or for the first segmented slice.

2) Testing:

Testing proceeds slice by slice through a sub-stack, using the previous slice’s image and predicted mask as context. Predictions are post-processed before being used in the next iteration, with validity checks based on cardiac structures.

  • Iterative testing: The trained LVRV-net and LV-net segment testing sub-stacks from UK Biobank and other datasets.For UK Biobank, the segmented sub-stack starts at different slices for LVRV-net and LV-net; for other datasets, it is the whole stack.
  • Iterative testing: Each slice prediction uses the preceding slice’s image and predicted mask as contextual input.The process iteratively predicts M′[i] from S′[i] together with S′[i-1] and M′[i-1].
  • Mask validation: A predicted mask is successful when LVM is present and LVC is mostly surrounded by LVM.The success criteria are applied during post-processing while segmenting a stack.
  • Mask validation: Successful LVRV-net masks retain only the largest RVC connected component, whereas unsuccessful predictions are reset to null masks.The post-processed mask becomes contextual input for the next iteration when applicable.

2) Padding to Square and Resize:

Inputs are standardized through square padding, resizing, intensity normalization, and Dice-based training objectives. The loss definitions incorporate stabilization terms to handle class imbalance and avoid division by zero.

  • Preprocessing: Images and masks are zero-padded to squares and resized with nearest-neighbor interpolation to 128 × 128 or 192 × 192.ROI-net uses 128 × 128 inputs, while LVRV-net and LV-net use 192 × 192 inputs.
  • Preprocessing: Each network input image is normalized by subtracting its cropped intensity-histogram mean and dividing by its standard deviation.The histogram is cropped between the 5th and 95th percentiles.
  • Training losses: Dice loss functions are used to address training difficulties caused by class imbalance.The paper uses two Dice loss functions to train the three networks.
  • Training losses: DL1 compares ROI-net pixel probabilities with binary ground truth, with ϵ = 1 improving stability when both are zero.DL1 ranges from 0 to -1, with better ROI-net performance corresponding to values close to -1.
  • Training losses: DL2 uses four class probabilities and one-hot ground truth for BG, LVC, LVM, and RVC, with ϵ = 1 serving the same stabilization role.The four class probabilities sum to 1 for each pixel.

3) DL3 for LV-net Training:

The experiments train the networks on UK Biobank and evaluate them on UK Biobank testing cases, while comparing segmentation behavior across structures, slices, and propagation variants. Results highlight distance-based performance, ground-truth variability, and differences between basal, apical, end-diastolic, and end-systolic settings.

  • Training and evaluation: DL3 averages Dice terms for BG, LVC, and LVM rather than all four classes.Its formula differs from DL2 by omitting RVC and averaging three Dice terms.
  • Training and evaluation: The networks are trained with 3078 UK Biobank cases and evaluated on 756 UK Biobank testing cases.They are also applied to ACDC, Sunnybrook, and RVSC without fine-tuning or further training.
  • Segmentation results: The UK Biobank results report Dice index and Hausdorff distance for 3D volumes.Mean values are reported with standard deviations, and LV-epi denotes the union of LVC and LVM.
  • Segmentation results: LVM Dice performance is lower because its shape, small volume, and variable ground truth make segmentation and evaluation more difficult.The reported ground-truth variability affects learning and evaluation, especially for LVM.
  • Slice-level evaluation: Slice-level evaluation divides stacks into five levels from near the base to near the apex and measures heart presence rate and 2D Hausdorff distance.G1 is closest to the base and G5 closest to the apex.
  • Slice-level evaluation: LV-net may perform slightly worse because its basal-slice ground truth has greater variability, while segmentation is generally better on ED than ES stacks.The paper suggests the larger heart at ED may make it easier to segment.

3) LVRV-net vs. Its Variants: Justification of the Top-Starting-Propagation Procedure:

LVRV-net’s top-starting propagation improves Hausdorff-distance robustness and spatial consistency over its variants, especially on difficult slices where independent segmentation can fail.

  • LVRV-net and LVRV-mid-starting-net have nearly identical Dice scores, while LVRV-no-propagation-net is slightly higher by 0.01 to 0.02.
  • LVRV-net achieves the lowest mean and standard deviation of Hausdorff distance among the three networks.
  • Hausdorff-distance means differ by less than 1mm, but LVRV-net’s standard deviation is several mm lower for some LV structures.
  • LVRV-net versus LVRV-no-propagation-net yields significant 3D Hausdorff-distance differences, with p-values below or equal to 0.001 for LVC, LVM, and LV-epi, and 0.042 for RVC.
  • Among 73 testing stacks, examples show propagation resolving faint apices and confusing neighboring structures that cause incorrect predictions without propagation.
  • Qualitative mesh comparisons suggest LVRV-net maintains 3D consistency better than the reconstructed human-expert ground truth in the presented examples.

C. Generalization Ability to Other Datasets

The trained networks generalize to ACDC, Sunnybrook, and pathological cases without finetuning, although dataset conventions and image differences affect accuracy and finetuning can overfit.

  • All three trained networks are applied to three external datasets without finetuning to demonstrate generalization and avoid overfitting on small datasets.
  • Finetuning LVRV-net on ACDC is beneficial briefly but overfitting occurs soon afterward, obviously from the 50th epoch.
  • The ROI determination succeeds on 100% of the 100 ACDC cases, with ROIs containing the heart while remaining reasonably small.
  • Experiments on ACDC: On ACDC, LV-net is less accurate than state-of-the-art methods on Dice but comparable in mean Hausdorff distance and better in standard deviation.
  • Experiments on ACDC: Performance differences across the five ACDC pathological groups remain limited, and image quality appears more influential than pathology.
  • Experiments on Sunnybrook: On Sunnybrook, LV-net is somewhat less accurate on Dice and APD but comparable or better on percentage of good contours.

3) Experiments on RVSC:

On RVSC, the method achieved better Hausdorff-distance performance than state-of-the-art methods while emphasizing spatial consistency and broader generalization. The discussion notes that distance robustness can coexist with lower Dice precision and that human-performance comparisons were not directly measured.

  • Experiments on RVSC: The proposed spatial-propagation method achieved a better mean Hausdorff distance and smaller standard deviation than state-of-the-art methods on RVSC.The evaluation used 16 RVSC cases with slices located below the base and above the apex.
  • Conclusion and Discussion: The method enforces spatially consistent segmentation by propagating predictions across adjacent slices.Its predictions depend on previously predicted segmentations rather than assuming a unique correct segmentation.
  • Conclusion and Discussion: The method is robust in distance measures but less precise than state-of-the-art methods in Dice index.The authors attribute this partly to variability in UK Biobank ground-truth annotations.
  • Conclusion and Discussion: Small false-positive regions can slightly reduce Dice index while causing a large increase in Hausdorff distance.This illustrates why distance-based metrics may expose inconsistencies that overlap-based metrics understate.
  • Conclusion and Discussion: The authors did not directly measure human performance using 3D metrics on UK Biobank.Reported 2D human-performance results were not directly comparable to the method’s results.
  • Conclusion and Discussion: Correct segmentation of difficult apical slices matters for applications that require constructing 3D meshes.The method is described as particularly robust on these slices because it segments them spatially consistently.

APPENDIX

The appendix describes the datasets, annotation splits, and overlap-based evaluation used in the study. UK Biobank provides the largest cohort, while ACDC, Sunnybrook, and RVSC contribute additional cardiac MRI cases and annotations.

  • Datasets: UK Biobank contains 4875 participants, with 3078 cases used for training and 756 for evaluation after excluding incomplete or unconvincing ground-truth.The dataset consists of short-axis cine MRI with expert annotations for LVC, LVM, and RVC at ED and ES.
  • Datasets: ACDC contains 100 subjects spanning five cardiac-condition groups, with expert LVC, LVM, and RVC annotations at ED and ES.The groups include dilated cardiomyopathy, hypertrophic cardiomyopathy, myocardial infarction, abnormal right ventricle, and normal subjects.
  • Datasets: Sunnybrook contains 30 subjects with healthy, hypertrophy, heart-failure, and infarction-related conditions.The dataset provides short-axis cine MRI stacks for automated left-ventricle segmentation evaluation.
  • Datasets: RVSC contains 16 training short-axis cine MRI stacks with ground-truth ventricular border delineations at ED and ES.The stacks consist of slices located across the ventricle.
  • Metrics: The Dice index measures overlap between two areas or volumes and ranges from 0 for complete mismatch to 1 for perfect match.The formulation applies to both 2D areas and 3D volumes.

2) Hausdorff Distance:

Hausdorff distance measures the distance between two areas or volumes, with smaller values indicating better agreement. The study uses it alongside APD, PGC, Dice index, and presence rate across different datasets.

  • Hausdorff Distance: Hausdorff distance measures the distance between two areas or volumes using Euclidean point-to-point distances.A smaller Hausdorff distance implies a better match.
  • Related Metrics: Average perpendicular distance measures the mean distance in millimeters from one contour to another across contour points.APD is a contour-distance metric rather than an area- or volume-overlap metric.
  • Related Metrics: Percentage of good contours is the fraction of predicted contours whose APD is less than 5mm from ground-truth contours.PGC uses a fixed 5mm distance threshold.
  • Related Metrics: Presence rate is the ratio of sub-stack slices whose predictions contain a structure to the total number of slices.It captures whether a method misses a structure entirely on difficult slices.
  • Evaluation Protocol: The evaluation uses 3D Dice and 3D Hausdorff distance for UK Biobank and ACDC, while Sunnybrook and RVSC use challenge-specific 2D metrics.Sunnybrook uses 2D Dice, APD, and PGC; RVSC uses 2D Dice and 2D Hausdorff distance.
Loading 1804.09400v1…