PaperlayerView exact revision on arXiv

Source-linked AI summary

Cardiac Segmentation with Strong Anatomical Guarantees

Nathan Painchaud, Youssef Skandarani, Thierry Judge, Olivier Bernard, Alain Lalande, Pierre-Marc Jodoin

arXiv:2006.08825v1cs.CVcs.LGeess.IV

TL;DR

CNNs can produce anatomically implausible cardiac segmentations even with shape priors. This paper post-processes segmentations with a cVAE-based latent-space method that steers invalid shapes toward valid ones. Across MRI and ultrasound evaluations, the method yields anatomically valid results while keeping overall metrics nearly unchanged.

  • Problem

    CNN segmentation methods can produce anatomically impossible cardiac shapes, and shape-prior or cleanup approaches do not guarantee plausibility for every result.

    Source: §I · p. 1

  • Method

    A cVAE-based post-processing method replaces invalid shapes' latent vectors with nearby valid vectors, using anatomically constrained sampling to augment valid latent vectors.

    Source: C. Anatomically-Constrained Data… · p. 5, §V · p. 9

  • Results

    Across MRI and ultrasound tests, the method produced anatomically valid results while leaving EF error and Dice index almost identical to the original methods.

    Source: 2) ACDC Postprocessing results · p. 8, §V · p. 9

  • Takeaways & Limitations

    Methods within inter-expert variation can, with this post-processing, produce segmentations that follow user-defined anatomical guidelines.

    Source: §V · p. 10

Abstract

from arXiv · show

Convolutional neural networks (CNN) have had unprecedented success in medical imaging and, in particular, in medical image segmentation. However, despite the fact that segmentation results are closer than ever to the inter-expert variability, CNNs are not immune to producing anatomically inaccurate segmentations, even when built upon a shape prior. In this paper, we present a framework for producing cardiac image segmentation maps that are guaranteed to respect pre-defined anatomical criteria, while remaining within the inter-expert variability. The idea behind our method is to use a well-trained CNN, have it process cardiac images, identify the anatomically implausible results and warp these results toward the closest anatomically valid cardiac shape. This warping procedure is carried out with a constrained variational autoencoder (cVAE) trained to learn a representation of valid cardiac shapes through a smooth, yet constrained, latent space. With this cVAE, we can project any implausible shape into the cardiac latent space and steer it toward the closest correct shape. We tested our framework on short-axis MRI as well as apical two and four-chamber view ultrasound images, two modalities for which cardiac shapes are drastically different. With our method, CNNs can now produce results that are both within the inter-expert variability and always anatomically plausible without having to rely on a shape prior.

I. INTRODUCTION

CNNs can produce anatomically impossible cardiac segmentations despite strong average accuracy and shape priors. The paper presents a framework intended to guarantee anatomical plausibility while preserving overall accuracy.

I. INTRODUCTION

  • MRI is a reference exam for cardiac function and viability, while ultrasound offers flexible, low-cost imaging but has poor signal-to-noise ratio and other image limitations.Source: §I · p. 1
  • CNN segmentations can contain holes, abnormal concavities, and duplicated regions, despite average scores near inter- and intra-observer variability.The authors argue these errors make the methods unfit for day-to-day clinical use.Source: §I · p. 1
  • Shape priors and morphological or connected-component cleanup still cannot guarantee anatomical plausibility in every result.Source: §I · p. 1
  • The proposed framework targets zero anatomically invalid outputs and is designed to preserve overall accuracy across segmentation methods, MRI, and echocardiography.Source: §I · pp. 1–2

II. PREVIOUS WORK

Prior cardiac segmentation work includes CNNs and models using soft shape priors, but those approaches do not ensure anatomically valid outputs. The proposed framework instead post-processes segmentations using a latent-space transformation toward a nearby valid shape.

A. MRI segmentation

  • MRI segmentation work includes generic and specialized CNNs, while shape-prior approaches use learned representations, registered spatial priors, or atlas propagation.The atlas-based refinement depends strongly on the accuracy of located landmarks, and prior approaches impose soft rather than guaranteed constraints,,.Source: CNNs · p. 2, CNNs with shape prior · p. 2

B. Echocardiographic segmentation

  • Echocardiographic segmentation research includes U-Nets, localization networks, autoencoder shape constraints, deep atlases, and image translation with soft priors.Reported U-Net outliers fell from 18% to 11% with LUNet localization,.Source: B. Echocardiographic segmentation · p. 2

III. PROPOSED FRAMEWORK

  • The proposed system combines a constrained VAE, anatomically constrained rejection-sampling augmentation, and a post-processing VAE that warps invalid shapes toward nearby valid ones.The latent-space transformation replaces an incorrect shape's latent vector with a close valid vector; the segmentation-method block can stand for any cardiac segmenter.Source: §III · p. 2
  • The framework defines correctness through complementary anatomical criteria, and Fig. 1 depicts the same framework for short-axis MRI and two- and four-chamber ultrasound.Source: §III · p. 2, Figure 1 · p. 3

A. Anatomical Criteria

The framework identifies implausibility using distinct short-axis and long-axis anatomical criteria. Thresholds are informed by each dataset's training set and are selected so clinically relevant segmentations are not marked invalid.

A. Anatomical Criteria

  • Short- and long-axis criteria are evaluated algorithmically on every latent-space sample, and their thresholds are based on ACDC and CAMUS training data.The datasets include healthy and pathological cases, and thresholds are set so no clinically relevant segmentation is marked invalid.Source: A. Anatomical Criteria · p. 3
  • Short-axis validity is checked with 16 criteria covering holes, disconnected or duplicated structures, contact errors, acute concavities, and excessive circularity.The criteria apply to the LV, RV, and myocardium; Fig. 2 gives examples of erroneous shapes in short- and long-axis views.Source: Long-Axis Criteria · p. 3, Figure 2 · p. 3
  • Long-axis validity is checked with 12 criteria covering holes, duplicated structures, excessive contact areas, thin myocardium, and LV width relative to myocardium thickness.The width and thickness comparison uses measurements at the midpoint of the structure's embedded bounding box.Source: Long-Axis Criteria · pp. 3–4

B. Constrained Variational Autoencoder (cVAE)

The cVAE learns a latent representation of expert-outlined valid cardiac shapes. A linear regression constraint encourages a latent space suited to interpolation and smooth shape changes.

B. Constrained Variational Autoencoder (cVAE)

  • A VAE encodes an input as a Gaussian latent distribution and decodes a sampled latent vector into a reconstruction; this work uses 32-dimensional vectors for 2D cardiac shapes.Source: B. Constrained Variational Autoencoder… · p. 4
  • Training on expert-outlined ground-truth shapes makes encoded centroids and points on the learned valid-shape manifold decode to anatomically valid shapes.Source: B. Constrained Variational Autoencoder… · p. 4
  • The latent representation is intended to support valid interpolation between shape vectors and smooth, anatomically coherent changes after small latent translations.Source: B. Constrained Variational Autoencoder… · p. 4
  • A single-neuron linear regression network is trained with the encoder and decoder to make the valid-shape latent manifold more linear.The cVAE loss combines the usual ELBO terms with an L2 regression loss; the regression target is slice position for MRI and time point for ultrasound.Source: B. Constrained Variational Autoencoder… · p. 4

C. Anatomically-Constrained Data Augmentation

Anatomically-constrained rejection sampling expands the sparse manifold of valid cardiac latent vectors; cardiac shape warping then moves invalid segmentations toward nearby valid ones.

C. Anatomically-Constrained Data Augmentation

  • Rejection sampling addresses the sparse valid-shape manifold by generating additional latent vectors from an easier proposal distribution.The method estimates the unknown valid-vector distribution with a Parzen window and accepts proposal samples according to the rejection criterion.Source: C. Anatomically-Constrained Data… · p. 4
  • Anatomically-constrained sampling accepts only decoded vectors whose segmentation maps satisfy the anatomical criteria, generating 4 million valid vectors for each dataset.The resulting MRI and ultrasound samples are illustrated in Fig. 3.Source: C. Anatomically-Constrained Data… · p. 5, Figure 3 · p. 5

D. Cardiac shape warping

  • Cardiac shape warping moves an invalid latent vector toward a nearby valid vector found by nearest-neighbor search, then uses a five-iteration dichotomic search to select the smallest valid interpolation.The valid vector is obtained from the augmented latent space; the anatomical criterion determines which half of the search interval to explore.Source: D. Cardiac shape warping · p. 5

E. Robust VAE

The robust VAE directly maps erroneous segmentation maps to plausible configurations by learning from synthetic valid/invalid shape pairs. Its alternative to nearest-neighbor warping reduces memory use and processing time, but does not retain the earlier method’s anatomical guarantees.

E. Robust VAE

  • The robust VAE is an alternative to cardiac shape warping that avoids storing millions of latent vectors and running nearest-neighbor searches, but gives up anatomical guarantees.The paper presents the choice between methods as application-dependent.Source: E. Robust VAE · p. 5
  • The robust VAE is fine-tuned from a pretrained cVAE as a denoising autoencoder, mapping implausible segmentation maps to valid ones while keeping the decoder and regression-network weights fixed.The encoder is thereby trained to project erroneous inputs near their corresponding valid latent vectors.Source: E. Robust VAE · pp. 5–6
  • Training uses 10,000 synthetic valid/invalid shape pairs, created by perturbing valid latent vectors along the regression-network axis and decoding them.The perturbations are intended to produce out-of-distribution latent samples that decode to anatomically implausible maps.Source: E. Robust VAE · p. 6
  • An additional KL loss term encourages the robust encoder’s latent vector for a noisy input to remain close to the original cVAE vector for its clean counterpart.The stated training objective is to recreate the valid shape from its paired invalid shape.Source: E. Robust VAE · p. 6

F. Implementation Details

The implementation uses convolutional encoder–decoder networks with a 32-dimensional latent space, trained end-to-end. Before post-processing, segmentation maps are resized and registered, making the method dependent on reasonably accurate initial heart position and orientation.

F. Implementation Details

  • The cVAE encoder uses four convolutional blocks and two fully connected posterior heads, with a 32-dimensional latent space chosen to balance compactness and reconstruction accuracy.Each block has two 3×3 convolutional layers; the first downsamples by half.Source: F. Implementation Details · p. 6
  • The decoder mirrors the encoder with four upsampling blocks and produces pixel-wise class scores.It begins with a fully connected projection; its blocks use transposed convolutions for upsampling followed by 3×3 convolutions.Source: F. Implementation Details · p. 6
  • The encoder and decoder are trained end-to-end with Adam, using learning rates of 6×10−5 for ACDC and 5×10−4 for CAMUS, and L2 regularization λ = 0.01.The ablation-study AE uses the same architecture and hyperparameters, but replaces the posterior heads with one fully connected latent-vector head.Source: F. Implementation Details · p. 6
  • Maps are resized to 256 × 256 and registered using dataset-specific heart landmarks, so inference depends on the initial segmentation being somewhat accurate in heart position and orientation.Registration is performed from the segmentation method’s output before post-processing.Source: F. Implementation Details · p. 6

A. Datasets, evaluation criteria, and other methods

The evaluation covers short-axis MRI and long-axis echocardiography, with segmentation methods assessed on dataset-specific anatomical and clinical measures. A latent-space interpolation ablation found that combining image registration with a regression constraint most strongly limited anatomically implausible shapes.

1) MRI dataset:

  • The ACDC dataset contains 150 short-axis cine-MR patients, split into 100 training and 50 test patients; evaluation reports 3D Dice, average HD, and LV/RV EF absolute error.The study compares its post-processing with results from ten ACDC challengers and an ACNN implementation.Source: 1) MRI dataset · pp. 6–7

2) Echocardiographic dataset:

  • CAMUS contains 500-patient apical two- and four-chamber ultrasound exams divided into 10 folds; the framework was tested on seven segmentation methods.The seven methods include four convolutional networks and three non-deep-learning methods.Source: 2) Echocardiographic dataset · p. 7

B. Experimental Results

  • The interpolation ablation generated 25 intermediate latent vectors between randomly selected valid cases and measured their anatomical error rates across 300 repetitions.It compared cVAE configurations with and without image registration and the one-neuron regression network.Source: B. Experimental Results · p. 7
  • 1.25% (ACDC) and 0.12% (CAMUS) of interpolated results were anatomically implausible with registration plus regression, more than 4× lower than any other configuration.The ablation repeated the interpolation process 300 times for each dataset.Source: B. Experimental Results · p. 7
  • On the ACDC test set, Table II compares invalid-segmentation counts across 11 methods, while Table III reports Dice, HD, and LV/RV EF errors.The tables compare results with and without post-processing.Source: 2) ACDC Postprocessing results · p. 7, Table II · p. 7

1) Constrained variational autoencoder:

The complete post-processing method eliminates anatomical-criteria violations while keeping ACDC clinical metrics close to the original methods. Simpler correction variants reduce errors too, but nearest-neighbor variants without data augmentation can compromise accuracy.

2) ACDC Postprocessing results:

  • The simple VAE significantly reduced ACDC anatomical errors, and rVAE reduced them by almost 2× without significantly affecting overall anatomical metrics.The rVAE processed images 10 times faster than the most accurate method, making it a stated compromise for real-time applications.Source: 2) ACDC Postprocessing results · pp. 7–8
  • Nearest-neighbor latent-vector correction, with or without rejection sampling, reduced anatomical errors to zero; data augmentation systematically improved results, while correction without an augmented latent space could hurt some methods’ overall accuracy.For Zotti-2, Dice changed from .899 to .909 and HD from 14.4 to 11.0; for Wolterink, Dice changed from .752 to .887 and HD from 32.2 to 13.5.Source: 2) ACDC Postprocessing results · p. 8
  • All complete-method ACDC results respected the anatomical criteria; Dice and EF error were almost unchanged, and HD increased by at most 1.3 mm.The reported maximum HD increase is less than one pixel at the stated average voxel size.Source: 2) ACDC Postprocessing results · p. 8, Table III · p. 8
  • Post-processing smoothly warped erroneous regions while barely affecting correct ones, and processing a 2D image took roughly one second on a Titan X GPU.Source: 2) ACDC Postprocessing results · p. 8
  • On CAMUS, the simple VAE significantly reduced anatomical errors without affecting average Dice index and HD too much.Table IV covers invalid-slice counts for seven methods in cross-validation on 2000 images.Source: 3) CAMUS Postprocessing results · p. 8, Table IV · p. 8

3) CAMUS Postprocessing results:

On CAMUS, postprocessing preserves segmentation accuracy while removing anatomical errors; the reported results also remain within observer variability. The authors discuss nearest-neighbor behavior and a limitation for severely degenerated inputs.

3) CAMUS Postprocessing results:

  • The robust VAE did not further reduce errors on CAMUS, especially for non-deep-learning methods, where errors were already low with a basic VAE.Source: 3) CAMUS Postprocessing results · p. 9
  • All three nearest-neighbor methods reduce anatomical errors to zero and have almost the same Dice index, Hausdorff distance, and anatomical errors.The authors suggest long-axis valid latent vectors are close enough that a nearest-neighbor swap can enforce criteria while preserving overall shape.Source: 3) CAMUS Postprocessing results · p. 9
  • The CAMUS results show no significant degradation in anatomical or clinical metrics after postprocessing.Table V reports accuracy and clinical metrics for segmentation methods with and without postprocessing.Source: 3) CAMUS Postprocessing results · p. 9, Table V · p. 9

4) Inter-observer variability:

  • The first four echocardiographic methods remain within observer variability after postprocessing, against an average Dice of 0.899 and Hausdorff distance of 7.34 mm.The passage identifies these as the inter-observer benchmarks reported in.Source: 4) Inter-observer variability · p. 9
  • The authors conclude that anatomical guidelines are followed without degrading the overall accuracy of highly effective methods.Source: 4) Inter-observer variability · p. 9
  • For inputs with little or no ground-truth overlap, correction seeks the closest valid shape and may not move toward ground truth.The authors describe a severely degenerated example in which inaccurate registration also produced a sideways shape.Source: 5) Post-processsing degenerated results · p. 9

5) Post-processsing degenerated results:

The conclusion describes a post-processing cVAE that replaces invalid latent vectors with nearby valid ones. Tests across 18 segmentation methods covered short-axis MRI and long-axis ultrasound, with anatomical criteria used to define validity.

V. CONCLUSION

  • The post-processing cVAE replaces an invalid shape’s latent vector with a nearby valid vector to produce a close, correct shape.Source: §V · p. 9
  • The framework was tested on 18 segmentation methods and was effective for short-axis MRI and long-axis ultrasound views.Source: §V · p. 9
  • Figure 7 compares input, ground truth, original prediction, and postprocessed prediction for degenerated cases.Source: Figure 7 · p. 10
  • The criteria comprise 16 anatomical checks for short-axis views and 12 for long-axis views, and need not be differentiable because they are not included in the loss.Source: §V · p. 10
  • Warping did not significantly change overall geometrical or clinical metrics; selected methods were within inter-expert variation and guaranteed to follow user-defined anatomical guidelines.The cited methods are Isensee, Zotti-2, Khened, and Baumgartner for ACDC, and LUNet for CAMUS.Source: §V · p. 10
Loading 2006.08825v1…