PaperlayerView exact revision on arXiv

Source-linked AI summary

SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining

Benjamin Billot, Douglas N. Greve, Oula Puonti, Axel Thielscher, Koen Van Leemput, Bruce Fischl, Adrian V. Dalca, Juan Eugenio Iglesias

arXiv:2107.09559v4eess.IVcs.CV

TL;DR

Brain-segmentation CNNs struggle with unseen contrasts and resolutions, limiting analysis of heterogeneous clinical scans. SynthSeg trains once on segmentation-conditioned synthetic data with fully randomised acquisition properties, and achieves broad generalisation across brain and cardiac MRI/CT settings. The evaluation supports strong cross-domain performance, while the authors note that automated label maps comprise a substantial part of evaluation.

  • Problem

    CNN-based brain segmentation is sensitive to contrast and resolution changes, creating a need for robust analysis across heterogeneous clinical scans.

  • Method

    SynthSeg trains a segmentation network on synthetic scans generated from label maps, fully randomising contrast, resolution, orientation, and artefacts.

  • Results

    Across 5,000 scans spanning eight datasets, six modalities, and 10 resolutions, SynthSeg maintained uniform accuracy and outperformed domain adaptation and Bayesian segmentation overall.

  • Takeaways & Limitations

    SynthSeg enables segmentation of varied brain MRI and CT scans without retraining or fine-tuning, supporting analysis of heterogeneous clinical data.

  • Takeaways & Limitations

    A substantial proportion of evaluation label maps were automated, although the authors report similar generalisation with manual and automated ground truths.

Abstract

from arXiv · show

Despite advances in data augmentation and transfer learning, convolutional neural networks (CNNs) difficultly generalise to unseen domains. When segmenting brain scans, CNNs are highly sensitive to changes in resolution and contrast: even within the same MRI modality, performance can decrease across datasets. Here we introduce SynthSeg, the first segmentation CNN robust against changes in contrast and resolution. SynthSeg is trained with synthetic data sampled from a generative model conditioned on segmentations. Crucially, we adopt a domain randomisation strategy where we fully randomise the contrast and resolution of the synthetic training data. Consequently, SynthSeg can segment real scans from a wide range of target domains without retraining or fine-tuning, which enables straightforward analysis of huge amounts of heterogeneous clinical data. Because SynthSeg only requires segmentations to be trained (no images), it can learn from labels obtained by automated methods on diverse populations (e.g., ageing and diseased), thus achieving robustness to a wide range of morphological variability. We demonstrate SynthSeg on 5,000 scans of six modalities (including CT) and ten resolutions, where it exhibits unparalleled generalisation compared with supervised CNNs, state-of-the-art domain adaptation, and Bayesian segmentation. Finally, we demonstrate the generalisability of SynthSeg by applying it to cardiac MRI and CT scans.

1. Introduction

SynthSeg addresses the clinical variability that limits automated brain-scan segmentation by training once on highly diverse synthetic data. It segments scans across contrasts, resolutions, ages, and pathologies without retraining or fine-tuning.

  • 1.1. Motivation: Clinical scans vary widely in orientation, slice spacing, resolution, and partial voluming, limiting supervised CNN deployment.Thick slices mix tissue types within voxels, complicating segmentation.
  • 1.1. Motivation: The lack of robust tools hinders clinical morphometry and leaves large collections of heterogeneous hospital scans unexplored.The authors identify a need for fast, accurate, reproducible segmentation across contrasts, resolutions, and populations.
  • 1.2. Contributions: SynthSeg trains a single network on synthetic scans with fully randomised orientation, contrast, resolution, and artefacts.The network is never exposed to real scans during training and learns from segmentation-conditioned samples.
  • 1.2. Contributions: The same network segments test scans spanning varied contrasts, resolutions, ages, atrophy, and white matter lesions without retraining or fine-tuning.Figure 1 illustrates synthetic training samples and test-time segmentations across these conditions.
  • 1.2. Contributions: SynthSeg requires only anatomical label maps for training, which can include automatically generated maps from diverse populations.This avoids requiring paired real images and labels for every new domain.
  • 1.2. Contributions: SynthSeg achieves nearly supervised-CNN accuracy on training domains while generalising across domains and modalities.The paper reports consistent advantages over domain adaptation and Bayesian segmentation, including cross-modality cardiac segmentation.

2. Related works

Prior approaches address contrast or domain shifts through Bayesian models, augmentation, or domain adaptation, but generally remain domain-specific. SynthSeg instead combines a segmentation-conditioned generative model with domain randomisation to target broad unseen-domain robustness.

  • Bayesian segmentation: Bayesian segmentation achieves contrast robustness by estimating an unsupervised intensity likelihood on each test scan.Its generative model combines an anatomical prior with an intensity likelihood, often a Gaussian Mixture Model.
  • Supervised CNNs: Supervised CNNs are accurate and fast on training domains but generalise poorly to unseen contrasts and resolutions.They commonly require new labelled data and retraining for each domain combination.
  • Augmentation and representation learning: Data augmentation and contrastive or adversarial methods improve generalisation, but conventional augmentation relies on handcrafted parameters.Semi-supervised methods can optimise augmentation parameters during training.
  • Domain adaptation: Domain adaptation bridges a source domain and a specific target domain but still requires retraining for each new domain.Approaches include shared latent spaces and image-to-image translation.
  • Domain randomisation: Domain randomisation fully randomises generative-model parameters to enlarge synthetic-data diversity beyond any specific target domain.The strategy aims to bridge multiple domain gaps within a shared semantic space rather than solve them separately.
  • Domain randomisation: SynthSeg combines domain randomisation with a Bayesian-segmentation-inspired generative model to segment brain scans across contrasts and resolutions without retraining.Synthetic images are generated with perfect ground-truth label maps by construction.

3.1. Generative model

SynthSeg generates synthetic training pairs from label maps while randomising morphology, contrast, artefacts, and resolution. This domain-randomised process is designed to expose the network to diverse acquisition conditions without using real training images.

  • Synthetic scans are generated on the fly with fully randomised combinations of contrast, resolution, morphology, artefacts, and noise.
  • Label map selection and spatial augmentation: The generative process samples a label map, applies affine and diffeomorphic spatial deformations, and forms an augmented map for image synthesis.The nonlinear deformation is constructed from an integrated stationary velocity field that avoids holes or foldings.
  • Contrast and intensity synthesis: A Gaussian mixture model conditioned on the deformed labels generates the initial high-resolution image with minibatch-randomised tissue means and standard deviations.The GMM standard deviations model both tissue heterogeneity and scanner thermal noise.
  • Bias field and intensity augmentation: Bias-field corruption, exponential intensity augmentation, and random Gamma transforms further vary the synthetic intensity distributions.The final high-resolution image is rescaled to [0, 1] before the Gamma transform.
  • Simulation of resolution variability: Resolution variability is simulated by randomly selecting acquisition direction, slice spacing, and thickness, then blurring, downsampling, and resampling to high resolution.The CNN is trained to produce crisp high-resolution segmentations despite simulated low-resolution inputs.
  • Training targets: Training targets retain only the K′ ≤ K labels selected for segmentation, with unwanted structures reset to background.
  • Domain randomisation: The central contribution is domain randomisation, with uniform-prior hyperparameters tuned on validation data and assessed through sensitivity analysis.

3.2. Segmentation network and learning

The segmentation model is a 3D UNet trained on synthetic image-target pairs using soft Dice loss and Adam optimisation. The generative model and network are implemented as one concatenated model.

  • Segmentation architecture: SynthSeg uses a five-level 3D UNet with batch normalisation, max-pooling on the contracting path, and upsampling on the expanding path.Each level contains two convolutional layers with 3 × 3 × 3 kernels.
  • Objective: Training uses average soft Dice loss between the network’s soft predictions and one-hot encoded target labels.
  • Optimisation: The network is optimised with Adam for 300,000 steps using a learning rate of 10^-4 and batch size 1.Training is repeated twice, with weights saved every 10,000 steps and the retained model selected using validation data.

3.3. Inference

At inference, scans are resampled and intensity-normalised before segmentation, with test-time augmentation used to slightly improve validation results.

  • Preprocessing: Test scans are resampled to the high-resolution grid with trilinear interpolation and normalised between 0 and 1 using the 1st and 99th percentiles.
  • Prediction: The preprocessed scans are fed to SynthSeg to obtain soft prediction maps for each label, using two augmented versions of each test scan.Test-time augmentation slightly improved results on the validation set.

4. General Experimental Setup

The evaluation covers heterogeneous brain datasets spanning modalities, populations, and resolutions, using manual or quality-controlled automated labels. SynthSeg is compared with supervised, adaptation, and Bayesian alternatives using Dice and surface-distance metrics.

  • Datasets: The experiments use eight datasets comprising 5,000 scans across six modalities and ten resolutions.
  • Datasets: Test data include FLAIR scans at 1 × 1 × 5 mm axial resolution and CT scans at 1 × 1 × 3 mm axial resolution.
  • Resolution evaluation: Additional resolution tests downsample 1 mm isotropic T1-39, FSM, and MSp scans to 3, 5, and 7 mm spacing in axial, coronal, and sagittal directions.These simulations assess segmentation accuracy as a function of resolution.
  • Ground truths: Ground truths combine manual labels with FreeSurfer segmentations that undergo visual quality control, enabling evaluation across large heterogeneous datasets.FreeSurfer Dice scores are reported in the range 0.85-0.88, supporting reliable assessment below 0.85.
  • Training data: SynthSeg training uses 20 T1-39, 500 HCP, and 500 ADNI label maps, with no real scans included in training.Mixing label maps increases morphological variety, while synthetic images remain perfectly aligned with their segmentations.
  • Competing methods: The comparison includes a supervised T1 baseline, nnUNet, test-time adaptation, SIFA, and the Bayesian SAMSEG framework.
  • Evaluation: Methods are assessed using hard Dice scores and the 95th percentile surface distance in millimetres.

5. Experiments and Results

SynthSeg is evaluated across datasets, contrasts, resolutions, and competing methods, showing strong accuracy and generalisation without exposure to real images during training. It also learns consistent representations across contrast and resolution changes.

  • SynthSeg reaches 0.88 Dice on T1-39 versus 0.91 for supervised networks, despite never seeing real images during training.
  • SynthSeg generalises better than supervised networks to intra-modality contrast changes, with an average 2.5-point Dice advantage over the T1 baseline on other T1 datasets.
  • SynthSeg outperforms tested domain adaptation methods across contrasts and resolutions without requiring retraining, whereas TTA fluctuates for larger domain gaps and SIFA improves CT SD95 by 14.92 mm.
  • SAMSEG averages 0.83 Dice at 1 mm but declines to 0.71 on 3 mm CT and 0.64 on 5 mm FLAIR because it does not model partial volume effects.
  • SynthSeg produces nearly identical final-layer features for 1 mm T1 and 5 mm axial T2 scans of the same subject.
  • SynthSeg maintains accurate segmentations across tested resolutions, losing 3.8 Dice points from 1 mm to 7 mm, compared with 7.6 points for SAMSEG.

T1mix FSM-DBS FLAIR CT

Experiments examine SynthSeg across contrast and resolution shifts, training-map quantity, morphological variability, neuroimaging group analysis, and cardiac MRI and CT. The results show robust segmentation and transfer across these settings, with some accuracy reductions for thin structures and low-resolution data.

  • T1mix FSM-DBS FLAIR CT: SynthSeg produces very accurate segmentations across all tested contrasts and resolutions, while competing methods show domain-specific errors or partial-volume sensitivity.
  • T1mix FSM-DBS FLAIR CT: SynthSeg’s advantage over the T1 baseline narrows as resolution decreases and nearly vanishes at 7 mm, making it particularly useful for low-resolution scans.
  • Ablations on DR and training label maps: Narrowing synthetic intensity distributions reduces mean accuracy by 1.4 and 2.6 Dice points for resolution-specific and contrast-resolution-specific variants, respectively.
  • Ablations on DR and training label maps: One training label map yields Dice scores between 0.68 and 0.80, while accuracy generally plateaus after five maps, except for MSp-PD at ten.
  • Ablations on DR and training label maps: Adding automated maps increases Dice by 1.9 points on ADNI and 2.0 points on FLAIR, and by 4.7 points for ADNI subjects with the largest ventricular volumes.
  • Alzheimer’s Disease volumetric study: For hippocampal atrophy analysis, SynthSeg and SAMSEG produce Cohen’s d values close to the ground truth on high-resolution T1 scans, while 5 mm FLAIR is more difficult.
  • Cardiac generalisability: SynthSeg segments all seven cardiac regions above 0.8 Dice, with mean Dice scores of 0.84 for MMWHS MRI and 0.88 for MMWHS CT without retraining.

6. Discussion

SynthSeg achieves contrast- and resolution-robust segmentation through domain randomisation and synthetic training, while reducing labeling demands and extending across morphologies and imaging domains.

  • SynthSeg trains once on synthetic scans with fully randomised contrast and resolution, encouraging domain-independent features without retraining or fine-tuning.The strategy deliberately generates highly diverse samples rather than realistic ones.
  • Contrast- and resolution-specific SynthSeg variants perform more poorly, supporting domain randomisation over domain-constrained training.The authors attribute this partly to mitigating generative-model assumptions and to evidence favoring augmentation beyond realism.
  • SynthSeg reduces labeling labor because it needs one anatomical label set rather than paired images and labels for every domain, and can use quality-checked automated labels.Synthetic scans are generated directly from their ground truths, ensuring alignment; automated labels also improve robustness to morphological variability.
  • The Gaussian model assumes training label maps include all tissues present at test time, though intensity clustering can supply labels for missing structures.This strategy enabled state-of-the-art cardiac segmentation despite finer structures absent from the original label maps.
  • Evaluation used a high proportion of automated label maps, although manual datasets and visual quality control supported comparable generalisation findings.The paper also cites indirect evaluation as evidence of accuracy and clinical utility.
  • SynthSeg addresses variable clinical acquisition procedures that leave current segmentation routines unavailable for many clinical scans.Its generalisation supports analysis of heterogeneous clinical data without relying on companion 1 mm T1 scans or domain-specific retraining.

7. Conclusion

The paper concludes that SynthSeg generalises across brain MRI and CT contrasts and resolutions without retraining, while also showing promise for cardiac segmentation and broader medical imaging.

  • SynthSeg maintained uniform accuracy across 5,000 scans spanning eight datasets, six modalities, and 10 resolutions, nearly matching supervised CNNs on their training domain.It also outperformed domain adaptation methods for larger domain shifts and Bayesian segmentation across tested datasets.
  • SynthSeg was more accurate than Bayesian segmentation, robust to partial-volume effects, and much faster in the reported evaluations.
  • SynthSeg precisely detected Alzheimer’s disease atrophy patterns on both high- and low-resolution scans, supporting clinical neuroimaging use.
  • State-of-the-art cardiac cross-modality segmentation indicates potential application beyond brain imaging.
  • Future work targets higher accuracy through improved synthetic-scan quality, output denoising, alternative architectures, and more precise evaluation across lesions and pathologies.
  • The trained model is distributed with FreeSurfer, removing retraining requirements and facilitating quantitative analysis of existing clinical scans.

Supplementary materials

The supplementary materials document synthetic examples, label-map variants, generative-model settings, competing-method modifications, and cardiac-training resources.

  • Examples and label maps: Supplementary figures show diverse generated scans and training label maps varying in lesions and skull-stripping quality.
  • Hyperparameters: Tables provide generative-model hyperparameters for brain and heart experiments, including intensity, rotation, and spatial-measure conventions.
  • Labels: Supplementary label tables list structures used for synthesis, prediction, and evaluation, including randomized skull-stripping variants.
  • Competing methods: Additional materials describe modifications to nnUNet, TTA, and SIFA, with validation-set improvements reported for each modified method.The listed changes include right/left flipping for nnUNet, a deeper TTA normalizer and atlas registration, and online augmentation for SIFA.
  • Training-size evaluation: The supplement reports retraining counts used to assess performance as the number of training subjects varied.
  • Cardiac segmentation: The cardiac extension combines manual foreground delineations with clustered sub-regions and automated background labels.
Loading 2107.09559v4…