Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets

Introduction

• Brain-structure segmentation is a prerequisite for analyses including volumetry, morphology, connectivity, physiology, and molecular studies, and MRI enables noninvasive in-vivo imaging with soft-tissue contrast.Source: Introduction · p. 1
• Clinical MRI scans vary in contrast, real-world artifacts, slice spacing, thickness, and orientation, making them difficult to analyze with conventional neuroimaging software.Clinical scans often use sparse 2D images in parallel planes rather than high-resolution 3D acquisitions.Source: Introduction · p. 1
• 10 million US clinical brain MRI scans acquired in 2019 alone were orders of magnitude more numerous than the largest research studies, motivating robust automated segmentation.Analyzing clinical data could increase sample size and statistical power and enable study of populations underrepresented in research datasets.Source: Introduction · p. 1
• Manual brain MRI segmentation requires costly expertise and is untenable at clinical scale, while restricting analysis to 1 mm T1-weighted scans would sharply reduce effective sample sizes.Such high-quality scans are expensive and seldom acquired in clinical settings.Source: Introduction · p. 1

• Available with FreeSurfer for out-of-the-box use, SynthSeg+ analyses scans of any contrast and resolution without retraining.The authors state that this availability facilitates use and enhances reproducibility.Source: Significance Statement · p. 1
• SynthSeg+ was evaluated on more than 15,000 heterogeneous clinical scans and enables automated segmentation and volumetry of large, uncurated clinical datasets.The suite is designed to handle variation in MR contrast, resolution, clinical artefacts, and subject populations.Source: Significance Statement · p. 2
• A hierarchical-network and denoiser architecture trained on synthetic data with domain randomisation increases robustness to clinical artefacts.The approach builds on the domain-randomisation strategy introduced by SynthSeg.Source: Significance Statement · p. 2
• The suite adds cortical parcellation, automated failure detection, and intracranial volume estimation, alongside whole-brain segmentation.Automated detection of faulty segmentations enables scalable quality control, and intracranial volume is a crucial covariate in volumetry.Source: Significance Statement · pp. 1–2

Whole-brain segmentation on clinical data

• Using denoiser D to correct S1 improved results by 2.3–5.1 Dice points over cascaded networks, while integrating D within SynthSeg+ beat postprocessing variants by 3.1–6.2 points.Although D smooths segmentations, S2 recovers sharp boundaries.Source: Results · p. 3
• On 500 heterogeneous clinical scans, SynthSeg+ achieved a mean 76 Dice points and outperformed SynthSeg by 23.5 points on big fails, 5.1 on mild fails, and 2.4 on passes.It yielded the best scores in all three visual-QC categories.Source: Results · p. 3, Figure 3 · p. 4

Robustness against MR contrast and resolution

• Both methods stayed above 80 Dice points across tested contrasts and resolutions; SynthSeg+ scored up to 1.6 points higher at lower resolutions, while SynthSeg led by up to 1.4 at 1 mm.SynthSeg+ scored higher at lower resolutions for all MRI modalities.Source: Results · p. 3, Figure 3 · p. 4

Cortex parcellation

• For cortical parcellation, SynthSeg+ beat SynthSeg by 17.1 Dice points on big fails and 1.8 on good cases, and led by 2.6 points on 7 mm proton density scans.SynthSeg led by 1.6 points on 1 mm T1-weighted scans; parcellation scores were below whole-brain scores but similar to those of a supervised T1-trained CNN (19).Source: Results · pp. 3–4, Figure 3 · p. 4

Automated quality control

• On 594 clinical scans, SynthSeg+’s QC regressor achieved accuracy above 96% and slightly outperformed Liu et al. (31) on every metric except specificity, with no significant AUC difference.The simpler D-versus-S2 strategy achieved 88.4% accuracy and 66.8% specificity; the table reports sensitivity, specificity, accuracy, and AUC.Source: Table 1 · p. 4, Results · p. 5

Alzheimer’s Disease volumetric study

• In the Alzheimer’s Disease study, hippocampal-volume effect sizes were 1.36–1.40 on T1 scans and 1.23 for SynthSeg and 1.20 for SynthSeg+ on 5 mm axial FLAIR scans.AUC comparisons showed no statistical differences between methods (all DeLong-test p-values above 0.3).Source: Table 2 · p. 5, Results · p. 5

Ageing study on over 14,000 clinical scans

• Across 14,752 clinical scans, SynthSeg+ reproduced ageing patterns reported in higher-quality studies (39–41), including white-matter volume peaking around 30, rising ventricular volume, and earlier thalamic than hippocampal or amygdala atrophy.Compared with SynthSeg, it produced fewer outliers and similar trajectories, including on scans with slice spacing above 6.5 mm.Source: Results · pp. 5–6, Figure 5 · p. 6

Discussion

• Hierarchical domain-agnostic CNN modules support robust segmentation across clinical contrasts, resolutions, artefacts, and subject populations, improving Dice scores and reducing ageing-study outliers.The denoiser-based approach consistently improves on cascaded networks and substantially outperforms postprocessing denoisers (28): it uses both the input scans and denoiser priors, while S2 refines their smooth boundaries.Source: Discussion · pp. 6–7
• Across more than 14,000 uncurated clinical scans, SynthSeg+ accurately replicated volume trajectories observed on higher-quality data.The comparison includes all scans, scans passing automated QC, and low-resolution scans; SynthSeg+ produced fewer outliers than SynthSeg and detected atrophy patterns at very low resolutions.Source: Figure 6 · p. 7, Discussion · p. 7
• Residual smoothing can slightly reduce accuracy relative to SynthSeg on high-resolution scans, which the discussion describes as a minor limitation compared with SynthSeg+'s robustness gains.Future work will address cortical surface placement for thickness estimates, multimodal processing, and performance assessment across pathologies.Source: Discussion · p. 7
• SynthSeg+ is the first method for volumetric cortical parcellation of clinical scans in the wild and maintains high performance across tested domains.On high-quality scans, it achieves scores competitive with supervised CNNs, while generalising across heterogeneous clinical acquisitions without retraining for each domain.Source: Discussion · p. 7
• Regression-based quality control detects the few SynthSeg+ failures, mainly linked to poor-quality acquisitions such as very low SNR or insufficient brain coverage.It slightly outperforms the tested approach (31); other automated QC methods were not evaluated because their lengthy iterative processes (29, 32) were considered impractical at the target scale.Source: Discussion · pp. 6–7
• SynthSeg+ runs out of the box on scans of any contrast and resolution, avoiding retraining and hyperparameter tuning, and is distributed publicly with FreeSurfer.The authors propose that robust, reproducible analysis could support larger clinical neuroimaging studies and applications including portable low-field MRI and clinical morphometry.Source: Discussion · pp. 7–8

Training datasets and population robustness

• Training uses 1,020 anatomical maps and domain-randomised synthetic images to expose the segmenters to variable shapes, contrasts, resolutions, artefacts, and morphologies.The training maps include OASIS, ADNI, and HCP segmentations. Synthetic examples apply spatial transforms, randomised intensity generation, bias fields, noise, blurring, and subsampling; low-resolution images are resampled to the 1 mm grid for training.Source: Materials and Methods · pp. 8–9

Brain MRI test datasets

• The main clinical cohort comprises 15,346 uncurated scans across varied contrasts, orientations, and resolutions, with 520 scans carrying ground-truth labels and 14,752 held out for indirect evaluation.Ground truths cover whole-brain segmentation, cortical parcellation, and ICV estimation; 500 labelled scans are used for testing and 20 for validation.Source: Materials and Methods · p. 8
• Additional test cohorts comprise 66 scans spanning T1-weighted, T2-weighted, DBS, and proton-density acquisitions, plus 100 ADNI participants with paired T1-weighted and FLAIR scans.The 66 scans have manual or semi-automated labels for 31 regions; cortical labels were obtained using FreeSurfer on T1 scans or companion T1 acquisitions. The ADNI subset includes healthy participants and participants diagnosed with Alzheimer’s disease.Source: Materials and Methods · p. 8

Deep hierarchical segmentation

• Four CNN modules produce coarse tissue labels, correct preliminary segmentations, refine 31 brain regions, and parcellate the cortex into 68 substructures.S1 segments four tissue groups; denoiser D addresses topological inconsistencies and larger errors; S2 uses corrected segmentations as priors while refining boundaries; S3 divides left and right cortex labels.Source: Materials and Methods · p. 8

Training of the denoiser; Automated QC module

• The denoiser learns to correct S1 errors from degraded real images, while a regressor flags 10 regions when predicted Dice falls below 0.65.For denoiser training, frozen S1 predictions are mapped to ground truth after on-the-fly image degradation. The regressor predicts performance scores from S2 segmentations, with the failure threshold selected on the validation set.Source: Materials and Methods · p. 9

Inference; Individual volumes and ICV estimation; Regression model for ageing

• Inference resamples scans to 1 mm isotropic resolution and takes 12–16 seconds per scan; regional volumes sum soft predictions, and ICV sums predicted intracranial structures.For multimodal inputs, each channel is segmented separately. The ageing model fits age B-splines, directional slice-spacing terms, and a gender bias by least squares using L-BFGS-B.Source: Materials and Methods · p. 9

Supplement 1: Demographic information and imaging protocols for the large clinical cohort.

• Fig. S7 summarizes age and gender per scan and the distributions of contrast, acquisition direction, and resolution across 15,346 scans from 1,367 MGH subject sessions.“Other” contrasts include unidentified contrasts (9.7%) and sequences with sub-percentage occurrence.Source: Figure S7 · p. 11

• Failures on low-contrast, low-SNR, or very-low-resolution scans included missing cerebellar or subcortical regions, amygdala labeled cortex, a missing ventricle labeled white matter, and posterior cerebellar over-segmentation.Source: Figure S8 · p. 12
• SynthSeg+’s regression-based QC very slightly outperformed Liu et al.’s method, with no statistical difference; denoiser-based D vs. S2 performed worst.Source: Figure S9 · p. 12
• 0.01 was the accuracy variation across thresholds from 0.55 to 0.75, showing relative robustness to threshold choice.Source: Figure S9 · p. 12

Supplement 4: Quantitative evaluation of the robustness to different subject populations

• Neighboring age groups showed no significant Dice-score differences at p=0.05; the minimum p-value was 0.062 between ages 45–60 and 60–75.The evaluation used two-sided Wilcoxon signed-rank tests on 500 clinical scans with ground truth.Source: Figure S10 · p. 13