Source-linked AI summary
Joint super-resolution and synthesis of 1 mm isotropic MP-RAGE volumes from clinical MRI exams with scans of different orientation, resolution and contrast
Juan Eugenio Iglesias, Benjamin Billot, Yael Balbastre, Azadeh Tabari, John Conklin, Daniel C. Alexander, Polina Golland, Brian L. Edlow, Bruce Fischl
TL;DR
Clinical morphometry tools are poorly suited to thick-slice scans with varying contrasts, resolutions, and orientations, leaving large clinical MRI collections difficult to analyze. SynthSR trains a CNN on synthetic images to reconstruct isotropic reference-contrast scans without high-resolution target-contrast training data. The reconstructions support subcortical volumetry and registration, while cortical thickness requires adequate image quality and joint synthesis from FLAIR remains limited.
Problem
Existing morphometry algorithms generally require near-isotropic, approximately 1 mm scans with specific contrasts, limiting quantitative analysis of thick-slice clinical MRI exams.
Method
SynthSR trains a CNN on synthetic images generated from 3D segmentations to combine scans with differing contrasts, resolutions, and orientations into an isotropic reference-contrast volume without high-resolution target-contrast data.
Results
SynthSR reconstructions support subcortical segmentation, volumetry, and registration; SR alone remains reliable at 5–7 mm spacing for these analyses, whereas cortical thickness is reliable at 3 mm spacing.
Takeaways & Limitations
SynthSR can make large collections of routine clinical MRI scans usable with existing research neuroimaging tools and may support studies with currently unattainable sample sizes.
Takeaways & Limitations
For 5 mm FLAIR synthesis, cortical thickness is not reliable and subcortical results are weaker than T1 super-resolution, although hippocampal-volume correlations exceed 0.75.
Abstract
from arXiv · showhide
Most existing algorithms for automatic 3D morphometry of human brain MRI scans are designed for data with near-isotropic voxels at approximately 1 mm resolution, and frequently have contrast constraints as well - typically requiring T1 scans (e.g., MP-RAGE). This limitation prevents the analysis of millions of MRI scans acquired with large inter-slice spacing ("thick slice") in clinical settings every year. The inability to quantitatively analyze these scans hinders the adoption of quantitative neuroimaging in healthcare, and precludes research studies that could attain huge sample sizes and hence greatly improve our understanding of the human brain. Recent advances in CNNs are producing outstanding results in super-resolution and contrast synthesis of MRI. However, these approaches are very sensitive to the contrast, resolution and orientation of the input images, and thus do not generalize to diverse clinical acquisition protocols - even within sites. Here we present SynthSR, a method to train a CNN that receives one or more thick-slice scans with different contrast, resolution and orientation, and produces an isotropic scan of canonical contrast (typically a 1 mm MP-RAGE). The presented method does not require any preprocessing, e.g., skull stripping or bias field correction. Crucially, SynthSR trains on synthetic input images generated from 3D segmentations, and can thus be used to train CNNs for any combination of contrasts, resolutions and orientations without high-resolution training data. We test the images generated with SynthSR in an array of common downstream analyses, and show that they can be reliably used for subcortical segmentation and volumetry, image registration (e.g., for tensor-based morphometry), and, if some image quality requirements are met, even cortical thickness morphometry. The source code is publicly available at github.com/BBillot/SynthSR.
1. Introduction
Clinical MRI exams often contain thick-slice, variably oriented scans with differing contrasts, while existing morphometry tools generally expect near-isotropic, contrast-specific data. SynthSR addresses this gap by training CNNs on synthetic multimodal images to reconstruct isotropic reference-contrast scans without real high-resolution target-contrast training data.
- Clinical imaging gap: Most neuroimaging software assumes near-isotropic MRI acquisitions, limiting segmentation and registration on routine clinical scans.Clinical exams commonly contain 20–30 slices spaced 5–7 mm apart, often with 0.5 mm in-plane resolution.
- Clinical imaging gap: Contrast-specific algorithms improve accuracy but struggle when clinical scans differ from the training intensity profiles, creating domain shift.This limitation is especially relevant across varying sequences, platforms, field strengths, and longitudinal protocols.
- Clinical imaging gap: The inability to analyze clinical scans quantitatively prevents precise longitudinal measurement and restricts research use of millions of archived MRI exams.Clinical consequences include subjective slice-position variability, while large clinical datasets could support substantially larger studies.
- Related approaches: Existing super-resolution and synthesis methods often require paired or high-resolution target-contrast data that are unavailable for diverse clinical protocols.Related approaches also may require paired data or cannot combine multiple scans with different resolutions and contrasts.
- Proposed approach: SynthSR trains CNNs with synthetic multimodal images that reproduce varied contrasts, resolutions, orientations, and acquisition artifacts without observing real high-resolution target-contrast scans.The generative process supports training for super-resolution, synthesis, or both across desired MRI configurations.
- Evaluation: The method is evaluated through downstream segmentation, volumetry, registration, and cortical-thickness analyses rather than relying only on image-similarity metrics.This evaluation targets the analyses ultimately used in neuroimaging applications.
2.1. Synthetic data generator
SynthSR uses a GPU-based generator to create diverse, artifact-corrupted multi-contrast scans from segmented high-resolution brains. The generator varies anatomy, contrast, resolution, orientation, registration, and acquisition artifacts while producing reliability maps for CNN training.
- Synthetic data generation: The generator samples synthetic scans at every minibatch, enabling CNN training for super-resolution and synthesis across resolutions and contrasts.Synthetic inputs are generated on the GPU, so sampling does not significantly slow training.
- Synthetic data generation: Training begins with high-resolution brain scans and corresponding segmentations, whose reference contrast typically defines the synthesized target contrast.For super-resolution alone, these high-resolution reference images are not required.
- Spatial augmentation: Spatial augmentation applies affine and diffeomorphic transformations to images and labels, using rotations, scaling, shearing, and smooth nonlinear deformation.Translations are excluded from the affine model because they are not helpful for dense prediction.
- Intensity generation: A label-conditioned multivariate Gaussian mixture model generates high-resolution intensities across multiple contrasts, followed by random gamma augmentation.The generator intentionally produces a wider contrast distribution than typically encountered in practice to improve CNN robustness.
- Acquisition and registration variability: The generator simulates bias fields, partial-volume effects, coordinate-frame variability, registration errors, and resampling artifacts in low-resolution inputs.Registration errors are included because test-time registration is imperfect.
- Reliability maps: Reliability maps encode voxel-wise confidence by indicating whether observations are measured or interpolated, improving CNN performance in practice.The generator produces reliability maps alongside the corrupted low-resolution volumes.
2.2. Learning and inference
SynthSR trains a CNN to predict either a high-resolution reference-contrast volume or a synthesized standard-contrast image from registered low-resolution scans and reliability maps. Training uses residual targets when a related input contrast exists, an L1 loss, and synthetic variability tuned to broaden robustness.
- Training objectives: The CNN predicts the desired output from registered low-resolution scans and their reliability maps, supporting separate super-resolution or joint super-resolution and synthesis modes.The synthetic inputs and reliability maps are generated on the fly during training.
- Training objectives: In super-resolution mode, the network regresses the residual between the synthetic high-resolution reference volume and the reference input, requiring no real images for training.Residual prediction is used because it is an easier optimization problem than direct image prediction.
- Training objectives: In joint super-resolution and synthesis, the network predicts a residual from a similar input contrast or directly predicts target intensities when no similar contrast exists.The target standard contrast is typically MPRAGE.
- Optimization: The CNN is optimized with Adam using the expected L1 error, chosen because pilot experiments produced more visually realistic results than L2 loss or structural similarity.The objective minimizes the discrepancy between the target and CNN output.
- Architecture: The model uses a five-level 3D U-net with convolutional layers, ELU activations, and feature counts that expand after pooling and contract after upsampling.The convolution kernels are 3×3×3.
- Hyperparameters: Generator hyperparameters are set by visual inspection to produce broad variation in shapes, artifacts, and intensity profiles, while contrast parameters are estimated from a small dataset-specific scan set.The intensity estimates use SAMSEG-derived class statistics, and mean ranges are widened fivefold to improve resilience.
2.3. Inference
At inference, SynthSR removes the synthetic generator and applies the trained CNN to preprocessed scans together with their reliability maps.
- Inference: Inference resamples all input scans to the target resolution while computing corresponding reliability maps before applying the trained model.The reference channel is handled first, and other channels are prepared using the same preprocessing procedure as during training.
2.4. Other practical considerations
Practical design choices improve realism and training stability without changing the jointly synthesized output. Inputs are normalized consistently, and training uses cropped volumes to fit GPU memory.
- Synthetic-image realism: Blurring synthetic high-resolution volumes with a 0.5 mm Gaussian kernel introduces spatial correlation and improves visual appeal in purely super-resolution mode.The blurring does not affect outputs when super-resolution and synthesis are performed jointly.
- Intensity normalization: Training and testing min-max normalize input volumes to the interval [0,1], with target normalization differing between super-resolution and joint synthesis modes.Super-resolution targets are normalized like the input to center residuals, whereas synthesis targets are scaled using white-matter intensity.
- Computational burden: Random 192 × 192 × 192 crops enable training on a 16GB GPU, with a learning rate of 10^-4 and 200,000 iterations.Training takes approximately 12 days on a Tesla P100 GPU, while inference is substantially less burdensome.
3. Experiments and results
Across synthetic downsampling, single-scan synthesis, and clinical exams, SynthSR produced isotropic images that improved downstream segmentation, volumetry, registration, and selected morphometric analyses over cubic interpolation or direct low-quality inputs. Performance remained strongest for subcortical measurements and registration, while cortical thickness depended on input contrast and resolution.
- Synthetic downsampling: SynthSR recovered high-resolution folding patterns and nearly original subcortical segmentations from 7 mm scans, whereas cubic interpolation lost folding and distorted structures.The CNN was trained only on synthetic data, yet its output retained minimal blurring relative to the original scan.
- Synthetic downsampling: SynthSR kept hippocampal volume errors under 3.5%, correlations above 0.97, and AD-control effect sizes above 1.30 at 7 mm spacing.The 1 mm reference effect size was 1.38; cubic-interpolation errors grew rapidly with slice spacing.
- Synthetic downsampling: SynthSR maintained hippocampal-volume agreement across the full volume range, avoiding the overestimation of severely atrophied hippocampi seen without super-resolution.The comparison used 7 mm scans against 1 mm reference volumes.
- Synthetic downsampling: At 3 mm spacing, SynthSR produced cortical-thickness maps similar to the isotropic reference and preserved many group-level clusters at 5 and 7 mm.Without super-resolution, cortical-thickness results were nearly unusable at 3 mm; SynthSR still showed reduced significance at larger spacings.
- Joint synthesis: For FLAIR inputs, SynthSR improved hippocampal volumetry and registration but produced unusable cortical-thickness maps because limited contrast caused segmentation errors.Hippocampal volume error improved to 8.4%, correlation to ρ = 0.76, and effect size to 0.90; TBM results were nearly identical to those from real 1 mm T1 scans.
- Clinical exams: In clinical multi-scan exams, SynthSR yielded plausible cortical surfaces, more precise subcortical segmentation, and detection of expected age-related atrophy and ventricular expansion.It detected negative age slopes with p<10^-4 for all structures, whereas the comparison method generally yielded p>10^-2 and missed the pallidum effect.
4. Discussion and conclusion
SynthSR generates isotropic reference-contrast volumes from routine anisotropic clinical MRI using synthetic training data, without requiring preprocessing or high-resolution input-modality training data. Across downstream analyses, it supports reliable registration and subcortical measurements in many settings, while cortical thickness and structurally disruptive pathology remain important boundaries.
- Method: SynthSR trains a regression CNN on synthetic scans that vary contrast, resolution, orientation, motion, bias fields, and registration errors.The simulations also include extracerebral tissue, allowing the method to avoid skull stripping, denoising, and bias-field correction.
- Method: SynthSR can use anisotropic clinical scans to produce isotropic volumes of a predefined reference contrast without high-resolution training data for the input modalities.The method is intended for routine clinical exams containing multiple 2D acquisitions with different contrasts and orientations.
- Downstream analyses: Registration and subcortical segmentation remain usable with SynthSR at 5–7 mm slice spacing, whereas cortical thickness requires substantially better input quality.For super-resolution alone, reliable cortical thickness analysis was demonstrated at 3 mm spacing; combined synthesis and super-resolution was less reliable.
- Downstream analyses: Hippocampal volumes from synthesized scans remained usable, with correlation over 0.75, but cortical thickness analysis was not reliable for 5 mm FLAIR inputs.The FLAIR orientation was approximately parallel to the hippocampal major axis, and subcortical results were weaker than for T1 super-resolution.
- Clinical-data evaluation: SynthSR improves image contrast for difficult boundaries and can recover subtler aging signatures missed by lower-resolution competing outputs, but cortical aging patterns remained undetected when image quality was insufficient.The reported example concerns improved contrast in regions such as the putamen and FreeSurfer detection of subtler pallidum aging effects.
- Scope and limitations: SynthSR is not intended to replace subject-specific MRI acquisition, and its output contrast depends on input quality rather than providing full explicit harmonization.The authors also caution that evaluation of abnormalities such as tumors may still require specific acquisitions or contrast agents.
- Future work: Future improvements include more robust architectures and augmentation, modality dropout for missing scans, and simulations of pathology such as tumors.The authors identify pathology modeling as crucial for clinical-scale deployment and note that more realistic simulations may be needed.
- Implications: SynthSR is publicly available and is intended to enable existing research-grade neuroimaging tools to analyze large collections of routine clinical MRI scans.The authors frame this as a route toward sample sizes and statistical power not attainable with conventional in vivo studies.