Source-linked AI summary
Automatic segmentation of MR brain images with a convolutional neural network
Pim Moeskops, Max A. Viergever, Adriënne M. Mendrik, Linda S. de Vries, Manon J. N. L. Benders, Ivana Išgum
TL;DR
Automatic MR brain segmentation supports quantitative assessment in large-scale studies, but existing methods can still contain inaccuracies. This paper develops a multi-scale CNN that learns relevant information from training data and achieves accurate segmentation across images acquired at different ages and with different protocols.
Problem
Automatic MR brain segmentation is needed for quantitative assessment in large-scale studies, while existing automatic methods still show various inaccuracies.
Method
A multi-scale CNN segments anatomical MR images by combining multiple patch and convolution-kernel sizes while learning relevant intensity and spatial characteristics without explicitly defined spatial features.
Results
Accurate segmentation results were obtained across preterm neonatal and adult images acquired at different ages and with coronal or axial T1- or T2-weighted protocols.
Takeaways & Limitations
Using a single T1- or T2-weighted image allows segmentation across acquisition protocols without requiring registration between multiple acquisitions.
Takeaways & Limitations
The method might be limited for images containing abnormalities that were not included in the training set.
Abstract
from arXiv · showhide
Automatic segmentation in MR brain images is important for quantitative analysis in large-scale studies with images acquired at all ages. This paper presents a method for the automatic segmentation of MR brain images into a number of tissue classes using a convolutional neural network. To ensure that the method obtains accurate segmentation details as well as spatial consistency, the network uses multiple patch sizes and multiple convolution kernel sizes to acquire multi-scale information about each voxel. The method is not dependent on explicit features, but learns to recognise the information that is important for the classification based on training data. The method requires a single anatomical MR image only. The segmentation method is applied to five different data sets: coronal T2-weighted images of preterm infants acquired at 30 weeks postmenstrual age (PMA) and 40 weeks PMA, axial T2- weighted images of preterm infants acquired at 40 weeks PMA, axial T1-weighted images of ageing adults acquired at an average age of 70 years, and T1-weighted images of young adults acquired at an average age of 23 years. The method obtained the following average Dice coefficients over all segmented tissue classes for each data set, respectively: 0.87, 0.82, 0.84, 0.86 and 0.91. The results demonstrate that the method obtains accurate segmentations in all five sets, and hence demonstrates its robustness to differences in age and acquisition protocol.
I. INTRODUCTION
Automatic MR brain segmentation supports quantitative analysis across ages, but existing methods still show inaccuracies. The paper proposes a multi-scale CNN that learns spatial and intensity characteristics from training data without explicitly defined spatial features.
- Automatic MR brain segmentation is important for quantitative assessment in large-scale studies spanning images acquired at all ages.
- Existing segmentation methods use explicit spatial and intensity information, supplied through atlases, handcrafted features, or intensity matching.
- Despite accurate overall segmentations in recent challenges, different methods perform best for different tissues and various inaccuracies remain.
- The proposed method uses multiple patch and kernel sizes in a multi-scale CNN to estimate local intensity and spatial characteristics.
- The method is evaluated across developing neonates, young adults, ageing adults, and both coronal and axial images to test adaptation based on training data.
II. METHOD
The method classifies each voxel with a multi-branch CNN that combines patches and kernels at multiple scales. Separate branches preserve scale-specific processing before a shared final tissue-classification output.
- Each voxel is classified from centered image patches at multiple sizes, with larger patches supplying spatial context and smaller patches supplying local detail.
- The network uses 9, 8, or 7 output classes for neonatal, ageing adult, or young adult images, respectively, including background.
- Separate CNN branches are used for each patch size, allowing scale-specific weights and biases, while only the output layer is shared.
- The branches connect through separate fully connected layers to one softmax node per tissue class and background.
- Class-balanced sampling extracts a defined number of samples per tissue class from each training image to avoid bias toward larger tissues.
III. DATA
The study evaluates the CNN on five volumetric MR image sets: three T2-weighted neonatal sets and two T1-weighted adult sets. Neonatal images were manually segmented into eight tissue classes.
- The evaluation comprises three volumetric T2-weighted MR image sets from preterm infants and two volumetric T1-weighted sets from adults.
- Neonatal T2-weighted images provide tissue contrast comparable to adult T1-weighted images because WM and GM contrast is inverted between age groups.
- A total of 22 preterm-infant images were acquired on a Philips Achieva 3T scanner under standard neonatal clinical practice.
- Ten coronal neonatal images were acquired at 30.9 ± 0.6 weeks postmenstrual age.
- Experts manually segmented neonatal images into eight tissue classes, including cerebellum, white matter, basal ganglia and thalami, CSF, brain stem, and cortical grey matter.
B. Ageing adult images
The adult data comprise axial ageing-adult images and young-adult images from separate acquisition sources. Their manual labels were adapted to the tissue definitions used in this study.
- Ageing adult images: The ageing-adult participants had varying degrees of atrophy and white matter lesions.
- Ageing adult images: Ageing-adult images were manually segmented into seven classes because all adult white matter is myelinated and therefore represented by one WM class.
- Ageing adult images: Fifteen young-adult images were acquired at 23.0 ± 4.1 years on a Siemens Vision 1.5T scanner and resized to 1.0 × 1.0 × 1.0 mm3 isotropic resolution.
- Ageing adult images: Young-adult manual labels originally containing 134 classes were merged into the study’s tissue classes, yielding six tissue classes because eCSF was not segmented.
D. Evaluation
The method was evaluated using 3D overlap and surface-distance metrics, with preprocessing, training, and test configurations defined across multiple image sets.
- The automatic segmentations were evaluated in 3D using Dice coefficients and mean surface distances.
- Parameter settings were defined in preliminary leave-one-subject-out experiments using five 30-week-PMA images, with five independent images reserved for evaluation.
- The images underwent bias correction, brain masking, and intensity scaling before classification.
- Acquisition parameters for the image sets are listed in Table I.
- Figure 2 visualizes trained 5 × 5, 7 × 7, and 9 × 9 convolution kernels and their responses on a T2-weighted test image.
- Figure 3 compares Dice coefficients across training epochs for 25 000 and 50 000 randomly selected samples per class.
- Figure 4 presents tissue-class segmentations across neonatal and adult datasets, comparing input images, manual labels, and automatic results.
B. CNN parameters
The CNN combines multiple in-plane patch sizes and convolution-kernel scales, balances class sampling during training, and reports results across several benchmark configurations.
- Three in-plane patches of 25 × 25, 51 × 51, and 75 × 75 voxels are extracted around each voxel of interest.
- The first CNN layers use 24 kernels per patch size, with kernel sizes of 5 × 5, 7 × 7, and 9 × 9 voxels respectively.
- The network produces softmax outputs for tissue classes and background, with class counts varying between neonatal and adult images.
- Class-balanced training samples are extracted from each class because tissue classes contain very different numbers of voxels.
- Table II reports Dice coefficients and mean surface distances for eight experimental training and testing configurations.
- Table III provides literature and challenge Dice results only as an indication because evaluations used different datasets and tissue definitions.
- Table IV ranks methods by average Dice coefficient over all eight tissue classes using three test images.
- The selected training configuration uses 10 epochs and 50 000 samples per class from each training image.
C. Evaluation experiments
Evaluation covered neonatal and adult images using independent testing, leave-one-subject-out validation, and varying training-set sizes.
- The 30-week-PMA evaluation used five training images and five independent test images.
- The 40-week-PMA coronal and axial evaluations used leave-one-subject-out cross-validation because few training images were available.
- Axial 40-week-PMA results were separately reported for five test images included in the NeoBrainS12 challenge.
- Ageing-adult performance was evaluated with five training images and 15 test images, enabling indirect comparison with MRBrainS13.
- Young-adult performance used five training and 10 test images, with an additional evaluation using 15 training and 20 independent test images.
D. Comparison with previous methods
The paper compares its segmentation results with prior literature, challenge results, and alternative multi-scale configurations, while noting limits on direct comparability.
- Published neonatal segmentation results are presented as indicative because studies used different scans and tissue definitions.
- Challenge tables list the best results per tissue class, which may come from different methods rather than one method.
- The proposed method was directly compared with methods segmenting the same eight tissue classes in the NeoBrainS12 images.
- The proposed multi-scale approach was also compared with a layer-combination approach using the largest 75 × 75 voxel patch.
F. Single-plane approach
The study tested whether additional slice directions and 3D inputs improved segmentation beyond in-plane patches. These alternatives were evaluated across neonatal and young-adult images, with limited or no improvement over the single-plane approach.
- F. Single-plane approach: The method was additionally evaluated with consecutive-slice, orthogonal-plane, and 3D patches to motivate using patches from acquisition planes only.The experiments included three consecutive slices, interpolated orthogonal planes, and interpolated 3D patches.
- F. Single-plane approach: 0.7353 overall average Dice coefficient was obtained for 134-class segmentation of 20 young-adult test images using orthogonal patches.The result was 0.7170 for cortical regions and 0.7850 for non-cortical regions.
V. DISCUSSION
The multi-scale CNN produced accurate segmentations across ages and acquisition protocols, while its patch-scale combination balanced local detail with spatial consistency. The discussion also identifies scope boundaries involving tissue visibility, training diversity, brain masking, architecture choices, and unseen abnormalities.
- V. DISCUSSION: Accurate segmentation was obtained across preterm neonatal and adult images acquired at different ages and with coronal or axial T1- or T2-weighted protocols.The method required no postprocessing of the segmentation results.
- V. DISCUSSION: mWM was the neonatal exception to accurate Dice-based segmentation because it contains few voxels, is poorly visible in T2-weighted images, and is difficult to annotate manually.Its location was nevertheless recognized relatively well according to mean surface distances and visual results.
- V. DISCUSSION: Only in-plane information was used because anisotropic voxel sizes made orthogonal or 3D patches non-square, cubic, or strongly resampled.Additional experiments did not provide information enabling more accurate classification; performance remained 0.87 at 30 weeks PMA and increased from 0.90 to 0.91 for young adults.
- V. DISCUSSION: 0.84 average Dice coefficient was achieved for cGM with three combined patch sizes, compared with 0.78 for each patch size separately.The smallest patch captured local texture but missed spatial consistency, whereas the largest produced smoother segmentations but missed small details.
- V. DISCUSSION: Application may be limited for abnormalities absent from the training set, and brain masking can increase computational load if omitted.The authors also note that additional architecture optimization remains possible.
- V. DISCUSSION: More diverse training images could improve performance or enable cross-dataset generalization, but generalization might reduce performance on individual datasets.The CNN training samples came from a limited number of training images despite voxel classification providing many samples.
VI. CONCLUSION
The presented CNN method produced accurate MR brain image segmentations across different ages and acquisition protocols.
- VI. CONCLUSION: The CNN method showed accurate segmentation results for images acquired at different ages and with different acquisition protocols.