Source-linked AI summary
Automated sub-cortical brain structure segmentation combining spatial and deep convolutional features
Kaisar Kushibar, Sergi Valverde, Sandra Gonzalez-Villa, Jose Bernal, Mariano Cabezas, Arnau Oliver, Xavier Llado
TL;DR
Manual sub-cortical MRI segmentation is time-consuming and prone to variability, while smaller structures are often omitted from deep-learning segmentation approaches. This paper combines 2.5D CNN features with atlas-based spatial priors and trains with restricted sampling of difficult areas; it reports performance comparable to or better than established and recent methods across MICCAI 2012 and IBSR 18.
Problem
Manual sub-cortical MRI segmentation is time-consuming and variable, and many deep-learning approaches focus on larger structures rather than the hippocampus, amygdala, and accumbens.
Source: §1 · pp. 1–2Method
Results
Takeaways & Limitations
The method is publicly available, and the reported experiments find that spatial priors and restricted sampling significantly affect its accuracy.
Source: §4 · p. 8, §5 · p. 9CNN accuracy tends to decrease on image domains different from training, and the authors note limited evidence for performance when radiological training data are small or absent.
Source: §4 · p. 9
Abstract
from arXiv · showhide
Sub-cortical brain structure segmentation in Magnetic Resonance Images (MRI) has attracted the interest of the research community for a long time because morphological changes in these structures are related to different neurodegenerative disorders. However, manual segmentation of these structures can be tedious and prone to variability, highlighting the need for robust automated segmentation methods. In this paper, we present a novel convolutional neural network based approach for accurate segmentation of the sub-cortical brain structures that combines both convolutional and prior spatial features for improving the segmentation accuracy. In order to increase the accuracy of the automated segmentation, we propose to train the network using a restricted sample selection to force the network to learn the most difficult parts of the structures. We evaluate the accuracy of the proposed method on the public MICCAI 2012 challenge and IBSR 18 datasets, comparing it with different available state-of-the-art methods and other recently proposed deep learning approaches. On the MICCAI 2012 dataset, our method shows an excellent performance comparable to the best challenge participant strategy, while performing significantly better than state-of-the-art techniques such as FreeSurfer and FIRST. On the IBSR 18 dataset, our method also exhibits a significant increase in the performance with respect to not only FreeSurfer and FIRST, but also comparable or better results than other recent deep learning approaches. Moreover, our experiments show that both the addition of the spatial priors and the restricted sampling strategy have a significant effect on the accuracy of the proposed method. In order to encourage the reproducibility and the use of the proposed method, a public version of our approach is available to download for the neuroimaging community.
1. Introduction
Manual sub-cortical MRI segmentation is time-consuming and poorly reproducible, motivating accurate automated methods. The paper addresses coverage of smaller structures by proposing a 2.5D CNN that combines spatial features with a restricted sample-selection strategy.
1. Introduction
- Manual MRI segmentation is time-consuming and poorly reproducible because of inter- and intra-operator variability.The introduction links sub-cortical morphology to disorders and describes segmentation as useful for quantitative analysis.Source: §1 · p. 1
- Many reviewed deep-learning approaches focus on larger sub-cortical structures, while hippocampus, amygdala, and accumbens are often not considered.The paper notes these smaller structures are used in examining neurological disorders.Source: §1 · p. 2
- The proposed method combines three orthogonal-view CNN features with atlas-probability spatial features and introduces sample selection targeting difficult structural areas.The authors describe atlas-probability input for sub-cortical segmentation as, to their knowledge, the first such deep-learning method.Source: §1 · p. 2
- The method is evaluated on MICCAI 2012 and IBSR 18 and compared with classical and recent CNN strategies.The authors also state that the method is publicly available to the community.Source: §1 · p. 2
2. Method
The method combines appearance patches from three orthogonal MRI views with probabilistic atlas features in a CNN. Training emphasizes difficult boundary negatives, while testing predicts labels and filters spurious regions within an ROI.
2.1. Input features
- Three orthogonal-view 32 × 32 patches approximate 3D neighborhood information with lower computational and memory cost than 3D patches.Atlas probabilities are extracted after registering the Harvard-Oxford atlas into each subject's image space and are supplied as a 15-value vector per voxel.Source: §2.1 · pp. 2–3, Figure 1 · p. 3
2.2. CNN architecture
- Three patch-processing branches and an atlas-probability branch feed fully connected layers and a 15-class softmax output.The architecture uses convolutional and max-pooling layers for the image branches, with ReLU activations.Source: §2.2 · p. 3, Figure 1 · p. 3
2.3. CNN training
- Training uses categorical cross-entropy optimized with Adam and balances positive structure voxels against negative samples selected from nearby structure boundaries.Boundary-focused negatives are intended to emphasize low-contrast structure borders; the selected negatives lie within five voxels of the boundaries.Source: §2.3 · p. 3
2.4. CNN testing
- At testing, the CNN assigns each voxel its maximum-probability class, and post-processing retains only each class's largest connected region within the ROI.The ROI is automatically defined using the known central location of sub-cortical structures; boundary-only negative training can produce spurious outputs in unseen background areas.Source: §2.4 · p. 3, Figure 2 · p. 4
2.5. Implementation and technical details
- The method is implemented in Python using Lasagne and Theano, with training performed on a single TITAN-X GPU.The authors state that the implementation is publicly available for download.Source: §2.5 · p. 4
3. Results
The results section evaluates the proposed method on the MICCAI Multi-Atlas Labeling challenge dataset and the IBSR dataset.
3. Results
- Evaluation includes the MICCAI Multi-Atlas Labeling challenge dataset and the publicly available IBSR dataset.The corresponding dataset details and results are assigned to Sections 3.2 and 3.3.Source: §3 · p. 4
- The section presents results obtained by the proposed method across two datasets.The passage identifies the datasets as MICCAI and IBSR.Source: §3 · p. 4
- Dataset details and corresponding results are deferred to Sections 3.2 and 3.3.The passage does not report performance values.Source: §3 · p. 4
3.1. Evaluation measures
The evaluation uses overlap and spatial-distance measures to compare automatic and manual segmentations. It also applies Wilcoxon signed-rank tests to method differences and the effects of spatial features and sample selection.
3.1. Evaluation measures
- Dice Similarity Coefficient (DSC) measures overlap between automatic and manual segmentations on a scale from 0 to 1.Zero represents no overlap and one represents 100% overlap with the ground truth.Source: §3.1 · p. 4
- Hausdorff Distance (HD) measures spatial discrepancy as the maximum of the minimum distances between segmentation and ground-truth boundaries.The metric is based on Euclidean distances between voxels of the two segmentations.Source: §3.1 · p. 4
- Wilcoxon signed-rank tests assess differences in DSC and HD between the proposed and state-of-the-art methods and test the effects of spatial features and sample selection.Source: §3.1 · p. 4
3.2. MICCAI 2012 Dataset
The MICCAI 2012 evaluation tests the proposed method on the challenge split against established tools and PICSL. It reports performance comparable to PICSL, stronger results than FreeSurfer and FIRST, and shorter training and testing times than PICSL.
3.2.1. Experimental details
- The evaluation used 15 training and 20 testing volumes; training patches came from subcortical voxels and structure boundaries, with early stopping based on validation accuracy.Around 1.5M patches were extracted, with 75% used for training and 25% for validation.Source: §3.2.1 · p. 5
3.2.2. Comparison with other available methods
- The comparison included FreeSurfer and FIRST with default parameters, and PICSL, the MICCAI 2012 challenge winner; other CNN methods lacked individually reported structure-level numerical results for this dataset.The dataset provides manual ground truth, and the experiments evaluated 14 classes representing seven bilateral structures.Source: §3.2 · p. 4, §3.2.2 · p. 5
3.2.3. Results
- The method achieved an overall mean DSC of 0.869, significantly above FIRST (0.799) and FreeSurfer (0.725), and not significantly different from PICSL (0.867).Its HD performance was also significantly better than FIRST and FreeSurfer, with reductions of 1.17 mm and 2.86 mm, respectively.Source: §3.2.3 · p. 5
- Qualitatively, the proposed method and PICSL had consistent structure boundaries close to ground truth, unlike FreeSurfer's coarse boundaries and FIRST's poor overlap.Figure 3 compares outputs for six structures on the MICCAI 2012 dataset.Source: §3.2.3 · p. 5, Figure 3 · p. 5
- Training took around 8 hours and testing less than 5 minutes per subject volume, compared with PICSL's reported 330 CPU hours for training and more than 50 minutes per subject for segmentation.The proposed method's testing time includes atlas registration.Source: §3.2.3 · p. 5
3.3. IBSR 18 Dataset
On IBSR 18, the proposed method was significantly better than FreeSurfer and FIRST on overall DSC, and it compared favorably with several recent CNN methods. The evaluation used leave-one-subject-out cross-validation on a dataset with lower contrast and resolution than MICCAI 2012.
3.3. IBSR 18 Dataset
- IBSR 18 contains 18 manually segmented volumes with 32 classes; the experiments evaluated 14 classes for seven bilateral subcortical structures.The images have lower contrast and resolution than MICCAI 2012, making segmentation more challenging.Source: §3.3 · p. 5
3.3.1. Experimental details
- Because IBSR 18 had no prescribed train-test split, the method used leave-one-subject-out cross-validation, extracting around 1.1M patches per fold and applying early stopping.Each fold used 825K training patches and 220K validation patches.Source: §3.3.1 · p. 6
3.3.2. Comparison with other available methods
- Comparisons covered FreeSurfer and FIRST as well as four recent CNN approaches, but most CNN reports covered only selected structures and did not report HD values.HD comparisons were therefore available only for FreeSurfer, FIRST, and the proposed method.Source: §3.3.2 · p. 6
3.3.3. Results
- The proposed method's overall mean DSC was 0.843, significantly above FreeSurfer (0.740) and FIRST (0.808); its overall mean HD was 4.49, versus 5.21 and 4.50, respectively.The HD difference was significant versus FreeSurfer but not versus FIRST.Source: §3.3.3 · p. 6
- Against other CNN approaches, the method outperformed Shakeri et al. on eight structures and Mehta et al. on six; on six structures it scored 0.859 mean DSC versus MS-CNN's 0.807.Its performance was comparable to Dolz et al., whose averaged DSC was slightly higher on the four biggest structures.Source: §3.3.3 · p. 6
3.4. E
The spatial-prior ablation compares the proposed method with and without atlas-probability features. Including spatial features significantly improved overall DSC and helped segment difficult regions in some images.
3.4. E
- Including spatial features significantly improved overall DSC (p < 0.001) and improved results for almost all structures.The reported ablation analyzed DSC on MICCAI 2012; the authors omitted IBSR 18 results for simplicity, stating that they produced a similar outcome.Source: §3.4 · pp. 6–7
- The spatial-prior comparison examined difficult caudate, pallidum, and accumbens regions, where the method with spatial features segmented areas better.The authors associate the benefit with images having intensity and shape irregularities absent from the training images.Source: §3.4 · p. 7
3.5. Effect of sample selection
The sample-selection experiment tests training with random background voxels against selecting samples from structure boundaries. Boundary-focused sampling improved average and per-structure performance and produced better segmentation at structure borders.
3.5. Effect of sample selection
- Random sampling drew 1.5M voxels per epoch, split equally between subcortical structures and background; background samples came from the whole brain rather than only structure boundaries.Spatial features were included in this experiment.Source: §3.5 · p. 7
- Mean DSC was 0.869 with the final method versus 0.792 without the proposed sample-selection technique (p < 0.001).The random-sampling comparison retained spatial features; the final method used boundary sampling and spatial features.Source: §3.5 · p. 7
- The sample-selection technique significantly improved performance on average and for each structure, with better segmentation at the boundaries than random sampling.The authors note that boundary intensities are confounding, so voxel assignment to structure or background depends strongly on ground truth.Source: §3.5 · p. 8
4. Discussion
The discussion reports strong performance across MICCAI 2012 and IBSR 18, and explains how spatial priors and restricted sampling contribute to segmentation. It also identifies comparison constraints and limits on generalization to new imaging domains and disease-affected images.
4. Discussion
- Across MICCAI 2012 and IBSR 18, the method outperformed or matched classic and CNN approaches, slightly overperformed PICSL, and performed comparably to Dolz et al.Spatial agreement with manual segmentation was higher than FreeSurfer and FIRST across evaluated datasets.Source: §4 · p. 8
- Adding spatial information increases execution time because it requires atlas registration.Spatial priors can filter oversized misclassified regions that occur in unobserved brain areas after restricted sampling.Source: §4 · p. 8
- Comparisons with prior CNN methods are constrained because studies did not always cover all sub-cortical structures or use the same training methodology.On IBSR 18, a six-fold strategy differed from leave-one-out by only 0.005 DSC.Source: §4 · p. 8
- Atlas spatial priors guide difficult areas and add location information, helping the network account for anatomical shape despite intensity irregularities.The priors may also reduce effects from low contrast, poor resolution, noise, and artifacts near structure boundaries, but remain prone to registration errors.Source: §4 · pp. 8–9
- Restricted sampling trains on the most difficult negative samples, while spatial priors restrict inference to a sub-cortical region of interest and remaining false positives are post-processed.Without restricted sampling, the method performed similarly to Shakeri et al. and MS-CNN under the same conditions.Source: §4 · p. 9
- Accuracy may decrease on different scanners or imaging protocols, and applicability to images with neurological disease, including white-matter lesions, remains a constraint.The discussion notes that FIRST may be more appropriate when little or no training data is available.Source: §4 · p. 9
5. Conclusion
The conclusion presents a CNN approach that fuses convolutional and prior spatial features and uses restricted sampling. Across MICCAI 2012 and IBSR 18, it reports performance that outperforms or matches state-of-the-art methods.
5. Conclusion
- The proposed CNN combines convolutional and prior spatial features, with restricted sample selection aimed at learning difficult parts of structures.The conclusion reports experiments on the public MICCAI 2012 and IBSR 18 datasets.Source: §5 · p. 9
- Across MICCAI 2012 and IBSR 18, the method outperformed or showed comparable performance to FreeSurfer, FIRST, and recent CNN approaches.The conclusion attributes a significant impact on effectiveness to spatial priors and restricted sampling.Source: §5 · p. 9
- A public version of the method is available to the neuroimaging community to encourage reproducibility and use.Source: §5 · p. 9