Source-linked AI summary
3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study
J. Dolz, C. Desrosiers, I. Ben Ayed
TL;DR
Subcortical MRI segmentation needs methods that are accurate, automated, reproducible, and scalable beyond slow manual or registration-heavy approaches. The paper develops a 3D FCNN using small kernels and multiscale features, achieving state-of-the-art IBSR performance and robust large-scale results across 1112 ABIDE subjects from 17 sites. Its outputs were consistent with atlas-based segmentations while requiring a fraction of their runtime and avoiding registration or normalization steps.
Problem
Manual segmentation is prohibitively time-consuming, variable across raters, and impractical for very large multi-center datasets, motivating fast, accurate, reproducible, fully automated methods.
Method
The method uses a 3D fully convolutional network with small kernels, intermediate-layer multiscale features, end-to-end GPU training, and alignment-independent processing.
Results
Mean IBSR DSC values ranged from 0.83 to 0.91 and mean MHD values from 0.13 mm to 0.26 mm, while the method achieved state-of-the-art performance and robust multi-site results.
Takeaways & Limitations
The method produced atlas-consistent segmentations in less time and was evaluated on heterogeneous, unregistered data from 17 sites, supporting its convenience for massive multi-site studies.
Takeaways & Limitations
The architecture has a small receptive field constrained by the memory requirements of 3D convolutions, and clinician-based validation remains necessary for full validation.
Abstract
from arXiv · showhide
This study investigates a 3D and fully convolutional neural network (CNN) for subcortical brain structure segmentation in MRI. 3D CNN architectures have been generally avoided due to their computational and memory requirements during inference. We address the problem via small kernels, allowing deeper architectures. We further model both local and global context by embedding intermediate-layer outputs in the final prediction, which encourages consistency between features extracted at different scales and embeds fine-grained information directly in the segmentation process. Our model is efficiently trained end-to-end on a graphics processing unit (GPU), in a single stage, exploiting the dense inference capabilities of fully CNNs. We performed comprehensive experiments over two publicly available datasets. First, we demonstrate a state-of-the-art performance on the ISBR dataset. Then, we report a {\em large-scale} multi-site evaluation over 1112 unregistered subject datasets acquired from 17 different sites (ABIDE dataset), with ages ranging from 7 to 64 years, showing that our method is robust to various acquisition protocols, demographics and clinical factors. Our method yielded segmentations that are highly consistent with a standard atlas-based approach, while running in a fraction of the time needed by atlas-based methods and avoiding registration/normalization steps. This makes it convenient for massive multi-site neuroanatomical imaging studies. To the best of our knowledge, our work is the first to study subcortical structure segmentation on such large-scale and heterogeneous data.
1. Introduction
Subcortical segmentation is important for studying brain disorders, but manual and conventional automated approaches remain too slow, variable, or computationally demanding for large heterogeneous datasets. The study proposes a 3D fully convolutional approach and evaluates it across standard and large-scale multi-site datasets.
- Accurate segmentation supports studying brain disorders, structural abnormalities, disease progression, and treatment outcomes.
- Manual subcortical delineation is time-consuming, rater-dependent, inconsistent, and impractical for very large multi-center neuroimaging studies.
- 3D CNNs preserve anatomical context across MRI volume directions but have largely been avoided because of computational and memory demands during inference.
- The proposed 3D FCNN uses small kernels, multiscale intermediate features, GPU training, dense inference, and alignment-independent processing.
- Across IBSR and ABIDE, the method achieved state-of-the-art IBSR performance and was evaluated on 1112 unregistered subjects from 17 sites.
- The segmentations were highly consistent with a standard atlas-based approach, ran in a fraction of its time, and were positioned for massive multi-site studies.
2. Methods and materials
The methods section presents the proposed 3D FCNN architecture, its smaller-kernel and multiscale extensions, preprocessing and postprocessing, and the study design.
- The study first presents the proposed 3D FCNN architecture at the core of the segmentation method.
- The architecture is extended with additional smaller-kernel convolution layers and intermediate-layer multiscale information.
- The methods also describe data preprocessing, output postprocessing, and the study design.
2.1. The proposed 3D FCNN architecture
The baseline architecture is a 3D fully convolutional network that replaces fully connected layers with 1^1^1 convolutions, preserves spatial information, and produces class probabilities from convolutional feature maps.
- The baseline CNNbase uses three convolutional layers with 7^7^7 kernels before converting fully connected layers into 1^1^1 convolutional filters.
- Each convolutional filter produces feature maps from the previous layer or input volume, with biases and nonlinear activation functions.
- A convolution reduces each spatial dimension by the filter size minus one when no padding is used; for example, a 3^3^3 filter removes two voxels per dimension.
- The architecture avoids pooling and deconvolution because reducing and recovering spatial resolution can produce coarse segmentations.
- PReLU replaces ReLU by learning a coefficient for negative inputs, adapting rectification to inputs at negligible extra computational cost.
- The classification layer groups neurons into C feature maps and converts their outputs into normalized class probabilities with softmax.
2.2. Deeper architecture via small convolution kernels
Replacing large kernels with smaller successive kernels creates a deeper FCNN while preserving finer spatial resolution and reducing parameter count.
- Each original convolutional layer is replaced by three successive layers using 3^3^3 kernels instead of 7^7^7 kernels.
- The smaller-kernel design yields a deeper network with fewer parameters, enabling more complex feature hierarchies and reducing overfitting risk.
2.3. Combining features across multiple scales
The model combines deeper small-kernel convolutions with intermediate feature maps to integrate fine and coarse information into segmentation.
- CNN layers encode increasingly abstract features, motivating segmentation across multiple levels of abstraction.
- CNNmulti injects feature maps from convolutional layers 3, 6, and 9 into the fully connected layers.
- The multiscale model uses one highest-resolution 3D input and combines intermediate features rather than separately resampling inputs at multiple resolutions.
- Center-cropped 9^3 volumes from layers 3, 6, and 9 are stacked to align feature-map sizes while representing different resolutions and receptive fields.Layer 3 retains higher resolution and a smaller receptive field than layers 6 and 9.
2.4. Pre- and post-processing steps
The pipeline minimizes alignment-dependent preprocessing and removes small isolated predictions after segmentation, targeting robustness and cleaner outputs.
- Registration and strict preprocessing can be time-consuming and reduce generalization to differently processed MRI data.
- Preprocessing uses volume-wise intensity normalization, bias-field correction, and registration-free skull stripping.Normalization and bias correction reduce sensitivity to contrast and intensity bias; skull stripping mainly reduces input size.
- Post-processing keeps only the largest connected component for each class to remove small isolated segmentation regions.CRF regularization was tested but did not significantly improve accuracy.
2.5. Study design and experiment setup
The study evaluates three 3D FCNN architectures across controlled IBSR validation and heterogeneous ABIDE multi-site experiments, with defined training, validation, and testing procedures.
- Datasets: Experiments use two publicly available datasets: IBSR for quantitative comparison and ABIDE for heterogeneous multi-site evaluation.
- Datasets: IBSR contains 18 T1-weighted MRI scans with expert labels, focusing on eight bilateral subcortical structures and using 6-fold cross-validation.
- Datasets: ABIDE I includes 1112 subjects from 17 international sites, including 539 participants with ASD and 573 typical controls aged 7–64 years.
- Datasets: Because ABIDE lacks ground-truth labels, evaluation uses automatic FreeSurfer recon-all segmentations and tests imaging, demographic, and clinical factors.
- Datasets: ABIDE training uses 10 control subjects from each of 15 sites, with one subject per site for validation and remaining subjects for testing.
- Architectures: The architecture comparison includes CNNbase, CNNsingle with successive 3^3^3 convolutions, and CNNmulti with multiscale features; each predicts nine classes.
- Implementation: Training uses GPU-based 3D FCNN implementation, batches of five, momentum 0.6, and early termination after validation performance stopped improving beyond 30 epochs.
- Evaluation: Segmentation accuracy is assessed with overlap and boundary-oriented metrics, including Dice similarity coefficient and mean Hausdorff distance.
3. Results
On IBSR and ABIDE, the proposed CNNmulti architecture achieved strong segmentation accuracy, improved over simpler FCNN variants, and remained robust across structures, sites, demographics, and clinical groups. It also produced segmentations quickly and showed age- and structure-related performance differences.
- IBSR evaluation: Pallidum segmentation was significantly less accurate than thalamus, caudate, and putamen segmentation, likely because the pallidum is smaller.All four structures were slightly more accurate in the right hemisphere, but the differences were not statistically significant.
- IBSR evaluation: The IBSR evaluation found state-of-the-art performance, with DSC improvements of 5% to 13% over a recent 2D FCNN baseline.The reported improvements ranged from 5% for the thalamus to 13% for the caudate.
- ABIDE evaluation: Deeper networks with smaller kernels and multiscale intermediate-layer features significantly improved both DSC and MHD over the baseline architecture on ABIDE.Both comparisons were statistically significant with p < 0.01.
- ABIDE evaluation: Across ABIDE subject groups, mean DSC exceeded 0.90 for every structure except the pallidum, which reached 0.85, while mean MHD stayed below 0.25 mm.Figure 4 reports these values across age, diagnosis, and site-related groups.
- Clinical and demographic factors: Control subjects generally achieved higher accuracy than ASD subjects, while thalamus and caudate segmentation improved with age and younger-subject variability may be incompletely captured during training.For example, putamen DSC was 0.9127 in young controls versus 0.9055 in young ASD subjects, while caudate MHD was 0.1397 versus 0.2568.
- Generalization and efficiency: Performance on sites excluded from training showed no statistically significant difference from training-site performance, supporting robustness across acquisition protocols and site-related factors.The method took 2–3 minutes per ABIDE subject on a GPU and less than 5 minutes on a CPU, compared with several hours for FreeSurfer’s full pipeline.
4. Discussion
The 3D FCNN achieved strong and consistent segmentation across IBSR and heterogeneous, multi-site ABIDE data, including subjects with different diagnoses, ages, and acquisition settings. Its performance was comparable across aligned and unregistered data, although image quality exclusions and architecture choices define important boundaries.
- Evaluation: 0.83–0.91 mean DSC and 0.13–0.26 mm mean MHD on IBSR established state-of-the-art performance against recently proposed methods.On ABIDE, mean DSC ranged from 0.86 to 0.92 and mean MHD from 0.14 mm to 0.22 mm relative to FreeSurfer.
- Evaluation: 0.86–0.92 mean DSC and 0.14–0.22 mm mean MHD on ABIDE indicated segmentations consistent with FreeSurfer across target structures.The ABIDE evaluation covered data acquired across multiple sites and subject groups.
- Subject factors: Similar mean DSC and MHD for control and ASD subjects suggested robustness to diagnosis-related morphological differences.Younger subjects showed slightly lower accuracy, consistent with greater variability during brain development.
- Multi-site robustness: Comparable performance on training and previously unseen sites demonstrated robustness to varied imaging parameters and acquisition protocols.Training on samples spanning broader site variability helped alleviate multi-site appearance differences.
- Registration: Comparable performance on IBSR and ABIDE suggested that pre-alignment did not significantly influence the proposed architecture’s performance.IBSR subjects were pre-aligned, whereas ABIDE subjects were not.
- Limitations: Poor-quality scans, including motion-artifact cases, were excluded after producing substantially different segmentations from other subjects.Figure 11 shows axial slices from two excluded subjects.
- Limitations: Chosen architecture parameters performed well on the test data but might not be optimal for other datasets.The authors identify automatic task- and dataset-specific tuning as future work.
- Limitations: A small receptive field remains a limitation because memory demands constrain the use of larger 3D convolutions.Dilated kernels are proposed as a future way to enlarge the receptive field without losing resolution or increasing trainable parameters.
5. Conclusion
The study presents a multiscale 3D fully convolutional approach for automatic subcortical segmentation. It achieves strong performance across datasets and heterogeneous sites while offering computational advantages over atlas-based methods.
- The method uses fully convolutional networks with 3D filters for automatic subcortical brain-region segmentation.Small kernels enable a deeper network with fewer parameters, while intermediate-layer outputs model local and global context.
- Small convolution kernels produce a deeper, lower-parameter network that is less prone to overfitting.
- 1112 unregistered subjects from 17 sites were used to evaluate robustness across acquisition site, age, and diagnosis group.The evaluation reported outstanding accuracy across all subject groups.
- The multiscale FCNN achieved state-of-the-art performance on the IBSR dataset.
- The approach produced consistent segmentations in less time than atlas-based methods, supporting automatic segmentation in large-scale neuroimaging studies.