Source-linked AI summary
QuickNAT: A Fully Convolutional Network for Quick and Accurate Segmentation of Neuroanatomy
Abhijit Guha Roy, Sailesh Conjeti, Nassir Navab, Christian Wachinger
TL;DR
Whole-brain MRI segmentation is important for morphological analysis but can take hours with existing approaches, delaying image-derived markers. QuickNAT combines a densely connected fully convolutional architecture with auxiliary-label pre-training and manual-label fine-tuning, and evaluations report accurate, reliable segmentation with much faster processing.
Problem
Whole-brain MRI segmentation is computationally intense, while training fully convolutional models is constrained by limited manually annotated data.
Method
QuickNAT uses a fully convolutional encoder-decoder network and pre-trains on auxiliary labels before fine-tuning on manual labels.
Results
QuickNAT achieves superior segmentation accuracy and reliability across heterogeneous datasets while being orders of magnitude faster than existing approaches.
Takeaways & Limitations
QuickNAT makes brain segmentation results available within seconds and supports processing large repositories, group analyses, and longitudinal studies.
Takeaways & Limitations
Performance deteriorated for scans with strong motion artifacts, and the demonstrated age range was 5 to 95 years.
Abstract
from arXiv · showhide
Whole brain segmentation from structural magnetic resonance imaging (MRI) is a prerequisite for most morphological analyses, but is computationally intense and can therefore delay the availability of image markers after scan acquisition. We introduce QuickNAT, a fully convolutional, densely connected neural network that segments a \revision{MRI brain scan} in 20 seconds. To enable training of the complex network with millions of learnable parameters using limited annotated data, we propose to first pre-train on auxiliary labels created from existing segmentation software. Subsequently, the pre-trained model is fine-tuned on manual labels to rectify errors in auxiliary labels. With this learning strategy, we are able to use large neuroimaging repositories without manual annotations for training. In an extensive set of evaluations on eight datasets that cover a wide age range, pathology, and different scanners, we demonstrate that QuickNAT achieves superior segmentation accuracy and reliability in comparison to state-of-the-art methods, while being orders of magnitude faster. The speed up facilitates processing of large data repositories and supports translation of imaging biomarkers by making them available within seconds for fast clinical decision making.
1. Introduction
QuickNAT addresses the slow, sometimes sub-optimal brain segmentation produced by atlas-based methods with a fast fully convolutional network and a training strategy that uses auxiliary and manual labels. The paper presents this combination as accurate, robust, and suitable for heterogeneous neuroimaging data.
- Atlas-based brain segmentation can require hours per scan because estimating 3D deformation fields is computationally intense and anatomical non-homologies can cause errors.
- QuickNAT is a deep fully convolutional neural network designed to segment MRI T1 scans in seconds on GPUs rather than hours.
- QuickNAT is reported to achieve state-of-the-art accuracy across varied ages, scanners, field strengths, and pathologies while supporting effect-size and longitudinal analyses.
- Limited manual annotations motivate pre-training on auxiliary segmentations from existing software, followed by fine-tuning on smaller manually annotated datasets.
- The method combines an encoder-decoder F-CNN architecture with auxiliary-label training as its two main methodological innovations.
2. Methods
QuickNAT uses a 2D encoder-decoder F-CNN design with dense connections, skip connections, unpooling, and a classifier that produces class probabilities for 27 anatomical structures.
- QuickNAT maps an MRI brain scan to a segmentation map representing 27 cortical and subcortical structures.
- Architectural Design: The architecture uses four encoders and four decoders separated by a bottleneck, with skip connections between matching spatial resolutions.
- Architectural Design: Dense blocks contain three convolutional layers with dense connections that improve gradient flow and promote feature reuse.
- Architectural Design: Unpooling recovers spatial locations of maximal activations during upsampling, supporting segmentation of small subcortical structures without learnable upsampling parameters.
- Classifier Block: A 1 × 1 convolution maps decoder features to an N-channel feature map, with N equal to 28 classes in the described architecture.
2.2. Loss Function
QuickNAT jointly optimizes weighted logistic and multi-class Dice losses to estimate label similarity while addressing class imbalance and anatomical-boundary errors.
- QuickNAT simultaneously optimizes weighted logistic loss and multi-class Dice loss.
- The logistic loss provides pixel-wise probabilistic similarity estimates, while the Dice loss measures overlap between estimated and manually annotated labels.
- Pixel weights address class imbalance and errors at anatomical boundaries.
- Median-frequency balancing up-weights rare classes, while a boundary term assigns greater weight to anatomical contours.
2.3. Model Learning
QuickNAT is trained with stochastic gradient descent and uses auxiliary labels from large unlabeled repositories before fine-tuning on limited manual annotations. This strategy addresses the scarcity of labeled data while exposing the network to broad anatomical variation.
- The network is optimized with stochastic gradient descent with momentum, using validation-based learning-rate reduction during pre-training.
- F-CNN training is difficult because each image is one training sample, while manual annotation remains limited and labor intensive.
- A large neuroimaging dataset is processed with existing software to create auxiliary labels for QuickNAT pre-training.
- Pre-training produces a network that imitates the software segmentations and provides a strong prior initialization.
2.5. Multi-View Aggregation
QuickNAT combines predictions from separate 2D networks trained along the coronal, axial, and sagittal views to produce the final voxelwise segmentation.
- Three 2D F-CNNs process coronal, axial, and sagittal views before their predictions are combined.
- The aggregation step combines the three networks’ probability maps to assign the final label for each voxel.
- Aggregating view-specific votes regularizes voxel labels and reduces spurious predictions.
- Sagittal training merges left- and right-hemisphere structures into 16 classes because symmetric sagittal slices cannot distinguish them.
3. Experimental Datasets
The experiments use diverse MRI datasets spanning manual annotations, clinical diagnoses, developmental and aging populations, and multi-site scanner variation.
- Nine brain MRI datasets support evaluations of segmentation accuracy and reliability across varied subjects, diagnoses, ages, and annotated structures.
- MALC contains 30 T1 scans with manual whole-brain annotations, split into 15 training and 15 testing volumes.
- ADNI-29 includes 29 subjects comprising 15 controls and 14 Alzheimer’s patients.
- The Travelling Human Phantom dataset scans three healthy subjects at eight US centers using scanners from different vendors.
- FreeSurfer annotations from IXI are used for pre-training, while MALC manual annotations are used for fine-tuning under aligned anatomical definitions.
4. Experiments and Results
QuickNAT is evaluated in eight experiments covering segmentation accuracy, reproducibility, and sensitivity across diverse neuroimaging datasets.
- Eight experiments assess QuickNAT’s accuracy, reproducibility, and sensitivity across a broad variety of neuroimaging datasets.
- Five experiments evaluate segmentation accuracy, while three evaluate other properties of the segmentation framework.
4.1. Evaluation of segmentation accuracy with training and testing on same dataset
On MALC, QuickNAT was evaluated against convolutional and atlas-based methods for accuracy, consistency, qualitative segmentation, and speed. Fine-tuning auxiliary-label pre-training improved performance, while view aggregation and rapid inference supported strong results across the 27 structures.
- Segmentation accuracy: p < 0.001: Fine-tuned QuickNAT significantly outperformed training with limited manual annotations alone, and exceeded U-Net and FCN by 5% and 12% mean Dice points.These comparisons used 15 MALC testing scans and Dice overlap scores.
- Segmentation accuracy: QuickNAT significantly outperformed Spatial STAPLE, slightly outperformed PICSL without statistical significance, and exceeded DeepNAT for the 25-structure comparison.The comparison followed the MALC challenge setup of 15 training and 15 testing scans.
- Structure-wise analysis: 14 of 27 structures had significantly higher Dice scores than Spatial STAPLE, and 23 of 27 exceeded U-Net; no structure differed significantly from PICSL.Figure 4 used two-sided Wilcoxon rank-sum tests with p < 0.05 significance markers.
- Qualitative analysis: Qualitative examples showed fewer background misclassifications after fine-tuning and under-inclusion of the left putamen for PICSL.The examples highlighted the left putamen and left pallidum.
- Inter-run consistency: QuickNAT showed lower volume distance than PICSL and Spatial STAPLE for hippocampus, white matter, and grey matter, while lateral-ventricle and amygdala performance remained within 2−3%.Volume distance measures error in estimated structure volume, with higher values indicating poorer estimation.
- View aggregation: Aggregating orthogonal views produced the best performance, with axial aggregation improving on the coronal-only model and the full aggregation significant at p < 0.05.The view experiments compared coronal, axial, coronal-plus-axial, and all-orthogonal-view configurations.
4.2. Evaluation of segmentation accuracy with training and testing on different dataset
QuickNAT was evaluated on unseen datasets spanning pathology, field strength, age, and acquisition quality. It generally achieved higher segmentation accuracy than FreeSurfer and remained competitive with alternative methods.
- ADNI-29: 0.841 ± 0.064 mean Dice: QuickNAT exceeded FreeSurfer’s 0.778 ± 0.097 on ADNI-29 by 6% points (p < 10^-7).QuickNAT was significantly better for 24 of 27 structures.
- ADNI-29: Fine-tuning increased global Dice by 6% points on unseen ADNI-29, while training on combined IXI and MALC data scored 0.814 ± 0.080.The authors report this as evidence favoring pre-training followed by fine-tuning.
- ADNI-29: QuickNAT outperformed FreeSurfer across both 1.5T and 3.0T scans and showed superior performance for both controls and AD patients.These comparisons assessed robustness to field strength and pathology.
- IBSR: 0.835 ± 0.080 mean Dice: QuickNAT exceeded FreeSurfer’s 0.794 ± 0.093 on IBSR, significantly for the brain-wide comparison and 16 structures.QuickNAT also scored about 2–3% points higher than a CNN baseline for three of four compared structures.
- CANDI: 0.842 ± 0.084 mean Dice: QuickNAT exceeded FreeSurfer’s 0.798 ± 0.092 on CANDI by 5% points, significantly overall and for 22 of 27 structures.CANDI tested generalization to children aged 5–15 years, outside the MALC training age range.
- Comparison with FSL: QuickNAT outperformed both FreeSurfer and FSL across all scans for 13 common structures, but FSL performed best on CANDI after excluding its failed scans.FSL failed on 44% of IBSR, 17% of ADNI, and 23% of CANDI scans; in IBSR, FSL’s 0.08% point advantage was not significant.
4.3. Evaluation of segmentation reliability
The reliability evaluations examined volume consistency across repeated scans, diagnoses, and imaging centers. QuickNAT generally produced reliable estimates, with performance comparable to or better than FreeSurfer depending on the setting.
- ALVIN Dataset: QuickNAT’s volume estimates were evaluated on control and Alzheimer’s disease subjects using the ALVIN protocol and manually annotated lateral ventricles.The dataset contained 7 young adults and 9 patients with Alzheimer’s disease.
- TRT Dataset: QuickNAT’s intra-session and inter-session coefficients of variation were both below 2% across the repeated-scan volume experiment.FreeSurfer showed higher variation for several structures, including thalamus (6%), pallidum (5%), amygdala (5%), and putamen (4%).
- TRT Dataset: FreeSurfer performed better for cerebral white-matter volume, with CV < 1%, possibly reflecting its later surface-processing and correction stage.The comparison covered eight structures and reported intra-session CVs, inter-session CVt, and their absolute difference.
- ALVIN Dataset: QuickNAT showed the best performance for both control subjects and AD patients in the ALVIN ventricle-volume evaluation.Performance was assessed using intra-class correlation.
- HTP Dataset: QuickNAT and FreeSurfer were equally robust on the heterogeneous Human Travelling Phantom dataset spanning eight imaging centers.The scans included substantial variation in data quality and motion artifacts across sites.
5. Discussion
QuickNAT combines auxiliary-label pre-training with a fully convolutional architecture to achieve accurate, reliable brain segmentation across heterogeneous datasets. Its evaluations show broad robustness, while motion artifacts and out-of-training tissue classes remain important boundaries.
- Pre-training with Auxiliary Labels: QuickNAT uses auxiliary labels from FreeSurfer to pre-train, then fine-tunes on small manually annotated datasets.This strategy leverages large unlabeled repositories while correcting errors inherited from automatically generated labels.
- Pre-training with Auxiliary Labels: QuickNAT achieved higher accuracy than FreeSurfer on ADNI-29 despite being pre-trained only on FreeSurfer-generated annotations.The authors suggest that the large IXI pre-training dataset could explain this behavior.
- Architecture: The fully convolutional design labels all voxels in a slice simultaneously, providing faster processing and larger context than patch-based DeepNAT.Dense connections also promote feature re-usability within encoder and decoder blocks.
- Segmentation Accuracy: QuickNAT achieved high accuracy across five experiments spanning acquisition parameters, pathology, field strengths, resolution, contrast, and age.On MALC, it showed similar accuracy and inter-run consistency to the best atlas-based methods; on ADNI-29, its effect sizes were more similar to manual segmentations than FreeSurfer.
- Segmentation Reliability: Less than 2% variation occurred for most brain structures in same-scanner test-retest data, while multi-site variation increased but remained comparable to FreeSurfer.QuickNAT also did not fail on any scan or produce a segmentation requiring rejection across the evaluated datasets.
- Limitations: Motion artifacts at the Dartmouth site increased variance and deteriorated segmentation performance in the human traveling phantom dataset.The authors note that motion correction could be applied when more than one source volume exists.
- Limitations: QuickNAT cannot segment tissue classes absent from its training set, such as tumors, without training data containing those annotations alongside all brain structures.The paper states that such a publicly available dataset was not known to the authors.
6. Conclusion
The conclusion presents QuickNAT as a fast deep-learning method for brain segmentation with strong generalization and accuracy across diverse datasets and acquisition conditions. It also emphasizes its potential to make imaging biomarkers available within seconds.
- Conclusion: QuickNAT runs in seconds, delivers superior performance to existing methods, and is orders of magnitude faster than patch-based CNNs and atlas-based approaches.The method generalized to unseen datasets and maintained high accuracy across diagnostic groups, scanner field strengths, and age.
- Conclusion: QuickNAT's speed supports processing large data repositories and translating imaging biomarkers into fast clinical decision making.The conclusion connects rapid segmentation with practical availability of quantitative imaging information.
Appendix
The appendix lists the 27 brain structures segmented by QuickNAT and explains how FreeSurfer and manual labels are remapped to QuickNAT's label IDs for evaluation.
- List of classes: QuickNAT segments 27 cortical and subcortical brain structures assigned IDs 1 through 27.The appendix provides the structure list, including cortical matter, ventricles, subcortical nuclei, hippocampus, amygdala, and related regions.
- Label remapping strategy: FreeSurfer and manual annotations are mapped to QuickNAT IDs for training, testing, and evaluation.FreeSurfer mappings correspond to aseg.mgz, while manual annotations include cortical parcellations and require a separate remapping procedure.