Source-linked AI summary
3D Whole Brain Segmentation using Spatially Localized Atlas Network Tiles
Yuankai Huo, Zhoubing Xu, Yunxi Xiong, Katherine Aboud, Prasanna Parvathaneni, Shunxing Bao, Camilo Bermudez, Susan M. Resnick, Laurie E. Cutting, Bennett A. Landman
TL;DR
High-resolution whole-brain segmentation is constrained by GPU memory and limited detailed manual annotations. SLANT distributes segmentation across spatially localized 3D networks and augments training with multi-atlas labels for 5111 initially unlabeled scans. It achieved better overall segmentation performance than multi-atlas methods while reducing processing time from more than 30 hours to about 15 minutes.
Problem
High-resolution whole-brain segmentation is limited by 3D GPU memory constraints and the scarcity of manually traced scans with detailed annotations.
Method
SLANT combines spatially distributed 3D network tiles with registration, harmonization, label fusion, and auxiliary multi-atlas labels for 5111 initially unlabeled scans.
Results
SLANT-27 achieved better overall segmentation performance than multi-atlas segmentation methods and required about 15 minutes versus typically more than 30 hours.
Takeaways & Limitations
Network tiles provide a strategy for high-resolution brain segmentation that can be adapted to higher-resolution MRI or histology scans.
Takeaways & Limitations
When only one GPU is available, training and testing time increase linearly with the number of network tiles.
Abstract
from arXiv · showhide
Detailed whole brain segmentation is an essential quantitative technique, which provides a non-invasive way of measuring brain regions from a structural magnetic resonance imaging (MRI). Recently, deep convolution neural network (CNN) has been applied to whole brain segmentation. However, restricted by current GPU memory, 2D based methods, downsampling based 3D CNN methods, and patch-based high-resolution 3D CNN methods have been the de facto standard solutions. 3D patch-based high resolution methods typically yield superior performance among CNN approaches on detailed whole brain segmentation (>100 labels), however, whose performance are still commonly inferior compared with multi-atlas segmentation methods (MAS) due to the following challenges: (1) a single network is typically used to learn both spatial and contextual information for the patches, (2) limited manually traced whole brain volumes are available (typically less than 50) for training a network. In this work, we propose the spatially localized atlas network tiles (SLANT) method to distribute multiple independent 3D fully convolutional networks (FCN) for high-resolution whole brain segmentation. To address the first challenge, multiple spatially distributed networks were used in the SLANT method, in which each network learned contextual information for a fixed spatial location. To address the second challenge, auxiliary labels on 5111 initially unlabeled scans were created by multi-atlas segmentation for training. Since the method integrated multiple traditional medical image processing methods with deep learning, we developed a containerized pipeline to deploy the end-to-end solution. From the results, the proposed method achieved superior performance compared with multi-atlas segmentation methods, while reducing the computational time from >30 hours to 15 minutes (https://github.com/MASILab/SLANTbrainSeg).
I. INTRODUCTION
Whole brain segmentation is valuable but difficult to automate at high resolution because GPU memory limits 3D networks and detailed manual annotations are scarce. SLANT addresses these constraints by distributing spatially localized networks and using auxiliary labels from multi-atlas segmentation.
- Manual delineation is the long-held gold standard but is resource- and time-intensive.
- Traditional multi-atlas segmentation can provide strong performance but has high computational cost, typically limiting whole-brain processing to fewer than 100 atlases.
- High-resolution whole-brain volumes cannot practically fit into prevalent 3D FCN architectures because of GPU memory limitations.
- Detailed manually traced whole-brain scans with more than 100 labels are rare for individual laboratories.
- SLANT uses multiple independent 3D FCNs, each responsible for a fixed spatial location, simplifying learning by reducing spatial variation within each network.
II. METHODS
The SLANT pipeline combines registration, bias correction, intensity harmonization, and learned segmentation preparation to place scans in a common anatomical and intensity space. This preprocessing supports consistent downstream network training and inference.
- SLANT accepts a single 3D T1-weighted MRI scan and uses 45 manually traced scans for network, multi-atlas, and intensity-normalization training.
- Affine registration maps the target scan to the MNI305 template, followed by N4 bias-field correction.
- Intensity normalization first standardizes each volume using its mean and standard deviation, then harmonizes intensities with a pretrained robust regression model.
B. Network Tiles
Network tiles divide the normalized MNI space among independent 3D U-Nets so high-resolution segmentation fits GPU constraints. Overlapping tiles require majority-vote fusion to produce one final segmentation.
- All scans are mapped to 1 mm isotropic MNI305 space with dimensions 172 × 220 × 156 voxels.
- Each independent 3D U-Net covers a defined subspace, with decoder outputs expanded to support 133 labels.
- SLANT-8 uses 8 non-overlapping tiles, whereas SLANT-27 uses 27 overlapping tiles with subspaces of 96 × 128 × 88 voxels.
- Majority-vote label fusion combines overlapping tile segmentations into a single final segmentation in MNI space.
D. Boost Learning on Unlabeled Data
SLANT expands training data by generating auxiliary labels for initially unlabeled scans with multi-atlas segmentation. These labels pretrain the network tiles before fine-tuning on manually traced scans.
- NLSS-based multi-atlas segmentation generated auxiliary labels for 5111 multi-site scans using 45 OASIS T1-weighted atlases.
- PCA-based atlas selection chooses 15 atlases closest to each testing image in a learned intensity manifold.
- Auxiliary labels are used for pretraining each network tile, followed by full-network fine-tuning on manually traced images.
III. CONTAINERIZED IMPLEMENTATION
SLANT was packaged as a Docker-based, end-to-end segmentation pipeline to simplify deployment of its preprocessing, network-tile inference, and image-processing components.
- The containerized implementation provides an end-to-end SLANT segmentation solution that can be deployed on MRI T1w scans with one command.The implementation was developed because repeating the integrated processing outside the authors’ laboratory could be time-consuming.
- Docker provides an operating-system-independent deployment capsule, while NVIDIA-Docker supplies the GPU capability required by SLANT.
- The Docker pipeline packages N4 bias-field correction, intensity normalization, affine registration to MNI space, and trained network tiles into deployable components.
- The published Docker image can be retrieved with the command `sudo docker pull masidocker/spiders:deep_brain_seg_v1_0_0`.
- Figure 3 distinguishes training components using 45 manually traced OASIS images versus 5111 auxiliary label images and identifies JLF and NLSS as baselines.
B. Run SLANT in Docker
SLANT can be run by mounting input and output directories into its NVIDIA-Docker container, with evaluation spanning internal OASIS validation and independent Colin27 and CANDI cohorts.
- Users obtain final output files by running the SLANT container with mounted input and output paths and the supplied deep-brain-segmentation script.
- The study used T1w 3D MRI volumes for one training cohort and three validation or testing cohorts.
- Training used 45 manually traced OASIS scans with 133 BrainCOLOR labels, alongside 5111 multi-site scans used to obtain auxiliary training data.
- Five withheld OASIS scans provided same-site validation, while Colin27 and CANDI served as independent testing cohorts.
- The CANDI cohort tested external validation across a different population aged 5–15 years, outside the OASIS training range of 18–96 years.
V. EXPERIMENTAL DESIGN
The experiments compared atlas-based and CNN baselines with SLANT variants using common network settings, training strategies, and sensitivity-selected epochs.
- V. EXPERIMENTAL DESIGN: JLF and NLSS were used as state-of-the-art multi-atlas label-fusion baselines, while patch-based, naïve U-Net, and HC-Net methods provided CNN comparisons.
- V. EXPERIMENTAL DESIGN: Figure 4 compares qualitative segmentations from manual, multi-atlas, patch-based DCNN, HC-Net, U-Net, and SLANT methods.
- V. EXPERIMENTAL DESIGN: SLANT was evaluated with 8 non-overlapping or 27 overlapping independent U-Net-shaped network tiles.
- V. EXPERIMENTAL DESIGN: The U-Net and SLANT experiments used the same 3D segmentation network, hyperparameters, preprocessing, and registration procedures for a fair comparison.
- V. EXPERIMENTAL DESIGN: Figure 5 selected epoch 5 for initialization and epoch 28 after fine-tuning for SLANT-8 and SLANT-27.
- V. EXPERIMENTAL DESIGN: Models trained from scratch ran for 1000 epochs, whereas fine-tuned models ran for 30 epochs, with testing parameters selected from five OASIS validation images.
B. Evaluation Metrics
Evaluation used voxel-overlap and surface-distance measures to compare automatic segmentations with manual ground truth, with statistical significance defined at p<0.05.
- B. Evaluation Metrics: Dice similarity coefficient measures twice the voxel intersection divided by the total voxels in automatic segmentation A and manual segmentation M.
- B. Evaluation Metrics: Mean surface distance complements overlap evaluation by measuring distances between vertices of automatic and manual segmentations.
- B. Evaluation Metrics: In the MSD definition, inf denotes the infimum and avg denotes the average.
- B. Evaluation Metrics: Method differences were evaluated with the Wilcoxon signed-rank test, using p<0.05 as the significance threshold.
VI. RESULTS
The evaluation compares baseline and SLANT methods using segmentation overlap, surface distance, regional performance, and computational time. SLANT-27 is used as the reference method because it achieved the strongest median DSC and MSD results.
- The results included comparisons across validation settings involving manual atlases, auxiliary atlases, and their combination.
- SLANT-27 with 5111 auxiliary labels and fine-tuning on 45 manual labels achieved the highest median DSC and served as the statistical reference.
- SLANT-27 with 5111 auxiliary labels and fine-tuning on 45 manual labels achieved the lowest median MSD and served as the statistical reference.
- The validation used multiple cohorts and reported DSC, MSD, and Hausdorff distance summaries.
- The study also evaluated 129 brain regions and presented regional comparisons between SLANT-27 and representative baseline methods.
- A Docker output report was designed to let users review segmentation quality immediately after scanning.
B. Testing
Testing showed that registration improved U-Net performance, auxiliary-label pretraining improved the same network, and SLANT-27 performed best across regional comparisons.
- Testing: Affine registration significantly improved U-Net performance compared with naïve U-Net.
- Testing: Training with 5111 auxiliary labeled scans outperformed training with only 45 manual labels for the same registered U-Net.
- Testing: Larger brain regions typically yielded better segmentation performance.
- Testing: SLANT-27 consistently achieved the largest number of best-performing regions across the evaluated ROIs.
VII. CONCLUSION
The conclusion presents SLANT as a high-resolution whole-brain segmentation method that combines medical image processing with deep neural networks. It uses spatially distributed network tiles and auxiliary labels, achieving better overall performance with substantially shorter computation time than multi-atlas segmentation.
- VII. CONCLUSION: SLANT combines registration, harmonization, label fusion, and deep neural networks for high-resolution whole-brain segmentation.
- VII. CONCLUSION: Multiple spatially distributed 3D networks address GPU memory limits by learning segmentation for different spatial locations.
- VII. CONCLUSION: 5111 initially unlabeled MRI T1w scans were converted into auxiliary training data through multi-atlas segmentation.
- VII. CONCLUSION: SLANT-27 achieved better overall segmentation performance in internal and external validation.
- VII. CONCLUSION: ~15 minutes were required by the proposed method, compared with >30 hours typically required by multi-atlas segmentation methods.
VIII. DISCUSSION
The discussion presents SLANT as a scalable high-resolution segmentation strategy whose tile design, registration, and auxiliary labels improve performance while reducing per-scan runtime. Its main trade-offs are greater computational-resource demands, registration overhead, and limited evaluation of tile configurations.
- Scalability: 27 network tiles address high-resolution segmentation memory constraints and could extend to 0.5 mm MRI or histology scans.The study used 3×3×3 tiles for 1 mm isotropic MRI.
- Limitations: SLANT requires greater computational resources, and with one GPU both training and testing time increase linearly with the number of network tiles.The stated examples require eight GPUs for SLANT-8 and 27 GPUs for SLANT-27 to match single-GPU method times.
- Computational cost: 15 pairwise registrations and non-local search label fusion make the NLSS multi-atlas benchmark computationally intensive.The benchmark typically requires around 36 hours per scan, including approximately 2 hours per registration pair and 6 hours for label fusion.
- Computational cost: 15 minutes are sufficient for SLANT to segment a single scan after training, compared with more than 30 hours for multi-atlas processing.The containerized pipeline enabled a single workstation with one GPU to process more than 5000 scans within two months.
- Performance and data: Affine registration substantially improved U-Net performance, while training with 5111 automatically labeled scans outperformed training with 45 scans.The larger auxiliary dataset required more computational resources than the manually labeled dataset.
- Limitations: Only SLANT-8 and SLANT-27 were evaluated because they offered a better performance–computational-cost balance than larger tile configurations.Additional validation-based hyperparameter tuning could further improve both benchmark and proposed methods.