Source-linked AI summary
Deep Neural Networks for Anatomical Brain Segmentation
Alexandre de Brebisson, Giovanni Montana
TL;DR
Whole-brain anatomical segmentation from MRI is valuable but manual labelling is costly, while prior deep-learning methods had not addressed the whole 3D brain. The paper introduces SegNet, which combines multi-scale local and global features without non-linear registration, and reports competitive MICCAI 2012 results.
Problem
Manual brain segmentation is time-consuming and expensive, and deep neural networks had not yet been applied to whole-brain anatomical segmentation.
Method
SegNet classifies each voxel using 3D and orthogonal 2D intensity patches, downscaled broad-context patches, and distances to regional centroids.
Results
Mean dice coefficient 0.725 was obtained on the MICCAI 2012 challenge benchmark, with no non-linear registration of MR images.
Takeaways & Limitations
The approach demonstrates the potential of deep neural networks for automated whole-brain anatomical segmentation with global spatial consistency.
Takeaways & Limitations
The authors identify severe class imbalance and limited training-atlas variability as areas requiring more sophisticated costs and additional data.
Abstract
from arXiv · showhide
We present a novel approach to automatically segment magnetic resonance (MR) images of the human brain into anatomical regions. Our methodology is based on a deep artificial neural network that assigns each voxel in an MR image of the brain to its corresponding anatomical region. The inputs of the network capture information at different scales around the voxel of interest: 3D and orthogonal 2D intensity patches capture the local spatial context while large, compressed 2D orthogonal patches and distances to the regional centroids enforce global spatial consistency. Contrary to commonly used segmentation methods, our technique does not require any non-linear registration of the MR images. To benchmark our model, we used the dataset provided for the MICCAI 2012 challenge on multi-atlas labelling, which consists of 35 manually segmented MR images of the brain. We obtained competitive results (mean dice coefficient 0.725, error rate 0.163) showing the potential of our approach. To our knowledge, our technique is the first to tackle the anatomical segmentation of the whole brain using deep neural networks.
1. Introduction
The paper targets fully automated whole-brain anatomical segmentation from MRIs, motivated by the clinical and research value of regional measurements and the cost of manual labelling. It proposes a deep neural network that classifies voxels using learned representations.
- Abnormal brain-region volumes or shapes are associated with disorders including Alzheimer’s disease and Parkinson, making MRI analysis relevant for detection, monitoring, and treatment evaluation.
- Manual brain segmentation is expensive and time-consuming, limiting large-scale use; automation could enable segmentation as soon as an MRI is acquired.
- The proposed deep artificial neural network automatically segments the entire brain by assigning each voxel to its anatomical region.
- Unlike shallow classifiers, the deep-learning approach learns hierarchies of relevant features directly from raw inputs rather than relying primarily on hand-engineered features.
2. Background
Whole-brain segmentation classifies every voxel into protocol-defined anatomical regions using manually labelled atlases. Existing approaches include registration-heavy multi-atlas methods and machine-learning methods, while prior deep-learning work had not addressed whole-brain 3D segmentation.
- A segmentation protocol defines the anatomical regions and delineation rules, while an atlas pairs a 3D MR image with its manual segmentation.
- Multi-atlas methods select similar atlases, non-linearly register them to the query image, propagate labels, and fuse the resulting segmentations.
- Machine-learning methods train classifiers on atlases to assign each voxel to its corresponding anatomical region from features such as neighbourhood intensities or location.
- Prior medical-imaging deep networks mostly used 2D intensity patches and sometimes added spatial consistency through post-processing.
- This work extends deep neural segmentation to the whole 3D brain and introduces multi-scale inputs to enforce spatial consistency.
3. Architecture of the network
This section introduces the inputs and architecture of the proposed network.
- The section describes the proposed network’s inputs.
- The section describes the proposed network’s architecture.
- The inputs and architecture are presented together as components of the proposed network.
3.1. Input Features
SegNet combines local intensity features with global spatial cues to classify voxels accurately while preserving anatomical layout. Large downscaled patches and centroid distances provide broad context without relying on absolute image coordinates.
- 3.1. Input Features: The input design balances sufficient task information against computational cost and overfitting by combining features for local precision and global spatial consistency.
- 3.1. Input Features: Local context comes from a centered 3D patch plus sagittal, coronal, and transverse 2D patches around each voxel.
- Architecture: The architecture uses eight feature pathways whose learned representations are merged into a joint representation.
- Features to ensure global spatial consistency.: Large 2D orthogonal patches are downscaled by averaging s × s windows, preserving broad image coverage at lower resolution and reducing input dimensionality.
- Features to ensure global spatial consistency.: Relative distances from each voxel to anatomical-region centroids provide spatial information without requiring a common reference space or initial registration.
- Features to ensure global spatial consistency.: Centroid distances are invariant to rotations and translations, and scaling both centroids and coordinates normalizes brain-size differences.
3.2. Deep Neural Network
SegNet is a feed-forward deep network that merges specialized multi-scale representations from eight input pathways to classify brain voxels. Convolutional and pooling layers capture local features, while shared low-level weights reduce parameters and slightly improve performance.
- SegNet stacks K artificial-neuron layers to learn hierarchical feature representations for anatomical voxel classification.Deeper neurons detect features formed from earlier feature detectors.
- Eight input-feature pathways learn specialized representations before merging into a common higher-level representation.Inputs other than centroid distances use 2D or 3D convolutional and pooling layers.
- Convolutional layers detect local features within t × t receptive-field windows of voxel intensities.Each neuron depends only on a spatially contiguous subset of layer inputs.
- Max-pooling layers reduce feature-map size by retaining the most responsive neuron within each p × p window.Pooling discards the precise receptive-field location of the detected feature.
- Rectified linear units are used in non-top layers because they are less prone to vanishing gradients than sigmoid or tanh functions.The top layer instead uses softmax, whose outputs are interpreted as probabilities and whose highest-probability output labels each voxel.
- Sharing weights for orthogonal 2D patches and downscaled patches halves first-layer parameters and slightly improves performance experimentally.The constraint reflects the assumption that lowest-level patch features should be orientation-independent and reduces overfitting risk.
3.3. Training Algorithm
Training minimizes a negative log-likelihood error using stochastic gradient descent with momentum. The updates use individual datapoints or small batches rather than the full training set for each gradient calculation.
- Training minimizes the negative log-likelihood error ECE over the network parameters θ.The target output y(i) is a one-hot vector for the classification label of input x(i).
- Stochastic gradient descent computes each error-function gradient from one datapoint or a small batch instead of all training datapoints.This is presented as a variant of gradient descent commonly used for large networks and datasets.
- The optimization includes a momentum term, with α and m denoting the learning rate and momentum, respectively.The momentum update rule applies to the weights at each iteration.
3.4. Estimation of the Centroids
Because centroid distances are unavailable for new brains, the method estimates them iteratively from an initial segmentation. Training adds Gaussian noise to these distances, and updated approximations can improve segmentation.
- A sub-network without the centroid pathway first produces an initial segmentation for a new MR image.That segmentation is used to compute approximated regional centroids and voxel-to-centroid distances.
- The estimated centroid distances are then used by a second neural network in an iterative procedure for segmenting the new brain.The distances are recomputed from the evolving segmentation.
- Gaussian noise corrupts centroid distances during training so the network relies less on individual distances and more on group-level statistical properties.This training strategy is intended to improve robustness to noisy centroid approximations.
- In a toy fully-connected network example, updated approximated centroid distances significantly improve segmentation despite poor initial accuracy.The example uses data from the MICCAI dataset 4.
4. An application to the MICCAI dataset
The SegNet evaluation used the MICCAI 2012 multi-atlas labelling dataset and tested multi-scale voxel inputs under single-GPU memory constraints. The best model achieved a validation error of 0.105, a testing mean dice coefficient of 0.725, and a testing error rate of 0.163.
- Training setup: Memory constraints motivated sampling approximately 20k voxels from each atlas, producing 300k training voxels, or about 1.5% of available voxels.All computations ran in memory on a single NVIDIA Tesla K40 GPU with 12GB memory.
- Training setup: Each voxel input combined a 3D patch, three orthogonal 2D patches, three downscaled orthogonal patches, and distances to regional centroids.The resulting input vector had 7,377 dimensions; the downscaled patches used averaged intensities.
- Training setup: The seven-layer SegNet architecture used early stopping on validation error, with a patience of 10 epochs.The model contained 30,565,555 parameters and used batch size 200, momentum 0.5, and learning rate 0.05.
- Results: 0.105 validation error selected SegNet, which achieved a 0.725 mean dice coefficient and 0.163 error rate on the 20 testing MRIs.The three orthogonal 2D patches substantially outperformed individual 2D or 3D patches, while centroid distances outperformed coordinates in validation comparisons.
- Results: Downscaled patches and centroid distances supported global spatial consistency, while misclassified voxels tended to lie on region boundaries.An example comparison also showed that using only orthogonal 2D patches wrongly classified parts of the opposite hemisphere, unlike SegNet.
5. Conclusion
The proposed deep neural network achieves competitive whole-brain MRI segmentation without non-linear registration, while combining local boundary detail with globally consistent spatial information. The authors identify class imbalance, limited atlases, and boundary sampling as priorities for future improvement.
- Results: Mean dice 0.725 was achieved against the MICCAI 2012 multi-atlas benchmark, without relying on non-linear MRI registration.The method was benchmarked against multi-atlas methods from the challenge.
- Architecture: Multi-scale intensity patches and centroid-distance inputs jointly delineate region boundaries and enforce global spatial consistency.Orthogonal patches captured 3D information with less memory, while centroid distances provided robust location cues.
- Limitations: The plain negative log-likelihood cost function optimized mean dice only indirectly, leaving class imbalance insufficiently addressed.Region sizes ranged from 0.01% to 16.9% of brain volume in the MICCAI dataset.
- Limitations: Volume-weighted error terms and equal voxel sampling across anatomical regions did not produce substantial performance improvements.The attempted weighting used true and predicted region volumes.
- Limitations: The networks generalized fairly well despite overfitting, but the small number of available atlases limits variability captured during training.The authors expect more training atlases or plausible artificial atlas transformations to improve performance.