Source-linked AI summary
Sub-cortical brain structure segmentation using F-CNN's
Mahsa Shakeri, Stavros Tsogkas, Enzo Ferrante, Sarah Lippe, Samuel Kadoury, Nikos Paragios, Iasonas Kokkinos
TL;DR
Accurate segmentation of frontostriatal subcortical structures can support quantitative analysis of neurological diseases. The paper uses full-slice fully convolutional networks with an MRF for volumetric consistency, and reports stronger results than learned-prior and state-of-the-art alternatives, with a caudate exception.
Problem
Accurate segmentation of frontostriatal subcortical structures matters because it enables extraction of volumetric and morphological indicators for quantitative disease characterization.
Source: §1 · p. 1Method
A fully convolutional network segments full 2D MRI slices, and its probability maps become unary potentials in a 3D-grid MRF inferred approximately with alpha-expansion.
Source: §1 · pp. 1–2, §2.1 · p. 2Results
The CNN-prior approach achieves higher Dice and lower Hausdorff and contour mean distance than the RF-prior method; it outperforms state-of-the-art methods on three subcortical structures in both datasets, while FreeSurfer is better for IBSR caudate.
Source: §3 · p. 3Takeaways & Limitations
The resulting segmentations can support extraction of volumetric and morphological indicators for quantitative characterization of neurological diseases and their evolution.
Source: §1 · p. 1The model does not accurately capture thin caudate tails, and symmetric structures confound CNN training.
Source: §4 · p. 4, §3.1 · p. 4
Abstract
from arXiv · showhide
In this paper we propose a deep learning approach for segmenting sub-cortical structures of the human brain in Magnetic Resonance (MR) image data. We draw inspiration from a state-of-the-art Fully-Convolutional Neural Network (F-CNN) architecture for semantic segmentation of objects in natural images, and adapt it to our task. Unlike previous CNN-based methods that operate on image patches, our model is applied on a full blown 2D image, without any alignment or registration steps at testing time. We further improve segmentation results by interpreting the CNN output as potentials of a Markov Random Field (MRF), whose topology corresponds to a volumetric grid. Alpha-expansion is used to perform approximate inference imposing spatial volumetric homogeneity to the CNN priors. We compare the performance of the proposed pipeline with a similar system using Random Forest-based priors, as well as state-of-art segmentation algorithms, and show promising results on two different brain MRI datasets.
1. INTRODUCTION
The paper adapts fully convolutional networks to segment frontostriatal subcortical structures from full 2D MRI slices. It adds an MRF with a volumetric neighborhood to encourage spatial homogeneity.
1. INTRODUCTION
- Segmenting frontostriatal structures supports extraction of volumetric and morphological indicators for quantitative analysis and characterization of neurological diseases and their evolution.The cited disorders include schizophrenia, Alzheimer’s disease, attention deficit, and subtypes of epilepsy.Source: §1 · p. 1
- Full-slice fully convolutional networks produce dense, multi-label brain segmentations in a single 5 ms forward pass.Unlike patch-based classification, the method processes full 2D slices and outputs segmentation maps for the whole image.Source: §1 · p. 1
- The 2D CNN remains agnostic to 3D context, so an MRF over a 6-neighborhood grid imposes volumetric homogeneity on its output.The authors avoid direct 3D CNN processing because it would reduce training data and increase computational requirements; alpha-expansion performs inference with stated optimality bounds for the defined energies.Source: §1 · pp. 1–2
2. USING CNNS FOR SEMANTIC SEGMENTATION
The network adapts a semantic-segmentation architecture to MRI by training from scratch and preserving output resolution. CNN probability maps then serve as priors in a voxel-graph MRF whose energy is minimized approximately with alpha-expansion.
2. USING CNNS FOR SEMANTIC SEGMENTATION
- Training from scratch adapts the Deeplab-inspired network to grayscale MR images with varying backgrounds rather than fine-tuning weights learned from natural RGB images.The authors cite the differing image characteristics as the reason for this choice.Source: §2 · p. 2
- A shallower five-pair convolutional/max-pooling network addresses limited scans and fine-structure resolution by subsampling only in the first two pooling layers and using stride 1 thereafter.For a 256 × 256 input, the network produces 64 × 64 × L probability maps; bilinear interpolation resizes them to the input dimensions before pixel labels are assigned by highest probability.Source: §2 · p. 2
2.1. Multi-label segmentation using CNN-based priors
- Stacked slice-wise CNN softmax maps provide voxel-level class probabilities, which the MRF treats as priors for assigning labels across the 3D volume.The maps encode the probability of each pixel belonging to a brain-structure label.Source: §2.1 · p. 2
- The MRF represents voxels as nodes in a 3D 6-neighborhood graph and combines unary and pairwise potentials to favor likely labels and spatial homogeneity.The pairwise formulation encourages neighboring variables to share a semantic label and is made inversely proportional to voxel intensity differences to align boundaries with intensity edges.Source: §2.1 · p. 2
- Alpha-expansion approximately minimizes the metric label-space energy, with the stated guarantee that the resulting energy is within a factor of 2 of the optimum.The energy combines unary and pairwise terms.Source: §2.1 · p. 2
3. EXPERIMENTS AND DISCUSSION
Across IBSR and Rolandic Epilepsy MRI data, the proposed method outperformed Random Forest priors and generally surpassed two automatic segmentation toolboxes, with a caudate-tail limitation. The experiments used 2D slices and compared CNN and RF priors under the same MRF refinement.
3. EXPERIMENTS AND DISCUSSION
- IBSR supplied 18 labeled 3D T1-weighted scans, while the Rolandic Epilepsy dataset included 17 children with epilepsy and 18 matched healthy participants.IBSR evaluation used eight primarily subcortical labels; RE annotations covered left and right putamen.Source: §3 · pp. 2–3
- CNN priors achieved higher Dice and lower Hausdorff and contour mean distance than RF priors; Figures 1 and 2 show comparisons on IBSR structures and RE putamen, respectively.The RF baseline used the same MRF refinement as the CNN approach.Source: §3 · p. 3, Figure 1 · p. 3, Figure 2 · p. 3
- The proposed method segmented three subcortical structures better than FreeSurfer and FSL-FIRST on both datasets, while FreeSurfer did better for IBSR caudate, possibly because CNNs miss thin tails.The evaluated structures were thalamus, caudate, putamen, and pallidum; RE annotations covered the left and right putamen.Source: §3 · pp. 2–3
3.1. CNN Training and Evaluation Details
- Training used single resized 256 × 256 grayscale slices with corresponding label maps, augmented by horizontal flips and translations of 5, 10, 15, and 20 pixels.IBSR was split into three sets, with two for training and one for testing; RE was split into equal training and testing subsets.Source: §3.1 · p. 3, §3 · p. 3
- Sagittal and coronal slice models performed poorly because symmetric left and right putamen can appear alike, leading the CNN to assign them the same label.The example illustrates why individual-slice evaluation needs priors that use full-volume structure.Source: §3.1 · pp. 3–4
4. CONCLUSION
The conclusion presents a 2D fully convolutional network with MRF refinement for frontostriatal subcortical MRI segmentation. It reports superiority to learned-prior and state-of-the-art methods, while identifying thin caudate tails, symmetry, and volumetric consistency as limitations.
4. CONCLUSION
- The method applies a fully convolutional network to 2D slices and uses its output as voxel likelihoods and MRF potentials to impose spatial volumetric homogeneity.The paper reports that this framework outperforms other learned-prior approaches and state-of-the-art segmentation methods.Source: §4 · p. 4
- The model does not accurately capture thin caudate tails, and symmetric structures confound training on views parallel to their plane of symmetry.Because training uses 2D slices, graph-based methods are needed for volumetric consistency; volumetric network layouts may help address these limitations.Source: §4 · p. 4