PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy

Christian Wachinger, Martin Reuter, Tassilo Klein

arXiv:1702.08192v1cs.CVcs.AIcs.LG

TL;DR

Accurate neuroanatomical segmentation supports MRI-based measurements of brain structure, but manual segmentation is time-consuming. DeepNAT addresses this with hierarchical, multi-task 3D convolutional segmentation augmented by intrinsic coordinates and a fully connected CRF, and reports high potential for segmenting neuroanatomy.

  • Problem

    Manual brain MRI segmentation is time-consuming, motivating computational tools for large-scale studies and quantitative analyses of structural brain changes.

  • Method

    DeepNAT combines multi-task neighborhood prediction, hierarchical foreground-to-structure segmentation, spectral coordinates, and a fully connected CRF for 3D brain segmentation.

  • Results

    DeepNAT's results demonstrate high potential for convolutional neural networks to segment neuroanatomy, with coordinates and multi-task learning improving segmentation accuracy.

  • Takeaways & Limitations

    The approach combines contextual coordinates, class-imbalance handling, neighborhood prediction, and voxel-label agreement within one brain-segmentation pipeline.

  • Takeaways & Limitations

    The limited dataset prevented a separate validation set, creating a risk of overfitting to the testing data in contribution comparisons.

Abstract

from arXiv · show

We introduce DeepNAT, a 3D Deep convolutional neural network for the automatic segmentation of NeuroAnaTomy in T1-weighted magnetic resonance images. DeepNAT is an end-to-end learning-based approach to brain segmentation that jointly learns an abstract feature representation and a multi-class classification. We propose a 3D patch-based approach, where we do not only predict the center voxel of the patch but also neighbors, which is formulated as multi-task learning. To address a class imbalance problem, we arrange two networks hierarchically, where the first one separates foreground from background, and the second one identifies 25 brain structures on the foreground. Since patches lack spatial context, we augment them with coordinates. To this end, we introduce a novel intrinsic parameterization of the brain volume, formed by eigenfunctions of the Laplace-Beltrami operator. As network architecture, we use three convolutional layers with pooling, batch normalization, and non-linearities, followed by fully connected layers with dropout. The final segmentation is inferred from the probabilistic output of the network with a 3D fully connected conditional random field, which ensures label agreement between close voxels. The roughly 2.7 million parameters in the network are learned with stochastic gradient descent. Our results show that DeepNAT compares favorably to state-of-the-art methods. Finally, the purely learning-based method may have a high potential for the adaptation to young, old, or diseased brains by fine-tuning the pre-trained network with a small training sample on the target application, where the availability of larger datasets with manual annotations may boost the overall segmentation accuracy in the future.

1. Introduction

DeepNAT addresses the limits of manual, atlas-based, and conventional patch-based brain segmentation with an end-to-end 3D convolutional approach. It combines hierarchical classification, multi-task neighborhood prediction, intrinsic spectral coordinates, and fully connected 3D CRF inference.

  • Motivation: Accurate neuroanatomical segmentation enables MRI-based measurements of brain volume, thickness, and shape for studying aging, disease, and individual phenotypes.
  • Motivation: Atlas-based methods can require deformable registration whose smoothness constraints may impede correct spatial alignment between subjects.
  • Motivation: Patch-based methods reduce image-wide segmentation to local prediction but lose broader spatial context and may rely on handcrafted representations.
  • DeepNAT: DeepNAT uses a 3D deep convolutional network with larger patches and more layers to model relationships needed for fine-grained brain-structure identification.
  • DeepNAT: Multi-task learning predicts the center voxel and neighboring labels simultaneously, while hierarchical segmentation separates foreground from background before classifying 25 brain structures.
  • DeepNAT: Spectral coordinates encode intrinsic brain location using Laplace-Beltrami eigenfunctions, and a fully connected CRF imposes label agreement across the 3D image domain.

2. Method

DeepNAT performs whole-brain segmentation with a hierarchical, patch-based 3D DCNN that combines multi-task predictions, intrinsic spectral coordinates, and fully connected CRF refinement.

  • Patch-Based DCNN: The network estimates probabilistic labels from skull-stripped 3D image patches using cascaded convolutional networks.The two cascaded networks share the same architecture but use different numbers of output neurons for their classification tasks.
  • Hierarchical Segmentation: DeepNAT first separates foreground from background, then classifies the foreground into 25 brain structures to address class imbalance.The foreground contains the target structures, while background within the brain mask is substantially larger than any individual structure.
  • Network Architecture: The DCNN architecture uses three convolutional layers, pooling, batch normalization, nonlinearities, fully connected layers, dropout, and Xavier weight initialization.The network contains 2,687,200 learned parameters, excluding negligible bias parameters.
  • Multi-task Learning: Multi-task learning predicts labels for the patch center and neighboring locations, allowing multiple patch-based predictions to contribute to voxel labeling.Tasks share the network and differ in their final inner-product layer; experiments use 7- and 27-location neighborhoods.
  • Spectral Brain Coordinates: Spectral brain coordinates preserve spatial context by augmenting patches with the first three Laplacian eigenfunctions computed inside the 3D brain mask.These intrinsic coordinates are invariant to image rotations and translations, although additional eigenfunctions may require reordering strategies.
  • Conditional Random Field: A fully connected 3D CRF converts DCNN probabilities into the final segmentation by encouraging label agreement while incorporating image appearance and spatial smoothness.Efficient approximate inference makes the all-pairs model practical on the entire brain volume.

3. Results

DeepNAT’s evaluation shows that its hierarchical, coordinate-augmented, multi-task configuration improves segmentation accuracy, while the fully connected CRF performs comparably to PICSL and better than several alternatives.

  • Configuration analysis: Seven-task multi-task learning substantially outperformed single-task prediction and slightly outperformed using 27 tasks.DeepNAT’s improvement over the single-task and 27-task variants was significant, with p < 0.001 for all variants except the spectral-only comparison, where p < 0.05.
  • Optimization analysis: 0.897 median Dice was obtained by DeepNAT, compared with 0.888 at learning rate 0.005, 0.895 with minibatch size 512, and 0.881 using ADAGRAD.These results show the reported sensitivity of accuracy to optimization settings.
  • Overall comparison: DeepNATcrf achieved significantly higher Dice scores than DeepNAT, FreeSurfer, and STAPLE, while its difference from PICSL was not significant.Across all structures, the comparisons yielded p < 0.001 against DeepNAT, FreeSurfer, and STAPLE, and p = 0.27 against PICSL.
  • Additional evaluation: With 15 training and 15 test images, DeepNATcrf remained significantly better than DeepNAT, FreeSurfer, and STAPLE, while its difference from PICSL was not significant.The median Dice was 0.007 points higher than PICSL, while the mean Dice points were the same.

4. Discussion

DeepNAT combines 3D convolutional modeling with spatial coordinates, hierarchical classification, multi-task prediction, and CRF refinement. Discussion results support gains over several comparators, while highlighting dataset, architecture, and end-to-end-training limitations.

  • Architecture: 3D convolutional networks increase modeling complexity but can capture more complex input-output relationships when trained with batch normalization, dropout, and Xavier initialization.The architecture was not fully explored because of memory constraints and long training times.
  • Hierarchical segmentation: Hierarchical segmentation addresses background-class imbalance by separating foreground from background before identifying individual brain structures.The authors report a benefit over directly segmenting brain structures.
  • Location information: Adding spectral and Cartesian coordinates substantially improves segmentation accuracy because patch-based methods otherwise lose larger image context.Spectral coordinates improve over Cartesian coordinates, while their combination provides further gains from complementary information.
  • Multi-task learning: Multi-task learning significantly improves accuracy for all brain structures by predicting neighboring labels alongside the center voxel.Shared network parameters produce multiple predictions per voxel and may yield more robust segmentations; the center task remains slightly more accurate than surrounding tasks.
  • Comparison to state-of-the-art: DeepNAT improves significantly over FreeSurfer and spatial STAPLE, while its improvement over PICSL is not statistically significant.DeepNATcrf also shows more accurate caudate labeling in the illustrated examples, and the fully connected CRF benefits final discrete segmentation.
  • Limitations and future directions: The evaluation is constrained by a small medical-imaging training set, overlapping-patch computational overhead, and the absence of a separate validation set.The authors note that direct whole-image prediction could speed inference but would require many more manually segmented images.

5. Conclusion

DeepNAT combines multi-task learning, hierarchical segmentation, spectral coordinates, and a 3D fully connected conditional random field for structural MRI brain segmentation. The authors report high potential for convolutional neural networks and propose fine-tuning as a route toward tailoring segmentations to specific populations.

  • DeepNAT integrates multi-task learning, hierarchical segmentation, spectral coordinates, and a 3D fully connected conditional random field.These components respectively support neighborhood label prediction, class-imbalance handling, patch context, and label agreement between nearby voxels.
  • The conditional random field ensures label agreement between close voxels after the network produces probabilistic outputs.
  • DeepNAT demonstrates the high potential of convolutional neural networks for segmenting neuroanatomy.
  • Fine-tuning a pre-trained network is proposed as a promising avenue for tailoring segmentations to young, old, or diseased brains.The authors note that larger datasets with manual annotations could further improve segmentation accuracy.
  • The authors make DeepNAT extensions, network definitions, and trained networks available for download.
Loading 1702.08192v1…