Source-linked AI summary
Brain Tumor Segmentation with Deep Neural Networks
Mohammad Havaei, Axel Davy, David Warde-Farley, Antoine Biard, Aaron Courville, Yoshua Bengio, Chris Pal, Pierre-Marc Jodoin, Hugo Larochelle
TL;DR
Brain tumor segmentation from MR images is important for diagnosis, growth prediction, and treatment planning, but tumors vary widely in shape, size, location, and contrast. The paper develops efficient deep CNNs combining local and global context, and reports improved accuracy and speed over the published state of the art on BRATS 2013.
Problem
Brain tumor segmentation from MR images is important for diagnostics, growth prediction, and treatment planning.
Method
The method uses cascaded CNNs with local and global pathways, plus two-phase training to address imbalanced tumor labels.
Results
The best model improved accuracy and speed over the published state of the art on the BRATS 2013 evaluation.
Takeaways & Limitations
The resulting CNN architectures provide practical automatic brain segmentation, processing an entire brain in 25 seconds to 3 minutes.
Takeaways & Limitations
Evaluation was limited to BRATS 2013 because BRATS 2014 could not be used due to evaluation-system and labeled-data quality problems.
Abstract
from arXiv · showhide
In this paper, we present a fully automatic brain tumor segmentation method based on Deep Neural Networks (DNNs). The proposed networks are tailored to glioblastomas (both low and high grade) pictured in MR images. By their very nature, these tumors can appear anywhere in the brain and have almost any kind of shape, size, and contrast. These reasons motivate our exploration of a machine learning solution that exploits a flexible, high capacity DNN while being extremely efficient. Here, we give a description of different model choices that we've found to be necessary for obtaining competitive performance. We explore in particular different architectures based on Convolutional Neural Networks (CNN), i.e. DNNs specifically adapted to image data. We present a novel CNN architecture which differs from those traditionally used in computer vision. Our CNN exploits both local features as well as more global contextual features simultaneously. Also, different from most traditional uses of CNNs, our networks use a final layer that is a convolutional implementation of a fully connected layer which allows a 40 fold speed up. We also describe a 2-phase training procedure that allows us to tackle difficulties related to the imbalance of tumor labels. Finally, we explore a cascade architecture in which the output of a basic CNN is treated as an additional source of information for a subsequent CNN. Results reported on the 2013 BRATS test dataset reveal that our architecture improves over the currently published state-of-the-art while being over 30 times faster.
1. Introduction
Brain tumor segmentation from MRI supports diagnosis, growth prediction, and treatment planning but is difficult because gliomas and glioblastomas are diffuse, variable, poorly contrasted, and affected by nonstandardized voxel values. The paper therefore develops task-adapted CNN architectures that combine local and contextual information and evaluates them on BRATS 2013.
- Motivation: MRI-based brain tumor segmentation can improve diagnostics, growth-rate prediction, and treatment planning.MRI provides detailed brain images and is commonly used to diagnose brain tumors.
- Challenges: Gliomas and glioblastomas are difficult to segment because they are diffuse, poorly contrasted, variable in location, shape, and size, and may have tentacle-like structures.Their surrounding edema further complicates localization, while MRI voxel-value scales vary with scanner and acquisition protocol.
- Approach: Deep neural networks learn increasingly complex, task-adapted feature hierarchies directly from in-domain data, unlike pipelines using hand-designed features.The paper specifically investigates CNNs, which are DNNs adapted to image data, and reports their advantages, disadvantages, and performance.
- Contributions: The proposed CNN architectures use Maxout units, Dropout regularization, and both local tumor shape and broader contextual information.The architectures are designed to exploit recent advances in CNN design and training.
- Experimental setting: The experiments use the fully annotated BRATS 2013 dataset with established training and testing splits for direct quantitative comparison with other methods.The study focuses its experimental analysis on the MICCAI brain tumor segmentation challenge 2013 dataset.
- Contributions: The method is fully automatic, ranks second on the BRATS 2013 scoreboard, and segments a brain in 25 seconds to 3 minutes.The reported runtime is one order of magnitude faster than most state-of-the-art methods.
2. Related work
Prior brain-tumor segmentation methods are broadly divided into generative and discriminative models, with deep CNNs emerging as a promising alternative to hand-designed feature pipelines. This work extends preliminary two-pathway CNN results while noting that its BRATS 2014 experiments were incomplete and not fully comparable.
- Brain tumor segmentation methods are broadly categorized as generative or discriminative models.
- Generative models: Generative models use domain-specific tissue priors and typically detect tumors as deviations from normal anatomy using atlas- or template-aligned MR images.
- Discriminative models: Discriminative models use limited anatomical prior knowledge and directly model voxel labels from low-level image features such as pixels, histograms, textures, and alignment-based descriptors.
- Discriminative models: Hand-designed discriminative pipelines assume sufficiently discriminative features, but generic edge-based features may lack brain-tumor adaptation and require feature computation.
- Deep CNNs: Deep CNNs are presented as a promising approach for brain tumor segmentation, and this work extends preliminary two-pathway results using that architecture as a building block.
- Limitations: BRATS 2014 experiments were not reported as complete and fair in the manuscript because the authors could not provide complete experiments at submission time.
3. Our Convolutional Neural Network Approach
The approach performs slice-wise, patch-based CNN segmentation of multimodal axial MR images, using convolutional architectures that combine local and global context. It further improves efficiency and label modeling through convolutional output layers, cascaded CNNs, and two-phase training for severe class imbalance.
- CNN formulation: The model segments each 2D axial slice by predicting a pixel’s class from an M×M patch across T1, T2, T1C, and FLAIR modalities.The input is an M×M×R tensor, with the spatial topology modeled in the axial X-Y plane.
- CNN formulation: The final prediction layer is convolutional rather than fully connected, producing softmax-normalized label probabilities without pooling for more efficient segmentation inference.The convolutional output layer avoids the inefficient test-time behavior of a conventional fully connected output layer.
- Multi-path architectures: The architecture concatenates feature maps from different layers to create multiple computational paths, including local 7 × 7 and global 13 × 13 receptive-field pathways.The local pathway captures visual details near the pixel, while the global pathway incorporates broader contextual information about its brain location.
- Cascaded architectures: A second CNN receives the first CNN’s output probabilities as additional inputs, allowing efficient modeling of dependencies between adjacent segmentation labels.In the pre-output concatenation variant, the first and second CNN outputs correspond to successive mean-field-like iterations.
- Two-phase training: 98% of voxels are healthy, so training first uses equiprobable labels and then retrains only the output layer on a representative label distribution.The pathological classes comprise 0.18% necrosis, 1.1% edema, 0.12% non-enhanced tumor, and 0.38% enhanced tumor.
4. Implementation details
The implementation uses GPU-accelerated deep learning with minimal preprocessing, validation-based hyperparameter selection, and connected-component post-processing. Full-image convolutional inference substantially accelerates segmentation, with model-specific prediction times reported for the tested architectures.
- Implementation platform: Pylearn2 provides the deep-learning implementation and GPU support for accelerating execution.The library is open source and specializes in deep learning algorithms.
- Preprocessing: Preprocessing removes the highest and lowest 1% of intensities, applies N4ITK bias correction to T1 and T1C, and normalizes each input channel.The preprocessing follows the procedure used by Tustison et al., the 2013 BRATS challenge winner.
- Post-processing: Connected components remove flat prediction blobs caused by bright brain corners near the skull.This is the stated post-processing method for suppressing these artifacts.
- Hyperparameter tuning: Hyperparameters were selected by grid search and cross-validation on a validation set, while max pooling used stride 1 to preserve per-pixel accuracy.The authors observed that max pooling in the global path did not improve accuracy.
- Inference speed: 25 seconds per brain is the reported segmentation time for the full-image TwoPathCNN implementation on a Titan black card.GPU execution and convolutional output enable processing a full image rather than individual patches.
- Inference speed: 1.5 minutes, 1.7 minutes, and 3 minutes are the average prediction times for MFCascadeCNN, LocalCascadeCNN, and InputCascadeCNN, respectively.These timings are reported for test-time predictions of the three cascade models.
5. Experiments and Results
Experiments on BRATS2013 evaluated CNN variants and cascaded architectures using online Dice, sensitivity, and specificity measures across three tumor regions. InputCascadeCNN* achieved the strongest reported performance, ranking second on the BRATS2013 scoreboard and exceeding published methods while substantially reducing prediction time.
- Dataset and evaluation: Experiments used BRATS2013 real-patient data, whose training set contained 30 subjects with pixel-accurate ground truth and whose test set contained 10 high-grade cases.The authors excluded BRATS2014 because of evaluation-system and label-quality problems, and reported worse performance when training on its older dataset.
- Dataset and evaluation: Online evaluation computed Dice, sensitivity, and specificity for complete, core, and enhancing tumor regions.The regions comprise all tumor structures, all except edema, and enhanced tumor alone, respectively.
- TwoPathCNN experiments: Jointly trained TwoPathCNN, combining local-detail and global-context pathways, was compared with single-path models, two-phase training, and averaged pathway outputs.The experiments also examined separately trained pathway outputs to assess the benefit of joint training and imbalance handling.
- Cascaded architectures: InputCascadeCNN* achieved the best overall performance, improving Dice across all tumor regions and reaching second place on the BRATS2013 scoreboard.MFCascadeCNN* produced smoother class boundaries, whereas LocalCascadeCNN* reduced complete-tumor false positives without improving tumor-core or enhancing-tumor performance.
- Comparison with state of the art: 3 minutes was the InputCascadeCNN* prediction time per brain versus 100 minutes for Tustison’s method, making it over 30 times faster.TwoPathCNN* required 25 seconds per prediction and was reported as over 200 times faster than Tustison’s method.
6. Conclusion
The paper concludes that deep convolutional neural networks achieve accurate, fast, and practical automatic brain tumor segmentation. Performance is attributed to architectures combining local and global information, stacked CNNs for label dependencies, and two-phase training for label imbalance.
- Conclusion: The best model improved on the published state-of-the-art in both accuracy and speed on the BRATS 2013 online evaluation.The conclusion reports confirmation from the BRATS 2013 online evaluation system.
- Conclusion: A novel two-pathway architecture models local details and global context, while stacking two CNNs models local label dependencies.These architectural choices are identified as contributing to the method’s high performance.
- Conclusion: Two-phase training enables efficient CNN training when tumor labels are distributed unevenly.The procedure addresses the difficulty of unbalanced label distributions.
- Conclusion: 25 seconds to 3 minutes is the time required to segment an entire brain, making the CNN systems practical.The reported range applies across the described CNN architectures and is enabled by convolutional models and efficient GPU implementation.