Source-linked AI summary
Brain Tumor Segmentation using an Ensemble of 3D U-Nets and Overall Survival Prediction using Radiomic Features
Xue Feng, Nicholas Tustison, Craig Meyer
TL;DR
Brain tumor sub-region segmentation is difficult because gliomas are heterogeneous and patch-based approaches can have limited efficiency and receptive fields. The paper uses an ensemble of six varied 3D U-Nets for segmentation and linear regression with imaging and clinical features for survival prediction; the ensemble improved overall segmentation performance, while the survival model achieved first place in the final competition stage. The study notes computational limits on ensemble size and missing histological and genetic features for survival prediction.
Problem
Heterogeneous glioma sub-regions make accurate multimodal MRI segmentation challenging, while survival prediction with small datasets risks overfitting.
Method
The study combines an ensemble of six 3D U-Nets with varied hyper-parameters for segmentation and linear regression using imaging and non-imaging features for survival prediction.
Results
The ensemble had the overall best segmentation performance among the evaluated individual models, and the survival model won 1st place in the final competition stage.
Takeaways & Limitations
Ensemble modeling improved segmentation results, while a simple linear model provided a robust survival-prediction approach against overfitting.
Takeaways & Limitations
The survival model lacked histological and genetic features that may be important for overall survival, limiting expressiveness and accuracy.
Abstract
from arXiv · showhide
Accurate segmentation of different sub-regions of gliomas including peritumoral edema, necrotic core, enhancing and non-enhancing tumor core from multimodal MRI scans has important clinical relevance in diagnosis, prognosis and treatment of brain tumors. However, due to the highly heterogeneous appearance and shape, segmentation of the sub-regions is very challenging. Recent development using deep learning models has proved its effectiveness in the past several brain segmentation challenges as well as other semantic and medical image segmentation problems. Most models in brain tumor segmentation use a 2D/3D patch to predict the class label for the center voxel and variant patch sizes and scales are used to improve the model performance. However, it has low computation efficiency and also has limited receptive field. U-Net is a widely used network structure for end-to-end segmentation and can be used on the entire image or extracted patches to provide classification labels over the entire input voxels so that it is more efficient and expect to yield better performance with larger input size. Furthermore, instead of picking the best network structure, an ensemble of multiple models, trained on different dataset or different hyper-parameters, can generally improve the segmentation performance. In this study we propose to use an ensemble of 3D U-Nets with different hyper-parameters for brain tumor segmentation. Preliminary results showed effectiveness of this model. In addition, we developed a linear model for survival prediction using extracted imaging and non-imaging features, which, despite the simplicity, can effectively reduce overfitting and regression errors.
1 Introduction
Glioma sub-regions vary biologically and radiographically, making accurate multimodal MRI segmentation difficult but clinically useful for tumor phenotyping and treatment planning. CNNs and U-Net-based approaches address this challenge, with ensembles offering a way to improve segmentation over individual models.
- Gliomas contain heterogeneous sub-regions, including peritumoral edema, necrotic core, and enhancing and non-enhancing tumor core.
- Different tumor sub-regions exhibit distinct intensity profiles across multimodal MRI scans, reflecting differences in tumor biology.
- Quantitative imaging features such as segmented tumor volumes can support image-based tumor phenotyping and surgical or treatment planning.
- CNN-based models have become effective approaches for brain tumor segmentation, while patch-based methods can have limited computation efficiency and receptive fields.
- U-Net predicts labels for all input voxels and ensembles of models trained with varied datasets or hyper-parameters can improve segmentation over a single model.
2 Methods
The method preprocesses multimodal MRI, extracts tumor-focused patches, trains six varied 3D U-Nets, ensembles full-volume predictions, and fits linear survival regression from imaging and clinical features. It combines segmentation and survival-prediction pipelines while addressing memory, class imbalance, and overfitting constraints.
- Segmentation: The segmentation pipeline preprocesses images, extracts patches, trains multiple 3D U-Nets, predicts full volumes, and averages model outputs for final labels.
- Image Pre-processing: Bias correction and non-local means denoising were applied to multimodal MRI before normalization and contrast fusion.
- Non-uniform Patch Extraction: Non-uniform patch extraction assigns higher sampling probability to foreground voxels to emphasize variable tumor labels while reducing whole-image memory and imbalance problems.
- Network Structure and Training: The network uses a 3D U-Net with encoding and decoding blocks, skip connections, 3D convolutions, PReLU activation, batch normalization, dropout, and cross-entropy loss.
- Network Structure and Training: Six models varied encoding/decoding blocks, loss weights, and patch sizes; weighted loss used 1.0 for background and 2.0 for each foreground class.
- Deployment and Ensemble: Sliding-window deployment averages overlapping predictions, while ensemble probabilities are averaged and the highest-probability class becomes each voxel's label.
- Survival Prediction: Survival prediction fits normalized linear regression using sub-region volumes and surface areas plus age and resection-status features.
3 Results
The ensemble of six 3D U-Net models achieved the strongest overall segmentation performance among the evaluated models, while the linear survival model showed mixed validation performance but won the final test stage.
- Brain Tumor Segmentation: The ensemble achieved the overall best performance compared with each individual 3D U-Net model.Table 2 reports mean Dice scores and 95 Hausdorff distances for enhancing tumor, whole tumor, and tumor core.
- Brain Tumor Segmentation: All six individual 3D U-Net models performed similarly before ensembling.
- Survival Prediction: The survival model achieved validation accuracy 0.321, MSE 99115.86, median SE 77757.86, std SE 104291.596, and Spearman Coefficient 0.264.The training coefficient of determination was 0.259, based on 163 training subjects and 28 evaluated cases.
- Survival Prediction: The method won 1st place on the testing dataset despite lower validation accuracy than other leading teams.The authors attribute this outcome to significant overfitting by other teams on validation data.
4 Discussion and Conclusions
The study combines an ensemble of 3D U-Nets for tumor segmentation with linear regression for survival prediction. Results indicate strong typical segmentation quality and robustness against overfitting, while computation and feature availability constrain performance.
- Brain Tumor Segmentation: The proposed pipeline ensembles six 3D U-Nets trained with varied architecture, patch-size, and loss-weight hyper-parameters.Each model produces full-volume predictions during deployment before final ensemble modeling.
- Brain Tumor Segmentation: Median Dice scores for the final ensemble were 0.867 for ET, 0.923 for WT, and 0.904 for TC.These median values were substantially higher than the corresponding mean metrics.
- Brain Tumor Segmentation: Some cases had Dice scores as low as 0 for ET and TC and 0.6 for WT, mainly because low sensitivity failed to recognize corresponding tumor regions.The authors suggest these failures arise when region characteristics deviate substantially from the training dataset.
- Brain Tumor Segmentation: Batch normalization improved 3D U-Net stability and performance, with batch statistics at deployment outperforming global statistics in this implementation.With batch size 1, the operation performs per-channel normalization by subtracting the channel’s own mean; the mechanism remains uncertain.
- Brain Tumor Segmentation: Overlapping sliding windows improve deployment accuracy by averaging predictions from multiple windows.This addresses unstable boundary-voxel predictions caused by padding during full-volume inference.
- Limitations: Only six networks were trained because of computation-time limitations, although more networks may improve results with diminishing marginal gains.
- Survival Prediction: Linear regression was selected for survival prediction because the small dataset was considered highly prone to overfitting and lacked histological and genetic features.The simpler model reduced expressiveness, while additional clinical features are expected to improve accuracy.
- Conclusions: The survival model won first place in the competition’s final stage.