Source-linked AI summary
nnU-Net for Brain Tumor Segmentation
Fabian Isensee, Paul F. Jaeger, Peter M. Full, Philipp Vollmuth, Klaus H. Maier-Hein
TL;DR
Brain tumor segmentation is difficult, and BraTS provides a major benchmark for comparing methods. This work applies nnU-Net as both a baseline and development framework, adds BraTS-specific modifications, and reimplements the competition ranking for model selection. The final ensemble won BraTS 2020 with reported Dice and HD95 scores across whole tumor, tumor core, and enhancing tumor.
Problem
Brain tumor segmentation is difficult, while accurate delineations could support diagnosis, therapy planning, monitoring, and biomarker-based disease stratification.
Method
The authors use nnU-Net as a baseline and development framework, incorporate BraTS-specific modifications, and reimplement the competition ranking for model selection.
Results
88.95, 85.06 and 82.03 Dice and 8.498, 17.337 and 17.805 HD95 were obtained for whole tumor, tumor core and enhancing tumor, respectively, with first place in BraTS 2020.
Takeaways & Limitations
nnU-Net provided high segmentation accuracy out of the box, while BraTS-specific modifications produced substantially higher validation performance.
Takeaways & Limitations
The study includes few modifications and insufficiently extensive validation to identify which aspects contributed most to performance.
Abstract
from arXiv · showhide
We apply nnU-Net to the segmentation task of the BraTS 2020 challenge. The unmodified nnU-Net baseline configuration already achieves a respectable result. By incorporating BraTS-specific modifications regarding postprocessing, region-based training, a more aggressive data augmentation as well as several minor modifications to the nnUNet pipeline we are able to improve its segmentation performance substantially. We furthermore re-implement the BraTS ranking scheme to determine which of our nnU-Net variants best fits the requirements imposed by it. Our final ensemble took the first place in the BraTS 2020 competition with Dice scores of 88.95, 85.06 and 82.03 and HD95 values of 8.498,17.337 and 17.805 for whole tumor, tumor core and enhancing tumor, respectively.
1 Introduction
Brain tumor segmentation is difficult but clinically valuable, supporting diagnosis, treatment planning, monitoring, biomarker discovery, and disease stratification. BraTS provides a large public benchmark, and this work evaluates nnU-Net as both a baseline and development framework for the challenge.
- Brain tumor segmentation is difficult, while accurate delineations could support diagnosis, therapy planning, and therapy response monitoring.
- Tumor and subregion segmentation can support imaging biomarker identification, disease stratification, and treatment response prediction.
- BraTS provides the largest fully annotated, publicly available database for model development and objective comparison of segmentation methods.
- BraTS 2020 includes 369 training and 125 validation cases, with validation references withheld and evaluation performed through an online platform.
- The authors investigate nnU-Net as a general-purpose baseline and framework for developing BraTS-specific segmentation methods.
2 Method
The paper argues that model selection should follow the competition’s actual ranking scheme rather than relying only on mean-aggregated segmentation metrics. It therefore reimplements BraTS ranking to select suitable nnU-Net variants.
- BraTS ranks submissions by aggregating per-case ranks across regions and metrics, rather than by directly ranking mean-aggregated scores.
- Each submission receives six ranks per test case, covering three regions and two metrics, which are averaged across cases and metrics.
- BraTS uses a rank-then-aggregate approach to combine Dice and HD95 across test cases.
- For empty enhancing-tumor references, false positives receive Dice 0 and HD95 373.13, whereas an empty prediction receives Dice 1 and HD95 0.
- The authors reimplemented BraTS ranking to rank their models against one another and select the best-performing variants.
2.2 nnU-Net baseline
The nnU-Net baseline is an automatically configured, plain 3D U-Net-like system used without modifications before later BraTS-specific changes. Its pipeline combines standardized preprocessing, multi-scale architecture, supervised training, and on-the-fly augmentation.
- nnU-Net is used without modifications as a baseline and automatically configures segmentation pipelines for biomedical datasets.
- The baseline uses a 3D U-Net-like encoder-decoder with skip connections, plain convolutions, strided-convolution downsampling, and transposed-convolution upsampling.
- Auxiliary segmentation outputs branch from nearly all decoder resolutions to provide deep supervision.
- The BraTS configuration uses 128 × 128 × 128 input patches, batch size 2, five downsampling operations, and a 4 × 4 × 4 bottleneck feature map.
- Training minimizes the sum of Dice and cross-entropy losses using stochastic gradient descent with on-the-fly data augmentation.
2.3 BraTS-specific optimizations
The authors integrate promising BraTS-specific modifications into nnU-Net, emphasizing method development beyond its standardized baseline. One modification trains directly on the overlapping regions used for evaluation rather than only the provided class labels.
- nnU-Net is extended with selected BraTS-specific modifications to serve as a framework for method development.
- BraTS supplies three class labels, but evaluates whole tumor, tumor core, and enhancing tumor as partially overlapping regions.
- The method replaces softmax with sigmoid to optimize the evaluated regions directly rather than the individual class labels.
Region-based training
The method removes enhancing-tumor predictions under a specific empty-reference condition to exploit how BraTS evaluation assigns Dice and ranks.
- When the reference enhancing tumor is empty, BraTS assigns Dice 1 to predictions with zero false positives, placing them in the shared first rank.This creates an incentive to remove enhancing tumor predictions entirely under the relevant condition.
2.4 Further nnU-Net Modifications
The authors modify nnU-Net’s configuration by increasing batch size, while using more aggressive augmentation to enlarge the effective training set and improve robustness.
- Further nnU-Net Modifications: Batch size increases from 2 to 5 to potentially improve model accuracy as the BraTS dataset grows.The change addresses noisier gradients and the trade-off between fitting accuracy and overfitting.
- Further nnU-Net Modifications: More aggressive data augmentation artificially enlarges the training set and is intended to increase model robustness.The baseline nnU-Net already uses a broad range of aggressive augmentation techniques.
More data augmentation
The authors intensify nnU-Net’s augmentation pipeline, motivated by the possibility that aggressive augmentation can improve robustness to scanner-related domain gaps when paired with batch normalization.
- Augmentations are applied on the fly during training using the batchgenerators framework.
- Rotation and scaling probabilities increase from 0.2 to 0.3, while the scale range expands from (0.85, 1.25) to (0.65, 1.6).Scaling factors are selected independently for each axis.
- Elastic deformation and additive brightness augmentation are each applied with probability 0.3, alongside more aggressive Gamma augmentation.
- Prior M&Ms observations suggested that aggressive augmentation could close scanner domain gaps effectively only with batch normalization rather than instance normalization.The authors relate this observation to lower BraTS test-case Dice scores than training and validation scores, which they interpret as possible evidence of a domain gap.
Batch dice
The authors replace sample-wise Dice averaging with a batch-level Dice computation that regularizes gradients from samples containing few annotated voxels.
- Standard sample Dice computes loss independently for each minibatch sample and then averages those losses.
- Batch Dice computes Dice loss across all minibatch samples as one combined sample rather than treating samples independently.Samples with few annotated voxels are thereby overshadowed by other samples in the batch.
- This batch-level computation regularizes the loss by reducing the dominance of large gradients from samples with few annotated voxels.The authors motivate this as protection against counterproductive updates when reference segmentations are imperfect.
Abbreviations
The paper abbreviates nnU-Net variants by the modifications applied, including region-based training, aggressive augmentation, batch normalization, and batch Dice.
- R denotes region-based training.
- DA/DA* denotes more aggressive data augmentation, with DA* applying brightness augmentation independently to each modality with probability 0.5.
- BD denotes training with batch Dice rather than sample Dice.
- Model names combine their applied modifications, and all reported models include postprocessing.
3 Results
The experiments compare nnU-Net variants using cross-validation ensembles, aggregated metrics, and a reimplemented BraTS ranking. The final ensemble was selected from validation performance and achieved first place in the BraTS 2020 competition.
- 3.1 Aggregated Scores: Five-fold cross-validation on 369 training cases estimates performance and selects the postprocessing threshold without using validation labels.
- 3.1 Aggregated Scores: Aggregated mean metrics can mislead model selection because enhancing-tumor false positives without reference tumors receive HD95 373.13, dominating averages.
- 3.2 Internal BraTS-like ranking: The internal BraTS-style ranking produces different model ordering from mean Dice, with DA* models ranking 1 and 2 on validation but 6 and 7 on training.
- 3.2 Internal BraTS-like ranking: The validation-selected final ensemble contained 25 models from three configurations, including 15 models of BL*+R+DA*+BN.
- 3.2 Internal BraTS-like ranking: 91.24, 85.06 and 79.89 were the ensemble’s validation mean Dice scores for whole tumor, tumor core and enhancing tumor, respectively, with HD95 3.69, 7.82 and 23.50.
- 3.3 Qualitative results: Qualitative cases were sampled by Dice-score percentiles, and the worst example’s enhancing lesion was removed during postprocessing despite being segmented beforehand.
- 3.4 Test set results: The submission achieved first place in the BraTS 2020 competition, with quantitative test-set results supplied by the challenge organizers.
4 Discussion
The final ensemble won BraTS 2020, while the authors identify limited experimental validation, uncertain modification contributions, label subjectivity, and ranking-related caveats.
- BraTS-specific modifications substantially improved validation performance over the nnU-Net baseline, although the manuscript covers few modifications and lacks extensive experimental validation.
- The experiments do not identify which nnU-Net modifications contributed most, and some design choices may have reduced overall performance.The authors believe more thorough hyperparameter optimization could yield further gains.
- Tumor-core errors may reflect subjective annotations of non-enhancing tumor, which often has little imaging evidence, particularly in LGG cases.
- Postprocessing that removes small enhancing-tumor lesions is favored by BraTS evaluation but may be undesirable when clinically important small lesions must be detected.The authors recommend omitting this postprocessing in that clinical scenario.
- Mean-aggregated metrics can misrepresent BraTS performance because outlier HD95 values and absent enhancing-tumor references distort model comparisons.The authors therefore reimplemented the competition ranking, whose ranks can also change when similar methods are added or removed.