Source-linked AI summary
ME-Net: Multi-Encoder Net Framework for Brain Tumor Segmentation
Wenbo Zhang, Guang Yang, He Huang, Weiji Yang, Xiaomei Xu, Yongkai Liu, Xiaobo Lai
TL;DR
Manual MRI tumor segmentation is time-consuming and varies across experts. ME-Net uses separate encoders for four MRI modalities and weighted Categorical Dice, reporting promising results on BraTS 2020. The authors note that ET segmentation remains weak, potentially because its features are blurred.
Problem
Manual MRI tumor segmentation takes considerable time, resources, and expertise, and experts can differ in their subregion segmentations.
Source: §1 · p. 3Method
ME-Net gives each of four MRI modalities its own encoder, merges their feature maps for decoding, and uses weighted Categorical Dice.
Source: §3.2.3 · p. 12, §6 · p. 24Results
The authors report promising results from experiments on the BraTS 2020 dataset.
Source: §6 · p. 24Takeaways & Limitations
The paper presents ME-Net as a 3D multimodal MRI segmentation approach combining modality-specific encoders with weighted Categorical Dice.
Source: §6 · p. 24ET segmentation was not very good, which the authors suggest may reflect blurred, indistinct ET features.
Source: §5.3.2 · p. 23
Abstract
from arXiv · showhide
Glioma is the most common and aggressive brain tumor. Magnetic resonance imaging (MRI) plays a vital role to evaluate tumors for the arrangement of tumor surgery and the treatment of subsequent procedures. However, the manual segmentation of the MRI image is strenuous, which limits its clinical application. With the development of deep learning, a large number of automatic segmentation methods have been developed, but most of them stay in 2D images, which leads to subpar performance. Moreover, the serious voxel imbalance between the brain tumor and the background as well as the different sizes and locations of the brain tumor makes the segmentation of 3D images a challenging problem. Aiming at segmenting 3D MRI, we propose a model for brain tumor segmentation with multiple encoders. The structure contains four encoders and one decoder. The four encoders correspond to the four modalities of the MRI image, perform one-to-one feature extraction, and then merge the feature maps of the four modalities into the decoder. This method reduces the difficulty of feature extraction and greatly improves model performance. We also introduced a new loss function named "Categorical Dice", and set different weights for different segmented regions at the same time, which solved the problem of voxel imbalance. We evaluated our approach using the online BraTS 2020 Challenge verification. Our proposed method can achieve promising results in the validation set compared to the state-of-the-art approaches with Dice scores of 0.70249, 0.88267, and 0.73864 for the intact tumor, tumor core, and enhanced tumor, respectively.
1. Introduction
Glioma’s aggressiveness and the cost and variability of manual MRI segmentation motivate automatic tools. ME-Net assigns separate encoders to the four MRI modalities before merging their features for decoding.
1. Introduction
- Glioma is invasive and fast-growing, with reported survival not exceeding 14 months after diagnosis, making early diagnosis and treatment crucial.Source: §1 · p. 3
- Manual MRI tumor segmentation requires substantial time, resources, and expertise, and different experts can produce significantly different subregion segmentations.Source: §1 · p. 3
- ME-Net uses four encoders for the four MRI modalities, extracting modality-specific features before merging them for the decoder.The authors argue that a shared-weight V-Net is not suited to extracting the different features of each modality.Source: §1 · p. 4
2. Related Works
Prior work spans traditional segmentation, general deep-learning architectures, and brain-tumor-specific networks. ME-Net responds to segmentation challenges with modality-specific encoders and weighted Categorical Dice, reporting promising BraTS 2020 performance.
2.1. Traditional machine learning
- Traditional methods included thresholding, edge-and-region segmentation, and k-means; cited limitations include restricted image suitability, lack of spatial information, and sensitivity to noise or intensity unevenness.Source: §2.1 · p. 5
2.2. Deep learning
- Deep-learning examples include pixel-level FCN, U-Net with skip connections, 3D V-Net, and ENet/ERFNet applications in medical imaging.Source: §2.2 · pp. 5–6
2.3. Brain tumor segmentation
- Brain-tumor approaches include patch-based DCNNs, varied 3D U-Nets, cascades, ensembles, and 2D slice-based WRN-PPNet.Source: §2.3 · pp. 6–7
2.4. Our work
- ME-Net addresses tumor variability, voxel imbalance, and MRI intensity differences with four modality-specific encoders and weighted Categorical Dice; BraTS 2020 results were promising.Source: §2.4 · pp. 7–8
3. Methods
ME-Net processes the four MRI modalities through separate encoder paths and a shared decoder, with preprocessing, feature fusion and a class-weighted loss designed for 3D tumor segmentation.
3.1. Data preprocessing and data augmentation
- ME-Net standardizes each modality separately, combines the four images into a four-channel volume, and trains on 128×128×64 patches, with 175 patches generated per case.The patch operation processes one patch at a time rather than the whole image.Source: §3.1 · pp. 9–10
3.2.1. Downsampling
- Four modality-specific encoders extract features separately, using skip connections to combine modal features during downsampling.Each encoder uses stride-2 convolution instead of pooling, doubling channels while halving resolution.Source: §3.2.1 · p. 10
3.2.2 Upsampling
- The decoder upsamples with stride-2 deconvolution, doubling feature-map resolution and halving channel count at each stage.Corresponding downsampling features are supplied to decoder stages; a final Softmax converts channels to class probabilities.Source: §3.2.2 · pp. 11–12
3.2.3. Fusion strategy
- ME-Net fuses features across corresponding encoder stages and between encoder and decoder stages to combine modalities and retain fine-grained details.The fused encoder-stage maps are adjusted with 1×1×1 convolutions before being passed to corresponding upsampling stages.Source: §3.2.3 · pp. 12–13
3.3. Loss function
- Categorical Dice addresses foreground-background imbalance by assigning fixed weights of 0.1 to background and 1 to necrosis, edema, and enhanced tumor.The authors report that background exceeds 99% of their dataset and use fixed class weights rather than weights based on label volume.Source: §3.3 · pp. 13–14
4. Dataset and Experiments
The experiments use BraTS 2020 multimodal MRI data to evaluate tumor-subregion segmentation. They report evaluation metrics and the ME-Net training setup, including its hyperparameters and computing environment.
4.1. Dataset
- Training used 369 labeled examples and 125 unlabeled validation cases from BraTS 2020, with final accuracy tested online on the official challenge website.The dataset includes HGG and LGG cases with FLAIR, T1, T1-CE, and T2 modalities; the task segments ET, WT, and TC.Source: §4.1 · p. 15
4.2. Evaluation metrics
- They evaluate Dice, sensitivity to segmented regions, specificity for background prediction, and Hausdorff95 distance between real and predicted surfaces, which emphasizes boundary differences.Sensitivity is described in relation to true positives and false negatives, while specificity concerns true negatives and false positives.Source: §4.2 · p. 15
4.3. Experimental detail
- ME-Net training used an initial learning rate of 0.0001, dropout of 0.5, about 450,000 steps, and learning-rate reductions at 200,000 and 400,000 steps.The experiments used TensorFlow, a TITAN RTX GPU, and 128 GB RAM; the dataset order was shuffled after each traversal.Source: §4.3 · p. 16
5. Results and Discussion
The model achieved the best WT accuracy among the compared methods and comparable results to other state-of-the-art BraTS 2020 teams. Results also show stronger WT Dice than ET and TC, while the discussion notes limits in ET segmentation, multi-class accuracy, and data augmentation.
5.1. Results
- WT Dice was 0.894 on the training set and 0.883 on the validation set, outperforming the other sub-regions.The authors suggest WT is easier to segment because it covers the largest area; they attribute lower ET and TC accuracy partly to uncertainty in their label boundaries.Source: §5.1 · pp. 16–17, Table 1 · p. 17
- The evaluation plots show small fluctuations and few outliers; WT sensitivity and specificity are similar, indicating balanced foreground and background prediction.The authors describe the distributions as slightly left-skewed, with results concentrated in the higher range.Source: §5.1 · p. 17, Figure 5 · p. 18, Figure 6 · p. 18
- Training examples show good segmentation overall, but the tumor-core prediction is slightly biased; validation examples span tumors of different positions, sizes, and shapes.The authors suggest the small tumor-core area may explain the bias; they also report good results on MRI images with different intensities.Source: §5.1 · pp. 19–20, Figure 7 · p. 19, Figure 8 · p. 20
5.2. Comparison
- The method achieved the best WT accuracy among compared methods and comparable results to state-of-the-art BraTS 2020 teams.The authors note that the methods in Table 2 used different datasets, so they also compare with BraTS 2020 challenge teams.Source: §5.2 · p. 21, Table 2 · p. 21, Table 3 · p. 22
5.3.1 Application areas / 5.3.2 Future work
- The authors describe potential clinical relevance for surgery and radiotherapy planning, while identifying weak ET segmentation, accuracy–complexity trade-offs, and untested data augmentation as future-work concerns.They propose attention modules for ET features, discuss region clipping and combining strengths across algorithms, and note that stronger data augmentation has not yet been tried.Source: §5.3.1 · pp. 22–23, §5.3.2 · pp. 23–24
6. Conclusion
ME-Net assigns one encoder to each of the four MRI modalities, combines their feature maps for decoding, and uses weighted Categorical Dice to address foreground-background voxel imbalance. On BraTS 2020 validation, it reports Dice scores of 0.70249 for enhanced tumor, 0.88267 for the entire tumor, and 0.73864 for the tumor core.
6. Conclusion
- 0.70249 for enhanced tumor, 0.88267 for the entire tumor, and 0.73864 for the tumor core were the reported validation Dice indices.The evaluation used the online BraTS Challenge verification tool.Source: §6 · p. 24
- ME-Net uses four modality-specific encoders whose feature maps are combined through skip connections and passed to a decoder.Each encoder corresponds to one of the four brain tumor MRI modalities.Source: §6 · p. 24
- Categorical Dice assigns weight 1 to each of the three tumor regions and 0 to the larger background region to address voxel imbalance.Source: §6 · p. 24