Source-linked AI summary
Mask-RCNN and U-net Ensembled for Nuclei Segmentation
Aarno Oskar Vuola, Saad Ullah Akram, Juho Kannala
TL;DR
Manual nuclei instance segmentation is important but laborious, and selecting an effective neural-network framework remains difficult. The paper compares U-Net and Mask-RCNN, then combines their predictions in an ensemble. The ensemble outperforms either model by combining their complementary strengths.
Problem
Manual nuclei labeling is subjective and laborious, while choosing between competing segmentation frameworks requires expert knowledge.
Method
The paper compares U-Net and Mask-RCNN and trains an ensemble that combines their predicted nucleus masks.
Results
The ensemble outperforms both U-Net and Mask-RCNN, while the two base models show different strengths and failures.
Takeaways & Limitations
Ensembling can improve nuclei segmentation by combining U-Net’s and Mask-RCNN’s complementary predictions.
Abstract
from arXiv · showhide
Nuclei segmentation is both an important and in some ways ideal task for modern computer vision methods, e.g. convolutional neural networks. While recent developments in theory and open-source software have made these tools easier to implement, expert knowledge is still required to choose the right model architecture and training setup. We compare two popular segmentation frameworks, U-Net and Mask-RCNN in the nuclei segmentation task and find that they have different strengths and failures. To get the best of both worlds, we develop an ensemble model to combine their predictions that can outperform both models by a significant margin and should be considered when aiming for best nuclei segmentation performance.
1 Introduction
Automated nuclei instance segmentation matters because manual labeling is subjective, laborious, and difficult to scale. The study compares U-Net and Mask-RCNN and develops an ensemble to address their differing errors.
- Manual nuclei segmentation is subjective and laborious, with images containing up to tens of thousands of nuclei requiring expert labeling.
- Neural networks can outperform classical watershed and active-contour methods when sufficient training data are available.
- U-Net and Mask-RCNN are accessible open-source frameworks, but achieving acceptable results across domains still requires expert tuning.
- The study compares where U-Net and Mask-RCNN excel or fail, then combines their predictions in an ensemble that can exceed both by more than 5 percent in some cases.
- The model designs were inspired by high-performing U-Net and Mask-RCNN approaches from Kaggle’s 2018 Data Science Bowl.
2 Method
The method combines U-Net’s detailed segmentation with Mask-RCNN’s direct instance-detection framework. A gradient-boosted ensemble selects or adds predictions using estimated mask quality.
- U-Net: U-Net uses skip connections across resolutions and achieved its best results with a ResNet101 backbone initialized from pretrained ImageNet weights.
- U-Net: U-Net’s binary image-level output makes instance segmentation difficult, so this method adds a border-prediction channel to separate nearby nuclei.
- U-Net: Figure 1 represents Mask-RCNN training with separate ground-truth nucleus instances, while U-Net receives segmentation and overlapping-border targets.
- Mask-RCNN: Mask-RCNN predicts nucleus bounding boxes and segments nuclei within them, directly addressing the instance-segmentation problem.
- Ensemble model: A gradient-boosting ensemble uses out-of-fold prediction features, including mask shape properties, to estimate IoU for U-Net and Mask-RCNN outputs.
- Ensemble model: At test time, non-overlapping masks above predicted IoU 0.3 are added, while overlapping masks retain the prediction with the highest estimated IoU.
3 Experiments
Experiments use diverse microscopy data and cross-validation, with post-processing tailored to each model. Evaluation combines mAP and object-level segmentation and detection measures.
- Data: The dataset contains 800 fluorescence and histology images and masks from varied staining and microscopy sources.
- Data: Only simple flips, rotations, shifts, and scaling augmentations were retained, while shearing, CLAHE, elastic deformations, and noise performed poorly.
- Training: Four-fold cross-validation trained four ResNet101-ImageNet models, with image classes balanced in each fold and test-fold results averaged.
- Post-processing: Post-processing removed masks under 10 pixels, filled holes morphologically, used watershed for U-Net instances, and split overlapping Mask-RCNN regions by distance.
- Evaluation: Evaluation reports mAP across IoU thresholds from 0.5 to 0.95 alongside object-level Dice, precision, recall, and under- or oversegmentation counts.
4 Results
U-Net and Mask-RCNN showed complementary strengths: U-Net produced more accurate masks, while Mask-RCNN detected nuclei and separated clustered instances better. Their ensemble achieved the strongest overall performance by selecting useful predictions from both models across thresholds and image conditions.
- Overall performance: At IoU threshold 0.7, U-Net’s Dice performance indicated accurate masks but more detection errors, whereas Mask-RCNN had better recall and precision but weaker segmentation masks.Mask-RCNN also performed better on under- and oversegmentation, suggesting stronger individual-nucleus detection in clusters.
- Overall performance: The ensemble achieved the best mAP and recall across situations and tracked the stronger model across different IoU thresholds.U-Net struggled at mid-range thresholds, while Mask-RCNN performed better at lower and higher thresholds.
- Overall performance: The ensemble improved recall but had slightly lower precision because false positives from U-Net sometimes entered the combined prediction.Its recall advantage came from selecting masks from both models, increasing true positives.
- Overall performance: The ensemble’s largest advantage appeared on histology images, where the basic models struggled differently and combining their best predictions increased segmentation accuracy.A challenging example showed mAP increasing from 0.28 to 0.32.
- Detailed analysis: Across nucleus area, eccentricity, and cluster size, the ensemble was at least as good as the individual models in almost every evaluated situation.U-Net was better for very large nuclei and lone nuclei, while Mask-RCNN was stronger for small or medium nuclei, grouped nuclei, and most eccentricities.
5 Conclusion
U-Net and Mask-RCNN had similar overall performance but made different errors. Combining their predictions produced better results than either model alone, though whether this generalizes to other instance-segmentation tasks remains open.
- U-Net and Mask-RCNN achieved similar overall performance but made different errors on nuclei segmentation.
- The ensemble produced better results than either individual model by combining their predictive strengths.
- Future work is needed to determine whether ensemble models also improve other biomedical or medical instance-segmentation tasks.