Source-linked AI summary
MILD-Net: Minimal Information Loss Dilated Network for Gland Instance Segmentation in Colon Histology Images
Simon Graham, Hao Chen, Jevgenij Gamper, Qi Dou, Pheng-Ann Heng, David Snead, Yee Wah Tsang, Nasir Rajpoot
TL;DR
Automated gland instance segmentation is difficult because gland appearance varies, structures can be hard to distinguish, and diagnostic use requires uncertainty information. MILD-Net preserves information during feature extraction, aggregates multi-scale features, and uses test-time random transformations to produce segmentation and uncertainty maps, achieving state-of-the-art performance on GlaS and an independent colorectal adenocarcinoma dataset.
Problem
Automated gland segmentation must handle variable gland appearance, difficult glandular/non-glandular distinctions, and uncertainty relevant to diagnostic decision making.
Method
MILD-Net re-introduces down-sampled images after max-pooling, uses atrous spatial pyramid pooling for multi-scale aggregation, and applies test-time random transformations for uncertainty-aware segmentation.
Results
MILD-Net achieves state-of-the-art performance on the MICCAI 2015 GlaS challenge and a second independent colorectal adenocarcinoma dataset.
Takeaways & Limitations
The framework supports gland instance segmentation while visualising ambiguous regions and enabling exclusion of predictions with high uncertainty.
Takeaways & Limitations
The method may fail to separate glands that are physically touching with no pixels between them.
Abstract
from arXiv · showhide
The analysis of glandular morphology within colon histopathology images is an important step in determining the grade of colon cancer. Despite the importance of this task, manual segmentation is laborious, time-consuming and can suffer from subjectivity among pathologists. The rise of computational pathology has led to the development of automated methods for gland segmentation that aim to overcome the challenges of manual segmentation. However, this task is non-trivial due to the large variability in glandular appearance and the difficulty in differentiating between certain glandular and non-glandular histological structures. Furthermore, a measure of uncertainty is essential for diagnostic decision making. To address these challenges, we propose a fully convolutional neural network that counters the loss of information caused by max-pooling by re-introducing the original image at multiple points within the network. We also use atrous spatial pyramid pooling with varying dilation rates for preserving the resolution and multi-level aggregation. To incorporate uncertainty, we introduce random transformations during test time for an enhanced segmentation result that simultaneously generates an uncertainty map, highlighting areas of ambiguity. We show that this map can be used to define a metric for disregarding predictions with high uncertainty. The proposed network achieves state-of-the-art performance on the GlaS challenge dataset and on a second independent colorectal adenocarcinoma dataset. In addition, we perform gland instance segmentation on whole-slide images from two further datasets to highlight the generalisability of our method. As an extension, we introduce MILD-Net+ for simultaneous gland and lumen segmentation, to increase the diagnostic power of the network.
1. Introduction
Automated gland segmentation supports objective extraction of morphology relevant to colorectal cancer grading, but remains difficult because glands vary in appearance and diagnostic ambiguity matters. MILD-Net addresses these challenges by retaining information during feature extraction, aggregating multi-scale features, and incorporating uncertainty estimation.
- Clinical motivation: Glandular formation is central to assessing colorectal adenocarcinoma differentiation, declining as cancer grade increases.Well-differentiated cases are described as more than 95% gland forming, whereas poorly differentiated cases lose typical glandular appearance.
- Clinical motivation: Computational pathology can objectively segment glands and extract morphological features from large histopathology images.These features would otherwise be impractical to obtain at scale through manual analysis.
- Challenges: Automated gland segmentation requires high-resolution boundaries and must handle glands with varying sizes and shapes.The difficulty increases with cancer grade, while gland-only outputs provide limited diagnostic information.
- Proposed framework: MILD-Net reintroduces the original down-sampled image after max-pooling through minimal information loss units.The design targets information loss that can harm pixel-level predictions, especially around glandular boundaries.
- Proposed framework: Atrous spatial pyramid pooling aggregates features at multiple scales, while random test-time transformations generate an uncertainty map alongside segmentation.The uncertainty map highlights ambiguous regions where predictions may be disregarded.
- Results: The framework achieves state-of-the-art performance on the 2015 MICCAI GlaS Challenge dataset and a second independent colorectal adenocarcinoma dataset.The framework is trained end to end with one minimal information loss dilated feature extraction network.
2. Methods
MILD-Net combines information-preserving residual units, dilated convolutions, multi-scale aggregation, and low-level feature fusion for gland segmentation. It also quantifies predictive uncertainty and extends the architecture to jointly segment glands and lumens.
- Information-preserving feature extraction: MIL units reintroduce a down-sampled version of the original image after max-pooling to preserve information important for pixel-level segmentation.The image is resized by bicubic interpolation, convolved, concatenated with pooled features, and incorporated into the residual summation.
- Multi-scale feature extraction: Dilated residual units and atrous spatial pyramid pooling preserve resolution while aggregating features across receptive-field sizes for morphologically variable glands.Global average pooling supplies context when a dilated kernel exceeds the feature-map size.
- Decoder and task-specific outputs: Progressive 2× upsampling concatenates low-level features before producing separate gland-object and contour outputs, supporting precise boundary localisation.The task-specific component branches after feature maps return to the original image resolution.
- Uncertainty quantification: Random input transformations at test time generate a predictive distribution, segmentation result, and uncertainty map for filtering highly uncertain gland predictions.Boundary pixels are removed from the uncertainty score, and glands below a selected global threshold τ may be retained.
- MILD-Net+ extension: MILD-Net+ minimally modifies the network to segment glands, gland contours, lumens, and lumen contours simultaneously.The extension applies the same uncertainty-refinement approach to gland and lumen outputs.
3. Experiments and Results
MILD-Net was evaluated on two gland-segmentation datasets, whole-slide images, and a lumen-segmentation extension. The experiments report strong benchmark performance, uncertainty-guided filtering, generalisation to unseen whole-slide data, and benefits from simultaneous lumen segmentation.
- Dataset and evaluation: The experiments used the GlaS challenge dataset and an independent CRAG colorectal adenocarcinoma dataset, with F1, object-level Dice, and object-level Hausdorff distance as evaluation criteria.These criteria measure gland-object detection, set similarity, and boundary-based segmentation accuracy.
- Results on GlaS and CRAG Datasets Using MILD-Net: MILD-Net achieved state-of-the-art performance on GlaS and overall better performance than competing methods on CRAG.The authors present this as evidence of performance across both datasets and of generalisation across datasets.
- Results on GlaS and CRAG Datasets Using MILD-Net: RTS improved predictions in high-uncertainty regions and reduced false positives through averaging, whereas Monte Carlo dropout produced insufficient variation and additional dropout harmed training.The authors attribute RTS’s performance increase to refined predictions and fewer false positives in ambiguous areas.
- Results on GlaS and CRAG Datasets Using MILD-Net: Selecting instances with uncertainty score τk below 0.25 retained around 75% of instances while increasing F1, object Dice, and object Hausdorff on both GlaS test sets.For Test A, the reported values were 0.930, 0.9359, and 28.658; for Test B, they were 0.913, 0.9567, and 22.70, respectively.
- Results on Whole-Slide Images Using MILD-Net: MILD-Net accurately segmented glands in 32 whole-slide-image high-power fields, including unseen data from different centres and images containing pathologist pen markings without preprocessing.The evaluation comprised 16 fields from COMET-1 and 16 from COMET-2.
- Results on GlaS and CRAG Datasets Using MILD-Net+: MILD-Net+ achieved superior lumen-segmentation performance across all reported statistical measures and outperformed MILD-Net on most gland-object measures.The authors suggest that lumen segmentation may provide additional cues for gland-object segmentation and adds lumenal features for classification.
5. Conclusion
The paper presents MILD-Net for gland instance segmentation, combining information-preserving feature extraction, multi-scale aggregation, and test-time uncertainty estimation. It reports state-of-the-art performance on the MICCAI 2015 gland segmentation challenge and an independent colorectal adenocarcinoma dataset.
- MILD-Net retains information during feature extraction and uses atrous spatial pyramid pooling to aggregate features across gland sizes.Random test-time transformations produce uncertainty maps and support an object-level uncertainty score for discarding high-uncertainty predictions.
- The method obtains state-of-the-art performance on the MICCAI 2015 gland segmentation challenge and a second independent colorectal adenocarcinoma dataset.