Source-linked AI summary
Gland Segmentation in Colon Histology Images: The GlaS Challenge Contest
Korsuk Sirinukunwattana, Josien P. W. Pluim, Hao Chen, Xiaojuan Qi, Pheng-Ann Heng, Yun Bo Guo, Li Yang Wang, Bogdan J. Matuszewski, Elia Bruni, Urko Sanchez, Anton Böhm, Olaf Ronneberger, Bassem Ben Cheikh, Daniel Racoceanu, Philipp Kainz, Michael Pfeiffer, Martin Urschler, David R. J. Snead, Nasir M. Rajpoot
TL;DR
GlaS addressed the difficulty and poor reproducibility of manual gland-based cancer grading by organizing a standardized challenge for automated segmentation in digitized H&E colon histology. It assembled common data, expert annotations, evaluation criteria, and competing methods, while documenting clinically relevant metrics and remaining dataset limitations.
Problem
Cancer grading uses intestinal gland morphology, but grading reproducibility remains difficult and manual gland segmentation is laborious.
Method
The paper overviews a standardized challenge in which researchers developed gland-segmentation algorithms using common images and expert pathologist annotations.
Results
The challenge evaluated competing algorithms on shared data and presented 10 leading entries from 19 final-round submissions.
Takeaways & Limitations
The publicly available dataset and standardized comparison platform support continued development and comparison of gland-segmentation methods.
Takeaways & Limitations
The dataset used a single expert’s ground truth and did not account for inter-observer or digitization variability.
Abstract
from arXiv · showhide
Colorectal adenocarcinoma originating in intestinal glandular structures is the most common form of colon cancer. In clinical practice, the morphology of intestinal glands, including architectural appearance and glandular formation, is used by pathologists to inform prognosis and plan the treatment of individual patients. However, achieving good inter-observer as well as intra-observer reproducibility of cancer grading is still a major challenge in modern pathology. An automated approach which quantifies the morphology of glands is a solution to the problem. This paper provides an overview to the Gland Segmentation in Colon Histology Images Challenge Contest (GlaS) held at MICCAI'2015. Details of the challenge, including organization, dataset and evaluation criteria, are presented, along with the method descriptions and evaluation results from the top performing methods.
1 Introduction
Cancer grading relies on gland morphology but remains difficult to reproduce consistently. The GlaS challenge addressed automated gland segmentation as a route to quantitative, computer-assisted analysis across benign and malignant tissue.
- Cancer grading informs prognosis and treatment planning but remains challenging to reproduce consistently in pathology.
- Intestinal gland architecture and formation are key criteria for grading colorectal cancer.
- Manual gland segmentation is laborious, motivating automated extraction of quantitative gland-morphology features.
- Consistent high-quality segmentation across cancer differentiation grades remained challenging and motivated the contest.
- GlaS brought researchers together to develop algorithms evaluated against expert pathologist annotations on benign tissue and colonic carcinomas.
2 Related Work
Prior gland-segmentation methods used diverse pixel, graph, polar-coordinate, and Bayesian representations. Because datasets and evaluation criteria varied, objective comparison remained difficult, motivating a common benchmark.
- Earlier methods included thresholding and region growing initialized from lumen-like empty regions.
- Graph-based methods represented tissue components as disks and used nuclear-object edges to constrain region growth.
- Polar-coordinate methods transformed closed gland boundaries into approximately vertical periodic paths for conditional random field inference.
- Bayesian segmentation modeled each gland as a polygon whose vertices were arranged epithelial nuclei.
- Different datasets and assessment criteria made objective comparison of prior algorithms difficult, while common benchmarks enable meaningful comparison and optimization.
3 Materials
The dataset comprised H&E-stained colorectal adenocarcinoma sections from multiple patients, with visual fields spanning benign and malignant tissue. Expert annotations supplied histologic grades and gland boundaries for training and evaluation.
- The dataset contained 165 images from 16 H&E-stained sections of stage T3 or T4 colorectal adenocarcinoma, each section from a different patient.
- The sections exhibited inter-subject variability in stain distribution and tissue architecture.
- An expert pathologist graded 52 visual fields as benign or malignant according to overall glandular architecture.
- The pathologist delineated each glandular object to provide ground truth for automatic segmentation.
- Data splits were stratified by histologic grade and visual field, preventing images from one visual field from appearing in multiple parts.
4 Challenge Organization
The challenge used a staged process combining registration, training-data release, method review, test-Part-A submissions, and an on-site Test-Part-B event. Participation narrowed from broad registration to evaluated submissions.
- The contest had four stages covering registration, training-data release, paper submission, Test-Part-A evaluation, and the challenge event.
- Stage 1 allowed academic or industrial individuals and groups of up to three people to register and download training data.
- Stage 2 required a two-page algorithm description and preliminary training-data results before access to Test Part A.
- Stage 2 review assessed method clarity and validation soundness rather than segmentation performance.
- Test Part B was evaluated at the MICCAI’2015 event, where teams had 45 minutes to produce results.
- Participation decreased from 110 registered teams to 21 Stage 2 submissions, 20 invitations, and 13 timely Test-Part-A submissions.
5 Evaluation
The challenge evaluated gland segmentation through complementary detection, volume-based, and boundary-based criteria, using object correspondences between algorithm outputs and pathologist annotations. These metrics distinguish pixel or volume agreement from gland-level and boundary similarity.
- Evaluation criteria: The evaluation measured gland detection, object volume agreement, and boundary similarity as three distinct segmentation criteria.Pixel labels can be accurate even when the predicted boundary curves differ substantially.
- Object correspondence: Corresponding segmented and ground-truth objects were paired by maximum overlap, with empty objects included when no overlap existed.The mappings need not be injective, surjective, or mutual inverses.
- Detection accuracy: The F1 score counted a segmented gland as a true positive when it intersected at least 50% of its ground-truth object.False negatives were the ground-truth objects not counted as true positives.
- Volume-based segmentation accuracy: The object-level Dice index measured agreement for corresponding glands in both directions and weighted each term by relative object area.This gives less emphasis to small segmented and ground-truth objects, and the whole-test score pooled objects across all images.
- Additional evaluation measure: The adjusted Rand index assessed pixel-pair labeling agreement between ground-truth and segmented objects, with an upper bound of 1 and possible negative values.Its agreement calculation uses pairwise concordance under a generalized hypergeometric assumption.
- Boundary-based segmentation accuracy: The object-level Hausdorff distance summarized the most extreme nearest-boundary discrepancy, so smaller values indicated greater boundary similarity.Euclidean distance was used, and unmatched objects were compared with the nearest object in the other set.
6 Ranking Scheme
The ranking scheme assigned separate ranks for every evaluation metric and test-data set, then used the resulting six scores to compare submitted entries.
- Ranking scores: Each entry received one ranking score for each of three metrics across each of two test-data sets.The metrics were gland-detection F1, object-level Dice, and object-level Hausdorff accuracy.
- Ranking scores: The best-performing entry received rank 1, the second-best rank 2, and subsequent entries followed in order.
7 Methods
The challenge included 13 methods, with seven selected for detailed discussion and six teams choosing to have their methods described. The approaches combined convolutional networks, contour or border modeling, preprocessing, classification, and post-processing.
- Overview: Seven methods were selected from 13 participants for detailed discussion, although only six chose to have their methods described.Selection stopped where the rank sums showed a substantial gap.
- Convolutional and contour-aware networks: The deep contour-aware network jointly predicted gland-object and contour masks using shared multi-level feature representations.Its two branches were designed to help separate touching glands through a multi-task learning framework.
- Convolutional and contour-aware networks: Another pipeline corrected stain variation with histogram matching and used a CNN for supervised pixel classification into glandular tissue classes.The three classes represented gland interiors, epithelial boundaries, and inter-gland tissue.
- Multi-path networks: A multi-path convolutional network captured local and global features through paths with different convolutional configurations.Simple-path networks also detected gland borders using training bands 5–10 pixels wide.
- Pipeline variants: The ExB submissions varied by including border detection, tissue-class binary classification, and post-processing for parameter selection or structural cleanup.Three entries compared pipelines with and without border networks and post-processing.
- U-net segmentation: The Freiburg method applied a 23-layer u-net to raw RGB images, followed by connected-component labeling or morphological post-processing.The second entry filled holes and removed segments smaller than 1000 pixels.
- Tissue-aware segmentation: A tissue-classification approach distinguished hollow, bounded, and crowded glands before segmentation using nuclear spatial distribution and tissue topology.It generated a closing map from morphological closings of k-means-segmented nuclear objects.
8 Results and Discussion
The challenge compared gland-segmentation methods using standardized evaluation and combined-test analyses, while examining performance across histologic grades and dataset-specific difficulties. Results highlighted strong entries, the importance of border separation, and limitations that prevent immediate clinical deployment.
- Evaluation results: CUMedVision2, ExB1, and ExB3 achieved the top three overall rank sums, while the leading entries differed between test Parts A and B.CUMedVision2, ExB2, and ExB3 led Part A; CUMedVision1, ExB1, and Freiburg2 led Part B.
- Evaluation results: Combined-test re-evaluation largely preserved the ranking order under both F1score + Diceobj + Hobj and F1score + ARI + Hobj schemes.The merged analysis addressed bias from weighting the two test parts equally and reported similar ranking orders for both schemes.
- General discussion: Lumen, non-glandular tissue resembling glands, dense nuclei, and small glands were identified as dataset features that negatively affect segmentation performance.These challenges can lead to over-segmentation or under-segmentation; CUMedVision, ExB, and Freiburg were observed to better avoid those errors.
- Additional experiments: Border separation improved malignant-gland segmentation but could cause benign under-segmentation when individual objects were already well separated.CUMedVision2 outperformed CUMedVision1 in benign and malignant cases, while ExB1 performed better on malignant glands but under-segmented benign objects relative to ExB2 and ExB3.
- Challenge objectives: The challenge standardized comparison of automatic and semi-automatic gland-segmentation algorithms and attracted many participating teams.Its stated objective was to raise awareness of intestinal gland segmentation and provide a platform for standardized performance comparison.
- General discussion: The contest metrics were clinically relevant because object-Hausdorff assesses gland shape, object-Dice assesses gland volume, and F1score assesses gland detection.These measures correspond to clinically relevant gland morphology, including shape and size, while F1score supports detection assessment.
9 Conclusions
The paper summarizes the GlaS Challenge Contest, which evaluated gland-segmentation algorithms on a shared standard dataset and made the dataset publicly available for further quantitative evaluation.
- The GlaS Challenge brought researchers together to validate gland-segmentation algorithms on the same standard dataset.
- 19 entries were submitted for evaluation in the final round, and 10 leading entries are presented in the paper.
- The challenge dataset was made publicly available so researchers could develop or improve approaches using quantitative evaluation.
A The Complete Contest Results
Figure 5 presents the ranking results from the GlaS Challenge Contest.
- Figure 5 summarizes the contest ranking results.