PaperlayerView exact revision on arXiv

Source-linked AI summary

Gland Instance Segmentation Using Deep Multichannel Neural Networks

Yan Xu, Yang Li, Yipei Wang, Mingyuan Liu, Yubo Fan, Maode Lai, Eric I-Chao Chang

arXiv:1611.06661v3cs.CV

TL;DR

Gland instance segmentation must separate glands from complex histology backgrounds and identify each gland individually. The paper uses a CNN-based multichannel framework that fuses region, boundary, and location cues. It reports state-of-the-art performance on the MICCAI 2015 challenge dataset and stronger results than several comparison methods.

  • Problem

    Irregular gland shapes, noisy backgrounds, and coalescing boundaries make it difficult to segment glands and distinguish individual instances.

  • Method

    A deep multichannel neural network extracts and fuses region, boundary, and location information for gland instance segmentation.

  • Results

    The method achieves state-of-the-art performance on the MICCAI 2015 Gland Segmentation Challenge dataset and outranks FCN, dilated FCN, and other participants by rank sum and weighted rank sum.

  • Takeaways & Limitations

    Combining region, location, and edge information improves separation of adjacent glandular structures in histological images.

Abstract

from arXiv · show

Objective: A new image instance segmentation method is proposed to segment individual glands (instances) in colon histology images. This process is challenging since the glands not only need to be segmented from a complex background, they must also be individually identified. Methods: We leverage the idea of image-to-image prediction in recent deep learning by designing an algorithm that automatically exploits and fuses complex multichannel information - regional, location, and boundary cues - in gland histology images. Our proposed algorithm, a deep multichannel framework, alleviates heavy feature design due to the use of convolutional neural networks and is able to meet multifarious requirements by altering channels. Results: Compared with methods reported in the 2015 MICCAI Gland Segmentation Challenge and other currently prevalent instance segmentation methods, we observe state-of-the-art results based on the evaluation metrics. Conclusion: The proposed deep multichannel algorithm is an effective method for gland instance segmentation. Significance: The generalization ability of our model not only enable the algorithm to solve gland instance segmentation problems, but the channel is also alternative that can be replaced for a specific task.

I. INTRODUCTION

Gland instance segmentation is needed to analyze gland morphology, but irregular shapes, heterogeneous staining, noise, and coalescence make individual recognition difficult. The paper proposes multichannel learning that combines region, boundary, and location cues, evaluated on the MICCAI 2015 challenge dataset.

  • Motivation: Gland instance segmentation assigns each foreground pixel to an individual gland rather than only distinguishing gland from background.This supports individual morphology analysis.
  • Challenges: Irregular gland shapes and heterogeneous histopathological backgrounds complicate segmentation and instance recognition.Intensity gradients and variable cellular matrices add background noise.
  • Challenges: Coalescence causes adjacent glands to be treated as one instance when their boundaries connect, even at a single pixel.The problem arises when nearby glands have tiny gaps or adhesive edges.
  • Research gap: Foreground segmentation alone is insufficient because glands must be recognized individually for morphology assessment and cancer grading.The paper distinguishes gland labeling/segmentation from instance recognition.
  • Contribution: The proposed method uses multichannel learning to extract region, boundary, and location cues for gland instance segmentation.A CNN fuses channel information to produce segmented instances.
  • Evaluation: The algorithm is evaluated on the MICCAI 2015 Gland Segmentation Challenge dataset and compared with other instance segmentation methods.The paper also reports ablation experiments.

II. PROBLEM

The paper formulates gland instance segmentation as foreground labeling and instance recognition. The first assigns pixels to gland or background, while the second differentiates individual gland regions and is approximated through edge and object detection.

  • Problem formulation: Gland instance segmentation is formulated as two subproblems: labeling/segmentation and instance recognition.This separates foreground identification from assigning pixels to individual glands.
  • Problem formulation: Instance recognition is more sensitive to small pixel errors than foreground labeling, because such errors can substantially increase its cost.The paper illustrates this contrast in Fig. 3.
  • Labeling/segmentation: Labeling/segmentation assigns each pixel a binary foreground-or-background label.The predicted labeling result is denoted by Ŷ.
  • Instance recognition: Instance recognition determines which gland each foreground pixel belongs to and evaluates predicted regions against labeled gland regions.A predicted region counts as an instance when its overlap ratio exceeds the threshold 0.5.
  • Optimization: Because the instance-recognition cost is nondifferentiable, the method approximates it using edge detection and object detection.Edge and object labels are generated from the segmentation and instance annotations.

III. RELATED WORK

Prior instance-segmentation methods largely follow detection-then-masking pipelines, while gland images pose distinctive shape and appearance challenges. The proposed approach extends an earlier version with object detection, ablation studies, rotation-based augmentation, and state-of-the-art challenge results.

  • Natural-image instance segmentation commonly detects objects first and then segments instances inside their bounding boxes.
  • Gland instance segmentation is difficult because gland morphology varies substantially and prior methods perform comparatively worse on malignant images.
  • The paper extends an earlier conference approach by adding an object-detection channel because object detection and edge detection complement each other.
  • The expanded study includes ablation experiments, rotation-invariant data augmentation, and state-of-the-art results on the 2015 MICCAI challenge dataset.
  • The method uses three channels to perform gland segmentation and instance recognition simultaneously, assigning each channel a different responsibility.

A. Foreground Segmentation Channel

The foreground channel separates glands from background, while the edge channel supplies boundary information that helps preserve contours and distinguish adjacent glands. Both are built from convolutional architectures, with dilated convolution or deep supervision addressing resolution and class-imbalance issues.

  • Foreground Segmentation Channel: The foreground segmentation channel distinguishes gland pixels from background pixels but alone cannot reliably separate adjacent glands of the same class.
  • Foreground Segmentation Channel: FCN provides end-to-end pixel-wise prediction, while dilated convolution enlarges the receptive field without further downsampling.
  • Foreground Segmentation Channel: The modified FCN-32s foreground channel sets pool4 and pool5 strides to 1 and uses dilated convolution in subsequent layers.
  • Edge Detection Channel: The edge channel detects gland boundaries, compensating for feature-resolution loss and separating connected regions of adjacent glands.
  • Edge Detection Channel: HED supplies multiscale edge predictions, and deep supervision addresses the imbalance between relatively few edge pixels and many background pixels.
  • Edge Detection Channel: The edge channel combines weighted outputs from multiple supervision scales and is trained with sigmoid cross-entropy loss.

C. Object Detection Channel

The object-detection channel identifies glands and their locations, providing instance-level information beyond foreground segmentation. It uses Faster R-CNN-style classification and bounding-box regression, followed by a filling operation that maps box coverage across pixels.

  • Object Detection Channel: The object-detection channel detects glands and estimates their locations in the image.
  • Object Detection Channel: Bounding-box confirmation supplies instance information, but restricting processing to boxes can lose context and worsen segmentation.
  • Object Detection Channel: The filling operation assigns each pixel the number of predicted bounding boxes covering it, including overlap counts.
  • Object Detection Channel: The channel uses Faster R-CNN parameters for box prediction, with classification and regression losses summed during training.

D. Fusing Multichannel

The fusion network combines region, location, and boundary outputs to produce the final instance-segmentation prediction. It uses a shallow convolutional architecture with dilated convolution to limit information loss while retaining a broad receptive field.

  • Fusing Multichannel: Fusion is necessary to combine the region, location, and boundary cues produced by the three channels.
  • Fusing Multichannel: A shallow seven-layer convolutional network combines the three channel outputs and yields the final prediction.
  • Fusing Multichannel: The fusion network replaces downsampling with dilated convolution to reduce information loss and preserve a sufficiently large receptive field.
  • Fusing Multichannel: The fusion architecture is selected by cross-validation, increasing layers and filters until performance no longer improves.
  • Fusing Multichannel: The method is evaluated on 165 labeled colorectal cancer histological images from the MICCAI 2015 Gland Segmentation Challenge dataset.

B. Data augmentation and Preprocessing

The method preprocesses gland labels into region, edge, and bounding-box information, then evaluates detection, segmentation, and morphology using competition-defined criteria.

  • Preprocessing: Per-channel zero-mean normalization precedes edge-label generation and subsequent edge-label dilation.
  • Preprocessing: Bounding-box ground truth is generated from each gland’s instance region using coordinate extrema.Each region’s minimum and maximum x- and y-coordinates define the smallest enclosing rectangle.
  • Evaluation: The competition score sums rankings across test sets and criteria, with a weighted rank sum accounting for test A comprising three-quarters of the images.The weighted evaluation combines the three criteria across test sets A and B.
  • Evaluation: The evaluation uses F1 score for gland detection, with predictions sharing more than 50% of area with ground truth counted as true positives.Unmatched predicted and ground-truth objects are treated as false positives and false negatives, respectively.
  • Evaluation: Dice evaluates segmentation overlap, while object-level matching is introduced because whole-image Dice cannot distinguish instances of the same class.Segmented objects are paired with ground-truth objects having maximal overlap.
  • Evaluation: Shape similarity is assessed with Hausdorff distance to reflect morphology-related performance.

E. Result and Discussion

The proposed algorithm outperforms FCN, dilated FCN, and other participants while better separating adjacent glands. Its performance benefits from combining region, location, and edge information, though ambiguous white regions and broken glands remain difficult cases.

  • Overall comparison: The proposed method outranks FCN, dilated FCN, and other participants on both rank sum and weighted rank sum.RS denotes rank sum, while WRS denotes weighted rank sum.
  • Comparison with FCN: Combining region, location, and edge information produces higher scores than FCN and dilated FCN.The method separates most adjacent glandular structures, supporting instance-level segmentation.
  • Error analysis: Adding pixel context while predicting object location abates false positives when foreground and background intensities are similar.FCN and dilated FCN sometimes classify background pixels as glands in such cases.
  • Comparison with prior methods: Combining edge and location context improves results on both test A and test B compared with CUMedVision1.CUMedVision2 improves test A but deteriorates on test B after adding edge information alone.
  • Limitations: The method can mistake white background regions for cytoplasm and confuse cytoplasm with background when glands are split by image cutting.These errors arise because white regions may represent either cytoplasm or tissue-free background.

1) Data Augmentation Strategy:

Data augmentation improves training outcomes by accommodating the varied shapes of normal and cancerous glands and reducing overfitting.

  • Data Augmentation Strategy: Adequate transformation of gland images improves performance and contributes to overfitting elimination.The effect is evaluated for the foreground segmentation channel using two augmentation strategies.

2) Plausibility of Channels:

The channel analysis supports dilated convolution and the combined three-channel design for gland instance segmentation. Edge processing addresses boundary imbalance and helps distinguish adjacent glands.

  • Dilated convolution: Dilated convolution enlarges the receptive field with fewer downsampling layers, reducing the resolution cost of downsampling.Downsampling enlarges receptive fields but loses resolution and information important for segmentation.
  • Dilated convolution: Dilated convolution improves segmentation precision in both the foreground segmentation channel and the fusion stage.The comparison includes FCN variants with and without dilated convolution.
  • Edge channel: Edge dilation alleviates the imbalance between sparse edge pixels and non-edge pixels, improving edge detection precision.Edge pixels occupy an extremely small proportion of the image, creating a training barrier.
  • Channel plausibility: The three-channel algorithm is favored over two-channel baselines combining foreground segmentation with either edge detection or object detection.The baselines are evaluated to test the contribution of the three channels.
  • Conclusion: The proposed network exploits edge, region, and location features to generate gland instance segmentations and achieves state-of-the-art results on the MICCAI 2015 dataset.The paper also reports baseline experiments supporting the method.
  • Future work: The algorithm is proposed for expansion to instance segmentation of other medical images.This is identified as future work rather than an evaluated result.
Loading 1611.06661v3…