PaperlayerView exact revision on arXiv

Source-linked AI summary

HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images

Simon Graham, Quoc Dang Vu, Shan E Ahmed Raza, Ayesha Azam, Yee Wah Tsang, Jin Tae Kwak, Nasir Rajpoot

arXiv:1812.06499v5cs.CV

TL;DR

Automated nuclear segmentation and classification in H&E histology is needed for large-scale analysis but is challenged by heterogeneous appearances and clustered nuclei. HoVer-Net jointly predicts nuclear pixels, centre-of-mass distance maps, and nuclear classes, achieving state-of-the-art instance-segmentation performance across multiple datasets and processing 1,000×1,000 tiles about 9.7× faster than Mask-RCNN. The paper also introduces an annotated colorectal dataset, while noting scope limitations from training on a single tissue type and tile-boundary issues for region-proposal methods.

  • Problem

    Automated nuclear analysis is difficult because nuclei are heterogeneous, clustered, and numerous, while accurate segmentation and classification support downstream histology analysis.

  • Method

    HoVer-Net jointly predicts nuclear pixels, horizontal and vertical distances to nuclear centres of mass, and nuclear classes through three up-sampling branches.

  • Results

    HoVer-Net achieves state-of-the-art instance-segmentation performance across multiple datasets and processes a 1,000×1,000 image tile 9.7× faster than Mask-RCNN.

  • Takeaways & Limitations

    Joint segmentation and classification enables analysis of spatial relationships between nuclear types together with nuclear morphological features.

  • Takeaways & Limitations

    The model is trained on a single tissue type, and region-proposal methods face difficulty merging instance predictions across neighbouring tiles.

Abstract

from arXiv · show

Nuclear segmentation and classification within Haematoxylin & Eosin stained histology images is a fundamental prerequisite in the digital pathology work-flow. The development of automated methods for nuclear segmentation and classification enables the quantitative analysis of tens of thousands of nuclei within a whole-slide pathology image, opening up possibilities of further analysis of large-scale nuclear morphometry. However, automated nuclear segmentation and classification is faced with a major challenge in that there are several different types of nuclei, some of them exhibiting large intra-class variability such as the tumour cells. Additionally, some of the nuclei are often clustered together. To address these challenges, we present a novel convolutional neural network for simultaneous nuclear segmentation and classification that leverages the instance-rich information encoded within the vertical and horizontal distances of nuclear pixels to their centres of mass. These distances are then utilised to separate clustered nuclei, resulting in an accurate segmentation, particularly in areas with overlapping instances. Then for each segmented instance, the network predicts the type of nucleus via a devoted up-sampling branch. We demonstrate state-of-the-art performance compared to other methods on multiple independent multi-tissue histology image datasets. As part of this work, we introduce a new dataset of Haematoxylin & Eosin stained colorectal adenocarcinoma image tiles, containing 24,319 exhaustively annotated nuclei with associated class labels.

I. INTRODUCTION

HoVer-Net addresses the difficulty of automating nuclear segmentation and classification in H&E histology, where nuclei vary substantially and often cluster. It jointly separates clustered instances, classifies each segmented nucleus, and reports strong performance across multiple datasets while introducing a new annotated colorectal dataset.

  • Manual H&E slide assessment has low throughput and is prone to intra- and inter-observer variability.
  • Nuclear analysis is difficult because nuclei vary in shape, size, and chromatin pattern across cell types, diseases, and tissue regions, while clustered nuclei complicate separation.
  • HoVer-Net predicts horizontal and vertical nuclear-pixel distances to centres of mass and uses them to separate clustered nuclei.
  • A dedicated up-sampling branch determines the nuclear type for each segmented instance within the same network.
  • HoVer-Net achieves state-of-the-art performance on multiple H&E histology datasets compared with over a dozen recently published methods.
  • The work introduces a colorectal dataset containing 24,319 exhaustively annotated nuclei within 41 colorectal adenocarcinoma image tiles.

II. RELATED WORK

Prior nuclear instance-segmentation methods use watershed, contour, distance-map, or region-proposal strategies, while nuclear classification is commonly handled separately. These approaches face challenges including unreliable thresholding, dependence on predefined nuclear geometry, tile-boundary merging, and difficult subtype discrimination.

  • Watershed methods commonly use thresholded markers and energy landscapes, but intensity thresholds can fail on complex images.
  • Contour-based methods predict nuclear clusters and contours or inner nuclei, contours, and background to separate instances.
  • Distance-map approaches regress nuclear distances to obtain watershed markers without predicting areas with indistinct contours.
  • Region-proposal methods segment nuclei independently but have difficulty merging instance predictions across neighbouring tiles.
  • Nuclear classification is often performed after segmentation or detection using separate feature-based or patch-based classifiers.

III. METHODS

The framework uses shared feature extraction and three up-sampling branches to perform nuclear pixel detection, clustered-instance separation, and nuclear classification. Horizontal and vertical distance predictions provide instance information, while classification is optional when labels are unavailable.

  • Network Architecture: The framework first detects nuclear pixels, then uses horizontal and vertical distance maps to support instance segmentation and obtain nuclear types.The distance maps encode each nuclear pixel’s distances to its centre of mass.
  • Network Architecture: The architecture combines a Preact-ResNet50-inspired encoder with up-sampling branches containing densely connected units and skip connections.The design reduces the down-sampling factor from 32 to 8 and uses dense units to build a large receptive field with relatively few parameters.
  • Network Architecture: The network uses three branches: nuclear pixel, HoVer, and nuclear classification.The NP branch predicts nuclei versus background, the HoVer branch predicts distances to nuclear centres of mass, and the NC branch predicts nuclear type.
  • Network Architecture: The NP and HoVer branches jointly segment instances by separating nuclear pixels from background and then separating touching nuclei.A shared encoder supplies features to the task-specific decoders.
  • Loss Function: The four weight sets for the encoder and three decoders are optimized jointly using weighted losses for HoVer, NP, and NC outputs.The HoVer branch uses regression losses, while NP and NC use cross-entropy and Dice losses; NC losses apply only when classification labels exist.

B. Post Processing

Post-processing uses gradients of the predicted horizontal and vertical maps to identify boundaries between neighbouring nuclei, then applies marker-controlled watershed to split instances. Per-pixel nuclear classifications are converted to instance labels by majority vote.

  • Post Processing: Gradients of the horizontal and vertical predictions highlight high-difference regions between neighbouring nuclei.The method uses Sobel derivatives to identify locations where nuclei should be separated.
  • Post Processing: The resulting markers are used with marker-controlled watershed to split the nuclear probability map into instances.The markers determine how the thresholded nuclear map is divided under the energy landscape.
  • Post Processing: Fig. 4 illustrates that DICE2 and AJI can respond differently to slightly different predictions because overlapping regions are penalized.Ground-truth instance contours are overlaid on predictions and original images for visual comparison.
  • Post Processing: Each segmented instance receives the nuclear type predicted most frequently among its pixels.This converts the NC branch’s per-pixel predictions into one class label per instance.

A. Nuclear Instance Segmentation Evaluation

The evaluation separates nuclear instance segmentation into background separation, instance detection, and instance segmentation, using DICE, PQ, and AJI. PQ additionally decomposes performance into detection and segmentation quality for interpretability.

  • Evaluation Tasks: Nuclear instance segmentation is evaluated through background separation, individual-instance detection, and segmentation of each detected instance.These sub-tasks provide a more detailed view than a single overall score.
  • Metric Limitations: DICE2 and AJI provide only overall instance-segmentation scores and do not reveal performance on the individual sub-tasks.Their treatment of overlapping regions can over-penalize slightly different predictions.
  • Metrics: PQ is decomposed into Detection Quality and Segmentation Quality to distinguish detection performance from overlap quality.DQ is an F1 score for instance detection, while SQ measures similarity of correctly detected instances to their matched ground truth.
  • PQ Matching: PQ matches prediction and ground-truth segments when IoU exceeds 0.5, producing true-positive, false-negative, and false-positive sets.The matching is unique over the complete prediction and ground-truth segment sets.
  • Metrics: DICE measures separation of all nuclei from background, while AJI provides direct comparison with previous publications.Panoptic Quality is used as a unified comparison score.

B. Nuclear Classification Evaluation

Nuclear classification evaluation incorporates detection and classification correctness rather than measuring class labels independently of instance localization. Detection is determined by distance to annotated nuclear centres when IoU-based matching is unavailable.

  • Classification Evaluation: Classification evaluation is performed on nuclear instances extracted through instance segmentation or detection, so the measurement includes both tasks.For each nuclear type, detected and missed instances are divided into true-positive, false-negative, and false-positive subsets.
  • Classification Evaluation: The classification formulation gives greater emphasis to nuclear type classification by setting α0 = α1 = 2 and α2 = α3 = 1.The resulting measure can be extended across all nuclear types by separating correctly and incorrectly classified detected instances.
  • Classification Evaluation: When classification labels are not exhaustively annotated, unlabelled ground-truth instances are subtracted from the incorrectly classified and missed-instance counts.This adjustment is specified for datasets such as CRCHisto.
  • Detection Criterion: Detection methods use centre distance rather than IoU to identify true positives for nuclear type classification.A prediction is positive when it falls within 6 pixels at 20× or 12 pixels at 40× of the annotated nuclear centre.

V. EXPERIMENTAL RESULTS

The experiments evaluate HoVer-Net across multiple nuclear segmentation datasets and compare it with diverse automated and software-based methods. Results emphasize instance-aware separation of clustered nuclei and report state-of-the-art segmentation performance.

  • Datasets: 41 CoNSeP tiles were acquired from 16 colorectal adenocarcinoma whole-slide images and include varied tissue regions and six nuclear types.The dataset provides segmentation annotations and class labels for colorectal adenocarcinoma images.
  • Experimental setting: Five datasets evaluate instance segmentation, while CoNSeP and CRCHisto evaluate nuclear classification.The study uses six independent datasets overall.
  • Experimental setting: The comparison includes computer-vision, medical-imaging, nuclear-segmentation, Cell Profiler, and QuPath methods.The evaluated methods span both learned approaches and open-source pathology software.
  • Comparative results: Overlapping nuclei produce especially large performance variation on Kumar and CoNSeP, where semantic-segmentation methods often achieve lower PQ because they fail to separate clustered instances.Instance-aware techniques are therefore favored in these challenging datasets.
  • Comparative results: HoVer-Net consistently improves over DIST and outperforms challenge-winning methods on the evaluated datasets, supporting state-of-the-art nuclear instance segmentation.The study places particular emphasis on panoptic quality (PQ).

D. Generalisation Study

The generalisation study tests whether models trained on Kumar transfer to unseen organs and independent staining sources. HoVer-Net successfully generalises across all three held-out test settings.

  • Generalisation study: The study assesses generalisation to new organs through nuclear-shape variation and to new centres through staining variation.These two settings target distinct sources of variation in unseen H&E images.
  • Experimental design: Kumar is used for training and validation, while combined CPM, TNBC, and CoNSeP datasets serve as three independent test sets.The test groups are separated according to dataset origin.
  • Experimental design: CPM tests generalisation to new organs, whereas TNBC tests transfer across preservation and staining practices from an independent source.TNBC contains samples from the already seen breast organ but differs in data source and specimen processing.
  • Comparative results: HoVer-Net successfully generalises to unseen data in all three test cases, while U-Net and DIST perform worse than competing methods across all three datasets.SegNet with watershed and Mask-RCNN also achieve competitive performance across the held-out sets.

E. Comparative Analysis of Classification Methods

The classification study converts leading instance-segmentation methods into multi-class systems and evaluates them on CoNSeP and CRCHisto. HoVer-Net provides good simultaneous instance segmentation and classification despite class imbalance and heterogeneous miscellaneous nuclei.

  • Comparative methods: The study compares HoVer-Net with Micro-Net, Mask-RCNN, DIST, and SC-CNN for miscellaneous, inflammatory, epithelial, and spindle-shaped nuclei.The models are evaluated in a multi-class setting.
  • Evaluation: Models trained on CoNSeP are evaluated on its test set and on the entire CRCHisto dataset.CoNSeP evaluation includes PQ, whereas CRCHisto lacks segmentation masks and is evaluated without PQ.
  • Comparative results: HoVer-Net achieves good-quality simultaneous instance segmentation and classification compared with competing methods.The reported conclusion covers both tasks jointly rather than classification alone.
  • Comparative results: The miscellaneous class has lower expected F1 because it contains fewer nuclei and combines diverse mitotic, necrotic, and uncategorisable cells.HoVer-Net nevertheless achieves satisfactory performance for this class.

VI. DISCUSSION AND CONCLUSIONS

HoVer-Net supports simultaneous nuclear instance segmentation and classification across multi-tissue histology images, including clustered nuclei. The paper also contributes a labelled colorectal dataset and evaluation measures, while noting scope and class-balance limitations.

  • Method: HoVer-Net uses three up-sampling branches to detect nuclear pixels, regress horizontal and vertical centre-of-mass distances, and classify nuclear types.The distance predictions help separate clustered nuclei, while the classification branch determines each instance’s type.
  • Results: State-of-the-art instance segmentation performance was achieved across multiple datasets prepared and stained under different conditions.The authors attribute this result to strong generalisation capacity and practical applicability as a prerequisite for nuclear feature extraction.
  • Results: 11.04 seconds versus 106.98 seconds: HoVer-Net processed a 1,000×1,000 tile approximately 9.7× faster than Mask-RCNN for segmentation and classification.DIST and Micro-Net were faster in the reported comparison, at 0.600 and 0.832 seconds respectively.
  • Evaluation: The paper proposes statistical measures because common measurements did not always reflect instance segmentation performance.The authors encourage their use to improve interpretability and enable fairer comparisons between methods.
  • Limitations: The model is trained on a single tissue type, and the miscellaneous classification category has low F1 because it has fewer samples and high intra-class variability.Future work is proposed to add samples, including necrotic and mitotic nuclei, to improve class balance.

APPENDIX A. ABLATION STUDIES

The ablation studies examine the loss strategy, post-processing, and classification-branch design. They report gains from the multi-term loss and from separating classification into its own branch.

  • Loss Terms: The proposed multi-term loss strategy increased all reported performance measures compared with the standard MSE-and-BCE strategy.The added terms include horizontal and vertical gradient MSE for the HoVer branch and dice loss for the nuclear-pixel branch.
  • Loss Terms: The gradient loss is intended to enforce the structure of horizontal and vertical map predictions and help separate neighbouring instances.Dice loss helps distinguish background from nuclear pixels under class imbalance.
  • Post Processing: The study compares Sobel-based post-processing with two further techniques for deriving the energy landscape and watershed markers.HoVer-Net derives both quantities from its predictions despite not being directly built on an energy landscape.
  • Classification Branch: A dedicated classification branch improved overall simultaneous nuclear instance segmentation and classification on both CoNSeP and CRCHisto.The comparison was against a two-branch setup that predicted nuclear types at the nuclear-pixel branch output.
  • Classification Branch: Classification performance was compromised when classification occurred at the nuclear-pixel branch output rather than in a dedicated branch.The authors connect this result to the difficulty of nuclear classification and the benefit of decoupling detection from classification.
Loading 1812.06499v5…