PaperlayerView exact revision on arXiv

Source-linked AI summary

Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images

Hui Qu, Pengxiang Wu, Qiaoying Huang, Jingru Yi, Zhennan Yan, Kang Li, Gregory M. Riedlinger, Subhajyoti De, Shaoting Zhang, Dimitris N. Metaxas

arXiv:2007.05448v1cs.CV

TL;DR

Pixel-wise nuclei segmentation requires laborious expert annotation, motivating a framework that uses only partial nuclear-point labels. The method detects nuclei semi-supervisedly, converts detected points into complementary coarse segmentation labels, and refines the trained model with CRF loss. Across the evaluated datasets, it achieves performance comparable to fully supervised methods while substantially reducing annotation effort.

  • Problem

    Existing weak annotations are often unsuitable for nuclei segmentation, while dense pixel-wise labels are laborious; the paper therefore investigates partial points annotation.

  • Method

    The method combines extended-Gaussian-mask initialization and background-propagation self-training for detection with Voronoi and cluster labels, CNN training, and CRF refinement for segmentation.

  • Results

    Comparable performance to fully supervised methods is achieved with much less annotation, including LC precision, recall, and F1 ratios of 98.2%, 100.3%, and 99.2%, respectively.

  • Takeaways & Limitations

    The framework supports nuclei analysis with substantially reduced annotation effort while retaining performance close to fully supervised models.

  • Takeaways & Limitations

    The approach still has room for improvement in nuclear shape accuracy, as indicated by AJI values.

Abstract

from arXiv · show

Nuclei segmentation is a fundamental task in histopathology image analysis. Typically, such segmentation tasks require significant effort to manually generate accurate pixel-wise annotations for fully supervised training. To alleviate such tedious and manual effort, in this paper we propose a novel weakly supervised segmentation framework based on partial points annotation, i.e., only a small portion of nuclei locations in each image are labeled. The framework consists of two learning stages. In the first stage, we design a semi-supervised strategy to learn a detection model from partially labeled nuclei locations. Specifically, an extended Gaussian mask is designed to train an initial model with partially labeled data. Then, selftraining with background propagation is proposed to make use of the unlabeled regions to boost nuclei detection and suppress false positives. In the second stage, a segmentation model is trained from the detected nuclei locations in a weakly-supervised fashion. Two types of coarse labels with complementary information are derived from the detected points and are then utilized to train a deep neural network. The fully-connected conditional random field loss is utilized in training to further refine the model without introducing extra computational complexity during inference. The proposed method is extensively evaluated on two nuclei segmentation datasets. The experimental results demonstrate that our method can achieve competitive performance compared to the fully supervised counterpart and the state-of-the-art methods while requiring significantly less annotation effort.

I. INTRODUCTION

The paper addresses the annotation burden of pixel-wise nuclei segmentation by proposing a weakly supervised framework using only partial nuclear locations. Its two-stage design combines semi-supervised detection with weakly supervised segmentation and aims to retain competitive performance while reducing annotation effort.

  • Pixel-wise nuclei segmentation labels are time-consuming to create and require expert domain knowledge.
  • Its first stage detects all nuclei from partial annotations using an extended Gaussian mask followed by self-training with background propagation.
  • The paper reports comparable performance to fully supervised methods while substantially reducing annotation effort, and analyzes learning strategies for weakly supervised segmentation.
  • Its second stage derives complementary Voronoi and cluster labels from detected points and trains a segmentation model with weak supervision.
  • The proposed framework uses only a small portion of annotated nuclear locations, such as 10%, for nuclei segmentation.

1) Nuclei detection:

The paper positions partial-point nuclei segmentation against existing learning-based and weakly supervised approaches, then introduces a two-step detection procedure for learning from incomplete point annotations. Extended Gaussian masks provide initial supervision, while self-training progressively incorporates background information.

  • Existing weak annotations are often unsuitable for nuclei because images contain many small, densely packed objects and fixed object classes.
  • The detection module first trains with extended Gaussian masks and then performs self-training with background propagation.
  • Extended Gaussian masks focus supervision around labeled points while ignoring most unlabeled pixels during initial training.
  • The mask treats pixels in the annular region r1 < Di < r2 as background, with r2 set to 2r1.
  • Detection outputs are obtained by thresholding the predicted probability map and finding connected-component centroids.

B. Self-training with background propagation

Self-training expands supervision beyond the partially labeled nuclei by propagating reliable background information into previously ignored regions. Iterative mask updates improve background coverage while preserving the original foreground annotations.

  • B. Self-training with background propagation: The initial detector is weakened by few labeled nuclei and extensive ignored regions, motivating self-training on unlabeled image areas.The method uses predicted nuclei locations together with originally labeled nuclei to supervise subsequent training.
  • B. Self-training with background propagation: Each self-training round derives a background map from the previous model and combines it with original labeled points to create a new training mask.Pixels with nuclei probability below 0.1 or above 0.7 are selected, with an area-based restriction for high-probability regions.
  • B. Self-training with background propagation: The updated mask adds newly identified background information to the original extended Gaussian mask.The mask-generation rule assigns labels based on distance to annotated points and selected probability thresholds, leaving other pixels ignored.
  • B. Self-training with background propagation: By the third round, the background area has grown and achieved high accuracy, while the ignored pixels are almost all nuclei.The original foreground nuclei annotation remains unchanged throughout the iterations.

IV. SEGMENTATION WITH POINTS

The segmentation stage converts detected nucleus points into weak pixel-level supervision rather than using the points directly. It combines Voronoi-based central-region labels with image-derived clustering labels to address missing background annotations and capture complementary information.

  • IV. SEGMENTATION WITH POINTS: The segmentation method uses the original image and detected-point information to generate weak supervision for a deep convolutional network.The point-level supervision is organized around coarse pixel-label generation from the detector output.
  • IV. SEGMENTATION WITH POINTS: Detected nucleus points cannot directly train a CNN with cross entropy because they provide positive nuclei labels but no negative background labels.The method therefore generates pixel-level labels for both classes before segmentation training.
  • IV. SEGMENTATION WITH POINTS: The method derives two coarse label types from detected points using Voronoi diagrams and k-means clustering.The labels exploit assumptions that points lie near nucleus centers, nuclei are nearly convex, and nuclei colors differ from background.
  • IV. SEGMENTATION WITH POINTS: Voronoi cells assign each detected point a region based on proximity, assuming the corresponding nucleus lies within its cell.This partitions the image into convex polygons around the detected seed points.
  • IV. SEGMENTATION WITH POINTS: Voronoi point-edge labels mark dilated detected points as nuclei and Voronoi edges as background, while ignoring remaining pixels.They emphasize central nucleus regions and help separate touching nuclei but cannot recover full masks without boundary and shape information.

2) Cluster labels:

Cluster labels add rough nuclear shape information to the central-region cues from Voronoi labels, using color and distance features with an ignored boundary class. The network is trained with both labels and refined using a dense CRF loss.

  • 2) Cluster labels:: K-means divides pixels into nuclei, background, and ignored clusters, assigning nuclei by maximum overlap with point labels and background by minimum overlap with dilated points.The remaining cluster is ignored because clustering struggles near nuclear boundaries.
  • 2) Cluster labels:: Combining RGB color with clipped distance-to-point features helps clustering distinguish nuclei whose colors are non-uniform.The feature vector contains normalized color values and a clipped distance value.
  • 2) Cluster labels:: Morphological dilation and erosion refine cluster labels separately within Voronoi cells, avoiding connections between nearby nuclei.Cluster labels provide more shape information than Voronoi labels but also introduce more errors and uncertainty.
  • 2) Cluster labels:: The segmentation network outputs background and nuclei probability maps and computes cross-entropy losses against cluster and Voronoi labels on non-ignored pixels.The balancing parameter α controls the relative weighting of the loss terms.
  • 2) Cluster labels:: Dense CRF refinement adds unary cross-entropy and pairwise label-consistency terms during fine-tuning, with fast mean-field inference avoiding direct affinity-matrix computation.Directly forming the N×N affinity matrix is prohibitive for large images, such as 1.6×10^9 elements at 200×200 pixels.

V. EXPERIMENTS

The framework is evaluated on two datasets of H&E-stained histopathology images. The supplied passage identifies the experimental setting but reports no quantitative outcomes.

  • V. EXPERIMENTS: Experiments evaluate the proposed framework on two datasets of H&E-stained histopathology images.The passage specifies the dataset type and count but gives no performance values.

A. Datasets

The study evaluates nuclei detection and segmentation on Lung Cancer and Multi-Organ datasets with partial point annotations. Detection uses standard precision, recall, F1, localization error, and sampled annotation ratios.

  • Datasets: The Lung Cancer dataset contains 40 900 × 900 images from eight carcinoma cases, split into 24 training, 8 validation, and 8 test images, with 24,401 nuclei masks.
  • Datasets: The Multi-Organ dataset contains 30 1000×1000 images from multiple hospitals and seven organs, split into 12 training, 4 validation, and 14 test images.Its nuclear appearances vary substantially because of organ and cancer-type heterogeneity.
  • Annotation: Both datasets provide full masks, while training detection uses nuclear-mask bounding-box centers and randomly sampled partial point annotations.
  • Evaluation: The experiments aim to detect all nuclei from models trained with partial points, using 10% annotation in one comparison and multiple ratios in another.
  • Evaluation: Detection performance is measured with precision, recall, F1 score, and localization error based on centroid matching within dataset-specific radii.The matching radius is 8 pixels for Lung Cancer and 11 pixels for Multi-Organ.

2) Implementation details:

The implementation compares fully supervised and progressively weaker detection strategies under partial point supervision. Extended Gaussian masks and background-propagation self-training are evaluated against simpler alternatives.

  • Detection strategies: The comparison includes full supervision, simple Gaussian masks, extended Gaussian masks, nuclei-updating self-training, and background-propagation self-training.Both self-training variants use extended Gaussian masks for initial training, with 10% points per training image.
  • Initial training: Extended Gaussian masks outperform simple Gaussian masks because they avoid treating all unlabeled nuclei as background and focus learning around labeled points.This enables correct predictions in similar unlabeled regions.
  • Self-training: Updating predicted nuclei decreases performance because newly added false positives are comparable in number to labeled nuclei.
  • Self-training: Background propagation improves recall, precision, F1 score, and localization accuracy by preserving labeled nuclei and gradually expanding the background without introducing false positives.
  • Comparison with full supervision: Using background propagation, precision, recall, and F1 reach 98.2%, 100.3%, and 99.2% of fully supervised results on LC, and 97.8%, 96.1%, and 97.0% on MO.Localization error is also very close to full supervision on both datasets.
  • Annotation ratios: With 50% points annotated, performance is nearly the same as full annotation; with 5%, F1 reaches 99.0% of full on LC and 96.3% on MO.

4) Different ratios of annotation:

Segmentation training uses weakly supervised and fully supervised settings with object- and pixel-level evaluation. The study examines generated-label weighting and dense CRF fine-tuning before comparison with supervised models.

  • Evaluation: Segmentation is evaluated with pixel accuracy, pixel-level F1, object-level Dice, and Aggregated Jaccard Index.Object-level correspondence requires more than 50% overlap.
  • Training: Weakly supervised models train for 100 epochs and receive 20 epochs of dense CRF fine-tuning, while fully supervised models train for 200 epochs with binary masks.
  • Training: The validation set selects the best model for testing in both weakly supervised and fully supervised settings.

3) Results and discussion:

Experiments analyze how complementary coarse labels and dense CRF parameters affect weakly supervised segmentation. The method performs close to fully supervised models, separates touching nuclei, and retains limitations in nuclear shape quality.

  • Label weighting: Combining cluster and Voronoi labels performs better than either label alone because it joins shape information with nuclei/background information.The best performance occurs around α = 0.5, which is used thereafter.
  • Label weighting: Voronoi-only labels improve object-level metrics but worsen pixel-level metrics by producing small, separated central-nuclei instances.
  • Dense CRF loss: Dense CRF performance depends on σpq, σrgb, and β: moderate spatial and color bandwidths balance local and large-object effects, while extreme settings are less effective.The best combinations are σpq = 9, σrgb = 0.2, β = 0.001 for LC and σpq = 9, σrgb = 0.1, β = 0.005 for MO.
  • Comparison with full supervision: Using all ground-truth points, weakly supervised segmentation is close to fully supervised performance, with LC gaps of 2.0% accuracy, 7.2% F1, 5.9% Dice, and 6.9% AJI.On MO, the accuracy and F1 gaps are 1.1% and 4.7%.
  • Comparison with full supervision: Without post-processing, the weakly supervised model separates most touching nuclei through Voronoi labels, whereas the fully supervised model fails to do so in this comparison.
  • Comparison with prior methods: Compared with DIST, the proposed method has higher pixel-level F1 but room to improve nuclear shapes, as indicated by AJI.

D. Segmentation results using detected points

Using detected points, segmentation remains satisfactory with sparse annotations but improves as point coverage increases. Performance is linked to nuclei–background contrast and within-nucleus color uniformity, while appearance variability widens the gap from ground-truth points.

  • Detected-point errors, including false positives, false negatives, and localization errors, degrade the generated Voronoi and cluster labels and therefore segmentation performance.
  • Even with 5% annotated points, the framework achieves satisfactory segmentation performance compared with fully supervised methods.
  • Segmentation performance improves as annotated points increase from 5% to 50%, because detection errors decrease with higher annotation coverage.
  • Larger nuclei-bg-diff and smaller nuclei-std correspond to better segmentation performance across LC and MO test images.nuclei-bg-diff measures nuclei–background pixel-value difference, while nuclei-std measures within-nucleus pixel-value variation.
  • Images with nuclei resembling background pixels and having large color variance show larger performance gaps between partial-point and ground-truth-point segmentation.Such images are harder to extract accurately and more sensitive to detected-point errors.

E. Sensitivity and generalization analyses

The method is relatively insensitive to which 10% points are initially selected and generalizes well between the LC and MO datasets. Point annotation also requires substantially less time than bounding-box or full-mask annotation.

  • Sensitivity: Small metric variances across ten random 10% point sets indicate that the method is not sensitive to the choice of initial points.
  • Generalization: 92% to 99% performance is achieved when MO-trained models are applied to LC, relative to models trained on LC.
  • Generalization: 95% to 99% performance is achieved when LC-trained models are applied to MO, relative to models trained on MO.
  • Annotation effort: For images averaging about 600 nuclei, full-mask annotation takes 115 minutes, bounding-box annotation 67 minutes, all-points annotation about 14 minutes, and 10% points annotation less than 2 minutes.
  • Conclusion: The proposed method uses partial nuclear locations for detection and segmentation, deriving Voronoi and cluster labels and refining training with dense CRF loss.
Loading 2007.05448v1…