PaperlayerView exact revision on arXiv

Source-linked AI summary

K-CLUE: A Segmentation-to-Clustering Framework for Ki67 Cell Typing

Ana Beatriz Vieira, Sara P. Oliveira, Diana Montezuma, João Fraga, Jaime S. Cardoso, Arlindo L. Oliveira

arXiv:2610.32493v2cs.CV

TL;DR

Manual Ki67 scoring is laborious and subjective, motivating annotation-free approaches to nucleus typing. K-CLUE clusters pretrained nucleus representations, and whitening reduces encoder sensitivity while supporting strong end-to-end F1 results.

  • Problem

    Ki67 quantification remains largely manual, with subjective interpretation of borderline nuclei contributing to inter-observer variability.

    Source: Introduction · p. 2

  • Method

    K-CLUE segments nuclei, encodes each crop with a frozen pretrained model, applies PCA and whitening, then clusters nuclei into Ki67-positive and Ki67-negative groups without task-specific training or target-domain annotations.

    Source: §3.3 · p. 5, §5 · p. 14

  • Results

    Within one F1-score point on MIST, the seven whitened encoders perform similarly, while whitened handcrafted color representations perform comparably to K-CLUE.

    Source: §5 · p. 14

  • Takeaways & Limitations

    Pretrained representations can support annotation-free Ki67 typing when their variance structure is accounted for.

    Source: §5 · p. 14

    Final cell-typing performance depends on upstream nuclei segmentation, since missed and spurious detections directly affect it.

    Source: §5 · p. 14

Abstract

from arXiv · show

Ki67 is a key marker of tumor proliferation in breast cancer, but its quantification from immunohistochemistry images normally depends on supervised classifiers trained per dataset. We examine the role of frozen pretrained visual representations in annotation-free Ki67 cell typing. To this end, we use K-CLUE, a framework that segments individual nuclei, represents each nucleus with a pretrained encoder, reduces the resulting embeddings with principal component analysis, and separates Ki67-positive from Ki67-negative nuclei using k-means clustering, without task-specific training or manual annotations. Clustering the raw components is highly sensitive to the choice of encoder: across seven encoders, the Ki67-positive F1-score on matched MIST nuclei varies by 44 points, although Ki67 status remains linearly decodable from every representation, with supervised probe AUCs above 0.98. Analysis of the embedding spaces shows that clustering succeeds when DAB-related variation occupies the leading directions of variance. Whitening the retained principal components removes this dependence, reducing the spread across encoders to 1.00 F1-score point. Handcrafted color representations are also competitive, confirming that Ki67 status is strongly reflected in chromatic information, but they require an explicit choice of color space. In end-to-end evaluation, K-CLUE reaches F1-scores of 85.00% and 81.91% for Ki67-positive and Ki67-negative nuclei on MIST, and 79.59% and 81.16% on BCData, outperforming DeepLIIF and IHCScoreGAN on both datasets. These results show that pretrained representations can support annotation-free Ki67 typing when their variance structure is accounted for.

Introduction

Ki67 assessment matters for characterizing tumor proliferation, yet manual scoring of stained nuclei is laborious and can vary between observers. K-CLUE uses frozen pretrained representations and unsupervised clustering, with whitening proposed to reduce encoder dependence.

Introduction

  • Ki67 is expressed during active cell-cycle phases and absent in resting cells; dual H-DAB staining marks proliferating nuclei brown and other nuclei blue.Accurate positive-versus-negative nucleus classification underlies assessment of proliferative activity, subtype stratification, and chemotherapy planning.Source: Introduction · p. 2
  • Manual Ki67 scoring relies on selected regions and visual estimates, while borderline cases require subjective interpretation and show inter-observer variability.Whole-slide images may contain hundreds of thousands of nuclei, making exhaustive counting impractical.Source: Introduction · p. 2
  • K-CLUE segments nuclei, embeds each crop with a frozen pretrained encoder, reduces embeddings with PCA, and clusters nuclei into Ki67-positive and Ki67-negative groups.The pipeline is presented as annotation-free; annotations are reserved for evaluation and explicitly identified supervised probes, not inference.Source: Introduction · pp. 2–3
  • Changing only the encoder shifts Ki67-positive F1 by more than forty percentage points, even with segmentation, crops, and clustering held fixed.The variation does not follow encoder capacity or pretraining domain.Source: Introduction · p. 2
  • Clustering can miss Ki67 signal when non-staining variation dominates leading principal components, although a supervised linear probe decodes Ki67 status from every evaluated encoder.K-means uses Euclidean distances, so high-variance directions have greater influence; whitening rescales retained components to unit variance.Source: Introduction · p. 2
  • Whitening principal components makes the pipeline largely encoder-independent, removing the need to select a representation or color descriptor per dataset.Handcrafted color descriptors are competitive, but their strongest representation varies across datasets.Source: Introduction · pp. 2–3
  • The resulting pipeline is evaluated end-to-end on two public cohorts under one configuration against supervised, generative, and handcrafted color baselines.The motivation includes the need for automated, reproducible methods that avoid costly nuclear annotations.Source: Introduction · p. 3

2 Related Work

Prior work addresses Ki67 scoring through handcrafted stain separation, generative IHC frameworks, and whitening methods developed for other representation tasks. These approaches differ in their data requirements and transfer assumptions.

2 Related Work

  • Geread et al. used unsupervised IHC color-histogram separation; on 30 TMA cores the average proliferation-index difference was 3.25%, and 74 of 80 Protein Atlas TMA images fell in the correct PI range.The Protein Atlas evaluation provided PI ranges rather than nucleus-level ground truth.Source: §2 · p. 3
  • Cellpose-SAM approaches a human-consensus nucleus-segmentation reference across imaging conditions, but methods developed for H&E and fluorescence may transfer poorly to DAB-stained IHC.Other cited segmentation models include CellViT++ [12], HoVer-NeXt, StarDist [25], and CellDETR.Source: §2 · p. 3
  • DeepLIIF uses paired IHC and multiplex immunofluorescence acquisitions, whereas IHCScoreGAN uses unpaired domain transfer for Ki67 scoring.DeepLIIF synthesizes protein-specific channels; IHCScoreGAN produces masks, cell centers, and biomarker maps.Source: §2 · p. 3
  • Whitening has been applied to image-retrieval descriptors, sentence embeddings [10], and sparse-autoencoder conditioning.The cited work associates it with improved retrieval, more uniform embeddings, and reduced influence of correlated or high-variance directions.Source: §2 · p. 4

3 Methodology

The methodology evaluates nuclei on two annotated breast-cancer cohorts, then uses Cellpose-DINOv3 and frozen pretrained encoders in an annotation-free pipeline for Ki67 typing. It also examines how embedding variance affects the clustering step.

3.1 Datasets

  • MIST provides pathologist point annotations on randomly cropped 256×256 tiles from 56 WSIs, while BCData supplies 402 test tiles from a dataset of 1,338 ROIs and 394 WSIs.BCData includes 181,074 annotated tumor cells: 62,623 Ki67-positive and 118,451 Ki67-negative.Source: §3.1 · p. 4

3.2 Object segmentation

  • Cellpose-DINOv3 ViT-B detects nuclei without early stain-based labeling, addressing the risk that H&E-oriented models miss DAB-stained nuclei in IHC images.The selected masks are used for subsequent experiments across IHC images.Source: §3.2 · pp. 4–5

3.3 K-CLUE

  • K-CLUE masks and crops each detected nucleus, encodes crops with frozen pretrained backbones, retains PCA components accounting for 90% of variance, whitens them, and applies pooled k-means with DAB-based cluster labels.Pixels outside each nuclear mask are filled with the dataset’s median background color to reduce interference from neighboring cells.Source: §3.3 · p. 5

3.4 Analysis of the embedding space

  • K-means follows high-variance directions, so Ki67 clustering depends on whether staining variation leads; first-component correlations with image covariates and tile origin probe that alignment.The analysis also measures whether Ki67 status is present in each representation using cross-validated logistic-regression AUC.Source: §3.4 · p. 6

4 Experiments

The experiments evaluate K-CLUE from nucleus detection through end-to-end Ki67 typing, including encoder, whitening, and external-dataset analyses. Whitening narrows encoder performance differences, and K-CLUE remains competitive with handcrafted color baselines and other methods across MIST and BCData.

4.1 Data Preprocessing

  • The comparisons adapt competitor inputs to their requirements: IHCScoreGAN tiles are upsampled and subdivided for MIST, while BCData inputs are adjusted for model size requirements.BCData has higher spatial resolution than MIST; K-CLUE does not require resolution adjustment.Source: §4.1 · pp. 6–7, §4.7 · p. 13

4.2 Evaluation protocol

  • End-to-end evaluation counts missed nuclei and unmatched predictions as classification errors, while representation-level metrics isolate matched nuclei.Predictions are matched to point annotations using one-to-one Hungarian assignment with a 3 µm tolerance.Source: §4.2 · p. 7

4.3 Nuclei segmentation

  • Cellpose-DINOv3 achieved the most reliable MIST nucleus detection, with the highest precision and recall and lowest MAE, and was used downstream.HoVer-NeXt and CellViT++ often missed brown DAB-stained nuclei; StarDist had higher recall but still missed many nuclei overall.Source: §4.3 · p. 7, Table 1 · p. 8, Figure 4 · p. 8

4.4 K-CLUE

  • K-CLUE detects nuclei, encodes each crop with a frozen pretrained model, reduces embeddings with PCA, and uses k-means to assign positive and negative groups.The cluster with higher mean DAB optical density is labeled positive; the assignment uses no annotations. Handcrafted alternatives summarize nuclear-pixel color channels before PCA and clustering.Source: §4.4 · pp. 8–9

4.5 Representation geometry and whitening

  • Ki67 status is linearly decodable from every evaluated encoder, but clustering quality tracks whether DAB variation occupies leading principal components.All supervised probe AUCs exceed 0.98; whitening changes the variance structure used by Euclidean k-means rather than adding information.Source: §4.5 · pp. 10–12
  • 44.15 F1-score points before whitening become 1.00 point across seven encoders after PCA whitening on MIST.HoVer-NeXt rises from 50.37% to 94.19% Ki67-positive F1, while encoders that already clustered well change only marginally.Source: §4.5 · pp. 9–10, Table 3 · p. 11

4.6 Results

  • 85.00% and 81.91%: K-CLUE achieves the highest MIST end-to-end F1-scores for Ki67-positive and Ki67-negative nuclei, respectively.It also has the highest Ki67-positive recall (81.00%) and Ki67-negative precision (88.25%).Source: Table 6 · p. 12, §4.6 · p. 12

4.7 Evaluation on an external dataset

  • 79.59% and 81.16%: K-CLUE's BCData F1-scores for Ki67-positive and Ki67-negative nuclei are slightly below the combined handcrafted baseline.K-CLUE remains within one F1 point of that baseline; the comparison is not significant on MIST, but is slightly below it on BCData.Source: §4.7 · pp. 13–14, Table 7 · p. 14

5 Conclusion

The conclusion presents K-CLUE as an annotation-free approach combining segmentation, frozen pretrained representations, PCA whitening, and unsupervised clustering. It emphasizes reduced encoder dependence and competitive end-to-end performance, while identifying upstream segmentation as a limitation.

5 Conclusion

  • K-CLUE combines nuclei segmentation, frozen pretrained representations, PCA whitening, and unsupervised clustering without task-specific training or target-domain annotations.Source: §5 · p. 14
  • Whitening brings seven encoders to within one F1-score point on MIST despite substantial variation in naive clustering, while handcrafted color representations perform comparably.The conclusion attributes this variation to embedding variance structure and notes that handcrafted representations require a task-specific color space and descriptor.Source: §5 · p. 14
  • Missed and spurious nucleus detections directly affect final cell-typing performance, making upstream segmentation K-CLUE's main limitation.The proposed future extension would distinguish tumor cells, lymphocytes, and other populations to estimate Ki67 indices within tissue compartments.Source: §5 · p. 14

A Supplementary Experimental Results

The appendix introduces supplementary experiments and analyses that support the main text.

A Supplementary Experimental Results

A.1 Segmentation network details

The segmentation appendix compares four Cellpose variants under default inference settings on MIST.

A.1 Segmentation network details

  • Table 8 compares four Cellpose variants using default inference parameters against 5,785 MIST annotations.It reports detection counts, matching-based precision, recall and F1-score, and per-image count MAE.Source: Table 8 · p. 18, §A.1 · p. 18

A.2 Features extraction

Nuclei of varying sizes are converted to fixed-size crops before encoding, and crop conversion determines how much surrounding tissue reaches the encoder.

A.2 Features extraction

  • Each detection is cropped in a fixed 64 × 64 px bounding box and upsampled to the encoder’s fixed input resolution.Encoders typically expect 224 × 224 px inputs.Source: §A.2 · p. 18
  • The configurations vary only in how the region outside the target nucleus is treated; segmentation output and the rest of the pipeline stay fixed.Source: §A.2 · p. 18

A.2.1 Neighboring/real background

The real-context strategy preserves the original tissue surroundings in each crop, but neighboring nuclei can obscure which cell is the classification target.

A.2.1 Neighboring/real background

  • Each crop is centered on the target cell’s centroid and cut as a 64 × 64 px patch from the original image without masking or inpainting.Source: §A.2.1 · p. 18
  • Real-context crops preserve realistic staining, tissue texture, and local context, but neighboring nuclei can obscure the target cell’s class.Because crops fit the largest dataset cell, they often include one or more neighbors that may be more densely stained or central-looking.Source: §A.2.1 · p. 18
  • Figure 6 illustrates examples of crops containing neighboring or real-context backgrounds.Source: Figure 6 · p. 19
  • Table 9 reports MIST Ki67 typing performance with real-context backgrounds after PCA whitening, evaluating Cellpose-DINOv3 detections matched to ground-truth nuclei.The table caption identifies the metrics as precision, recall, F1-score, MCC, and per-tile count MAE.Source: Table 9 · p. 19

A.2.2 White background

White fill removes surrounding-tissue information and centers each nucleus, but its sharper mask boundary can affect embedding geometry. The study favors median fill for tighter encoder convergence after whitening and less boundary influence.

A.2.2 White background

  • White fill replaces everything outside the target cell in the fixed 64 × 64 px box and centers the cell in the patch.Source: §A.2.2 · p. 19
  • For five of seven encoders, the first principal component is associated with nuclear area at 0.603–0.722 with white fill, versus 0.138–0.654 with median fill.The white fill’s brighter-than-tile background makes the mask-boundary edge more marked.Source: §A.2.2 · p. 20
  • Figure 7 shows examples of crops extracted with a white background.Source: Figure 7 · p. 20
  • Table 10 reports MIST Ki67 typing metrics for Cellpose-DINOv3 detections matched to ground-truth nuclei under white fill after PCA whitening.The caption lists precision, recall, F1-score, MCC, and per-tile count MAE.Source: Table 10 · p. 20
  • With real background retained, no encoder exceeds 67.97% Ki67-positive F1-score, and neighbors may have the opposite class with no cue marking the target nucleus.The real-context results are reported for MIST after PCA whitening in Table 9.Source: §A.2.2 · pp. 20–21
  • 93.8%–95.6% Ki67-positive F1-scores across all seven encoders are reached after whitening with either masked fill; median fill narrows the spread to 1.00 points versus 1.73 for white.White fill has a marginally higher best score, 95.57% against 94.78%, but median fill is selected for tighter convergence and less mask-boundary influence on leading directions.Source: §A.2.2 · p. 21

A.3 Handcrafted features

Handcrafted color descriptors provide a competitive Ki67 signal, but the most effective color space and response to whitening vary across datasets. HSV whitening on BCData is a notable exception, shifting clustering away from DAB-aligned variation.

A.3 Handcrafted features

  • The descriptors summarize pixel statistics within each nuclear mask, using HED, HSV, CIE L∗a∗b∗, or their concatenation; retained PCA components are clustered with k = 2.The cluster with higher mean DAB optical density is assigned Ki67-positive; results are reported with and without whitening.Source: §A.3 · p. 21
  • On MIST without whitening, HSV has the highest handcrafted Ki67-positive F1-score at 82.12%, while combined HED+HSV+CIE L∗a∗b∗ has the highest Ki67-negative F1-score at 80.47%.Whitening improves F1-score for all color-space sets on both classes, though Ki67-negative MAE rises for HSV and the combined representation; the single-channel DAB baseline has high positive precision but substantially lower recall.Source: §A.3 · p. 21, Table 11 · p. 22
  • On BCData without whitening, CIE L∗a∗b∗ leads both classes at 80.54% positive and 81.93% negative F1-score, while HSV is similar at 80.10% and 81.70%.Whitening makes the combined representation best for positive F1 among whitened choices at 80.33%, but HSV positive F1 falls from 80.10% to 57.65%.Source: §A.3 · p. 21, Table 12 · p. 22
  • For HSV on BCData, whitening shifts clustering from a DAB-correlated first component (r = 0.94) to a hue-dominated second component unrelated to DAB (r = 0.03).K-means then partly separates nuclei by hue rather than Ki67 status; other representations and HSV on MIST remain aligned with the first component after whitening.Source: §A.3 · pp. 21–22

A.4 Variance structure of the embeddings

On BCData, PCA whitening narrows the variation in Ki67 clustering performance across encoders. The analyses associate clustering performance with where DAB-related variation falls among the leading principal components.

A.4 Variance structure of the embeddings

  • DAB signal is strongest in PC1 for four encoders and PC2 for HoVer-NeXt, while slide identity is associated with all ten components for UNI2 and Virchow2.For HoVer-NeXt, the DAB association is 0.838 at PC2 versus 0.238 at PC1, which is occupied by nuclear area.Source: §A.4 · pp. 22–23, Figure 8 · p. 23

A.5 Effect of the whitening exponent

  • At α=0.7, HoVer-NeXt and Virchow2 gain more than 30 Ki67-positive F1 points; PC1-DAB encoders improve gradually, while UNI2 rises from α=0.5.The experiment scales each retained component by its standard deviation raised to α; α=0 leaves PCA components unchanged and α=1 gives them unit variance.Source: §A.5 · pp. 23–24, Table 13 · p. 23

A.6 Ablation studies on BCData dataset

  • After whitening, all seven encoders reach 90.89%–94.42% Ki67-positive F1-score, narrowing the spread from 40.91 to 3.53 points.Before whitening, HoVer-NeXt ranges at 52.76% Ki67-positive F1-score, while EfficientNet-B2 reaches 93.67%.Source: Table 14 · p. 24, §A.6 · p. 24, Table 15 · p. 25
  • Probe AUCs are 0.9945 for ResNet-101 and 0.9960 for Virchow2, while unwhitened Ki67-positive F1 correlates with PC1–DAB association at r=0.999.HoVer-NeXt and UNI2 show weaker PC1–DAB association but stronger DAB association in PC2.Source: §A.6 · pp. 24–25, Table 16 · p. 25
Loading 2610.32493v2…