Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology

1. Introduction

• Limited samples for supervised MIL motivate unsupervised slide representation learning, which may help mitigate data and label scarcity and improve generalization.MIL aggregates more than 10,000 patch embeddings per WSI, while current work also explores domain-specific self-supervised encoders and new aggregation strategies.Source: §1 · p. 1
• PANTHER assumes patch embeddings arise from a Gaussian mixture, using assignment probabilities to map patches to prototypes and mixture probabilities, means, and covariances to represent them.Concatenated mixture parameters form the slide representation, enabling per-prototype modeling and interpretation.Source: §1 · p. 2
• PANTHER is reported to outperform nearly all unsupervised and supervised baselines across four diagnostic and nine prognostic tasks, with prototype spread visualized and quantified.The framework is presented as a compact, unsupervised GMM-based WSI representation.Source: §1 · p. 2

2.1. Multiple instance learning in CPath

• PANTHER differs from supervised MIL by constructing an unsupervised set embedding agnostic to downstream tasks, while MIL learns task-specific slide embeddings from patches..Source: §2.1 · p. 2

2.2. Quantification of distance between sets

• PANTHER can be viewed as matching WSI patches to prototypes, whereas optimal transport measures the minimum-cost matching between two sets.The paper describes applications of set matching including slide retrieval, domain adaptation, and WSI-genomic fusion.Source: §2.2 · p. 2

2.3. Prototype-based set representation

• Prototype-based set representations should encode both cardinality—how many elements are associated with a prototype—and prototype identity.Prior pathology approaches include pooled cluster embeddings and prototype-assignment proportions; the paper identifies missing cardinality or deep visual features as limitations of H2T and HPL.Source: §2.3 · p. 3

3.1. Prototype-based aggregation

• PANTHER models patch embeddings with a Gaussian mixture, estimates slide-specific parameters, and concatenates them into a fixed-length slide representation.The representation is built from a small set of prototypes relative to the number of patches; typical settings use C = 8–32 prototypes and 10,000–20,000 patches.Source: §3.1 · pp. 3–4

3.1.1 Connection to optimal transport

• PANTHER’s aggregation relates to optimal transport, but estimates prototype mixture probabilities rather than imposing a uniform prototype distribution.The passage describes the OT solution as a special case of the GMM solution with a uniform prototype distribution.Source: §3.1.1 · p. 4

3.2. Downstream evaluation

• Downstream predictors can use a linear layer or MLP; a structured MLP decomposes the concatenated representation to learn a mapping for each prototype.The cited discussion contrasts this per-prototype mapping with typical MIL, where variable set size and permutation invariance make the mappings nontrivial.Source: §3.2 · pp. 4–5

3.3. Interpretability

• For interpretation, PANTHER assigns patches to their most probable prototypes to form tissue maps and visualizes patch similarity to a selected prototype.The prototype assignment probability also quantifies the extent of a prototype’s distribution within a slide.Source: §3.3 · p. 5

4.1. Datasets

• Subtyping evaluation covers EBRAINS fine and coarse brain-cancer subtyping, NSCLC subtyping on TCGA and CPTAC, and PANDA prostate grading; survival evaluation spans six TCGA cancer types.KIRC and LUAD models trained on TCGA are externally evaluated on CPTAC, with LUAD also evaluated on NLST; survival uses disease-specific survival and the c-index.Source: §4.1 · p. 5

4.2. Baselines

• Comparators include unsupervised representations followed by task-specific linear networks and supervised slide representations, including ABMIL, TransMIL, AttnMISL, and ILRA.The unsupervised baselines are DeepSets, ProtoCounts, H2T, and optimal transport; OT assumes uniform mixture probability.Source: §4.2 · p. 5

4.3. Implementation

• PANTHER tests concatenated, weighted-average, and largest- or smallest-mixture representations with linear or MLP heads, using UNI features shared across baselines.All patches are used, one EM step was found sufficient for convergence, and K-means prototypes are cohort-derived and shared across several prototype-based methods; Table 1 covers four subtyping tasks with C = 16 prototypes for the listed methods.Source: §4.2 · p. 5, §4.3 · p. 5, Table 1 · p. 6

5.1. Subtyping and survival prediction

• PANTHERAll+MLP outperforms or matches the best supervised baselines on subtyping and survival, while PANTHERAll+lin. remains competitive on subtyping and performs better on most cancer types in survival prediction.The reported results are in Tables 1 and 2; the supplied Table 2 caption identifies disease-specific survival c-index comparisons using UNI features and five runs.Source: §5.1 · p. 6, Table 2 · p. 7
• Concatenating prototype representations with their cardinalities is associated with stronger results than averaging, and both single-prototype variants perform consistently poorly.PANTHERTop performs relatively well for NSCLC subtyping, which the authors attribute to a simple binary task and tumor being the most populous component on average.Source: §5.1 · pp. 6–7

5.2. Interpretability

• Pathologist-reviewed prototypes reflected tumor, stromal, immune, and normal-tissue phenotypes; NSCLC adenocarcinoma and squamous prototypes appeared almost exclusively in LUAD and LUSC, respectively.In CRC, prototype distributions showed strong concordance with existing CRC-100K tissue annotations.Source: §5.2 · p. 7, Figure 3 · p. 8

5.3. Further ablations

• PANTHER was robust across choices of prototype count C, survival loss functions, and feature encoders.Source: §5.3 · p. 8

• PANTHER uses Gaussian mixtures over patch distributions to learn unsupervised slide representations.Source: §6 · p. 8
• Using C = 16 prototypes for every task may over- or under-cluster some cancers.Source: §6 · p. 8
• Future work includes more expressive mixture models, data-driven prototype counts, and evaluation on rare cancer cohorts with small sample sizes.Source: §6 · p. 8

• The derivation frames GMM parameter estimation as maximizing the log-likelihood and introduces a Jensen-inequality lower bound as a surrogate objective.Source: §1 · p. 14
• Increasing the lower bound with respect to the parameters monotonically increases the log-likelihood, and this optimization is performed by alternating E- and M-steps.Source: §1 · p. 14
• The E-step uses Bayes’ rule to compute posterior prototype-assignment probabilities and the corresponding expectations.Source: §1 · p. 14
• The M-step optimizes the surrogate Q function with respect to the GMM parameters, treating the prior-iteration entropy term as constant.Source: §1 · p. 14

• For training, the authors use the AdamW optimizer with a learning rate of 1 × 10−4 and weight decay of 1 × 10−5.Source: §2 · p. 14
• Slide classification uses cross-entropy for up to 20 epochs, with early stopping after 10 epochs without validation-loss improvement.Supervised baselines use batch size 1 with gradient accumulation over 32 steps; unsupervised baselines, including PANTHER, use batch size 32.Source: §2 · p. 15
• Survival experiments use Cox proportional hazards loss for unsupervised baselines and negative log-likelihood for supervised baselines.Unsupervised baselines use batches of 64 patients for 50 epochs; supervised baselines use one patient per batch for 20 epochs.Source: §2 · p. 15

3. Computational considerations

• PANTHER pre-extracts 32,784-dimensional slide features, 468× smaller than the patch embeddings used for MIL training.The features concatenate πc, µc, and Σc across 16 prototypes.Source: §3 · p. 15
• At batch size 1, PANTHER extracts features at 10 WSIs per second and compresses 11K TCGA slides (4 TB) to about 1.4 GB.Source: §3 · p. 15
• A linear classifier on PANTHER features has 32,784 parameters, compared with about 500K for ABMIL.Source: §3 · p. 15

4.1. Slide classification

• EBRAINS evaluates fine- and coarse-grained brain-tumor subtyping on 2,319 WSIs, while NSCLC classification uses TCGA training and CPTAC external validation.EBRAINS uses a 50:25:25 train/validation/test split; NSCLC uses TCGA 80:10:10 splits and balanced accuracy and weighted F1 for evaluation.Source: §4.1 · p. 15
• PANDA evaluates six-class ISUP grading on 10,616 prostate biopsies, with separate evaluation on KRLS and RUMC cohorts using quadratic weighted kappa.The dataset is split 80:10:10 for training, validation, and testing.Source: §4.1 · p. 15

4.2. Survival prediction

• Survival evaluation uses site-stratified five-fold TCGA cross-validation for six cancer types and external CPTAC or NLST cohorts for KIRC and LUAD.External evaluation is described as more difficult because of image-acquisition, geographic, demographic, and follow-up differences.Source: §4.2 · p. 15

5. Additional experiments

• Additional experiments assess feature encoders, prototype counts C = {8, 16, 32}, and survival losses including NLL and ranking loss.The experiments include representative classification and survival tasks selected for challenging subtyping, tissue-pattern proportions, CRC annotations, and out-of-domain generalization.Source: §5 · pp. 15–16

Stronger feature encoders improve supervised MIL baselines

• UNI improves supervised MIL baselines across classification tasks except DSMIL on PANDA-RUMC, and UNI-based AttnMISL ranks highly on survival tasks.AttnMISL reaches the second-highest CRC c-index and the highest LUAD c-index on TCGA and NLST; Table S1 covers encoder comparisons.Source: §6 · p. 16, Table S1 · p. 18

Stronger feature encoders enable unsupervised baselines to compete with MIL

• With UNI features, unsupervised slide-representation methods gain substantially and outperform AB-MIL and other MIL baselines in the reported comparisons.With CTransPath, OT and PANTHER underperform weakly supervised baselines on EBRAINS and PANDA-KRLS; Table S1 covers encoder comparisons on classification and survival tasks.Source: §6 · p. 16, Table S1 · p. 18

PANTHER trains stable survival models

• PANTHERAll+MLP consistently outperforms cluster-based methods on CRC survival, while its C=8 c-index of 0.691 exceeds TransMIL’s 0.684.On LUAD TCGA, PANTHERAll+MLP is consistently second to OT, whose best c-index is 0.715 with C=8; many baselines show unstable external-cohort performance.Source: §6 · p. 16, Table S2 · p. 19, Table S3 · p. 20

PANTHER prototypes capture distinct tumor morphologies

• PANTHER maps distinct tumor patterns, including muscle-invading BLCA, nested BRCA tumors, and KIRC with or without poorly differentiated glands, with tumor heatmaps concordant with supervised classifiers.The exemplar figures show prototype assignments and distributions alongside pathologist annotations.Source: §6 · p. 16, Figure S1 · p. 21, Figure S2 · p. 22

PANTHER prototypes capture distribution of tissue classes in CRC-100K

• PANTHER prototypes correspond to morphologically relevant patterns across all nine CRC-100K tissue classes in COADREAD, as assessed with a supervised tissue classifier.Figure S4 compares prototype assignments with predicted CRC-100K tissue labels for COADREAD slides and learned prototypes.Source: §6 · pp. 16–17, Figure S4 · p. 24