CellPath-Bench: A Multidimensional Benchmark for Whole-Slide Cellular Representations in Pathology Foundation Models

Introduction

• Positive CRA across all 30 models and incomplete alignment between CRA and CRT show that nucleus-anchored cell information and cross-domain transfer capture complementary properties.Magnification, taxonomic granularity, and contextual feature integration further differentiate model behavior.Source: Figure 1 · p. 1, Introduction · p. 2
• Conventional patch-, region-, and slide-level evaluations assess task utility but cannot isolate frozen cell-level representations, since outcomes depend on aggregation, prediction, adaptation, and optimization.Higher-level performance may rely on tissue composition or regional texture, while spatial aggregation can obscure cell-discriminative information in local features.Source: Introduction · pp. 1–2
• CellPath-Bench keeps encoders frozen, samples features at registered nuclear coordinates, and uses multiclass linear probes to measure decodability, within-section CRA, and cross-domain CRT.IOCD, MOCV, and LOOO assess generalization across datasets and organs; CRT summarizes performance across these protocols.Source: Introduction · p. 2
• The benchmark’s quality-controlled reference contains 25 spatially registered H&E–Xenium sections, 7,079,283 cells, and 11 organs, with harmonized fine- and coarse-grained taxonomies.The panel was selected from 52 candidate Xenium datasets.Source: Introduction · p. 2
• The study evaluates 30 pathology-specific and general-purpose foundation models using 304,920 controlled linear-probe runs across spatial readouts, magnifications, taxonomic granularities, and evaluation protocols, assessing cell-type decodability and transfer.Source: Introduction · p. 2

Benchmark Overview

• The benchmark combines a registered cellular reference, coordinate-aligned representation interface, and protocol-specific linear probing under shared evaluation settings.Each section supplies an H&E image, registered nuclear coordinates, and harmonized Xenium-derived labels; fixed readouts and common partitions, probe configurations, and metrics support comparisons across models.Source: Benchmark Overview · pp. 2–3

Benchmark Data and Task Construction

• The primary S25 panel contains 25 quality-ranked sections spanning 11 organs and 7,079,283 spatially indexed cells.Fourteen scRNA-seq references supported transfer of 68 molecular cell types to Xenium sections, then deterministic mapping to nine-class Tfine and three-class Tcoarse tasks; quality review retained the 25 highest morphology–annotation-concordance sections among annotation-qualified candidates.Source: Benchmark Data and Task Construction · p. 3

Problem formulation.

• Frozen encoders are evaluated by fitting a new multiclass linear probe for each model and condition, measuring cell-type information decodable without encoder adaptation.Evaluation conditions combine magnification, readout, taxonomy, and protocol; model selection uses validation data and each probe is evaluated once on its test partition.Source: Problem formulation · p. 4

Coordinate-aligned cellular feature extraction.

• Registered nuclear coordinates anchor Nuc feature sampling, while Mean and Cls summarize patches; their combinations test whether patch-level context complements Nuc.Available fusions use direct addition or concatenation without normalization, rescaling, or projection; Cls and its fusions are omitted when an encoder lacks a class token. Nuc is spatially anchored but does not imply a nucleus-restricted receptive field.Source: Coordinate-aligned cellular feature… · p. 4

Evaluation Protocols

• Four protocols test transfer across spatial regions, sections within an organ, held-out sections in multi-organ training, and entirely unseen organs.IS, IOCD, MOCV, and LOOO prevent cells from held-out spatial or anatomical units from entering probe fitting; Macro-F1 is primary, and paired Wilcoxon tests assess model comparisons.Source: Evaluation Protocols · pp. 4–5

Evaluated foundation models.

• The benchmark evaluates 30 models spanning pathology vision, pathology vision–language, pathology vision–omics, and general-purpose vision pretraining paradigms.Run calculations use 30 models, 3 magnifications, 2 taxonomies, and 7 representation modes.Source: Evaluated foundation models · p. 5, Table 3 · p. 5
• At 20×, Nuc outperformed Mean across both taxonomies and all four protocols, while 10× showed a clearer performance decline than 20× or 40×.Under IS, performance retention relative to 40× was 99.6% at 20× and 92.9% at 10×; combining Nuc with Mean or Cls provided no consistent improvement.Source: Figure 3 · p. 6, Multidimensional Analysis of… · p. 6

Within-Section Evaluation of Cell Representation Advantage

• H-Optimus-1 achieved the highest CRA, 10.14±6.05%, significantly outperforming all 29 competing models.CRA averages matched Nuc-minus-Mean differences within each organ, then weights the 11 organs equally; CRA captures nucleus-anchored advantage, complementary to absolute Nuc performance as a measure of decodable cell-type information.Source: Table 4 · p. 6, Within-Section Evaluation of Cell… · p. 6

Cross-Domain Evaluation of Cell Representation Transferability

• SEAL(UNI2) and UNI2 tied for the highest CRT at 26.33, followed by H-Optimus-1 at 25.33 and H-Optimus-0 at 24.33.UNI2, SEAL(UNI2), H-Optimus-0, and H-Optimus-1 formed the leading group across IOCD, MOCV, and LOOO; CRT summarizes significant wins and is panel-relative, not an absolute transfer-performance measure.Source: Table 5 · p. 7, Cross-Domain Evaluation of Cell… · p. 7
• IOCD tests held-out sections within known organs, MOCV tests held-out sections after multi-organ training, and LOOO tests transfer to unseen organs.Because their training composition, test domains, and aggregation units differ, absolute scores should be interpreted within rather than directly across protocols.Source: Cross-Domain Evaluation of Cell… · p. 7

Discussion and Conclusion

• CellPath-Bench aligns WSI feature maps with registered nuclear locations and uses standardized linear probes to evaluate cell-type information across readouts, resolutions, taxonomies, and domains.It characterizes where cell-type information is encoded in pathology foundation models and how reliably it transfers across biological domains.Source: Discussion and Conclusion · pp. 7–8
• Cell-level representation capability is multidimensional: Nuc Macro-F1 measures linearly accessible cell-type information, while CRA and CRT characterize nucleus-anchored advantage and cross-domain competitiveness.Together, these measures provide complementary views of local cellular information and its generalization.Source: Discussion and Conclusion · p. 7
• The benchmark is limited to frozen linear probing and discrete cell-type classification, and CRT depends on the evaluated model panel and statistical power.Future extensions may incorporate continuous cell states, molecular phenotypes, and additional cell-level tasks.Source: Discussion and Conclusion · p. 7

A. Evaluated Foundation Models

• Table 6 lists the 30 foundation models evaluated in CellPath-Bench.The passage identifies the panel size but does not provide model names.Source: Table 6 · p. 11

B. Dataset Composition and Cell-Type Annotation

• Of 52 Xenium slides, 51 retained at least two Coarse classes for ranking, and the 25-slide CellPath-Bench panel is separated from remaining candidates.The acute lymphoid leukemia slide was excluded from ranking because its labels collapsed to one Coarse class, making balanced accuracy undefined.Source: Table 7 · p. 12
• The panel taxonomy maps cell types through nine Fine classes to three Coarse classes, with pooled Fine-class cell counts and percentages shown.Figure 4 also represents each Fine class’s abundance with a horizontal bar.Source: Figure 4 · p. 14
• The finalized annotation taxonomy contains 72 labels, with primary and secondary markers retained for 68 biological cell types; four state or quality-control labels have none.The four labels are necrotic, apoptotic, mitotic, and uncertain cells.Source: Table 8 · p. 15

Intra-section results

• Complete intra-section results cover 30 models across four nested panels, three magnifications, three metrics, and two taxonomies.The tables are organized by panel, magnification, metric, and taxonomy; the panel sizes range from S25 (25 sections across 11 organs) to S10 (10 sections across 6 organs).Source: C. Complete Experimental Results · p. 26, Table 9 · p. 27

Taxonomies

• Tfine uses a 9-class taxonomy and Tcoarse its 3-class coarsening, with classes scored only when they meet training and test cell-count thresholds.For intra-section Tfine, the thresholds are 500 training and 200 test cells; for Tcoarse, they are 100 of each, so scored class counts can vary by evaluation unit.Source: C. Complete Experimental Results · pp. 26, 63

Readouts and CRA

• Readouts include nucleus-centred sampling, patch mean pooling, native class tokens, and additive or concatenative fusions; CRA measures paired nucleus-minus-mean performance differences.For intra-section tables, Wins counts models significantly exceeded in CRA using a two-sided Wilcoxon signed-rank test with p < 0.05 and a positive median difference.Source: C. Complete Experimental Results · p. 26

Aggregation

• Intra-section values are organ-balanced means, while reported standard deviations measure between-section variability; all values are in percentage points.Runs are averaged over folds within each section, then over sections within each organ, then over organs.Source: C. Complete Experimental Results · p. 26

Cross-section results

• Cross-section results cover IOCD, MOCV, and LOOO transfer protocols across magnifications, metrics, and taxonomies, with three seeds per protocol.IOCD holds out 23 sections for intra-organ cross-dataset testing, MOCV pools five folds across organs, and LOOO holds out each of 11 organs; results are balanced over protocol evaluation units.Source: C. Complete Experimental Results · pp. 62–63

• Wins counts how many of the other 29 models a model significantly exceeds on Nuc, using paired per-unit Wilcoxon signed-rank tests with p < 0.05 and a positive median difference.The maximum is 29; each test uses the metric and taxonomy of its own table.Source: C. Complete Experimental Results · p. 63
• The captions report complete IOCD results at 40× for Tfine Macro-F1 and Macro-AUROC, and Tcoarse Macro-F1; at 20× for Tfine Macro-F1; and at 10× for Tfine Macro-F1 and Tcoarse Macro-AUPRC.Source: Table 81 · p. 63, Table 82 · p. 64, Table 83 · p. 64
• MOCV complete results are reported across several table conditions: 40× Tfine and Tcoarse Macro-F1, 40× Tfine Macro-AUROC, 20× Tfine Macro-F1, and 10× Tfine Macro-F1 or Tcoarse Macro-AUPRC.Source: Table 99 · p. 72, Table 100 · p. 73, Table 101 · p. 73
• Complete LOOO results are reported for 40× Tfine and Tcoarse Macro-F1, 40× Tfine Macro-AUROC, 20× Tfine Macro-F1, and 10× Tfine Macro-F1 and Tcoarse Macro-AUPRC.Source: Table 117 · p. 81, Table 118 · p. 82, Table 119 · p. 82