PaperlayerView exact revision on arXiv

Source-linked AI summary

MOOZY: A Patient-First Foundation Model for Computational Pathology

Yousef Kotp, Vincent Quoc-Huy Trinh, Christopher Pal, Mahdi S. Hosseini

arXiv:2603.27048v3cs.CV

TL;DR

Computational pathology lacks WSI foundation models that jointly represent multiple patient slides and transfer across diverse clinical tasks. MOOZY uses patient-level case transformers with staged self-supervision and multi-task alignment, achieving the strongest macro weighted F1 and balanced accuracy across sixteen held-out tasks among slide encoders.

  • Problem

    WSI representations must transfer across cancer types, endpoints, and patient populations while jointly modeling intra-slide structure and multiple slides from each patient.

  • Method

    MOOZY uses a patient-first case transformer, decoupling vision-only slide pretraining from case-aware semantic alignment with multi-task supervision.

  • Results

    Across sixteen held-out tasks, MOOZY achieves the strongest macro weighted F1 (0.769) and balanced accuracy among slide encoders, and exceeds all trained MIL baselines across three macro metrics.

  • Takeaways & Limitations

    The results support patient-level pretraining as a source of competitive, transferable, and parameter-efficient pathology representations using public data.

  • Takeaways & Limitations

    Existing multi-slide approaches often rely on simple fusion heuristics that treat cases as unordered pools and may discard cross-slide interactions.

Abstract

from arXiv · show

Computational pathology needs whole-slide image (WSI) foundation models that transfer across diverse clinical tasks, yet current approaches remain largely slide-centric, often depend on private data and expensive paired-report supervision, and do not explicitly model relationships among multiple slides from the same patient. We present MOOZY, a patient-first pathology foundation model in which the patient case, not the individual slide, is the core unit of representation. MOOZY explicitly models dependencies across all slides from the same patient via a case transformer during pretraining, combining multi-stage self-supervision with scaled low-cost task supervision. In Stage 1, we pretrain a vision-only slide encoder on 77,134 public slide feature grids using masked self-distillation. In Stage 2, we align these representations with clinical semantics using a case transformer and multi-task supervision over 333 tasks from 56 public datasets, including 205 classification and 128 survival tasks across four endpoints. Across sixteen held-out tasks, MOOZY improves macro weighted F1, balanced accuracy, and macro weighted ROC-AUC relative to PRISM by +4.19\%, +7.93\%, and +6.95\%, respectively. MOOZY is also parameter efficient with 85.77M parameters, 14$\times$ smaller than GigaPath. These results suggest that patient-level pretraining yields transferable embeddings, providing a path toward scalable patient-first histopathology foundation models.

1 Introduction

Computational pathology needs transferable WSI representations that generalize across organs, scanners, endpoints, and patient populations, but existing approaches remain task-specific, proprietary, slide-centric, and weak at modeling multi-slide cases. MOOZY addresses these limitations with patient-first case modeling, two-stage pretraining, public multi-task supervision, and evaluation across held-out tasks.

  • Motivation: Computational pathology seeks WSI representations that transfer across cancer types, clinical endpoints, and patient populations without task-specific retraining.Historically, supervised pipelines must be rebuilt when the organ, scanner domain, or clinical objective changes.
  • Limitations: Existing foundation models often rely on proprietary data, unreleased checkpoints or recipes, and heavy tile encoders despite long-range WSI context being the central challenge.These limitations reduce transparency, scalability, and reuse.
  • MOOZY design: MOOZY makes the patient case, rather than an individual slide, the fundamental representation unit and explicitly models dependencies across all patient slides with a case transformer.A lightweight patch encoder compresses gigapixel slides into feature grids.
  • Training framework: MOOZY uses two stages that separate vision-only slide self-supervised pretraining from patient-aware semantic alignment through a case-level aggregator.The framework is presented as moving beyond naive early and late multi-slide fusion.
  • Supervision: 333 tasks from 56 public datasets span classification and four survival endpoints across 23 anatomical sites without private slides, paired reports, or expert annotations.The supervision regime supports scalable patient-aware semantic alignment using entirely public data.
  • Evaluation: MOOZY is evaluated on sixteen held-out tasks against slide encoders and MIL baselines, with attention-map analysis and embedding visualization assessing transferability and parameter efficiency.The evaluation combines quantitative and qualitative analyses.

2 Related Work

Related work spans pathology patch encoders, multi-instance learning, and slide-level pretraining. Existing methods improve representations and aggregation but remain largely slide-centric and task-specific, motivating universal pretrained slide representations.

  • Pathology Patch Encoders: Pathology patch encoders increasingly use DINOv2-style self-distillation, masked image modeling, regularization, and density-based objectives, alongside vision–language alignment and knowledge distillation.Architectures have scaled from ViT-Large to ViT-Huge and ViT-Giant using both proprietary and public data.
  • Multi-Instance Learning: Multi-instance learning represents a whole-slide image as a bag of patch features with one slide-level label, using pooling, attention, transformers, augmentation, and efficient approximations.These aggregators are trained from scratch for each task, motivating universal pretrained slide representations.
  • Slide Encoders: Slide-level pretraining operates on unordered sets of thousands of heterogeneous tile embeddings through vision-only methods including self-distillation, contrastive sampling, transformations, masked autoencoders, contextualizers, and state-space learning.These methods include dilated attention, lightweight contextualization, and contrastive objectives.
  • Slide Encoders: Multimodal slide encoders align slide representations with clinical text, genomic or transcriptomic profiles, and cross-stain sections.These directions complement vision-only slide-level pretraining.

3 Methodology

MOOZY separates representation learning from clinical semantic alignment in a two-stage framework: a self-supervised slide encoder is followed by case-level aggregation and multi-task prediction. The resulting case-centric embedding applies consistently across single- and multi-slide cases while supporting classification and survival objectives.

  • Stage 1: Self-Supervised Slide Encoder Pretraining: Stage 1 pretrains a slide encoder on spatial feature grids using multi-scale crops, block masking, EMA-teacher CLS distillation, and masked patch prediction.The total Stage 1 objective combines global CLS distillation and masked image modeling losses.
  • Stage 1: Self-Supervised Slide Encoder Pretraining: The slide encoder is a Vision Transformer adapted to precomputed feature grids, using projected patch features, [CLS] and register tokens, mask embeddings, and 2-D ALiBi spatial biases.Background-involving attention pairs receive an additive −∞ mask, while [CLS] and register tokens remain spatially neutral.
  • Stage 2: Case-Aware Semantic Alignment: Stage 2 decouples general-purpose slide representation learning from semantic alignment by steering the pretrained encoder toward clinical utility through multi-task supervision.This design contrasts with task-specific MIL pipelines that jointly learn aggregation and task adaptation from task labels.
  • Stage 2: Case-Aware Semantic Alignment: A lightweight case transformer prepends a learnable [CASE] token to all slide embeddings and produces one case representation, including when a case contains a single slide.Applying the aggregator uniformly keeps the embedding space case-centric and consistent regardless of slide count at inference.
  • Stage 2: Case-Aware Semantic Alignment: Task-specific linear or MLP heads support classification with weighted cross-entropy and survival prediction with a discrete-hazard objective, while averaging losses over currently supervised tasks.Unlabeled tasks are excluded per case, enabling sparse multi-task supervision; task heads are discarded at inference.

4 Experiment Setup

MOOZY is evaluated on 56 open-source datasets spanning 23 anatomical sites, with Stage 1 using 77,134 slide feature grids and Stage 2 using 333 supervised tasks. The experiments include both slide- and case-level supervision, with case-level training aggregating all slides through a case transformer.

  • Dataset: 56 open-source datasets cover 23 anatomical sites and are processed with AtlasPatch for tissue segmentation using SAM2.The collection includes REG, TCGA, CPTAC, BC-Therapy, BRACS, CAMELYON17, and other public datasets.
  • Dataset: 77,134 slide feature grids comprise the Stage 1 self-supervised pretraining data, including 53,286 at 20× and 23,848 at 40× magnification from approximately 31.8 TB of raw WSI data.The two magnification-specific grids are sampled uniformly as independent training samples.
  • Dataset: 333 Stage 2 tasks span 205 classification and 128 survival tasks across 56 datasets, covering OS, DSS, DFI, and PFI when endpoints are available.The task set averages 6 tasks per dataset.
  • Dataset: 41,089 supervised cases and 45,179 unique whole-slide images form the labeled Stage 2 subset, with 56 slide-level and 277 case-level tasks.The labeled slides are a subset of the 53,286 Stage 1 slides because unlabeled slides are excluded; the mean per-task slide-to-case ratio is 1.3308.
  • SSL Pretraining: Stage 1 trains a 6-layer, 12-head slide transformer for 200 epochs on 8 GPUs with an effective batch of 1,024 slides.Training uses 14,400 optimizer steps and approximately 436 GPU-hours.
  • Case-Aware Semantic Alignment: Stage 2 trains a 3-layer, 12-head case transformer for 20 epochs on 8 GPUs with an effective batch of 1,024 cases, pooling all slides in each supervised case.The case transformer uses a learnable [CASE] token; training takes 1,000 optimizer steps and approximately 512 GPU-hours.

5 Results

MOOZY is evaluated on sixteen held-out tasks spanning diverse clinical settings and achieves the strongest macro weighted F1 and balanced accuracy across all tasks among compared slide encoders. Its model totals 85.77M parameters, including separate slide/case and patch encoder components.

  • Evaluation Tasks: The evaluation spans sixteen held-out tasks covering diverse clinical settings, including mutations, treatment response, molecular subtypes, grade, and primary-versus-metastatic classification.Tasks include Residual Cancer Burden, TP53, BAP1, ACVR2A, KRAS, and IDH status, among others.
  • Comparison with Slide Encoders: Across all sixteen held-out tasks, MOOZY achieves the strongest macro weighted F1 (0.769) and balanced accuracy (0.702), while TITAN leads weighted ROC-AUC (0.768 versus 0.763).For case-level tasks, baseline encoders average per-slide embeddings, whereas MOOZY uses its native case-level embedding from the case transformer.
  • Parameter Efficiency: 85.77M parameters comprise 64.10M for the slide and case encoder and 21.67M for the patch encoder.The reported total model size is described as 4–14× smaller than comparison models, though the supplied passage does not identify them.

9 RC Burden is case-level task, but the slide to case ratio is 1.0

MOOZY’s case-level modeling improves held-out-task performance over MIL and slide-pooling alternatives, while remaining parameter-efficient and producing diagnostically broad, balanced attention patterns. Qualitative analyses further show strong class separation across representative tasks.

  • MIL comparison: 0.029 weighted F1, 0.043 weighted ROC-AUC, and 0.041 balanced accuracy separate MOOZY from the strongest MIL baseline, CONCH v1.5, across sixteen tasks.Each comparison entry averages over five MIL architectures: MeanMIL, ABMIL, CLAM, DSMIL, and TransMIL.
  • Case aggregator ablation: 0.769 macro weighted F1, 0.763 macro weighted ROC-AUC, and 0.702 balanced accuracy are achieved by full MOOZY after adding the case aggregator.Without the aggregator, the corresponding values are 0.749, 0.737, and 0.682.
  • Case aggregator ablation: 14 of 16 tasks improve in F1, 12 in AUC, and 13 in balanced accuracy when the case aggregator is added.The aggregator compares full MOOZY with mean slide pooling during the ablation.
  • Attention-map analysis: 1.00 mean gap score and 2.63 shift score indicate that MOOZY attends broadly while maintaining near-balanced focus between tumor and surrounding tissue.TITAN had the closest-to-balance shift score at 3.13, while PRISM scored 1.75 for semantic gap and 2.38 for shift.
  • Qualitative analysis: MOOZY shows the clearest cancer-type separation on CPTAC and TCGA, while TITAN is strongest for anatomical-site classification.t-SNE and PCA stability diagnostics confirm the same qualitative pattern.

6 Conclusion … G MLP Probe Evaluation Protocol

MOOZY is a patient-first, two-stage pathology foundation model that combines vision-only self-supervision with case-aware multi-task alignment and explicit inter-slide dependency modeling. It achieves strong held-out-task performance while avoiding proprietary slides, paired reports, and billion-parameter architectures, and uses a shared evaluation protocol for representation comparisons.

  • 6 Conclusion: MOOZY uses a two-stage patient-first framework that separates vision-only SSL pretraining from case-aware multi-task alignment with explicit inter-slide dependency modeling.The framework is designed to learn competitive representations without proprietary slides, paired clinical reports, or billion-parameter architectures.
  • B.2 Stage 2 Sparse Supervision Structure: Stage 2 handles sparse supervision by computing each task’s loss only on labeled cases and averaging optimization over active tasks with usable labels.The supervision matrix marks valid case-task pairs, while slide-level datasets without case identifiers are represented as single-slide cases.
  • B.1 Training Task Distribution by Anatomical Site and Category; B.3 TCGA Task Preparation: 240 TCGA tasks include 117 classification and 123 survival tasks, spanning 32 projects plus one pan-cancer task and covering 9,732 cases and 11,857 slides.The task preparation includes slide-level and case-level labels, with case-level tumor-grade, survival, and mutation labels linked to all slides from each case.
  • B.4 REG task preparation: REG provides 8,494 report–slide pairs across seven specimen groups, yielding 36 conservatively labeled tasks whose labels are derived only from explicit report wording.The task set includes organ, procedure, diagnosis, grading, malignancy, and related attributes, with samples omitted when task-specific evidence is insufficient.
  • C.1 Spatial Grid Construction; C.2 Multi-Scale Window Validity Constraint: Spatial grids assign patches to raster positions, fill tissue-free positions with zero vectors, and track tissue presence with a binary validity mask.Multi-scale crops must satisfy a minimum valid-token fraction; failed crops are resampled up to a fixed attempt limit.
  • C.3 Block Masking Algorithm; C.4 Adaptive Token Capping (Stage 2): Stage 1 masking distributes ratios across global crops and reaches each target by placing random rectangular blocks over valid tokens, then filling any remaining budget with individual tokens.Stage 2 caps excessive valid-token counts using stratified sampling that retains exactly one token per spatial bin, preserving whole-slide coverage.
  • C.5 Survival Bin Selection and Loss; C.6 Projection Head Formulation; C.7 Task Head Formulations: Survival tasks choose discrete time bins adaptively from observed events, use quantile cut-points, merge tied cut-points, and optimize a discrete-hazard likelihood.The projection head maps embeddings through a bottleneck, L2 normalization, and a prototype layer; task heads are either dropout-regularized linear heads or LayerNorm/GELU MLPs.
  • D Augmentation Strategy Visualizations; E SSL Pretraining Hyperparameters; F Semantic Alignment Hyperparameters; G MLP Probe Evaluation Protocol: Five-fold evaluation uses label-stratified 80%/20% train-validation splits, selects models by validation weighted F1, and reports mean ± standard deviation for weighted F1, weighted ROC-AUC, and balanced accuracy.The MLP probe uses adaptive hidden sizes, AdamW, 200 epochs, batch size 64, cross-entropy, dropout 0.25, and class-balanced sampling; augmentations target orientation robustness, token regularization, and masked prediction.

H Linear Probe Setup and Results · I Encoder Parameter Comparison · J MIL-Specific Task-wise Comparison

The sections define a stratified, case-grouped linear-probe evaluation across slide encoders and MIL methods, compare encoder parameter budgets, and report macro and task-wise benchmark results. MOOZY is substantially smaller than major baselines and remains strongest on macro metrics, while task-level performance varies by architecture.

  • H Linear Probe Setup and Results: Linear probing uses multinomial logistic regression with the same five-fold, label-stratified, case-grouped splits as the MLP comparison.Each fold uses an 80% train and 20% validation split.
  • H Linear Probe Setup and Results: 10−6 to 105: L2 regularization strength is selected over 45 logarithmically spaced values by minimizing validation loss.Optimization uses LBFGS for up to 500 iterations with class-balanced weighting.
  • H Linear Probe Setup and Results: Performance is reported across folds using weighted F1, weighted ROC-AUC, and balanced accuracy, with MIL results averaged over five architectures.The architectures are MeanMIL, ABMIL, CLAM, DSMIL, and TransMIL.
  • I Encoder Parameter Comparison: 85.77M total parameters: MOOZY is substantially smaller than GigaPath at 1.22B and PRISM at 742.06M.The comparison separates slide-encoder, patch-encoder, and total parameter counts.
  • I Encoder Parameter Comparison: MOOZY uses a comparatively lightweight patch encoder and compact case-level slide stack, whereas most baselines place most parameters in the patch encoder.The breakdown makes capacity allocation explicit across compared slide encoders.
  • J MIL-Specific Task-wise Comparison: Sixteen held-out tasks: MOOZY remains strongest on macro-average metrics, while task-wise results vary across architectures.Variation is particularly evident for Synaptophysin Grade and Non-Keratinizing SCC Grade; the tables provide full per-task and per-method breakdowns.

K Stage 1 Only and MOOZY Task-wise Comparison

MOOZY consistently outperforms the Stage 1-only model across sixteen tasks, improving all three reported metrics on eleven tasks and achieving its largest reported gain on UCEC Genomic Subtype F1.

  • Task-wise comparison: Across sixteen tasks, MOOZY improves weighted F1 on eleven, AUC on fifteen, and balanced accuracy on fourteen.The comparison reports five-fold means and standard deviations for Stage 1 and MOOZY, alongside relative gains for the three metrics.
  • Task-wise comparison: Eleven tasks improve on all three metrics: weighted F1, weighted ROC-AUC, and balanced accuracy.These metrics are evaluated in the task-level breakdown comparing Stage 1 and MOOZY.
  • Task-wise comparison: +12.00% is the largest reported gain, observed for UCEC Genomic Subtype F1.The passage also identifies BAP1 mutation AUC as another largest-gain category, but its value is not provided in the supplied text.

L Stage 2 Only and MOOZY Task-wise Comparison

MOOZY compares favorably with Stage 2 only, which trains the slide encoder from scratch without Stage 1 self-supervised pretraining. It improves all three metrics on fourteen of sixteen tasks, with the largest gains in treatment response and RAS/BRAF status.

  • Comparison setup: Stage 2 only trains the slide encoder with multi-task supervision from scratch, without Stage 1 SSL pretraining.This comparison isolates the contribution of Stage 1 self-supervised pretraining.
  • Largest gains: +18.37% Treatment Response F1, +17.07% balanced accuracy, and +32.76% RAS/BRAF Status AUC are the largest reported gains.These are the strongest task-metric improvements in the Stage 2 only versus MOOZY comparison.

M Stage 2 Only Without Case Aggregator and MOOZY Task-wise Comparison

MOOZY outperforms a Stage 2-only configuration that lacks Stage 1 self-supervised pretraining and replaces the case transformer with mean slide pooling. Across sixteen tasks, improvements are reported for weighted F1 on fourteen tasks, balanced accuracy on fourteen, and AUC on twelve, with twelve tasks improving on all three metrics.

  • Configuration: The comparison configuration trains the slide encoder from scratch with multi-task supervision, omitting Stage 1 SSL pretraining.It therefore evaluates Stage 2 supervision without the preceding vision-only self-supervised stage.
  • Configuration: The comparison configuration aggregates per-slide embeddings by mean slide pooling rather than using the case transformer.This removes explicit case-level aggregation from the evaluated configuration.
  • Task-wise comparison: 14 of 16 tasks improve in weighted F1 and balanced accuracy, while 12 improve in AUC versus this configuration.12 tasks improve on all three metrics simultaneously.

N Case Aggregator Ablation: Mean Slide Pooling vs MOOZY · O Attention Map Generation and Additional Examples · P Embedding Visualization Settings and t-SNE Results

The ablation shows that MOOZY’s case aggregator generally improves slide-level task performance over mean slide pooling, while the paper evaluates representations with matched intrinsic attention maps and reports t-SNE clustering patterns across datasets. These analyses indicate task gains from case-level aggregation and dataset-dependent embedding separation across models.

  • N Case Aggregator Ablation: Mean Slide Pooling vs MOOZY: Across sixteen tasks, the case aggregator improves weighted F1 on fourteen, AUC on twelve, and balanced accuracy on thirteen.The ablation compares full MOOZY with the slide encoder alone using mean slide pooling under the MLP probe protocol.
  • O Attention Map Generation and Additional Examples: Heatmaps use randomly sampled WSIs and matched visualizations across encoders with the same attribution objective and common relevance scale.The compared encoders are CHIEF, Madeleine, PRISM, TITAN, and MOOZY.
  • O Attention Map Generation and Additional Examples: Attention attribution targets the squared ℓ2 norm of each slide embedding, enabling encoder-intrinsic comparisons without downstream task heads.Each patch receives Grad×Input relevance based on the embedding’s sensitivity to perturbations of that patch.
  • O Attention Map Generation and Additional Examples: Normalized patch relevances are projected onto level-0 thumbnail boxes, averaged over overlaps, and displayed as matched slide-level overlays.Additional cross-model comparisons in Figures 11–15 illustrate the qualitative patterns across slides and cohorts.
  • P Embedding Visualization Settings and t-SNE Results: t-SNE uses perplexity 25 and 1000 iterations, while UMAP uses neighborhood size 120, minimum distance 0.3, and cosine distance.Effective sample counts are N=2152 for CPTAC cancer type, N=1172 for anatomical site, and N=1280 for TCGA cancer type.
  • P Embedding Visualization Settings and t-SNE Results: MOOZY shows the clearest separation for CPTAC and TCGA cancer type, TITAN is strongest for anatomical site, and Madeleine and PRISM have weaker boundaries.The t-SNE layouts are described as consistent with the UMAP results in Figure 6.

Q Unsupervised Embedding Geometry Analysis

MOOZY’s unsupervised embedding geometry is evaluated on 3,300 unique slides using PCA compactness and bootstrap neighborhood stability. It is the most compact encoder while maintaining neighborhood stability comparable to all baselines.

  • Geometry diagnostics: 3,300 unique slides are analyzed with PCA compactness and bootstrap neighborhood stability to assess embedding geometry.PCA compactness measures the number of orthogonal directions needed to explain variance, while bootstrap stability evaluates local-neighborhood robustness under sampling perturbations.
  • PCA compactness: 9/12/17 components at 80%/90%/95% variance make MOOZY the most compact encoder, versus TITAN (17/31/56), CHIEF (19/38/67), Madeleine (17/37/72), PRISM (22/45/81), and GigaPath (58/123/205).Lower component counts indicate less redundancy and more information-efficient embeddings.
  • Neighborhood stability: 0.7998 to 0.8032 is the stability-score span at k=30 across encoders, with a mean spread of only 0.0029 (0.8002 to 0.8031) across k.Higher µ_k indicates more stable local structure, and the encoders are described as nearly tied on this metric.
  • Overall finding: MOOZY has the strongest compactness while maintaining stability comparable to all baselines, indicating better representation efficiency without a meaningful robustness tradeoff.The stability comparison uses bootstrap neighborhood overlap after random subsampling of slides.
Loading 2603.27048v3…