PaperlayerView exact revision on arXiv

Source-linked AI summary

Can Protein-Derived Knowledge Improve Pathology Foundation Models?

Di Zhang, Zhangpeng Gong, Jiashuai Liu, Zhi Zeng, Jiusong Ge, Chunze Yang, Xitong Ling, Kai Yi, Kai He, Weimiao Yu, Mireia Crispin-Ortuzar, Chen Li, Zeyu Gao

arXiv:2609.33178v1cs.CV

TL;DR

Existing molecular encoders represent protein sequences or single-cell states rather than the sample-level bulk profiles paired with WSIs, and cross-modal supervision is restricted to paired samples. ProSlide combines independent proteomic pretraining with protein-supervised slide pretraining and relational distillation; across 12 tasks it achieves the highest mean accuracy and AUC within each of three cancer groups.

  • Problem

    Existing molecular encoders do not represent the sample-level bulk protein profiles paired with WSIs, and paired-only cross-modal supervision limits direct pathology training against standalone profiles.

    Source: §1 · p. 2

  • Method

    ProSlide uses independently pretrained PFE, protein-expression-supervised slide pretraining, and Prot2Path to align WSI and protein similarity distributions over paired and standalone references.

    Source: §1 · p. 2, §3.2 · p. 6, §3.3 · p. 7

  • Results

    Across 12 tasks, ProSlide achieves the highest mean accuracy and AUC within each of the breast, lung, and renal cancer groups.

    Source: §1 · p. 3, §5 · p. 11

  • Takeaways & Limitations

    Evaluation across 12 tasks supports the value of standalone proteomics for data-efficient pathology representation learning.

    Source: §5 · p. 11

Abstract

from arXiv · show

Molecularly guided pathology foundation models (PFMs) exploit transcriptomic or proteomic information to enrich whole-slide image (WSI) representations, yet effectively leveraging large standalone molecular corpora remains challenging. First, existing molecular foundation models encode protein sequences or single-cell states, not the patient-level bulk expression profiles paired with WSIs. Second, because cross-modal supervision is restricted to paired WSI-omics samples, knowledge from standalone molecular corpora reaches the pathology encoder only indirectly, creating a paired-support bottleneck. To address these challenges, we propose a three-stage framework that decouples proteomic knowledge acquisition from cross-modal transfer, yielding ProSlide, a slide-level hierarchical pathology foundation model. First, to close the modality gap, we pretrain a Proteomic Foundation Encoder (PFE) on 12,695 sample-level bulk protein profiles using virtual profile generation and expression-space multi-view pretraining. Second, we pretrain ProSlide, a patch-region-slide encoder, to predict protein expression from paired WSI-protein samples. Third, to relax the paired-support bottleneck, we introduce Prot2Path, a cross-modal relational distillation objective. For each paired sample, it aligns the similarity distributions of the WSI and its protein profile over a shared, frozen bank of PFE-encoded paired and standalone profiles. We evaluate ProSlide on 12 downstream tasks across breast, lung, and renal cancers. Despite being pretrained with only 2,229 WSIs and 12,695 sample-level protein profiles, ProSlide achieves the highest mean accuracy and AUC within each cancer group.

1 Introduction

Molecular supervision can enrich pathology representations, but existing molecular encoders and paired-only cross-modal training leave gaps in scale and support. ProSlide addresses these gaps with independent proteomic pretraining and relational distillation, achieving the highest mean accuracy and AUC across three cancer groups.

1 Introduction

  • Molecular profiles provide complementary biological supervision that connects tissue morphology with underlying molecular characteristics.Prior molecularly guided PFMs incorporate such information to enrich WSI representations.Source: §1 · pp. 1–2
  • Protein language models encode sequence-derived properties and single-cell models characterize cellular states, whereas WSI-associated proteomics measures sample-level bulk protein abundance.This granularity mismatch motivates learning a sample-level proteomic representation aligned with pathology supervision.Source: §1 · p. 2
  • Conventional cross-modal learning uses molecular representations only for samples paired with WSIs, leaving standalone profiles outside the pathology training objective.This paired-support bottleneck means pathology training does not directly preserve relationships to molecular states outside the paired cohort.Source: §1 · p. 2
  • ProSlide separates proteomic knowledge acquisition from transfer through PFE pretraining, protein-expression-supervised slide pretraining, and Prot2Path relational distillation.Prot2Path aligns ProSlide with PFE-defined relations over a frozen bank of paired and standalone protein profiles.Source: §1 · p. 2
  • Across 12 tasks, ProSlide achieves the highest mean accuracy and AUC within each of the breast, lung, and renal cancer groups.It was pretrained with 2,229 WSIs and 12,695 sample-level protein profiles.Source: §1 · p. 3

2 Related Work

Prior pathology work incorporates paired molecular supervision, while proteomic representation learning spans sequence and single-cell expression levels. Prot2Path instead transfers relations from an independently pretrained bulk-proteomics encoder using paired and standalone profiles.

Molecularly Guided Pathology Representation Learning

  • Molecularly guided approaches use paired WSI–omics data; Threads incorporates genomic and transcriptomic supervision, while CARE uses RNA and protein profiles.ProSlide complements these approaches by independently pretraining a proteomic encoder on standalone data before transferring knowledge through paired samples.Source: §2 · p. 3

Protein and Proteomic Representation Learning

  • Protein representation methods include sequence-based ESM-2 and single-cell scPROTEIN, whereas PFE targets bulk profiles of continuous measurements across identified proteins.PFE combines virtual profile generation with identity-preserving expression perturbations for subsequent pathology distillation.Source: §2 · p. 3

Knowledge Distillation

  • Prot2Path transfers relations from an independently pretrained proteomic teacher by aligning protein- and WSI-derived similarity distributions over shared paired and standalone references.This contrasts with other described distillation approaches, including pathology-teacher consolidation and genomic supervision through a memory for histology-only inference.Source: §2 · p. 3

3 Method

The method separates proteomic representation learning from cross-modal transfer in a three-stage framework: PFE learns from bulk protein profiles, ProSlide predicts protein expression from paired WSIs, and Prot2Path transfers proteomic relations.

3 Method

  • The three-stage framework first pretrains PFE on standalone protein profiles, then trains ProSlide with paired WSI–protein data, and finally transfers PFE relations through a shared bank.The bank contains both paired and standalone protein profiles.Source: §3 · pp. 3–4, Figure 2 · p. 5

3.1.1 PFE Architecture

  • PFE represents each bulk protein profile using protein identity, normalized expression, and an indicator of whether each measurement is observed.An L-layer Transformer produces profile-level and contextualized protein-level representations.Source: §3.1.1 · p. 4

3.1.2 Expression-Space Multi-View Pretraining

  • PFE uses virtual profile generation and expression-space multi-view pretraining, with complementary self-supervised objectives at profile and protein levels.Virtual augmentations change a small subset of observed values while preserving identities, missingness, and unselected values.Source: §3.1.2 · pp. 4–5

3.2 Stage II: Protein-Supervised ProSlide Pretraining

  • ProSlide is pretrained to predict protein expression from paired WSI–protein samples, then initialized for Prot2Path distillation.Its patch-to-region-to-slide hierarchy uses pre-extracted features from a frozen CONCH v1.5 patch encoder; the regression heads are removed after pretraining.Source: §3.2 · p. 6

3.3 Stage III: Prot2Path Cross-Modal Knowledge Distillation

  • Prot2Path aligns proteomic and pathology similarity distributions over the same sampled reference profiles to transfer PFE-defined neighborhoods.The reference bank includes paired and standalone profiles, allowing profiles without corresponding WSIs to contribute to the relational target.Source: §3.3 · pp. 6–7

4 Experiments

ProSlide is evaluated on WSI-level tasks across breast, lung, and renal cancers using frozen slide representations and comparisons with pathology foundation models. The experiments report strong cross-cancer results, examine the contributions of proteomic pretraining and distillation, and qualitatively assess attention heatmaps.

4.1 Experimental Setup

  • The benchmark covers 12 WSI-level classification tasks across eight cohorts, with four tasks each for breast, lung, and renal cancers.Stages II and III use TCGA for WSI pretraining, while downstream evaluations use non-TCGA cohorts; balanced accuracy and AUROC are reported.Source: §4.1 · pp. 8–9

4.3 Ablation Studies

  • Protein-expression supervision yields higher-rank representations than subtype supervision, and combining it with Prot2Path achieves the best ACC and AUC across all three cancer groups.Subtype-supervised representations remain low-rank after distillation; the combined approach outperforms either stage alone.Source: Table 3 · p. 9, §4.3 · p. 10

4.2 Main Results

  • ProSlide achieves the highest mean ACC and AUC across renal, breast, and lung cancer groups, although cross-cohort binary lung subtyping shows endpoint- and metric-dependent transferability.The comparison uses frozen slide representations against mean pooling and five pathology foundation models.Source: §4.2 · p. 9

4.3 Ablation Studies

  • PFE provides stronger distillation supervision than Protein-BERT even without virtual profiles, while moderate virtual profile generation improves retrieval and downstream performance.The largest virtual batch has the highest downstream AUC despite weaker retrieval, so retrieval accuracy alone does not characterize teacher value.Source: §4.3 · p. 9
  • Prot2Path outperforms hard matching and paired-bank KD in ACC and AUC across all cancer groups, while reference sampling trades bank size against predictive performance.Its ACC gains over direct alignment are consistent, but AUC favors different methods across cancer groups.Source: Table 4 · p. 10, §4.3 · p. 10

4.4 Interpretability Analysis

  • Across three representative CPTAC-BRCA cases, ProSlide attention favors relatively tumor-rich or better-preserved tissue and broadly overlaps localized tumor annotations.The examples also indicate sensitivity to tissue composition and image quality.Source: §4.4 · p. 10, Figure 3 · p. 11

5 Conclusion

The conclusion presents a three-stage framework combining independent proteomic pretraining, protein-supervised ProSlide pretraining, and Prot2Path relational distillation. Evaluation across 12 breast, renal, and lung cancer tasks supports the approach and the value of standalone proteomics for data-efficient pathology representation learning.

5 Conclusion

  • The framework combines independent PFE pretraining, protein-supervised ProSlide pretraining, and Prot2Path relational distillation.Standalone protein profiles contribute to proteomic representation learning and to the distillation reference bank.Source: §5 · p. 11
  • Evaluation across 12 tasks in breast, renal, and lung cancers supports the effectiveness of the approach.Source: §5 · p. 11
  • The conclusion highlights standalone proteomics as valuable for data-efficient pathology representation learning.Source: §5 · p. 11

Supplementary Material

The supplementary material describes harmonized RPPA data used for PFE pretraining, represented with a common panel of 38 cancer-related protein measurements.

Supplementary Material

A.1 Proteomic Data Resources

PFE is built from harmonized bulk-protein profiles and learns representations through expression-space augmentation and self-distillation. ProSlide then aggregates pathology features hierarchically and trains them with protein-expression supervision.

A.1 Proteomic Data Resources

  • The 13,586-profile corpus combines patient tumors and cancer-cell-line resources; 12,695 profiles train PFE and 891 TCGA profiles are held out.The resources are TCGA, TCPA, CCLE, and CPPA, including pharmacologically perturbed cell-line profiles.Source: §A.1 · p. 16

A.2 Cross-Resource Protein Harmonization; A.4 Selection of the 38-Protein Panel

  • A fixed panel of 38 cancer-related targets harmonizes protein names and preserves distinct phosphorylation sites as separate targets.The panel comprises 32 distinct proteins and six site-specific phosphorylation measurements.Source: §A.2 · p. 17, §A.4 · p. 17, Table S2 · p. 18

A.3 Missing Measurements and Expression Normalization

  • PFE marks unavailable measurements as missing, retains each resource’s processed scale, and standardizes valid values using protein-specific statistics during training.External cell-line and perturbation profiles are filtered by protein coverage to limit extremely sparse inputs.Source: §A.3 · p. 17

B Proteomic Pretraining; B.1 Details of Virtual Profile Generation

  • Virtual Profile Generation expands the effective cohort, while expression-space views preserve protein identities and perturb continuous measurements because bulk profiles lack spatial structure.The perturbations include posterization, Gaussian noise, and partial mixup; selected expression values change without altering identities or missingness.Source: B Proteomic Pretraining · p. 19, §B.1 · pp. 19–20

B.2 Details of Expression-Space Multi-View Construction

  • PFE constructs clean and augmented profile-wide views plus subset-perturbed views, with the student receiving the additional local views.All views retain the full set of protein identities, and transformations apply to observed measurements.Source: §B.2 · pp. 20–21

B.3 Profile- and Protein-Level Learning Objectives

  • Student–teacher self-distillation aligns profile representations across views and masked protein representations, while excluding virtual profiles from protein-level targets.The profile objective uses real and virtual profiles; protein-level distillation is computed only for real profiles, and the overall objective also includes a KoLeo term.Source: §B.3 · pp. 21–23

B.4 PFE Pretraining Configuration; B.5 Comparison of Protein Modeling Paradigms

  • PFE uses a seven-layer Transformer and is pretrained for 100 epochs; its expression-space self-distillation differs from masked prediction and graph contrastive learning.The stated configuration uses four NVIDIA RTX 4090 GPUs; the authors describe its objectives as promoting consistency under quantization, localized noise, and partial masking.Source: §B.4 · p. 23, §B.5 · p. 24

C.1 ProSlide Architecture

  • ProSlide encodes frozen patch features through region and slide representations, then predicts protein expression from its slide representation.The hierarchy uses 4 × 4 patch regions; the RSA and SSA encoders and expression-regression heads are trained while the CONCH v1.5 patch encoder remains frozen.Source: §C.1 · p. 25

C.2 Prot2Path Implementation Details

Prot2Path compares WSI and protein representations against a shared reference bank to transfer inter-sample relations. The supplementary material also describes additional evaluation and analysis areas.

C.2 Prot2Path Implementation Details

  • Prot2Path uses a frozen PFE-encoded bank of paired and standalone profiles and minimizes teacher-to-student KL divergence between cosine-similarity distributions.The reference pool contains 12,695 profiles: 1,338 paired training profiles and 11,357 standalone profiles; both temperatures are fixed at 0.05.Source: §C.2 · p. 25

D Downstream Evaluation and Additional Analyses

  • The additional analyses cover pretraining data efficiency, inference GPU memory use, and model interpretability alongside downstream evaluation.The passage identifies these analysis topics but does not report their findings.Source: D Downstream Evaluation and Additional… · p. 26

D.1 Datasets and Task Definitions

The benchmark spans 12 WSI-level tasks across renal, breast, and lung cancers, covering molecular prediction and histological subtyping. Evaluation includes patient-level repeated splits, cross-cohort tests, and comparisons with multimodal and visual pathology baselines.

D.1 Datasets and Task Definitions

  • Tasks 4, 11, and 12 evaluate cross-cohort transfer, including fixed external test cohorts for renal BAP1 and lung subtyping.Task 10 pools Local-LUNG and DHMC-LUNG samples for fine-grained lung adenocarcinoma subtyping.Source: §D.1 · pp. 26–27
  • The 12-task benchmark comprises nine molecular prediction tasks and three histological subtyping tasks across renal, breast, and lung cancers.Molecular tasks include mutation and receptor-status prediction; subtyping includes pooled and cross-cohort settings.Source: §D.1 · pp. 26–27
  • Splits are patient-level, keeping all WSIs from each patient in one partition; several tasks use repeated 80/20 train/test splits without a validation set.For Tasks 4, 11, and 12, test patients are fixed and the development pool is divided into training and validation sets at 3:1.Source: §D.1 · pp. 27–28

D.2 Evaluation Protocols and Baselines

  • The frozen-feature linear-probing comparisons include multimodal models CARE, TITAN, and PRISM, alongside visual self-supervised and supervised baselines.The described visual baselines include GigaPath and FEATHER-24K.Source: §D.2 · p. 28

D.3 Detailed Downstream Results

Across the task-level results, ProSlide leads on two specified endpoints but does not lead every molecular task. Its cross-cohort lung-subtyping results differ between AUC and ACC.

D.3 Detailed Downstream Results

  • Other encoders lead on several CPTAC molecular tasks, so ProSlide’s relative performance varies across endpoints.The supplied task-level passage does not identify the leading encoders or give their values.Source: §D.3 · p. 28
  • On cross-cohort binary lung subtyping, ProSlide has competitive AUC but weaker ACC, underscoring the need to consider both metrics.The passage does not provide the metric values.Source: §D.3 · p. 28
  • ProSlide achieves the highest ACC and AUC on SETD2 mutation prediction and cross-cohort fine-grained lung subtyping.Table S10 reports logistic-regression results across the selected 12-task set as mean±standard deviation.Source: §D.3 · p. 28, Table S10 · p. 29

D.4 Data Efficiency and Inference Memory Usage

ProSlide combines strong aggregate performance at limited slide-level pretraining scale with modest inference-memory growth across evaluated slide sizes.

D.4 Data Efficiency

  • With 2,229 WSIs, ProSlide achieves the highest aggregate performance among evaluated models in the data-efficiency comparison.The count refers to slide-level pretraining; ProSlide also draws on pretrained CONCH v1.5 features and PFE-derived proteomic knowledge.Source: Figure S1 · p. 30, §D.4 · p. 30

D.4 Inference Memory Usage

  • ProSlide’s inference memory grows modestly across evaluated patch counts, remaining below TITAN, PRISM, and GigaPath and below CARE at larger counts.FEATHER has the lowest memory usage among evaluated models.Source: §D.4 · p. 30, Figure S1 · p. 30
  • Hierarchical aggregation summarizes patch features into region representations before slide-level attention, consistent with ProSlide’s memory behavior.Source: §D.4 · p. 30

D.5 Analysis of PFE Protein Identity Embeddings

PFE protein identity embeddings show selected similarity patterns that qualitatively correspond to known biological relationships.

D.5 Analysis of PFE Protein Identity Embeddings

  • The clustered heatmap compares pairwise cosine similarities among 38 learned protein identity embeddings, with warmer colors indicating higher similarity.Both axes use the same clustering order, and the input embeddings are shared across samples.Source: §D.5 · p. 31
  • PDCD1 and CTLA4 have high embedding similarity and encode inhibitory immune-checkpoint receptors involved in T-cell regulation.Source: §D.5 · p. 31
  • BRCA2 and ATR also show relatively high similarity, alongside their stated functional connection through DNA damage signaling and repair.ATM/ATR-mediated PALB2 phosphorylation promotes RAD51 function within the PALB2–BRCA2 repair pathway.Source: §D.5 · p. 31

Related papers

Loading 2609.33178v1…