PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Trustworthy Biological Alignment in TabPFN-Probed Pathology Foundation Models

Ushashi Bhattacharjee, Alloy Das, Saria Hannan, Tirtho Roy, Koushik Howlader, Soumik Sarkar

arXiv:2609.29523v1cs.MAq-bio.QM

TL;DR

Strong downstream prediction by pathology foundation models does not establish whether their representations encode genuine, robust molecular information. This paper audits frozen representations with spatially paired H&E and transcriptomics and a pretrained TabPFN probe, without task-specific updates. Decodability varies across tissues and targets, and some apparent alignment is sensitive to distribution shift or section-level shortcuts.

  • Problem

    Strong downstream prediction does not establish whether frozen pathology representations encode biologically meaningful molecular information or whether that information is reliable under variation.

    Source: §1 · p. 1

  • Method

    The framework pairs H&E images with spatially matched HEST-1k transcriptomics and uses TabPFN to decode gene and pathway targets from frozen pathology representations without task-specific updates.

    Source: §1 · p. 1

  • Results

    Across 240 samples from three organs, pathway activity was more consistently recoverable than individual-gene expression, while decodability varied by tissue and degraded under distribution shift, with section identity remaining strongly encoded.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Molecular prediction alone is insufficient to evaluate biological alignment; interpretation should consider whether decodability survives controls, shift, and shortcut tests.

    Source: §5 · p. 7

    The primary probe is TabPFN, so absolute PCC and especially label-efficiency conclusions may remain probe-family dependent; the within-section control also uses a smaller context than the matched condition.

    Source: §6 · p. 7

Abstract

from arXiv · show

Histology and transcriptomics provide complementary views of tissue biology, capturing spatial morphology and molecular activity, respectively. Pathology foundation models (PFMs) learn rich morphological representations from H&E images, yet strong downstream performance alone does not establish whether these representations encode biologically meaningful and robust molecular information. We present a **training-free framework for auditing biological alignment in frozen PFMs** using spatially paired histology and transcriptomics from HEST-1k, evaluated on **240 samples spanning three organs**. Multiple frozen PFMs are used to extract H&E representations, while gene expression is aggregated into biologically interpretable pathway-level programs. We use TabPFN as a pretrained probe to quantify the extent to which these molecular programs can be decoded from frozen image representations without task-specific gradient updates. Beyond predictive performance, our audit examines whether pathway decodability generalizes across tissue sections, patient groups, and tissue types; whether representations exhibit section-level or other shortcut dependencies; and whether predictions remain stable under small image perturbations and context resampling. This multi-tissue evaluation distinguishes molecular programs that are consistently encoded from those that are tissue-specific, unstable, or shortcut-sensitive. Our framework therefore provides a systematic approach for assessing not only **what biological information pathology foundation models encode, but also how reliably that information persists under clinically relevant sources of variation**.

1 Introduction

Strong molecular prediction by pathology foundation models does not establish that their frozen representations encode genuine biology or remain reliable under variation. This work introduces a training-free audit and finds that decodability varies by tissue and target, with pathway activity more consistently recoverable than individual genes, while shift and section shortcuts can undermine trustworthiness.

Motivation

  • Existing molecular prediction and multimodal alignment approaches train task-specific components, so their performance cannot isolate biology already present in frozen representations.Prior shortcut studies also generally examine shortcuts separately from molecular prediction.Source: §1 · p. 1

Approach

  • The audit pairs spatially matched H&E and gene expression from HEST-1k, organizes expression into pathway activities, and uses TabPFN to probe frozen PFM representations without task-specific updates.The same pipeline evaluates negative controls, distribution shifts, image perturbations, and shortcut tests.Source: §1 · p. 1, Figure 1 · p. 2

Results

  • Across 240 samples from Breast, Skin, and Brain, pathway activity was more consistently recoverable than individual-gene expression, while decodability varied by tissue.Performance degraded under distribution shift, and section identity could remain strongly encoded.Source: §1 · p. 2

Contribution

  • The contribution combines general pathology foundation models, molecularly informed histology methods, and robustness auditing in one training-free evaluation rather than claiming novelty for any individual component.The authors report that section identity is linearly decodable up to 214× chance.Source: §1 · pp. 2–3

2 Method

The method treats molecular prediction from frozen pathology embeddings as an audit of decodability, then tests whether that signal survives controls, shifts, perturbations, and shortcut checks. The encoder and probe remain fixed while contexts, targets, and evaluation conditions vary.

Preliminaries

  • Decodability is the mean per-target Pearson correlation over held-out queries, interpreted as a property of the representation and biology because the encoder and probe are fixed.The probe decodes a query embedding from labeled examples in a context set without changing either model’s parameters.Source: Preliminaries · p. 3

Training-free audit framework

  • Contexts are drawn from other specimens using random, section-balanced, diverse, or nearest-neighbor strategies, with fixed seeded sampling and no query specimen in its own context.Section-balanced contexts check whether shortcut effects depend on section-imbalanced sampling.Source: Training-free audit framework · p. 3
  • Pathway scores use ssGSEA over Reactome and Hallmark sets, while gene targets use the top 1,000 highly variable genes; different technology-specific gene panels limit absolute PCC comparisons across organs.The study instead interprets matched-versus-within-section-control contrasts within each organ; this mitigates but cannot eliminate the panel confound.Source: Training-free audit framework · pp. 3–4
  • Specimen-level cross-validation keeps each query specimen out of its context, and confidence intervals resample at the specimen level rather than treating spots from one section as independent.Comparisons are paired across matched targets, with Wilcoxon and permutation tests and Holm correction.Source: Training-free audit framework · p. 4
  • The four audit operators test negative controls, distribution shift, benign image perturbations, and section-identity shortcuts on the same frozen pipeline.The protocol also tracks prediction movement when only the context draw changes.Source: Preliminaries · p. 3, Figure 2 · p. 3, Training-free audit framework · p. 4

3 Experimental Setup

The definitive audit uses three data-rich HEST-1k organs, with additional encoders evaluated under the same protocol to assess encoder specificity.

Experimental cohort

  • The audit covers 240 samples from Breast, Skin, and Brain, selected because each has enough sections to estimate the within-section control, and repeats the protocol with five frozen encoders.UNI is the primary encoder; the other four are Phikon, Phikon-v2, Lunit DINO ViT, and CTransPath-style SimCLR ResNet.Source: §3 · p. 4

4 Results

Across five frozen encoders, the audit finds molecular decodability varies by tissue and is more consistent for pathways than individual genes. It then evaluates label efficiency, distribution shift, shortcut controls, and spatial reconstructions using the frozen UNI encoder and TabPFN probe.

Decodability across encoders and tissues

  • Pathway decodability ranks UNI (0.303) far above Phikon, Phikon-v2, and CTransPath (near 0.054) and Lunit (0.044); UNI is significantly best in every organ×target cell (all p < 10−5).This is a ranking among the five tested encoders under the frozen probe, not a general ranking of pathology foundation models.Source: Figure 3 · p. 5, Decodability across encoders and tissues · p. 5

Label efficiency and in-context scaling

  • UNI+TabPFN reaches pathway PCC 0.303 versus 0.161 for the gradient-trained MLP and matches the reported HEST-1k trained-probe reference with ≈23× fewer labeled context examples.The reference is protocol-compatible, not a hard ceiling, because published probes differ in gene selection and preprocessing.Source: Figure 4 · p. 5, Label efficiency and in-context scaling · p. 5

Robustness under distribution shift

  • Organ holdout drops UNI pathway PCC from 0.303 to 0.114 (62%); it retains roughly half its signal on Breast and Skin but collapses on Brain, while substitutes have no signal to lose.Under image perturbations, geometric transforms retain at least 95% of clean PCC 0.294, while the harshest photometric corruptions retain about 72%; prediction agreement stays above 0.84.Source: Figure 5 · p. 6, Robustness under distribution shift · p. 6

Trustworthiness controls

  • UNI’s matched pathway decodability of 0.303 drops to 0.120 within section (K=256) but remains above the 0.016 shuffle floor; substitute controls meet or exceed their matched scores.The unequal context sizes limit direct effect-size comparison.Source: Figure 6 · p. 6, Trustworthiness controls · p. 6

Qualitative spatial predictions

  • Spatial reconstructions show TabPFN most faithfully reproduces ground-truth pathway organization, with the clearest structured signal in Skin and the weakest in Brain.The representative sections use the same frozen UNI features and K=512 context across TabPFN, kNN, Ridge, and MLP.Source: Figure 7 · p. 6, Qualitative spatial predictions · pp. 6–7

5 Discussion

A high frozen-probe score shows molecular variation is accessible, not what source produced it. The authors argue that model comparisons should also test control separation, shift retention, and shortcut sensitivity.

5 Discussion

  • A high PCC establishes accessible molecular variation but does not identify its source; UNI retains a within-section residual while showing an organ-OOD drop.The authors interpret this as alignment that survives the within-section control but is not uniformly portable.Source: §5 · p. 7
  • Matched decodability should be interpreted alongside shuffled and random-feature controls and the within-section residual, which test distinct sources of predictive signal.The within-section permutation asks whether local morphology carries information beyond section membership.Source: §5 · p. 7
  • The authors recommend a compact evaluation profile rather than one leaderboard score, using pathway-level probing as a reproducible first screen and gene-level analysis after shortcut controls.The profile includes matched decodability, negative-control separation, shift retention, and the within-section residual.Source: §5 · p. 7

6 Limitations and Outlook

The audit’s conclusions are bounded by its probe, control design, cohort, and available metadata. The authors identify specific follow-ups needed to assess portability and deployment-relevant robustness.

6 Limitations and Outlook

  • TabPFN is the primary probe, so absolute PCC and especially label efficiency may be probe-specific; the within-section control also uses K=256 versus K=512 matched context.Its lower PCC mixes shortcut removal with reduced context, so the authors interpret only the residual above the shuffle floor and call for a matched-K ablation.Source: §6 · p. 7
  • Organ-OOD tests portability across biological domains, not invariance of underlying biology, and the definitive audit covers only Breast, Skin, and Brain.Organ-specific programs, differing gene panels, and technology composition may contribute to the drop; the authors call for evaluation of additional lower-resource organs.Source: §6 · p. 7
  • HEST lacks harmonized scanner, stain, and site metadata, preventing controlled evaluation of the most deployment-relevant shifts.The authors say acquisition-labeled external cohorts are required to quantify this risk directly.Source: §6 · p. 7
  • Label-permutation controls do not model spatial autocorrelation, and context-reseed stability is a sensitivity diagnostic rather than formal uncertainty analysis.The authors propose spatially aware nulls and calibrated coverage diagnostics as future work.Source: §6 · p. 7

7 Conclusion

The audit finds molecular signal in frozen pathology representations, but its decodability varies by target and tissue and does not by itself establish trustworthy biological alignment. The authors conclude that evaluation should jointly assess signal, controls, shift robustness, and shortcut sensitivity.

7 Conclusion

  • UNI loses roughly 62% of its pathway signal under organ-level shift, while section identity is linearly decodable at up to 214× chance.The within-section control retains a molecular residual above the shuffle floor, but unequal context sizes prevent a clean estimate of shortcut magnitude.Source: §7 · p. 7
  • Pathway activity is more consistently recoverable than individual-gene expression, and Breast and Skin are more decodable than Brain in this cohort.These results span five encoders and three organs.Source: §7 · p. 7
  • Matched performance remains well above label-shuffled and random-feature floors, supporting decoded signal beyond those negative-control baselines.Source: §7 · p. 7
  • Decodability alone is not a certificate of biological understanding; the authors recommend jointly evaluating molecular signal, negative-control separation, distribution-shift robustness, and shortcut sensitivity.They describe the standardized, gradient-free audit as a practical routine diagnostic for new pathology foundation models.Source: §7 · p. 7
Loading 2609.29523v1…