MOSAIC: Towards a Universal Latent Space for Computational Pathology Foundation Models

1.1. Research Questions

• The central question is whether PFMs with different architectures, objectives, data, and dimensionalities learn fundamentally different feature spaces or coordinate systems of shared representations.Prior work reports representational convergence and alignment, but functional alignment across heterogeneous pathology encoders remains largely unexplored.Source: §1 · p. 1

1. Introduction

• MOSAIC studies similarity, alignment, and interoperability across 12 patch encoders and 6 slide encoders, including downstream molecular and morphological information and transfer without retraining.The study tests whether predictors trained with one encoder can operate on another encoder’s aligned representations.Source: §1 · p. 1

RQ1: How similar are the representation spaces of different pathology encoders?

• The study asks how similar different pathology encoders’ representation spaces are, extending prior similarity work to heterogeneous PFMs.Prior studies examined representational convergence and developed metrics including CKA, CCA-based methods, and representational similarity analysis [26] [29].Source: RQ1: How similar are the… · p. 2

RQ2: What determines latent geometry: model architecture or pretraining objective?

• MOSAIC examines whether latent geometry tracks architectural family or pretraining objective, including self-supervised, vision–language, and supervised learning.Prior evidence suggests that learning objectives shape representational similarity and may yield consistency across datasets.Source: RQ2: What determines latent geometry:… · p. 2

RQ3: Can independently learned representation spaces be aligned into a shared latent space?

• The study tests whether independently learned pathology representations can be aligned into a shared latent space.Earlier work has explored alignment and translation of independently learned latent spaces, including multi-model and geometry-preserving approaches [27] [28] [40].Source: RQ3: Can independently learned… · p. 2

RQ4: Does alignment produce representations that remain useful for downstream pathology tasks?

• MOSAIC tests whether geometric alignment also preserves discriminative information for molecular and morphological prediction tasks.Prior alignment studies have evaluated task-relevant information through downstream performance.Source: RQ4: Does alignment produce… · p. 2

RQ5: Can knowledge learned using one encoder transfer to another through the shared space?

• The study tests whether predictors trained with one pathology encoder can operate on another encoder’s representation after shared-space mapping, without retraining.This evaluates interoperability through transfer rather than alignment alone.Source: RQ5: Can knowledge learned using one… · p. 2

1.2. Related Studies in CPath

• MOSAIC goes beyond earlier PFM similarity and redundancy studies by explicitly aligning heterogeneous encoders and evaluating downstream information and cross-model transfer.Prior work examined similarity across six PFMs and redundancy and complementarity across patch- and slide-level PFMs.Source: §1.2 · p. 2

2.1. Datasets and Downstream Tasks

• MOSAIC-Bench uses TCGA and CPTAC whole-slide images across breast, lung, and colon cancer, with patient-level splits for histologic and molecular prediction tasks.The cohorts comprise 2,169 TCGA and 2,162 CPTAC WSIs; tasks include histologic subtype and stage prediction and recurrent-mutation prediction.Source: §2.1 · p. 2, Table 1 · p. 2

2.2. Cross-Encoder Representation Similarity

• Cross-encoder similarity is assessed with seven complementary metrics spanning kernel, subspace, geometric, relational, and statistical notions of representation similarity.Features are centered across samples, and encoder-by-encoder matrices summarize the metrics’ complementary views of convergence.Source: §2.2 · p. 3

2.3. Multi-Encoder Shared-Space Alignment

• Five alignment strategies map matched encoder representations into a common-dimensional space using correlation-based, geometric, shared-subspace, or nonlinear formulations.The methods are GCCA, MCCA, Generalized Procrustes Alignment, Joint PCA, and a shared autoencoder; fits use training samples only and are applied unchanged to validation and test data.Source: §2.3 · p. 3

2.4. Shared-Space and Downstream Evaluation

• Evaluation measures shared-space agreement and retrieval, geometry preservation, downstream AUC under patient-level MIL, and cross-model transfer between encoder feature spaces.Geometry preservation includes nearest-neighbor overlap and reconstruction fidelity; transfer maps a source representation through the shared space into the target encoder’s native space.Source: §2.4 · pp. 3–4

2.5. Factors Governing Representation Geometry

• The analysis compares representation similarity by pretraining objective and architecture, tracks layer-wise linear CKA across normalized relative depth, and independently repeats similarity analyses at 5×, 10×, and 20× magnification.Source: §2.5 · p. 4

3.1.1. Patch-Level Representational Similarity

• For 224-px patch encoders, Linear CKA on CPTAC decreases from 0.555 at 5× to 0.498 at 20×, while PWCCA remains between 0.772–0.786.Geometry-based similarity also declines from 5× to 20× on TCGA, whereas CCA-based measures are comparatively stable.Source: §3.1.1 · p. 5, Table 2 · p. 5

3.1.2. Slide-Level Representational Similarity

• On slide-level encoders, Linear CKA averages 0.689 (range 0.548–0.810), while PWCCA averages 0.828 but correlates weakly with Linear CKA (ρ = 0.129).Given the small slide count relative to embedding dimensions, the authors primarily interpret CKA, Procrustes, cosine RSA, and distance correlation.Source: §3.1.2 · p. 5

3.2. Downstream Utility of the Shared Representation Space

• Morphological and clinical tasks show relatively small differences across representations, while direct feature concatenation does not consistently beat the best single encoder on molecular tasks.Subtyping AUCs are near 0.95–0.97; staging follows the same general pattern at lower absolute performance.Source: §3.2 · p. 6, Figure 2 · p. 6
• Approximately 0.55 to 0.63 AUC for BRCA GATA3 and 0.61 to 0.69 for COAD PIK3CA, while MOSAIC underperforms on LUAD TP53 and COAD TP53.The 64-dimensional shared space can outperform both the best individual encoder and feature concatenation on selected targets, but task-specific signals are not equally preserved.Source: §3.2 · p. 6, Figure 2 · p. 6

3.3. Layer-wise Representational Alignment

• CONCH aligns increasingly with UNI2 and GigaPath through deeper layers, less with CTransPath, while ResNet50 alignment decreases toward later CONCH layers.The CONCH–UNI2 CKA matrix also shows high similarity across intermediate and deeper layers, not only between blocks at matching depths.Source: §3.3 · p. 6, Figure 3 · p. 6

3.4. Effect of Magnification on Cross-Model Similarity

• All seven similarity metrics decrease as magnification rises; Linear CKA falls from 0.609 at 5× to 0.553 at 20×.The patch sets were independently sampled across magnifications, so this compares cross-model similarity structure rather than patch-level correspondence across scales.Source: §3.4 · p. 7, Figure 4 · p. 7

4.1. Alignment Quality

• GCCA achieves the lowest alignment error (0.030), highest paired cosine similarity (0.962), and highest shared-space Recall@1 (0.997), while Procrustes best preserves reconstruction and neighborhoods.Procrustes reaches R2 = 0.588 reconstruction quality and 0.744 neighborhood preservation, but paired cosine similarity is 0.670 and shared-space Recall@1 is 0.604.Source: §4.1 · p. 7, Figure 5 · p. 7

4.2. Cross-Encoder Retrieval

• After alignment, cross-encoder retrieval improves dramatically, with GCCA strongest and most consistent across encoder pairs; Procrustes varies more and shows directional asymmetry.Before alignment, Recall@1 is approximately the 1/5000 chance level and median ranks lie near the candidate-set midpoint.Source: §4.2 · pp. 7–8, Figure 5 · p. 7

4.3. Cross-Encoder Representation Translation

• Cross-model Recall@1 reaches 58% of the self round-trip ceiling (0.470 vs. 0.807), and translation is direction-dependent: Virchow is an easier target than H-optimus-0.The self round-trip comparison indicates that some transfer loss arises from the 64-dimensional shared-space bottleneck itself.Source: §4.3 · p. 8, Figure 6 · p. 8, Table 3 · p. 8

4.4. Architecture or Pretraining Objective Analysis

• Across six groups and seven metrics, the objective effect is positive in all 42 combinations, whereas the architecture effect is negative in seven; mean effects are +0.076 versus +0.051.The objective effect exceeds the backbone effect at every TCGA magnification, while the architecture effect can turn negative at 20×.Source: §4.4 · p. 8, Table 4 · p. 8, Table 5 · p. 8
• Patch-level agreement varies jointly with model family, subcohort, input resolution, and magnification rather than remaining fixed for an encoder pair.The 512-px CONCH and CONCH v1.5 comparison remains consistently high, while pairings in the 256-px group vary across settings.Source: §4.4 · p. 9

A.1. Patch-Level Encoders

• Patch encoders are grouped by native input resolution rather than resized to a common patch size, avoiding morphology changes and interpolation artifacts that could confound comparisons.The groups are summarized by model count and pretraining-objective distribution.Source: §A.1 · p. 11, Table S1 · p. 11

A.2. Slide-Level Encoder

• Each slide encoder retains its original or TRIDENT-supported patch encoder and corresponding patch size and magnification, preserving its intended upstream feature conditions.The patch-feature mappings are summarized in Table S2.Source: §A.2 · p. 11, Table S2 · p. 11

A.3. WSI Processing with TRIDENT

• TRIDENT provides a shared workflow for tissue segmentation, patch-coordinate generation, patch-feature extraction, and slide-embedding generation.It enables consistent WSI processing across the evaluated encoders.Source: §A.3 · p. 11

A.3.1. Tissue Segmentation and Patch Extraction

• Tissue regions are segmented before patch coordinates are generated; each magnification receives an independent grid, so patches across scales are not spatially paired.Patch size follows each encoder’s native input resolution, and coordinates are stored with slide metadata.Source: §A.3.1 · p. 11

A.3.2. Patch-Level Feature Extraction

• Patch representations are extracted at each foundation model’s native input resolution, with magnification and patch size matched between coordinate generation and feature extraction.Source: §A.3.2 · p. 11

A.3.3. Slide-Level Feature Extraction

• For slide-level models, TRIDENT couples each encoder to its required upstream patch encoder, extracting those embeddings when needed before slide aggregation.Model-specific patch-to-slide configurations are retained rather than replaced with a common patch encoder.Source: §A.3.3 · pp. 11–12

A.4. Summary of Foundation Models Used in MOSAIC

• The evaluated collection comprises 12 patch-level and 6 slide-level encoders spanning varied architectures, objectives, data scales, and representation dimensions.Patch outputs range from 512 to 2560 dimensions; slide encoders also differ in how they aggregate patch information.Source: §A.4 · pp. 12–13, Table S3 · p. 12

B. Downstream Task Definitions

• The benchmark includes 13 tasks across histologic subtyping, pathologic staging, and molecular mutation prediction, testing tissue organization as well as subtler genotype-associated morphology.Source: B. Downstream Task Definitions · p. 13

B.1. TCGA-NSCLC: LUAD vs. LUSC Subtyping

• NSCLC LUAD-versus-LUSC and BRCA IDC-versus-ILC classification assess representation of characteristic tumor morphology and tissue organization.IDC is associated with cohesive tumor-cell organization, whereas ILC commonly shows dispersed, single-file infiltration.Source: §B.1 · p. 13, §B.2 · p. 13

B.3. TCGA-NSCLC: Pathologic Staging

• NSCLC and breast-cancer pathologic staging test whether slide representations capture morphological characteristics associated with disease progression.Source: §B.3 · p. 13, §B.4 · p. 13

B.5. CPTAC-LUAD: TP53 Mutation Prediction

• LUAD mutation tasks cover TP53, KRAS, and STK11, testing whether H&E representations encode genotype-associated morphology and signals from comparatively subtle targets.Source: §B.5 · p. 13, §B.6 · p. 13, §B.7 · p. 13

B.8. CPTAC-BRCA: PIK3CA Mutation Prediction

• BRCA mutation tasks cover PIK3CA, GATA3, and MAP3K1, assessing morphology associated with PI3K-pathway alterations, tumor biology, and genotype-associated tissue phenotypes.Source: §B.8 · p. 13, §B.9 · p. 13, §B.10 · p. 13

B.11. CPTAC-COAD: TP53 Mutation Prediction

• COAD mutation tasks cover TP53, KRAS, and PIK3CA, testing whether representations capture mutation-associated morphology and molecular signals beyond visually apparent histology.Source: §B.11 · pp. 13–14, §B.12 · p. 14, §B.13 · p. 14

• Seven complementary metrics assess distinct forms of correspondence, including global geometry, correlated subspaces, pairwise relationships, and nonlinear statistical dependence.Source: C. Representation Similarity Metrics · p. 14, Table S4 · p. 15
• SVCCA retains the smallest set of SVD components explaining 99% of variance before CCA, while PWCCA weights canonical directions by their contribution to the original representation.Source: C. Representation Similarity Metrics · pp. 14–15
• Procrustes similarity tests whether spaces match under an orthogonal transformation and global scaling, making it stricter than CCA-based measures.Source: C. Representation Similarity Metrics · p. 15
• Cosine RSA compares the rank ordering of pairwise sample relationships, while linear CKA measures sample-wise geometric agreement and is invariant to orthogonal transforms and isotropic scaling.Source: C. Representation Similarity Metrics · pp. 14–15
• Distance correlation captures general statistical dependence without requiring equal dimensions or a linear relationship; RBF-kernel CKA likewise captures nonlinear agreement in pairwise sample relationships.Source: C. Representation Similarity Metrics · pp. 14–16
• All encoder pairs populate an encoder-by-encoder similarity matrix using the same matched sample indices, separating representational differences from tissue-sampling differences.Source: C. Representation Similarity Metrics · p. 16

• GCCA learns linear projections and a shared latent representation that captures correlated structure across encoders, with an orthogonality constraint encouraging non-redundant dimensions.Source: D. Shared-Space Alignment Methods · p. 16
• MCCA maximizes total pairwise correlation among projected encoders, aligning them indirectly without explicitly introducing GCCA’s common latent representation.Source: D. Shared-Space Alignment Methods · p. 16
• GPA first reduces each representation to dimension d, then aligns them to a consensus using encoder-specific rotations or reflections and global scaling while preserving internal geometry.Source: D. Shared-Space Alignment Methods · pp. 16–17
• Joint PCA derives a shared target from leading components of concatenated representations, then fits per-encoder ridge projections that can map unseen samples without other encoder views at inference.Source: D. Shared-Space Alignment Methods · p. 17
• The shared autoencoder learns nonlinear encoder-specific mappings into a common bottleneck and balances reconstruction fidelity against alignment through a tunable trade-off.Source: D. Shared-Space Alignment Methods · p. 17
• Alignment parameters are fitted on training samples only; patch alignment uses matched spatial coordinates, while slide alignment uses matched whole-slide images.Source: D. Shared-Space Alignment Methods · p. 17

E.1. Cross-Model Agreement and Retrieval

• Cross-model agreement uses mean cosine similarity for matched patches, and retrieval tests identity or shared-label matches with ranking metrics across ordered encoder pairs.Source: §E.1 · p. 18

E.2. Geometry Preservation

• Geometry preservation is assessed because alignment could increase correspondence while discarding original structure, using neighborhood overlap and reconstruction fidelity measures.Source: §E.2 · pp. 18–19

E.3. Downstream Utility

• Downstream utility compares single-encoder, concatenated, and shared representations under patient-level MIL with ABMIL and TransMIL, using a common patient-disjoint split and training-only alignment.Source: §E.3 · p. 19

E.4. Cross-Model Transfer

• Cross-model transfer encodes a source representation into the shared space and decodes it into a target encoder’s native feature space, comparing the result with the same patch’s native target representation.Source: §E.4 · p. 19
• Self round-trip reconstruction isolates information lost through the shared bottleneck, while the gap to cross-model transfer reflects the additional difficulty of translating between encoder spaces.Source: §E.4 · p. 19

• Encoder agreement varies across cohorts and similarity metrics, while patch similarity also reflects image resolution, magnification, and cancer subcohort.Source: F. Additional Results · p. 19
• Aligned representations substantially improve cross-model retrieval over unaligned controls, with GCCA consistently strong across configurations, and shared-space translation reconstructs features in target encoders’ native spaces.Source: F. Additional Results · p. 19
• Shared features remain competitive on several downstream tasks and sometimes outperform single or concatenated representations, but gains depend on task and MIL architecture; deeper layers align more for several CLS/first-token pairs, while mean-pooled features stay broadly similar throughout.Source: F. Additional Results · p. 19

F.1. Similarity Metrics for Slide Encoders

• On CPTAC, CHIEF–GigaPath leads six of seven similarity metrics, while Feather–PRISM leads PWCCA.GigaPath–Madeleine and Madeleine–TITAN also show consistently high agreement across several metrics.Source: Table S9 · p. 21, §F.1 · p. 21
• Across CPTAC subcohorts, GigaPath features in the strongest pairings, with GigaPath–Madeleine dominant in BRCA and COAD across most metrics.PWCCA saturates at 1.000 in some smaller subcohorts and is interpreted cautiously.Source: Table S10 · p. 22, §F.1 · p. 22
• On TCGA, Feather–TITAN leads five metrics, CHIEF–GigaPath leads SVCCA, and Feather–PRISM leads PWCCA.The dominant encoder pair therefore differs between CPTAC and TCGA.Source: §F.1 · p. 21, Table S11 · p. 23
• Across TCGA subcohorts, CHIEF–GigaPath dominates LUAD and LUSC, while BRCA splits between CHIEF–GigaPath and GigaPath–Madeleine.PWCCA is interpreted cautiously because smaller subcohort sample sizes cause substantial saturation at 1.000.Source: Table S11 · p. 23, §F.1 · p. 23

F.2. Patch-Level Similarity Metrics for Patch Encoders

• Similarity is structured rather than uniform: strongest encoder pairs shift with magnification, while CONCH–CONCH v1.5 remains highly similar across scales.At 224 px, GPFM–H-optimus-0 agreement decreases from 5× to 20×; at 512 px, CONCH–CONCH v1.5 similarity remains stable across magnifications.Source: Table S12 · p. 24, §F.2 · p. 25, Table S16 · p. 25

F.3. Subcohort-level Patch Representational Similarity

• Subcohort analyses compare patch-encoder pairs separately across CPTAC BRCA, COAD, and LUAD and TCGA BRCA, LUAD, and LUSC, stratifying by resolution and magnification.The tables also include subcohort comparisons of CONCH and CONCH v1.5 at 512 px.Source: Table S22 · p. 28, Table S24 · p. 28, Table S28 · p. 31

F.4. Cross-Model Retrieval in the Shared Latent Space

• At 256 px, retrieval is harder with a larger, more heterogeneous encoder set; magnification effects are smaller than alignment strategy and encoder-set composition.Procrustes often falls below 0.5 Recall@1 in this setting.Source: §F.4 · p. 41
• Unaligned retrieval is near zero, whereas learned alignment improves it across CPTAC and TCGA; GCCA leads, with Joint PCA, MCCA, and the autoencoder strong and Procrustes weaker.The method ranking is highly consistent across the two cohorts.Source: §F.4 · p. 41

F.5. Cross-Encoder Representation Translation

• Translation fidelity is strongest at 512 px and most challenging at 256 px; reconstruction quality tracks retrieval accuracy across configurations.The shared representation can be decoded into encoder-specific feature spaces, with translation ease depending on resolution and encoder configuration.Source: Table S41 · p. 42, §F.5 · p. 42

F.6. Detailed Results of Downstream Utility

• Across 13 tasks, the best single encoder remains the strongest AUC baseline for most tasks, while MOSAIC improves selected molecular endpoints but does not universally outperform individual encoders or concatenation.The 64-dimensional shared space trades cross-model alignment against preservation of encoder-specific discriminative information.Source: §F.6 · pp. 43–44
• For NSCLC subtyping with TransMIL, MOSAIC achieves an AUC of 0.961 versus 0.973 for the best single encoder; for staging, best single representations consistently outperform both alternatives.Staging-related signals may depend more strongly on encoder-specific information attenuated by the shared bottleneck.Source: §F.6 · p. 43
• For CPTAC-BRCA GATA3, MOSAIC has the highest AUC with ABMIL (0.620) and TransMIL (0.647), versus best-single AUCs of 0.558 and 0.534, respectively.It also leads on ABMIL KRAS for COAD (0.583) and TransMIL PIK3CA for COAD (0.724), but not universally across molecular tasks.Source: §F.6 · p. 44
• Representation-fusion utility depends on MIL architecture: concatenation leads on several ABMIL tasks, while the best single encoder dominates most TransMIL tasks.MOSAIC remains competitive on several tasks and occasionally surpasses both alternatives.Source: §F.6 · p. 44

F.7. Layerwise encoder similarity Analysis

• Across an encoder’s depth, linear CKA compares its final representation with its first block or stage; lower similarity indicates greater representational change through the network.Source: Table S47 · p. 45
• Under CLS/first-token pooling, mean similarity rises from 0.626 to 0.680, but four of ten trajectories show terminal divergence; under mean pooling, average CKA stays high from 0.899 to 0.867 with no sustained terminal divergence.CTransPath lacks a semantic CLS token, so its first-token trajectory is not a true CLS-token trajectory; mean pooling is more directly comparable.Source: §F.7 · p. 45