PaperlayerView exact revision on arXiv

Source-linked AI summary

Conserved Immune Topology Improves Pathology Foundation Model Generalization for Cross-Cancer MSI-H Prediction

Dasari Naga Raju

arXiv:2609.05182v1cs.CV

TL;DR

Cross-cancer MSI-H prediction remains difficult because organ-specific histological and architectural differences create domain gaps for pathology foundation models and MIL. CIT augments frozen foundation-model embeddings with unsupervised spatial immune descriptors, improving zero-shot transfer across gastrointestinal cancer cohorts and suggesting an organ-invariant signal, while broader validation remains necessary.

  • Problem

    Cross-cancer MSI-H generalization remains unresolved because organ-specific histological and architectural differences create distribution shifts despite strong within-cohort performance.

  • Method

    CIT uses unsupervised clustering of frozen foundation-model embeddings and tile coordinates to encode TLS, peritumoral immune reactions, multi-scale immune density, and immune–tumor mixing without annotations or target-domain data.

  • Results

    0.0534 AUC increased zero-shot TransMIL cross-cancer performance from 0.6627 to 0.7161 (p = 0.003), with consistent improvements across all three MIL aggregators.

  • Takeaways & Limitations

    Spatial immune topology may provide an organ-invariant representation that reduces cross-cancer transfer degradation for pathology foundation models.

  • Takeaways & Limitations

    Evaluation was restricted to gastrointestinal COAD and STAD cohorts, and the unsupervised immune clustering lacked explicit histological validation.

Abstract

from arXiv · show

Pathology foundation models integrated with multiple instance learning achieve competitive accuracy within single-cancer cohorts, yet cross-cancer generalization remains unresolved due to organ-specific histological and architectural differences. In this paper, we propose Conserved Immune Topology (CIT), a lightweight spatial representation for cross-cancer MSI-H prediction that augments foundation-model embeddings with biologically motivated immune descriptors. CIT uses unsupervised clustering to identify immune-associated tiles, then encodes tertiary lymphoid structures, peritumoral immune reactions, multi-scale tumor-infiltrating lymphocyte density, and immune-tumor mixing from frozen foundation-model embeddings and tile coordinates without requiring annotations or target-domain data. The proposed method was evaluated under cross-site and cross-cancer settings using CPTAC-COAD and TCGA-STAD cohorts, which introduce scanner variability, distribution shifts, and organ-specific architectural variations. Zero-shot cross-cancer transfer with CIT increased TransMIL AUC from 0.6627 to 0.7161, an absolute gain of 0.0534 (p=0.003), with consistent improvements across all three MIL aggregators. These results suggest that spatial immune topology provides potentially an organ-invariant representation for MSI-H prediction, supporting cross-cancer generalization of pathology foundation models.

1 Introduction

MSI-H prediction from whole-slide images could reduce reliance on additional laboratory tests, but cross-cancer transfer is limited by organ-specific histological and architectural differences. Conserved Immune Topology (CIT) addresses this gap by adding biologically motivated spatial immune descriptors to foundation-model embeddings within existing MIL pipelines.

  • MSI-H guides prognosis, treatment selection, and immunotherapy decisions across multiple cancer types.Conventional molecular assays and mismatch-repair protein immunohistochemistry require additional laboratory tests.
  • Foundation models combined with multiple instance learning achieve competitive MSI-H prediction within individual cancer types.Recent computational pathology methods infer MSI-H status directly from whole-slide images.
  • Cross-cancer transfer faces domain gaps from organ-specific histological and architectural differences.Representations learned from one cancer type often transfer poorly to another because most methods are developed and evaluated within single-cancer cohorts.
  • CIT is a lightweight spatial representation that augments foundation-model embeddings with biologically motivated descriptors of immune organization.It integrates into existing MIL pipelines without annotations, target-domain adaptation, or modifications to the underlying foundation model.

2 Related Work

Prior work established image-based MSI-H prediction and strong within-cohort performance, but cross-cancer and external-cohort transfer remains limited. Spatial immune organization offers a complementary direction that CIT derives from frozen embeddings and tile coordinates without supervised structural annotations.

  • Early studies showed that mismatch-repair deficiency leaves morphological signatures detectable in histopathological images.
  • Within-cohort MSI-H validation typically reports AUCs ≥0.85, whereas cross-cancer studies remain sparse and report degradation of 0.15–0.25 AUC units.
  • Pathology foundation models provide transferable frozen representations and support MIL pipelines, but remain sensitive to domain-specific appearance.
  • ABMIL, CLAM, and TransMIL differ primarily in how they aggregate tile features into slide-level predictions.
  • External-cohort MSI-H transfer is often poor, while domain adaptation requires target-domain data and domain generalization seeks invariant representations without it.
  • CIT builds on spatial immune analyses by inferring immune organization from frozen embeddings and tile coordinates without supervised cell- or structure-level annotations.

3 Method

CIT constructs a ten-dimensional spatial descriptor from frozen foundation-model embeddings and normalized tile coordinates, then concatenates it with each tile embedding for MIL-based slide prediction. Unsupervised clustering identifies immune-associated regions without manual annotations, with clustering learned on the training cohort and fixed during transfer evaluation.

  • CIT concatenates a ten-dimensional descriptor with each frozen foundation-model embedding to create the augmented tile representation used by MIL.
  • A WSI is partitioned into N tiles represented by frozen foundation-model embeddings, with tile coordinates normalized by slide width and height.
  • Spatial neighborhoods use Euclidean distance between normalized coordinates, with k nearest neighbors selected from {10, 20, 30, 100} according to the descriptor.
  • Unsupervised clustering groups tiles with similar frozen-feature representations to infer immune-associated regions rather than individual immune cells.
  • K-means uses K = 50 clusters and up to 500 random tiles per slide from the training cohort to balance computational cost and cluster stability.
  • Cluster tightness assigns higher values to clusters with lower intracluster variation.The score is defined from mean distance to the cluster centroid with a small numerical-stability constant.
  • The top 30% of clusters by tightness, κ = 15 of 50, are designated immune-enriched and interpreted as computational proxies rather than explicit cell-type annotations.
  • Each tile receives a binary immune indicator and continuous enrichment score, while the clustering model and hyperparameters remain fixed for cross-site and cross-cancer evaluation.

3.3 Conserved Immune Topology Descriptors

CIT represents conserved immune topology through four spatial descriptor groups that together form a ten-dimensional vector. These descriptors include TLS organization, peritumoral immune reactions, multi-scale immune density, and immune–tumor mixing.

  • CIT computes four descriptor groups for conserved immune topology: TLS, peritumoral immune reactions, multi-scale immune density, and immune–tumor mixing.Together they define the ten-dimensional vector u_i.
  • TLS are detected as compact immune-enriched clusters using DBSCAN on normalized coordinates of immune-associated tiles.
  • Each detected TLS is represented by its centroid and size, while tiles receive membership and normalized distance-to-centroid descriptors.

Peritumoral Margin Reaction

CIT estimates peritumoral margin reactions from tile spatial density and immune scores, enabling boundary-sensitive descriptors without explicit tissue annotations.

  • Peritumoral Margin Reaction: Crohn’s-like lymphocytic reactions at invasive tumor margins motivate CIT’s peritumoral margin descriptor.This reaction is described as characteristic of MSI-H tumors.
  • Peritumoral Margin Reaction: Higher normalized margin-distance values identify immune-associated regions near tissue margins.Interior regions have denser neighborhoods, so local density inversely correlates with margin distance.
  • Peritumoral Margin Reaction: Local spatial density is estimated from average distances to the 20 nearest spatial neighbors.The density calculation uses neighboring tile coordinates and a small constant.
  • Peritumoral Margin Reaction: The peritumoral immune band combines normalized margin distance with each tile’s immune score.This descriptor integrates spatial position and immune association.

Multi-scale Immune Density

CIT measures immune-associated tile density and immune–tumor organization across complementary neighborhood scales, capturing both local abundance and spatial mixing.

  • Multi-scale Immune Density: CIT computes immune density as the fraction of immune-associated tiles within each tile’s k-nearest-neighbor neighborhood.The neighborhood sizes k ∈ {10, 30, 100} capture complementary local-to-regional scales.
  • Multi-scale Immune Density: The three neighborhood sizes capture complementary spatial scales from local to regional immune density.Each scale summarizes immune-associated tile abundance around a tile.
  • Immune–Tumor Mixing: Immune–tumor mixing is quantified using neighborhood heterogeneity and immune-associated tile abundance.The descriptors distinguish spatial intermixing from immune exclusion.
  • Immune–Tumor Mixing: Binary entropy is computed from the immune fraction within each 20-nearest-neighbor neighborhood.Higher entropy indicates greater immune–tumor mixing, whereas lower values indicate homogeneous neighborhoods.
  • Immune–Tumor Mixing: An immune–tumor ratio over the 30-nearest-neighbor neighborhood summarizes surrounding immune-associated tissue prevalence.The ratio is further normalized across the slide.

3.4 Integration with Multiple Instance Learning

CIT integrates spatial immune descriptors with frozen foundation-model embeddings by concatenation, producing augmented tile features for existing MIL aggregators.

  • 3.4 Integration with Multiple Instance Learning: Each tile’s descriptor vector is concatenated with its corresponding frozen foundation-model embedding.The augmented representation has dimension d+10.
  • 3.4 Integration with Multiple Instance Learning: The concatenated representation serves as input to the MIL aggregator for slide-level prediction.CIT therefore augments tile features before multiple-instance aggregation.
  • 3.4 Integration with Multiple Instance Learning: Concatenation preserves the frozen foundation model, adds no integration parameters, and limits overhead through a 10-dimensional augmentation.CIT remains independent of the aggregation method and does not modify the underlying feature extractor.

4 Experimental Setup

The study evaluates CIT across colorectal and gastric cohorts, including cross-site and zero-shot cross-cancer settings, with multiple foundation-model and MIL configurations.

  • 4 Experimental Setup: TCGA-COAD provided training and internal five-fold validation, CPTAC-COAD tested cross-site robustness, and TCGA-STAD tested zero-shot cross-cancer transfer.The cohorts contained 301, 105, and 308 patients, respectively.
  • 4 Experimental Setup: Tiles were extracted at 20× magnification as 224 × 224 images, with coordinates retained for spatial descriptor computation.UNI2-h produced frozen 1536-dimensional tile embeddings.
  • 4 Experimental Setup: ABMIL, CLAM-SB, and TransMIL were evaluated with identical baseline and CIT configurations.Internal validation used UNI2-h and CONCH, while cross-site and cross-cancer experiments used UNI2-h.
  • 4 Experimental Setup: Table 1 summarizes patient counts and MSI-H/MSS class distributions across the study cohorts.The table provides the cohort composition underlying the experimental protocol.
  • 4 Experimental Setup: Models were trained for 40 epochs with Adam, learning rate 10^-4, weight decay 10^-5, and hidden dimension 256.The highest-validation-AUC checkpoint was retained for each fold, and external predictions were averaged across folds.

5 Results

CIT preserved within-cohort accuracy while improving robustness under cross-site and zero-shot cross-cancer distribution shifts. Gains were consistent across MIL aggregators and depended on the full spatial descriptor representation.

  • Within-Cancer Validation: Internal TCGA-COAD validation showed minimal CIT effects, with AUC changes of +0.001 for ABMIL, −0.002 for CLAM-SB, and +0.017 for TransMIL.These changes fell within fold-wise variation, supporting preserved within-cohort accuracy.
  • Cross-Site Generalization: Cross-site gains ranged from −0.0003 AUC for ABMIL to +0.0407 for TransMIL on independent CPTAC-COAD data.CLAM-SB and TransMIL improved despite institution and scanner shifts, whereas ABMIL was unchanged.
  • Zero-Shot Cross-Cancer Generalization: 0.0534 AUC was the TransMIL gain under zero-shot transfer from TCGA-COAD to TCGA-STAD, increasing performance from 0.6627 to 0.7161.ABMIL and TransMIL improvements remained significant after Bonferroni correction, while CLAM-SB’s improvement was not significant.
  • Bidirectional Transfer: CIT improved AUC for all three MIL aggregators in reverse transfer from TCGA-STAD to TCGA-COAD, with TransMIL gaining +0.041.Bidirectional gains were +0.0534 from COAD to STAD and +0.0408 from STAD to COAD.
  • Ablation Study: Removing any descriptor group reduced zero-shot TCGA-STAD performance, with immune–tumor mixing producing the largest single-group decrease of −0.0111 AUC.The two interface-oriented descriptors together produced the largest combined drop, 0.0187 AUC, while internal effects were ≤0.0020 AUC.

6 Discussion

CIT appears to improve cross-cancer MSI-H transfer by encoding spatial immune organization that complements foundation-model appearance features. Its gains persist across descriptor ablations and transfer directions, but validation remains limited to gastrointestinal cohorts and lacks explicit histological validation.

  • 0.0365 AUC less degradation was observed with CIT-augmented features than with foundation-model features under cross-cancer transfer.Degradation was 0.2771 AUC for foundation-model features versus 0.2406 AUC with CIT augmentation.
  • TransMIL gained +0.0534 AUC cross-cancer, from 0.6627 to 0.7161 (p = 0.003), compared with +0.0169 AUC internally.The larger cross-cancer gain suggests CIT benefits particularly under distribution shift rather than uniformly across settings.
  • 0.0446 versus 0.0253 AUC was the mean CIT gain under cross-cancer versus cross-site transfer.The spatial descriptors appeared less sensitive to scanner, institution, and tissue-type variation.
  • Ablation results showed that removing any descriptor group reduced zero-shot TCGA-STAD performance, so no single group explained the full gain.CIT increased zero-shot transfer AUC by 0.0534 using ten additional descriptor dimensions per tile, with gains in both transfer directions.
  • The study’s main limitations are absent explicit histological validation for unsupervised immune clustering and evaluation restricted to gastrointestinal COAD and STAD cohorts.Broader testing on non-gastrointestinal MSI-H datasets is needed to assess pan-cancer generalizability.

7 Conclusion

The paper presents CIT as a lightweight spatial descriptor that captures conserved immune organization from frozen foundation-model embeddings without dense annotations. Integrated with existing MIL architectures, it improved zero-shot cross-cancer MSI-H prediction while maintaining within-cohort accuracy.

  • CIT captures conserved immune organization from frozen foundation-model embeddings without requiring dense annotations.
  • CIT integrates with existing MIL architectures by encoding key spatial immune patterns.
  • CIT improved zero-shot cross-cancer MSI-H prediction across all aggregators while maintaining within-cohort accuracy.The approach mitigated transfer degradation observed with appearance-based features alone.
  • Spatial immune organization may serve as a fundamental, potentially organ-invariant biomarker.Future work will evaluate CIT on non-gastrointestinal datasets to establish broader generalizability.
Loading 2609.05182v1…