Morphology-aware deep learning predicts colorectal cancer survival from whole-slide images

Findings

• Tissue patterns each carried modest prognostic information, suggesting useful signals were distributed across the slide rather than confined to tumor cells.The method learns from both image patterns and patients’ five-year outcomes.Source: Plain language summary · p. 2

Evaluation

• In an archival tissue cohort originating from a clinical trial, MorphContrast distinguished patients with different five-year outcomes.Source: Plain language summary · p. 2

Scope

• The method showed similar discrimination in an independent external cohort, but further prospective and independent validation is needed before clinical use.Source: Plain language summary · p. 2

Motivation

• Five-year survival prediction from whole-slide images could offer risk stratification using existing digitized data, but each slide-level outcome must supervise thousands of patches without patch-level annotation.Source: §1 · p. 2

Research gap

• The relative prognostic value of tissue morphologies remains insufficiently characterized, despite prognostic features extending beyond malignant epithelium.Whether signal is concentrated in one morphology or distributed across many affects whether analysis should be tissue-restricted or aggregate the tissue mixture with unequal weighting.Source: §1 · pp. 2–3

Morphology-specific analysis

• Morphology-specific models in TCGA-COAD/READ showed weak, graded prognostic signal across tumor and non-tumor morphologies, with no individual morphology strongly predictive.The analysis is methodological motivation, not a direct biological finding, because spatial context and correlated clinicopathologic factors are uncontrolled.Source: §1 · p. 3

MorphContrast

• MorphContrast jointly optimizes survival prediction with morphology-based within-patient contrasts, outcome-aware inter-patient contrasts, and a diversity regularizer, while keeping the MorphDistill patch encoder frozen.Matched morphologies from patients with concordant outcomes form positive pairs; discordant outcomes form hard negatives.Source: §1 · p. 3

Contribution

• Unlike approaches that weight patches from patient outcomes or use morphology as an input or intermediate annotation, MorphContrast jointly organizes representations by morphology and outcome.Its cross-patient comparisons target outcome-associated variation within matched histologic contexts.Source: §1 · pp. 3–4

Evaluation

• On Alliance/CALGB 89803, the single model achieved an AUC of 0.642 and the post hoc ensemble reached a pooled OOF AUC of 0.70; SurGen AUCs were 0.669 and 0.678, respectively.The ensemble result on SurGen was similar to FR-MIL, the strongest evaluated baseline (0.6766), and does not establish a clear external performance advantage.Source: §1 · p. 4

Multiple-instance learning

• MIL methods improve patch aggregation through learned weighting, clustering, critical-instance selection, self-attention, regional re-embedding, feature recalibration, or prototype modeling.Source: §2 · p. 4

Morphology-specific analysis

• Adipose tissue, stroma, normal epithelium, and muscle scored comparably to tumor-epithelial classes, while the displayed morphology-specific models remained modest predictors.The figure shows five-fold cross-validated AUCs for single-morphology ABMIL models in TCGA-COAD/READ; the dashed line marks chance at 0.50.Source: Figure 1 · p. 5

Positioning

• MorphContrast retains gated-attention pooling and adds morphology- and outcome-aware contrastive objectives with diversity regularization.The approach is presented as structural supervision for outcome-aware representations, with externally assessed discrimination comparable to the strongest evaluated baseline; prospective validation is required before clinical use.Source: §2 · pp. 5–6

Hard-negative design

• ACMIL and MHIM-MIL use attention-derived salience to select instances for masking, while MorphContrast defines hard negatives by same-morphology patches from patients with discordant outcomes.This changes what the representation encodes rather than which instances the aggregator receives.Source: §2 · p. 5

Morphology-aware methods

• MorphDistill and PRISM use morphology as an input, intermediate representation, or auxiliary supervision, whereas MorphContrast uses morphology and outcome jointly to define representation-learning relationships.The design targets prognostic variation within matched histologic contexts.Source: §2 · p. 6

3.2 Architecture

• A shared two-layer task-adaptation network maps frozen MorphDistill patch features to encoded representations used by both survival and contrastive pathways.MorphDistill provides a 768-dimensional feature vector and morphology label for each retained patch; it remains frozen during training.Source: §3.1 · p. 6, §3.2 · p. 7

3.1 Overview

• For each anchor patient, training samples one patient with a concordant outcome and one with a discordant outcome from a class-balanced pool built from the current training partition.Intra-patient morphology pairs, inter-patient outcome-aware pairs, and a diversity penalty are accumulated before one backward pass; the patient pool and projection head are removed at inference.Source: §3.1 · p. 7

3.2 Architecture

• Unit-L2-normalized projected patches have inner products equal to cosine similarity, which is temperature-scaled for the pairwise contrastive objectives.The projection head contains two linear layers separated by a ReLU activation.Source: §3.2 · p. 7
• Gated attention weights encoded patches for patient-level survival prediction, while a projection head maps patches into a lower-dimensional contrastive space during training.Patch and morphology contributions to the patient representation can differ; the projection head is removed at inference.Source: §3.2 · pp. 7–8

3.3 Class-Balanced Patient Pool

• The training pool balances five-year outcome groups, and each anchor patient is paired with one outcome-concordant and one outcome-discordant patient.Sampling uses only the training partition; up to 512 morphology-balanced patches are retained per patient.Source: §3.3 · p. 8

3.4 Patch-Level Contrastive Objectives

• Within each patient, the intra-patient objective treats same-morphology patches as positives and different-morphology patches as negatives.Anchors without at least one valid positive and one valid negative are omitted from the corresponding loss.Source: §3.4 · p. 9
• Across patients, morphology-matched patches with concordant outcomes are positives, while discordant-outcome patches are hard negatives that target outcome variation within comparable histologic contexts.Patches from different morphologies provide standard negatives.Source: §3.4 · p. 9
• Diversity regularization encourages dispersion of each patient's projected patch representations rather than exact pairwise orthogonality.The term operates over ordered pairs of distinct patches.Source: §3.4 · pp. 9–10

3.5 Survival Objective and Joint Optimization

• The joint objective combines patient-level survival loss after gated-attention pooling with auxiliary losses that update the shared feature encoder and projection head.The survival loss also updates the attention network and classifier, so patch-level contrastive supervision reshapes features later pooled for survival prediction.Source: §3.5 · p. 10

3.6 Study design and cohorts

• All three cohorts use non-overlapping 224 × 224 patches at 20×, with patches below 25% tissue discarded; the study assigns each cohort a distinct analysis role.TCGA-COAD/READ supports morphology-specific analysis, Alliance/CALGB 89803 is the primary development cohort, and SurGen is the external cohort.Source: §3.6 · p. 10

3.6.2 Digitization and quality assurance

• More than 98% of Alliance/CALGB 89803 cases contributed one representative tumor slide, and inadequate-staining slides were removed after quality checks.A pathologist blinded to clinical information and outcome confirmed tumor presence; checks included tissue coverage, color fidelity, focus, and scanning artifacts.Source: §3.6.2 · p. 11

3.6.3 Cohort construction and endpoint

• The Alliance/CALGB 89803 analysis includes 424 patients with known five-year outcomes after specified exclusions, including patients censored before five years and mucinous adenocarcinomas.There were 103 deaths within five years and 321 patients surviving at least five years; the primary cohort derives from a randomized phase III trial of adjuvant IFL versus FL in resected stage III colon cancer.Source: §3.6.1 · pp. 10–11, §3.6.3 · p. 11

3.6.4 Discovery cohort: TCGA-COAD/READ

• TCGA-COAD/READ provides 562 patients for five-fold, patient-level cross-validation of morphology-specific models, with mucinous cases retained.Its scanner protocol, patient population, and stage distribution differ from Alliance, while patch extraction uses the shared protocol.Source: §3.6.4 · p. 11

3.6.5 External cohort: SurGen

• External evaluation uses 218 colon cancer cases from SurGen's survival-annotated subset to align the disease site with the colon-only Alliance cohort.SurGen differs from Alliance in scanner resolution and case mix, creating potential distribution shift.Source: §3.6.5 · p. 12

3.7 Comparison Methods

• Alliance comparisons include attention- and transformer-based MIL, CRC prognosis and morphology-aware models, and frozen pathology encoders paired with the same gated-attention survival head.SurGen comparisons use TransMIL, FR-MIL, and MHIM-MIL; models are trained on Alliance and evaluated without additional training, and comparisons are cohort-specific.Source: §3.7 · p. 12

3.8 Statistical analyses

• Discrimination is measured with AUC and Harrell’s C-index, alongside threshold-based balanced accuracy, sensitivity, and specificity; risk stratification uses log-rank tests and univariate Cox models.Alliance fold-based results and pooled out-of-fold ensemble results use different reporting procedures; SurGen estimates use bootstrap confidence intervals.Source: §3.8 · pp. 12–13

3.9 Implementation and Training Details

• Alliance development uses patient-level five-fold cross-validation, with training, validation, and held-out test splits for optimization and performance reporting.The post hoc search generates 50 candidate ensembles; the selected ensemble has three Pareto-selected members whose out-of-fold risk scores are converted to percentile ranks and aggregated.Source: §3.9 · p. 13, §3.10 · pp. 13–14

4.1 Prognostic Discrimination Across Individual Tissue Morphologies

• Adipose achieved the highest morphology-specific AUC (0.622), followed by high-grade adenocarcinoma and high-grade adenoma dysplasia (0.605 each) and stroma (0.604).No individual morphology exceeded an AUC of 0.622; the leading classes included tumor-associated and non-tumor compartments.Source: §4.1 · p. 14, Table 1 · p. 15

4.2 Single-Model and Ensemble Performance

• 0.70 was the pooled OOF AUC for the Alliance/CALGB 89803 post hoc ensemble, versus 0.642 for the single MorphContrast model.The ensemble’s balanced accuracy, sensitivity, and specificity were each 0.68.Source: §4.2 · p. 14, Table 2 · p. 15

4.3 Comparison with MIL and Morphology-Aware Baselines

• TransMIL led conventional MIL at AUC 0.67 ± 0.05, while PRISM and MorphDistill achieved 0.70 ± 0.04 and 0.68 ± 0.08, respectively.The MorphContrast ensemble achieved a pooled OOF AUC of 0.70.Source: §4.3 · p. 14, Table 3 · p. 15

4.4 Comparison with Frozen Pathology Foundation Encoders

• 0.642 ± 0.05 was MorphContrast’s AUC under the shared survival-head design, compared with 0.63 ± 0.03 for the highest-scoring frozen encoder, UNI v2.Across the ten frozen pathology foundation encoders, AUCs ranged from 0.51 to 0.63.Source: §4.4 · p. 16, Table 4 · p. 16

4.5 Comparison with Clinical Variables

• The MorphContrast ensemble’s pooled OOF AUC was 0.70, versus 0.541 for the clinical model using all fourteen variables and 0.550 for the reduced model.The clinical models used the same five patient-disjoint folds as MorphContrast.Source: §4.5 · p. 16, Table 5 · p. 17

4.6 Risk-Group Stratification

• The MorphContrast ensemble stratified the Alliance cohort into risk groups with an HR of 3.79 (95% CI: 2.55–5.63, p < 0.0001).Figure 3 presents Kaplan–Meier risk stratification at each model’s operating threshold.Source: §4.6 · p. 16, Figure 3 · p. 17

4.7 External Evaluation on the SurGen Cohort

• 0.678 was the MorphContrast ensemble AUC on external SurGen evaluation, compared with 0.669 for the single model and 0.6766 for FR-MIL.Models were trained on Alliance/CALGB 89803 and evaluated on SurGen without fine-tuning or adaptation.Source: §4.7 · p. 18, Table 6 · p. 18

4.8 Ablation of the Contrastive Objectives

• The full objective achieved the highest mean AUC (0.64 ± 0.05); either contrastive term alone yielded only modest increases over 0.60 ± 0.07 without contrastive objectives.The full objective also narrowed the sensitivity–specificity gap from 59.41 to 8.12 percentage points.Source: §4.8 · p. 19, Table 7 · p. 19

• Adipose tissue (AUC 0.622) and stroma (0.604) performed comparably to adenocarcinoma compartments (0.601–0.605), but these results are not measures of biological importance.Patch abundance, spatial context, slide sampling, and correlated clinicopathologic factors were not controlled.Source: §5 · pp. 19–20
• Within patients, same-morphology patches form positive pairs and different-morphology patches form negatives; across patients, outcome-discordant morphology-matched patches serve as hard negatives.Source: §5 · p. 20
• The ensemble’s pooled OOF AUC of 0.70 exceeded MorphDistill (0.68) and matched PRISM (0.70), while MorphContrast’s model had a higher mean AUC than ten frozen encoders (0.51–0.63).Source: §5 · p. 20
• On SurGen, MorphContrast and its ensemble had higher AUC point estimates than TransMIL and MHIM-MIL, while the ensemble’s 95% confidence interval overlapped FR-MIL’s.The external cohort differed from Alliance in scanner resolution.Source: §5 · p. 20
• Frozen-encoder morphology labels may introduce noise into contrastive pairs, and excluding patients censored before five years discards time-to-event information and may introduce selection bias.The framework is also unimodal.Source: §5 · p. 20

• On Alliance/CALGB 89803, the model achieved an AUC of 0.642 and the post hoc ensemble a pooled out-of-fold AUC of 0.70; on SurGen, their AUCs were 0.669 and 0.678, respectively.Source: §6 · pp. 20–21
• The TCGA-COAD/READ analysis showed modest but distributed prognostic discrimination across tumor and non-tumor morphologies, motivating morphology- and outcome-aware contrastive learning.Source: §6 · p. 20
• Within-patient morphology matches and mismatches define positive and negative pairs, while cross-patient morphology-matched patches with concordant and discordant outcomes define positives and hard negatives.Source: §6 · p. 21

Competing interests

• The authors declared no financial or non-financial competing interests.Source: Competing interests · p. 21

Ethics approval and consent to participate

• The Ohio State University IRB approved the retrospective study, and waived informed consent because the data were archival and de-identified.Source: Ethics approval and consent to… · p. 21

Funding

• The project received support from R01 CA276301 and Pelotonia IRP CC13702, among other institutional support.Source: Funding · p. 21

Trial registration

• CALGB 89803 is registered as NCT00003835, with registry identifier NCI-2012-01844.Source: Trial registration · p. 22

Disclaimer

• The authors state that the manuscript’s analyses and conclusions are their sole responsibility and do not necessarily reflect the views of the trial investigators or named organizations.Source: Disclaimer · p. 22

Data availability

• Qualified investigators may request Alliance/CALGB 89803 slides and outcomes; TCGA-COAD/READ and SurGen data are publicly accessible through the named repositories.Source: Data availability · p. 22

Supplementary Material

• Each Supplementary Fig. 1 panel reports low- and high-risk group sizes, hazard ratio with 95% CI, and log-rank p-value for frozen encoders and MorphContrast.Foundation-model features use the same gated-attention survival head.Source: Supplementary Material · p. 27
• The Alliance/CALGB cohort comprised 424 slides from 424 patients, with a mean age of 60.47 years.Source: Supplementary Material · p. 28
• The cohort included 391 White and 33 Black patients, 238 male and 186 female patients, and evenly split treatment arms of 212 each.Source: Supplementary Material · p. 28
• Sigmoid colon was the most common listed primary site (147), followed by cecum (101) and ascending colon (64).Source: Supplementary Material · p. 28
• Hyperparameter search varied model dimensions, learning rate, decision threshold, auxiliary-loss weights, and survival loss; fixed training used AdamW, batch size 4, and up to 150 epochs.Source: Supplementary Material · p. 29
• Fourteen trial case-report-form variables informed full and reduced logistic-regression baselines; the reduced model used lymph-node count, T category, and grade.The cohort had 424 patients and 103 events; both models used ℓ2 regularization and five patient-disjoint folds.Source: Supplementary Material · p. 30