Source-linked AI summary
Deep learning-based survival prediction for multiple cancer types using histopathology images
Ellery Wulczyn, David F. Steiner, Zhaoyang Xu, Apaar Sadhwani, Hongwu Wang, Isabelle Flament, Craig H. Mermel, Po-Hsuan Cameron Chen, Yun Liu, Martin C. Stumpe
TL;DR
The paper asks whether histopathology images can improve cancer survival risk stratification beyond established clinical variables. It develops a weakly supervised deep learning system across 10 TCGA cancer types and evaluates survival associations and predictive value. The DLS was significantly associated with disease-specific survival, improved the combined-cohort c-index over baseline variables, and stratified risk within stages II and III, while limited events produced wide confidence intervals.
Problem
Improving prognostic risk stratification beyond staging and other available variables matters for treatment and monitoring decisions.
Method
A weakly supervised deep learning system learned prognostic features from whole-slide histopathology images without region-level annotations and used survival-specific modeling.
Results
The DLS predicted disease-specific survival across multiple cancer types, improved the combined-cohort c-index by 3.7 over the Baseline-only model, and stratified patients within stage II and stage III.
Takeaways & Limitations
Weakly supervised deep learning may provide prognostic information across multiple cancer types and within specific pathologic stages.
Takeaways & Limitations
Each cancer type’s test dataset contained fewer than 250 cases and fewer than 100 disease-specific survival events, resulting in wide confidence intervals and requiring larger-scale validation.
Abstract
from arXiv · showhide
Prognostic information at diagnosis has important implications for cancer treatment and monitoring. Although cancer staging, histopathological assessment, molecular features, and clinical variables can provide useful prognostic insights, improving risk stratification remains an active research area. We developed a deep learning system (DLS) to predict disease specific survival across 10 cancer types from The Cancer Genome Atlas (TCGA). We used a weakly-supervised approach without pixel-level annotations, and tested three different survival loss functions. The DLS was developed using 9,086 slides from 3,664 cases and evaluated using 3,009 slides from 1,216 cases. In multivariable Cox regression analysis of the combined cohort including all 10 cancers, the DLS was significantly associated with disease specific survival (hazard ratio of 1.58, 95% CI 1.28-1.70, p<0.0001) after adjusting for cancer type, stage, age, and sex. In a per-cancer adjusted subanalysis, the DLS remained a significant predictor of survival in 5 of 10 cancer types. Compared to a baseline model including stage, age, and sex, the c-index of the model demonstrated an absolute 3.7% improvement (95% CI 1.0-6.5) in the combined cohort. Additionally, our models stratified patients within individual cancer stages, particularly stage II (p=0.025) and stage III (p<0.001). By developing and evaluating prognostic models across multiple cancer types, this work represents one of the most comprehensive studies exploring the direct prediction of clinical outcomes using deep learning and histopathology images. Our analysis demonstrates the potential for this approach to provide prognostic information in multiple cancer types, and even within specific pathologic stages. However, given the relatively small number of clinical events, we observed wide confidence intervals, suggesting that future work will benefit from larger datasets.
Introduction
The paper addresses the challenge of predicting survival directly from whole-slide histopathology images without expert annotations, extending prior work across multiple cancer types.
- TNM staging is useful for prognosis, but improved prediction strategies incorporating clinical, genetic, and histomorphological information remain needed.
- Whole-slide survival prediction is challenging because images are extremely large and provide only global labels rather than region-level annotations.
- Prior TCGA studies used feature engineering, annotated regions, known histologic features, or models focused on individual cancer types.
- The study develops a deep learning system to predict survival across multiple cancer types from whole-slide images without expert annotations or known features of interest.
- The approach tests several survival loss functions, directly optimizes a convolutional neural network for prognostic features, and uses image subsampling to manage large slides.
- The evaluation compares risk stratification against TNM stage, age, and sex across 10 TCGA cancer types, but limited cases and events make effect sizes difficult to estimate precisely.
Methods
The methods train weakly supervised neural networks on sampled tissue patches from TCGA whole-slide images and evaluate survival prediction using several loss functions and Cox-based analyses.
- TCGA cases were split into train, tune, and test sets in a 2:1:1 ratio with stratification by event observation and discretized time-to-event.
- 4,880 cases comprising 12,095 images were used for training and tuning, while 1,216 cases comprising 3,009 images were used for evaluation.
- The DLS used shared-weight convolutional modules and average pooling to merge patch-level image features while accommodating different numbers of patches.
- Survival Loss functions: Three survival loss functions were tested, and censored cross-entropy was selected for final training because it performed best on the tune split.
- Survival Loss functions: Censored cross-entropy discretized survival time into intervals and used observed-event or censoring information to maximize the appropriate event likelihood.
- Survival analysis: Risk groups were formed within cancer type so each group had the same cancer-type distribution, and Kaplan-Meier comparisons were repeated within stages.
- Survival analysis: Cox regression assessed prognostic associations and predictive value, with leave-one-out cross-validation and c-index evaluation for models with and without the DLS.
Results
The DLS produced prognostic risk scores that separated survival groups overall and within stages, while adding predictive value beyond baseline clinical variables across the combined cohort and several cancer types.
- Risk-group stratification: p<0.001 separated high- and low-risk groups using DLS risk-score quartiles within each cancer type.Within-cancer-type binning preserved the distribution of cancer types across risk groups.
- Within-stage stratification: Stage II (p < 0.05) and stage III (p < 0.001) patients were further stratified by DLS risk, unlike stage I or stage IV patients.The stage-specific comparisons used Kaplan-Meier curves for low- and high-risk groups.
- Multivariable analysis: Hazard ratio 1.48 (p<0.0001) identified the DLS as a significant disease-specific survival predictor after adjustment for cancer stage, age, sex, and cancer type.The combined analysis included all 10 cancer types, with cancer type represented as an indicator variable.
- Multivariable analysis: 5 of 10 cancer types retained significant DLS associations with disease-specific survival in adjusted subanalyses.The reported p-values ranged from 0.0002 to 0.0257.
- Predictive value: 3.7 (95% CI [1.0, 6.5]) was the c-index delta between Baseline-only and Baseline+DLS models for all studies combined.The delta was interpreted as the DLS’s added predictive value over stage, age, and sex.
- Predictive value: 6.4 (95% CI [2.2, 10.8]) was the combined-analysis AUC improvement for predicting 5-year disease-specific survival.The AUC results were described as qualitatively similar to the c-index results.
- Model interpretation: Most confident high- or low-risk patch predictions primarily contained tumor, whereas intermediate predictions tended to contain fat, stroma, or fragmented tissue.More detailed associations between histologic features and patch-level risk predictions were not identified.
Discussion
The study evaluates weakly supervised deep learning for prognosis without region-level expert annotations and finds prognostic morphologic signals across multiple cancer types. The authors frame the results as promising but preliminary because TCGA’s limited events and setting constrain statistical precision and generalizability.
- Rationale: The DLS aimed to improve disease-specific survival prediction from histopathology images without human annotations for known morphologic features or regions of interest.The motivation was the clinical importance of prognostic information for treatment and monitoring.
- Rationale: Weakly supervised models may avoid the cost, tediousness, and difficulties of region-level annotations.The passage also describes their relatively unbiased nature as a potential advantage.
- Interpretation: The DLS learned morphologic features predictive of disease-specific survival in multiple cancer types, although factors explaining performance differences remain unclear.Future work should examine sample size, image-specific variables, and disease-specific variables.
- Interpretation: DLS survival associations remained significant after adjustment for age and cancer stage, suggesting prognostic morphologic features independent of those baseline variables.The most confident prognostic regions were primarily tumor, with minimal intervening stroma or other obvious histological structures.
- Limitations: Fewer than 250 test cases and 100 disease-specific survival events per cancer type produced wide confidence intervals that limit statistical conclusions.The authors characterize the work as a proof-of-concept requiring larger-dataset development and validation.
- Limitations: TCGA’s small number of images per case and high tumor purity leave effectiveness of random patch sampling uncertain in real-world clinical settings.The authors note possible variation in tumor purity, sectioning protocols, and the number of slides per case.
- Conclusion: The study demonstrates direct clinical-outcome prediction in a weakly supervised setting without region-level expert annotations.The authors present it as providing insights and benchmarks for survival-prediction datasets and modeling approaches using TCGA.
Tables
The tables describe the TCGA dataset and evaluate associations and predictive performance for DLS and baseline clinical variables across 10 cancer studies.
- Dataset overview: The dataset comprised TCGA cases with known disease-specific survival, pathologic stage, age, and sex assigned to train, tune, and test splits in a 2:1:1 ratio.The combined cohort represents all 10 studies.
- Survival associations: Table 2 reports multivariable Cox regression associations between disease-specific survival and DLS, stage, age, sex, with study included for the combined analysis.Hazard ratios are the reported association measure.
- Predictive performance: Table 3 compares c-index performance for DLS-only, Baseline-only, and Baseline + DLS Cox regression models.Baseline consists of stage, age, and sex.
Figures
The figures illustrate the DLS pipeline, risk-group and stage-specific Kaplan-Meier analyses, and patch-level visualization, while supplementary materials document training and dataset details.
- Model pipeline: Image patches are sampled from tissue-containing regions, transformed by a CNN, averaged, and passed to a fully connected layer producing discretized survival-time probabilities.The loss divides follow-up into four discrete bins based on censorship time and outcome.
- Risk groups: Kaplan-Meier curves compare DLS risk quartiles defined within each cancer type, with green, yellow, and red denoting low, medium, and high risk.The comparison uses a binary logrank test between low- and high-risk groups.
- Stage-stratified analysis: Kaplan-Meier curves stratify DLS risk groups within cancer stage and cancer type, preserving the cancer-type distribution across groups.This stage-specific analysis repeats the within-cancer-type grouping procedure.
- Patch visualization: Risk heatmaps and selected patches visualize regions associated with high- and low-risk patch-level predictions across cases.Rows in the selected-patch panels represent different cases, with patches in the same row drawn from the same case.
- Supplementary dataset details: Supplementary Table S1 reports pathologic-stage distributions by study, using clinical rather than pathologic stage for OV.The table is a dataset-composition reference for the cancer cohorts.
- Training analysis: Supplementary Figure S1 compares censored cross-entropy, Cox partial likelihood, and an exponential lower bound on concordance index across batch sizes and learning rates.Models were evaluated on the tune split using TCGA KIRC data.