CHIMERA Challenge Task 2 and 3: Response Subtypes Classification and Progression Survival Prediction in Bladder Cancer Patients using Multimodal Datasets

Highlights

• Cross-model error analysis identified consensus-hard patients, with prediction difficulty associated with T1 substage.Source: Highlights · p. 2
• Weighted F1 of 0.73 (Task BRS) and C-index of 0.68 (Task Progression) were achieved by top-performing models in CHIMERA, the first multimodal bladder-cancer benchmark.Thirteen top-performing models were selected from 159 submissions for benchmarking.Source: Highlights · p. 2
• Task BRS histopathology partly compensated for missing pathology-derived clinicopathological variables, while progression models depended on those variables; cohort shifts and missing data limited generalization.The findings motivate missingness-aware architectures and validation in fully independent multi-institutional cohorts.Source: Highlights · p. 2

Introduction

• ˜50% of HR-NMIBC patients experience recurrence and/or progression after BCG, while traditional clinicopathological risk groups cannot fully capture heterogeneous individual outcomes.Delaying definitive treatment in patients unlikely to benefit from BCG can expose them to toxicity and postpone potentially curative interventions.Source: §1 · p. 4
• BRS3 tumors have inferior recurrence-free and progression-free survival and more BCG failure, while BRS stratification outperformed guideline-recommended risk stratification in an independent cohort.However, transcriptomic subtyping requires time- and cost-intensive sequencing workflows and finite tumor tissue, limiting routine availability [5].Source: §1 · pp. 4–5
• Routine H&E WSIs preserve morphology associated with tumor biology, and computational pathology can infer molecular phenotypes, motivating subtype prediction without relying on RNA sequencing.CHIMERA Task BRS tests prediction of RNA-seq-defined BCG response subtypes using H&E WSIs and structured clinicopathological information.Source: §1 · pp. 5–6
• RNA-seq captures molecular activity but bulk profiles obscure cell-state heterogeneity and spatial organization, whereas histology preserves tissue architecture; clinical and treatment variables add patient context.Higher representation of the high-risk class 2a state was associated with worse progression-free survival in NMIBC, and prior multimodal studies support the potential of complementary inputs for prognosis and risk stratification.Source: §1 · p. 5
• CHIMERA addresses confounded comparisons and transportability concerns with two HR-NMIBC tasks: RNA-seq-defined subtype prediction from H&E and structured data, and time-to-progression prediction from H&E, RNA-seq, and structured data.Both use multi-institutional data, hidden validation and test sets, and prespecified evaluation; the paper presents this as the first digital pathology and multimodal analysis challenge for bladder cancer.Source: §1 · pp. 5–6

2.2.1. Tumor characteristics

• The cohort included 328 Erasmus MC and 94 UroLife patients; the final Task BRS and Progression splits contained 182/17/169 and 176/17/165 training/validation/test cases, respectively.Cases missing a modality and four UroLife intermediate-risk cases were excluded; six recurrent cases were additionally excluded from Task Progression. Erasmus patients had received at least five of six BCG induction instillations, and included tumors had at least 80% tumor purity.Source: §2.3 · p. 9, §2.2.1 · p. 9

2.3.1. Structured data and reference standard

• Structured variables were harmonized across cohorts, missing values were encoded as -1, and progression required advancement to ≥T2, N+, or M+; BRS labels and progression outcomes served as task-specific references.BRS was the RNA-seq-derived reference for Task BRS and an input for Task Progression, whose reference included progression status and time to progression or end of follow-up.Source: §2.3.1 · p. 10

2.3.2. RNA-seq

• RNA-seq cohorts were quantified against GRCh38 and GENCODE 39, then normalized with DESeq2 to stabilized log-like expression values for shared protein-coding genes.Sequencing depth and library protocols differed across cohorts, and no batch-effect adjustment was performed between Erasmus Cohorts A and B.Source: §2.3.2 · p. 10

2.4. Data quality inspection

• H&E slides were digitized at 0.25 µm/pixel, while image quality and potential cohort shift were assessed using pathologist annotations and UNI patch embeddings.RNA-seq cohort shift was also assessed with housekeeping-gene PCA and a cross-validated classifier trained on the 2,000 most variable genes.Source: §2.3.3 · p. 13, §2.4 · p. 13

2.4.1. Additional supporting data

• Optional supporting training data included tissue masks, 224×224-pixel patch coordinates and UNI embeddings, tumor annotations, and slide and tissue quality labels.Tumor-region annotations covered a subset of 169 tumors and were validated by pathology experts.Source: §2.4.1 · pp. 13, 15

2.5. Evaluation metrics

• Task BRS used weighted F1 for binary BRS classification, whereas Task Progression used censored C-index to rank patients by predicted progression risk.Each task was scored once over the full test set; submissions with missing results received no score.Source: §2.5 · p. 15

2.6. CHIMERA benchmarking design

• Benchmarking covered the top eight Task BRS and five Task Progression teams, cohort-shift robustness, structured-data missingness and variable ablations, and modality-overlap analyses.Cohort comparisons used bootstrap inference; structured entries were masked at 25%, 50%, or 75%, and individual variables were removed in separate ablations.Source: §2.6 · pp. 15–16
• Additional analyses compared cohort-specific performance of H&E-only and H&E-plus-RNA-seq models, identified shared patient-level errors, and evaluated reduced structured-data models.Benchmarking used platform test submissions or resubmitted final models; teams whose original test models were unavailable retrained them, while reduced models were voluntarily retrained by participating teams.Source: §2.6 · pp. 16–17, §2.7 · p. 17

3.1. Data quality inspection

• Image and RNA-seq quality checks exposed cohort structure: patch features separated Erasmus from Urolife, and 2,000-gene expression distinguished Cohort B from Urolife (AUC 1.00).Of 182 training-set H&E slides, 169 were annotated good quality; one out-of-focus slide remained available with its defect disclosed. Housekeeping-gene PCA did not clearly separate Cohort B from Urolife, despite the broader-gene result.Source: §3.1 · p. 18

3.2. Participants, top models and summary of models

• Hidden-test estimates ranged from 0.60 to 0.73 for Task BRS and 0.57 to 0.68 for Task Progression, below team-reported development ranges.Development estimates ranged from approximately 0.63 to 0.85 for Task BRS and 0.73 to 0.91 for Task Progression; models from the top eight BRS teams and top five Progression teams were selected for analysis.Source: §3.2 · pp. 18–19, Table 3 · p. 19

3.3. Robustness

• Most teams’ Task BRS F1 decreased on Urolife, with significant declines only for HKKH and Aillis; Progression C-index was higher in Cohort B across models.Adding RNA-seq to H&E did not significantly change C-index in Cohort B (0.550 vs. 0.575; ∆C-index = +0.025, p_adj = 0.743) but increased it in Urolife (0.405 vs. 0.540; ∆C-index = +0.134, p_adj = 0.068).Source: §3.3 · p. 22

3.4. Structured data: missing values

• Random structured-data masking caused no significant Task BRS changes, whereas Progression C-indices generally fell with increasing masking in Cohort B and varied less monotonically in Urolife.In Cohort B, significant declines occurred for HKKH at 50% and 75%, SMILE at 75%, NMIL at 50%, and WL at 25% and 75% masking; Urolife baselines were lower and responses less monotonic.Source: §3.4 · p. 24, Table 8 · p. 24, Table 9 · p. 25

3.5. Structured data: missing variables

• Single-variable masking generally changed Task BRS little in Cohort B, but effects were larger for some Urolife models and for Progression models in both cohorts.Task BRS examples include HKKH F1 falling from 0.689 to 0.602 after masking age or no_instillations in Cohort B, and MITEL-UNIUD falling from 0.714 to 0.615 after masking EORTC in Urolife. In Progression, NMIL’s C-index fell from 0.699 to 0.410 after masking BRS in Cohort B and from 0.479 to 0.319 after masking sex in Urolife; these inference-time ablations are not causal estimates, and no_instillations effects may reflect treatment-course information rather than the progression label.Source: §3.5 · pp. 25–26

3.6. Reduced models - removing modality-overlapping variables

• Removing image-derived variables had small or mixed Task BRS effects; Progression full models were numerically better in Combined and Cohort B, but no reduced-model comparison remained significant after correction.For Task BRS, MITEL-UNIUD’s reduced model outperformed its full model in Combined and Urolife, while BioToTem and BUAA_REMEX changes were small; all corrected p-values were at least 0.149. Progression differences were not significant after correction (all p_adj ≥0.670).Source: Table 6 · p. 23, Table 7 · p. 23, §3.6 · pp. 26–27

3.7. Error analysis

• T1 substage differed between hard and easy patients for both tasks; some patients had consistent errors or failures across multiple models.Task BRS identified 41 hard and 35 easy patients, with five misclassified by all eight models; T1 substage differed after correction (p = 0.004, q = 0.046). Progression identified 24 hard and 30 easy patients, including one failed by all five models; T1 substage differed after correction (p < 0.001, q = 0.012), while variant histology’s association was nominal only.Source: §3.7 · p. 28, Figure 6 · p. 28, Figure 7 · p. 29

4. Discussion

• Hidden-test performance reached a weighted F1 of 0.7261 for Task BRS and a C-index of 0.6828 for Task Progression.The discussion emphasizes that leaderboard scores alone do not characterize robustness or transportability.Source: §4 · p. 29

4.1. Cohort shift and model transportability

• Progression models had higher C-indices in Erasmus Cohort B than Urolife, but cohort shift did not establish which modality caused the performance loss.A single voluntary TIA-Pegasus comparison found no significant discrimination change after adding RNA-seq in Erasmus Cohort B and no corrected-significant improvement in Urolife; modality perturbation and domain-shift robustness are distinct.Source: §4.1 · pp. 29–30

4.2. Task-dependent effects of structured information

• Task BRS models showed small average effects from masking structured variables, whereas Task Progression models were more sensitive to clinical, BRS, and BCG-exposure inputs.Instillation counts can reflect treatment course accumulated during follow-up, not baseline causal prognosis; replacing values with the missing code also changes the input pattern. BRS contributed in some progression models, but this depended on representation and architecture.Source: §4.2 · pp. 30–31

4.3. Limited incremental value of image-derived structured variables in the evaluated models

• In the evaluated reduced models, pathology-derived structured variables showed no significant incremental endpoint value after multiple-comparison correction, especially for Task BRS.This does not show that H&E encoders recovered those variables; the comparison involved few teams, and Task Progression trends favored full models without reaching significance.Source: §4.3 · pp. 31–32

4.4. Missing data handling remains necessary

• Partial random masking generally left Task BRS performance near baseline, while Task Progression responses varied by cohort and could deteriorate substantially in Erasmus Cohort B.Urolife progression responses were non-monotonic; systematic variable absence, fewer events, lower baseline discrimination, and model-specific handling of missing codes may contribute to fluctuations.Source: §4.4 · pp. 32–33

4.5. Patient-level drivers of predictive error

• Cross-model error analysis found repeatedly misclassified BRS cases and enrichment of T1e tumors among consistently misclassified progression cases.Possible explanations for BRS errors include label uncertainty or biological overlap; T1 substaging variability also complicates interpretation. The C-index ranks comparable patient pairs but does not assess calibration or predicted event-time accuracy.Source: §4.5 · p. 33

4.6. Limitations

• Progression analyses were limited by 27 events in Erasmus Cohort B and 16 in Urolife, while voluntary post-challenge analyses and missing-code perturbations constrain modality and robustness conclusions.Urolife was training-external but not fully independent external validation; treatment predictors were not restricted to a uniform pretreatment landmark, so stronger external-validity claims require independent prospective evaluation.Source: §4.6 · p. 34

5. Conclusions

• Test-set performance reached 0.726 weighted F1 for BRS prediction and 0.683 C-index for progression prediction in CHIMERA’s standardized multimodal benchmark.The benchmark combines H&E whole slide images, structured clinicopathological information, and RNA-seq in the same HR-NMIBC population, with public training and hidden evaluation.Source: §5 · p. 35
• BRS models generally tolerated removal or masking of structured variables, whereas progression models were more sensitive to clinical context, BCG exposure, and molecular subtype.BRS results support partial informational overlap between H&E representations and curated pathology features, but do not demonstrate direct recovery; progression performance was lower and less stable in Urolife, amid missing variables and strong RNA-seq cohort signal.Source: §5 · p. 35
• CHIMERA does not establish clinical readiness or universal gains from added modalities; future work calls for larger multi-institutional cohorts, missingness-aware and domain-alignment methods, calibration, and independent prospective validation.The framework supports reproducible study of modality value, cohort shift, missing-input robustness, calibration, and patient-level failure modes.Source: §5 · p. 35

CRediT authorship contribution statement

• Chia and Wang are credited with conceptualization, formal analysis, investigation, methodology, software, validation, visualization, and writing; Chia also handled data curation and project administration.Both are credited with writing the original draft and reviewing and editing the manuscript.Source: CRediT authorship contribution statement · p. 35
• Spaans and Mohammadlou contributed conceptualization and project administration, while several authors are credited with data curation and writing review and editing.Faryna is credited with conceptualization and software; Wakkee and Vermeulen with resources and supervision, with Vermeulen also credited for data curation.Source: CRediT authorship contribution statement · pp. 35–36
• Khalili is credited with conceptualization, funding acquisition, methodology, project administration, resources, software, supervision, and writing review and editing.Source: CRediT authorship contribution statement · p. 36
• A large contributor group is credited with methodology, software, and writing review and editing.The group includes the listed authors from Quan Cap through Shan E Ahmed Raza.Source: CRediT authorship contribution statement · p. 36

• Ethics approval came from Erasmus MC (MEC-2018-1097) and Arnhem-Nijmegen committees (CMO 2013-494; CMO 2005-315), with BCG treatment and follow-up guided by SWOG and EAU protocols or the treating urologist.Source: Ethics Approval · p. 36

Funding

• Funding included Hanarth Fond, NWO Vidi AIPRECISE, EU Horizon Europe CLARIFY, the Digital Research Alliance of Canada, the Research Council of Finland, and Astellas Pharma-sponsored prizes.Hanarth Fond supported Catherine Chia; AIPRECISE (09150172310031) supported Tongjie Wang; CLARIFY (grant agreement 860627) supported Tahlita Zuiverloon and Farbod Khoraminia. The sponsors had no role in study design, data collection and analysis, the decision to publish, or manuscript preparation.Source: Funding · p. 37

Figure captions

• The Task BRS patient-level error analysis found that T1 substage differed between hard and easy patients after multiple-testing correction (p < 0.05).The figure ordered patients from hardest to easiest and marked model-level errors and patients misclassified by all models.Source: Figure captions · p. 43
• Figures 4 and 5 present single-variable structured-data ablations for Task BRS and Task Progression, stratified by Erasmus Cohort B and Urolife.They plot metric differences from baseline with 95% confidence intervals and indicate significance; some Task Progression ablations could not be evaluated.Source: Figure captions · p. 43
• Figures 2 and 3 depict task data distributions, multimodal dataset components, and quality checks for histopathology images and RNA-seq data.The flow diagram notes exclusion of four Urolife tumors labeled intermediate risk as out of distribution because that category was absent from the Erasmus Cohort.Source: Figure captions · p. 43
• Figure 1 outlines the challenge workflow from public-training model development through hidden validation and test evaluation to six-part benchmarking of top-performing solutions.Source: Figure captions · p. 43
• In Task Progression, T1 substage remained significant after correction (p < 0.001, q = 0.012), whereas histology did not (p = 0.034, q = 0.207).The patient-level analysis grouped patients by prediction difficulty and displayed model discordance, failures, observed outcomes, T1 substage, and histology.Source: Figure captions · p. 44

Task BRS - Team BioToTem - Multimodal model wih ACMIL

• BioToTem’s final H0-mini model achieved internal and external F1 scores of 0.80 and 0.73, while GRIS’s ensemble achieved 0.683 and 0.6921, respectively.GRIS trained multiclass predictions for BRS1, BRS2, and BRS3, then binarized them during inference as BRS1/2 versus BRS3.Source: Task BRS - Team BioToTem - Multimodal… · p. 46, S4.1. Task BRS - Team GRIS -… · p. 47
• BioToTem analyzed H&E WSIs at 0.5 mpp with provided tissue masks and 224 x 224 patches, using augmentation instead of stain normalization; missing values were retained as a dedicated unknown category.Clinical categorical variables were one-hot encoded, and continuous variables were normalized to zero mean and unit variance.Source: Task BRS - Team BioToTem - Multimodal… · p. 46
• BioToTem compared UNI and H0-mini encoders and fused ACMIL slide-level embeddings with encoded clinical features; it selected H0-mini after cross-validation and ensembled predicted probabilities.The workflow used a fixed binary threshold of 0.5, initial 3-fold cross-validation to compare encoders and fusions, then 5-fold cross-validation on the selected encoder.Source: Task BRS - Team BioToTem - Multimodal… · p. 46

S4.1. Task BRS - Team GRIS - Multimodal model with multiclass predictions

• GRIS processed tissue-segmented H&E WSIs with CLAM at 0.5, 1.0, and 2.0 mpp using 224 x 224 patches and Macenko stain normalization.Clinical variables were one-hot encoded and normalized, with an explicit missingness dimension, yielding 23 features.Source: S4.1. Task BRS - Team GRIS -… · p. 47
• GRIS encoded H&E patches using 1536-dimensional UNI2-h features and fused them with weighted clinical features by concatenation before an MLP.Models used staged optimization, focusing first on the WSI model, then clinical data, and finally the multimodal model.Source: S4.1. Task BRS - Team GRIS -… · p. 47
• GRIS trained with 10-fold stratified cross-validation and ensembling, then binarized its BRS1, BRS2, and BRS3 multiclass predictions during inference.The binarized output grouped BRS1/2 versus BRS3.Source: S4.1. Task BRS - Team GRIS -… · p. 47

S4.2. Task BRS - Team BUAA_REMEX - Multimodal model using PANTHER

• The BUAA_REMEX team’s multimodal framework integrating histopathology and clinical data achieved internal and external F1 scores of 0.669 and 0.6726, respectively, in 5-fold cross-validation.Clinical tabular data were converted into textual representations, with missing values denoted as "not available".Source: S4.2. Task BRS - Team BUAA_REMEX -… · p. 47

S4.3. Task BRS - Team CADGEN-BIIT - Multimodal model using Evidential-Mamba

• The CADGEN-BIIT multimodal BRS model achieved internal F1 0.85 and external F1 0.6593.It fused UNI histopathology embeddings with clinical embeddings using clinically guided cross-attention and Dempster-Shafer theory; missing clinical values were median-imputed.Source: S4.2. Task BRS - Team BUAA_REMEX -… · p. 47

S4.4. Task BRS - Team WL - Unimodal model using AutoGluon-Tabular

• The WL clinical-only BRS model selected Random Forests, achieving internal F1 0.626 and external F1 0.652.AutoGluon-Tabular base learners and ensembles were evaluated, with calibrated decision thresholds chosen to maximize F1; samples missing reference standards were removed.Source: S4.4. Task BRS - Team WL - Unimodal… · p. 48

S4.5. Task BRS - Team MITEL-UNIUD - multimodal model using GAT and table-to-text encoding

• The MITEL-UNIUD multimodal BRS model achieved internal F1 0.7594 and external F1 0.6344.The team selected one-hot clinical encoding over text encoding, which underperformed, and concatenated projected clinical and graph-based slide embeddings for classification.Source: S4.4. Task BRS - Team WL - Unimodal… · p. 48

S4.6. Task BRS - Team HKKH - multimodal model using MADMIL

• The HKKH multimodal BRS model achieved internal F1 0.77 and external F1 0.6105.It fused UNI slide embeddings from multi-head attention-based deep multiple instance learning with clinical variables; missing values were not explicitly handled.Source: S4.6. Task BRS - Team HKKH -… · p. 48

S4.7. Task BRS - Team Aillis - Unimodal model using UNI2-H with single-head attention

• The Aillis unimodal histopathology BRS model achieved internal performance 0.699 and external performance 0.5981.The selected model used UNI2-H with single-head attention; the team also evaluated EfficientNet-B5 variants and excluded four WSIs with blurriness or incorrect tissue masks.Source: S4.6. Task BRS - Team HKKH -… · pp. 48–49

S4.8. Task Progression - Team SMILE - Unimodal using a discrete-time survival

• The SMILE clinical MLP survival model achieved C-indices of 0.4783 on validation and 0.6828 on test.Clinical, RNA, and multimodal configurations were evaluated using a discrete-time model with four quartile-based intervals; clinical MLP was selected as final.Source: S4.8. Task Progression - Team SMILE -… · p. 49

S4.9. Task Progression - Team HKKH - Multimodal by evaluating three fusion

• ABMIL_Surv_PG achieved internal and external C-indices of 0.91 and 0.6792, respectively, among HKKH’s three multimodal fusion strategies.The strategies were simple concatenation, low-rank bilinear fusion across modality pairs, and extended residual fusion; models used Cox proportional hazards loss.Source: S4.9. Task Progression - Team HKKH -… · p. 49
• HKKH encoded slide patches with UNI and gated-attention ABMIL, encoded RNA-seq with a two-layer feedforward network, and compared three fusion strategies for survival prediction.The final model used Cox proportional hazards loss and converted patient-specific risk scores to survival times through Kaplan–Meier estimation.Source: S4.9. Task Progression - Team HKKH -… · p. 49

S4.11. Task Progression - Team WL - Unimodal using ensemble of models

• The WL clinical-only ensemble of an MLP and random survival forest achieved internal and external C-indices of 0.762 and 0.5977, respectively.The team developed a clinical-only model to avoid modality leakage and meet Grand Challenge constraints, using 10-fold stratified cross-validation and refitting on all development data.Source: S4.11. Task Progression - Team WL -… · p. 50

S4.12. Task Progression - Team TIA-Pegasus - Multimodal using ModalSurv

• TIA-Pegasus submitted its multimodal model despite a higher internal C-index for the clinical-only model (0.749 versus 0.733); the submitted model’s external C-index was 0.574.The clinical-only model outperformed the multimodal models, and the full three-modality model underperformed the clinical-and-histopathology model.Source: S4.12. Task Progression - Team… · pp. 50–51

S5. Extended Task BRS analysis

• The extended Task BRS analysis reports predicted-probability distributions for Cohort B versus Urolife and a table comparing full and reduced structured-data model AUC across Combined, B, and U cohorts.The table defines ∆AUC as AUC(Full) − AUC(Reduced); the supplied passages do not include the table’s cell values.Source: S5. Extended Task BRS analysis · p. 51, Table S1 · p. 52

S6. Additional Robustness analysis

• Figure S2 presents TIA-Pegasus Task Progression C-indices by cohort for H&E-only versus H&E+RNA-seq submodels, with bootstrap confidence intervals and adjusted p-values.The supplied caption describes the comparison but provides no cohort-specific values or significance outcomes.Source: Figure S2 · p. 53