PaperlayerView exact revision on arXiv

Source-linked AI summary

Leakage-Controlled Multimodal Learning for Diagnosis and Progression Prediction in Alzheimer's Disease Research

Akeem Temitope Otapo, Ghazaleh Khodabandelou, Zuheng Ming, Alice Othmani

arXiv:2610.10648v1cs.LG

TL;DR

Irregular visits, heterogeneous measurements, and missing modalities complicate Alzheimer’s disease prediction. This study integrates MRI and longitudinal clinical data in a leakage-controlled multitask framework. Three-seed internal diagnosis and stage-1 progression AUROCs are 0.935 and 0.884.

  • Problem

    Irregular visits, heterogeneous measurements, and missing modalities complicate longitudinal Alzheimer’s disease prediction, while existing work addresses these challenges in subsets.

    Source: §1 · p. 3

  • Method

    The framework combines adapted structural MRI and causal clinical streams with task-specific ODE-GRU dynamics, auxiliary comorbidity modelling, and subgroup objectives.

    Source: §1 · p. 4, §7 · p. 16

  • Results

    Three-seed internal diagnosis and stage-1 progression AUROCs are 0.935 and 0.884, with tabular-hybrid AUROCs of 0.951 and 0.909.

    Source: §7 · p. 16

  • Takeaways & Limitations

    The experiments provide a structured evaluation of multimodal multitask prediction across heterogeneous Alzheimer’s disease research cohorts.

    Source: §7 · p. 17

    Internal estimates may be optimistic because validation folds also guide model selection, threshold and temperature fitting, and hybrid weighting.

    Source: §6.4 · p. 16

Abstract

from arXiv · show

Alzheimer's disease prediction involves irregular visits, heterogeneous measurements and incomplete modalities. This study presents a multimodal multitask framework combining an adapted SFCN MRI encoder, four causal clinical Transformers, shared fusion and task-specific ODE-GRU dynamics. Fine-tuning and LoRA adapt the final two MRI blocks. Task-DRO balances task losses, while Group-CVaR targets cohort and comorbidity strata. Branch-specific input controls, subject-grouped partitions and empirical causality checks support longitudinal evaluation. Across 2,649 subjects and 17,317 visits from ADNI, OASIS-2 and MIRIAD, internal validation yields diagnosis, stage-1 progression and first-stage-1-visit progression AUROCs of 0.935 +/- 0.002, 0.884 +/- 0.003 and 0.870 +/- 0.005, respectively (mean +/- SD across three seeds). Corresponding hybrid AUROCs are 0.951, 0.909 and 0.896. Next-visit MMSE mean absolute error (MAE) is 1.61 points; worst-stratum diagnosis AUROC is 0.827 +/- 0.008. Sampled ADNI explanations identify task-specific input dependence. OASIS-3 external validation yields network and hybrid diagnosis AUROCs of 0.763 and 0.767, hybrid next-visit progression AUROC of 0.764, diagnosis calibration error decreasing from 0.197 to 0.052, and next-visit MMSE MAE of 0.86. Seed-42 paired ablations of six components yield pooled diagnosis and progression AUROC differences between -0.004 and +0.004; removing clinical encoder inputs lowers diagnosis AUROC by 0.272. The framework integrates longitudinal prediction, missing-modality handling, auxiliary comorbidity modelling and subgroup evaluation within a common pipeline.

Highlights

The framework combines task-specific input controls and irregular-visit dynamics with comorbidity-aware training. Internal and OASIS-3 results report diagnosis and progression AUROCs.

Highlights

  • ODE-GRU dynamics model irregular visits across multiple prediction tasks.The framework also uses task-specific input controls for multimodal longitudinal prediction.Source: Highlights · p. 2
  • Three-seed internal diagnosis and progression AUROCs reach 0.935 and 0.884.The OASIS-3 hybrid diagnosis and next-visit progression AUROCs are 0.767 and 0.764.Source: Highlights · p. 2
  • Comorbidity-aware training combines auxiliary supervision with subgroup auditing.Source: Highlights · p. 2

1. Introduction

AD prediction faces heterogeneous disease presentation, irregular visits, missing modalities, and risks of temporal leakage. The study proposes a joint framework with explicit controls and evaluation for these challenges.

1. Introduction

  • Irregular visits, missing modalities, and mismatched image and tabular dimensions complicate longitudinal clinical prediction.Comorbidities also contribute to heterogeneous clinical presentation and progression.Source: §1 · p. 3
  • Existing methods often separate diagnosis from prognosis and use fixed heuristic task weights that ignore task difficulty, label noise, and data availability.Source: §1 · p. 4
  • Temporal leakage can inflate reported performance and make post-hoc explanations unreliable, while few studies test leakage empirically.Source: §1 · p. 4
  • The proposed framework jointly integrates structural MRI, longitudinal clinical data, continuous-time dynamics, multitask learning, and explicit comorbidity modelling.Source: §1 · p. 4
  • The model combines an adapted SFCN encoder, four task-specific causal clinical Transformers, shared causal attention, ODE-GRU branches, and capped Task-DRO.It jointly predicts diagnosis, progression, MMSE, atrophy, and comorbidities.Source: §1 · p. 4
  • Subject-grouped partitions, causal input rules, branch-specific prohibited inputs, and empirical checks address leakage; evaluation includes seeds, ablations, subgroup audits, and OASIS-3.Source: §1 · p. 4

2. Related Work

Related work spans multimodal diagnosis and prognosis, continuous-time models, multitask learning, and robust subgroup objectives. The paper positions its architecture as a joint treatment of these areas rather than a matched-score comparison.

2.1. Multimodal Diagnosis and Prognosis

  • Published multimodal diagnosis and prognosis studies differ in outcomes, horizons, and validation protocols, so their scores are not treated as matched benchmarks.Source: §2.1 · p. 4

2.2. Continuous-Time Modelling

  • Continuous-time neural models represent elapsed time as an alternative to fixed-grid forecasting or missing-value handling.Source: §2.2 · p. 4

2.3. Multitask Learning and Task Balancing

  • Task-DRO adapts task weights to relative losses on top of fixed base weights, against prior work on adaptive weighting and shared clinical representations.Source: §2.3 · pp. 4–5

2.4. Comorbidities and Robust Learning

  • The proposed Group-CVaR penalty targets cohort and recorded-comorbidity strata, with its effect on worst strata assessed directly.Source: §2.4 · p. 5

2.5. Comparative Summary

  • The comparative table provides context across modalities, continuous-time modelling, tasks, comorbidity, and robustness, while the proposed architecture integrates these components and evaluates them through ablations.Source: §2.5 · p. 5

3. Methodology

The methodology combines MRI and task-specific causal clinical streams, then uses shared fusion and continuous-time task branches to make longitudinal predictions. MRI adaptation, input-specific clinical encoders, and ODE-GRU dynamics support this multi-task design.

3. Methodology

  • The model processes visit sequences to predict diagnosis, three-year conversion, MMSE, atrophy, and last-visit comorbidities, with each visit-level output limited to information available through that visit.The proposed-framework row reports AUROCs of 0.884 / 0.870 / 0.935 for 3-y MCI conversion / sMCI-pMCI / diagnosis; Table 1 is context rather than a head-to-head ranking.Source: §3 · p. 5, §3.2 · p. 6

3.1.1. MRI Encoder

  • The MRI stream adapts the pretrained SFCN by fine-tuning its last two blocks with LoRA and adding an auxiliary cohort-defined diagnosis objective.Visits without MRI use a learnable embedding, and modality dropout replaces a subject’s MRI stream during training with probability 0.05.Source: §3.1.1 · p. 5

3.1.2. Task-Specific Causal Clinical Encoders

  • Four task-specific clinical Transformers use distinct input sets and causal masks, so a visit cannot attend to later visits.Each encoder combines time-varying and temporal features and applies a two-layer, four-head Transformer with causal and key-padding masks.Source: §3.1.2 · p. 5

3.2. Shared Causal Attention Fusion

  • A shared fusion module combines MRI and each task’s clinical stream through causal, padding-masked attention, then projects and adapts the fused representation for that task.Its learned modality weights are global across subjects, visits, and tasks, rather than patient-specific explanations.Source: §3.2 · p. 5
  • Table 1 reports 0.884 / 0.870 / 0.935 for 3-y MCI conversion / sMCI-pMCI / diagnosis, but differing cohorts, endpoints, horizons, and validation protocols prevent head-to-head ranking.The table compares selected models’ capabilities and reported results as context.Source: §3.2 · p. 6, Table 1 · p. 6

3.3. Continuous-Time Dynamics: ODE-GRU

  • Each task branch evolves its hidden state between irregular visits with an ODE-GRU, then updates it with the fused observation at a visit.The evaluated configuration integrates between visits with midpoint RK2 and two substeps; the continuous-time state is defined between visits.Source: §3.3 · p. 6

3.4. Prediction Heads

Prediction heads use evolved task-branch states for classification and next-visit residual forecasts, while current MMSE and atrophy use task-adapted representations. The heads also model comorbidities from the diagnosis branch’s final-visit state.

3.4. Prediction Heads

  • Classification and next-visit residual heads use evolved branch states, while current MMSE and volume predictions use their task-adapted representations.The comorbidity head predicts nine labels from the diagnosis-branch state at the last visit.Source: §3.4 · pp. 6–7
  • The current diagnosis head predicts a cohort-defined binary endpoint, with an auxiliary cohort-mapped three-way stage head trained on visits including MCI.A direct head pathway uses visit-available demographics, baseline imaging and fluid biomarkers, creatinine, and family history; family history is treated separately from the participant’s medical history.Source: §3.4 · p. 7
  • The atrophy target is nWBV in OASIS-2 and intracranial-volume-normalized hippocampal volume in ADNI, standardized per cohort on the training fold.Source: §3.4 · p. 7
  • Next-visit MMSE and atrophy forecasts predict a residual relative to the current observed value, or the model’s current estimate when that value is unobserved.The head is not conditioned on the time to the next visit, so results refer to the next observed visit rather than a fixed horizon.Source: §3.4 · p. 7
  • The nine-output comorbidity head reads the diagnosis-branch state at the last visit, allowing comorbidity information to shape the diagnosis representation.Source: §3.4 · p. 7

3.5. Robust Optimisation Objective

The objective combines observed-label task losses with task balancing, group robustness, and trajectory regularisation.

3.5.8. Final Objective

  • The training objective combines task balancing, subgroup robustness, and trajectory regularisation with an image-level auxiliary loss and residual-forecast penalty.The residual-forecast penalty uses λ∆ = 0.05 and applies after the first five anchor epochs.Source: §3.5 · p. 7, §3.5.8 · p. 8

3.5.1. Base Task Losses

  • Diagnosis, conversion, MMSE, atrophy, and comorbidity use task-specific losses, and imputed values are never used as targets.Diagnosis uses label-smoothed cross-entropy, conversion focal loss, MMSE and atrophy Huber loss, and comorbidity binary cross-entropy.Source: §3.5.1 · p. 7

3.5.2. Comorbidity Mask

  • Comorbidities unavailable outside ADNI remain unknown rather than being treated as absent, and comorbidity losses and analyses use observed records only.This avoids conflating missing information with no comorbidity.Source: §3.5.2 · pp. 7–8

3.5.3. Task-DRO: Scale-Free Dynamic Task Balancing

  • Task-DRO balances relative losses using moving averages and capped softmax weights, preventing any single task from dominating.A task whose loss exceeds its recent average receives a larger adaptive weight; weights are capped at 0.3.Source: §3.5.3 · p. 8

3.5.4. Group-CVaR over Cohorts and Comorbidity Strata

  • Group-CVaR penalizes higher mean losses across cohort and recorded-comorbidity strata, using the 70th-percentile group-loss threshold.Comorbidity strata are used only for subjects with observed comorbidity records; the penalty is skipped when too few groups or subjects are present.Source: §3.5.4 · p. 8

3.5.5. Monotonicity Constraints

  • For subjects ever labelled with dementia, a soft penalty discourages increases between consecutive current-value MMSE and atrophy estimates.Atrophy targets are volume measures, so the discouraged direction is increasing volume; upward predictions and fluctuating measurements remain possible.Source: §3.5.5 · p. 8

3.5.6. Longitudinal Contrastive Regularisation

  • A symmetric NT-Xent loss on projected states from each subject’s last two visits encourages subject-specific, temporally consistent diagnosis representations.The contrastive loss uses cosine similarity, κ = 0.1, and λcon = 0.05.Source: §3.5.6 · p. 8

3.5.7. Components Deliberately Disabled

  • Uncertainty weighting is deliberately omitted to avoid adding a second task-weighting mechanism alongside Task-DRO.Source: §3.5.7 · p. 8

3.6. Network–Tabular Hybrid and Calibration

The hybrid combines network and tabular scores, with validation-fitted weights and temperature scaling; the training algorithm applies the joint objectives and selects checkpoints on validation predictions.

3.6. Network–Tabular Hybrid and Calibration

  • The hybrid rank-mixes the network with a training-fold tabular stack, whose validation-selected cohort weights combine class-balanced logistic regression and histogram gradient boosting.The tabular stack uses the same causal features as the corresponding network branch and no MRI.Source: §3.6 · p. 8
  • Temperature scaling is fitted on validation predictions, making internal calibration in-sample for temperature fitting, while external OASIS-3 calibration applies internal-fold temperatures unchanged.Unless stated otherwise, reported model claims refer to the network alone; the hybrid is complementary.Source: §3.6 · p. 8

3.7. Training Algorithm

  • Training encodes MRI and clinical streams, computes observed-label task losses and robustness and auxiliary terms, updates parameters, and selects checkpoints using validation predictions.The procedure fits temperature and conversion thresholds on validation predictions and the tabular stack on training-fold data.Source: §3.6 · p. 9, §3.7 · p. 9

4. Experimental Setup

The experimental setup combines three longitudinal development cohorts, cohort-specific endpoints and inputs, causal preprocessing, subject-grouped validation, and explicit leakage checks.

4.1. Datasets

  • The development set contains 2,649 subjects and 17,317 visits from OASIS-2, MIRIAD, and ADNI, with OASIS-3 held out for external testing.The assembled ADNI subset contains 2,430 subjects; the development set includes 1,081 indexed MRI acquisitions from 219 subjects.Source: §4.1 · p. 9
  • Diagnosis labels are cohort-defined, and the nominal 3-year progression label marks a first observed stage increase within the horizon under specified follow-up rules.A negative progression label requires at least 2.85 years of follow-up; shorter follow-up without progression is unknown, and MIRIAD has no stage-1 at-risk visits.Source: §4.1 · p. 9, Table 3 · p. 9

4.1.3. ADNI

  • Available inputs differ by cohort: OASIS-2 and MIRIAD lack several modalities and medical history, while ADNI supplies broader clinical, biological, and imaging measures.ADNI raw MRI and variables used to define comorbidity labels are excluded from inputs; its nine rule-derived comorbidities are unavailable for OASIS-2 and MIRIAD, where they remain unknown.Source: §4.1.1 · p. 9, §4.1.2 · p. 9, §4.1.3 · p. 9

4.2.3. Branch-Specific Prohibited Inputs

  • Branch-specific exclusions prevent target-related inputs, including cognitive measures and CDR from diagnosis and CDR from all branches.Forecasting may use cognitive tests and ADNI stage information known at forecast time; an assertion checks each branch against its prohibited inputs.Source: §4.2.3 · p. 10

4.2.5. Empirical Leakage Checks

  • Five-fold validation groups every subject’s visits and MRI files together, and per-fold checks test partition separation, feature causality, and model causality.Causality checks recompute features after deleting later visits and test whether outputs through visit k change when later visits are removed.Source: §4.2.4 · p. 10, §4.2.5 · p. 10

4.4. Evaluation Metrics

  • Training uses three seeds and a validation selection score combining diagnosis, conversion, MMSE, and comorbidity performance; evaluation includes pooled out-of-fold metrics and subgroup summaries.The evaluation protocol also defines calibration, regression, comorbidity, worst-stratum, and paired-comparison metrics; the supplied Table 5 caption identifies full-model results on internal five-fold validation.Source: §4.3 · p. 10, §4.4 · p. 11

4.2. Preprocessing and Leakage Control

  • Longitudinal features use last observation carried forward without back-filling, with availability and elapsed-time indicators; transformations are fitted on training folds only.MRI is processed per volume, without dataset-level intensity statistics, registration, or voxel-spacing resampling.Source: §4.2.1 · p. 10, §4.2.2 · p. 10, Figure 2 · p. 11

4.5. Ablation Protocol

  • The seed-42 ablations reuse the same five folds and remove individual components or modalities, including a clinical-input removal variant and a GRU without continuous-time integration.The Table 5 caption reports full-model internal validation, with seed-42 pooled out-of-fold estimates and three-seed mean and standard deviation.Source: §4.5 · p. 11, Table 5 · p. 12

5. Results

The full model performs across internal and external diagnosis and progression evaluations, while ablations, cohort and subgroup analyses, and sampled explanations qualify where performance and input dependence vary.

5.1. Main Results

  • Diagnosis AUROC is 0.936 [0.925, 0.948] (0.935±0.002 over three seeds), and 3-year stage-1 progression AUROC is 0.881 [0.858, 0.902] (0.884±0.003); the hybrid reaches 0.951 and 0.909.First-stage-1-visit AUROC is 0.872 (0.870±0.005) for the network and 0.896 for the hybrid. The tabular stack alone reaches 0.948, 0.908 and 0.894; next-visit MMSE MAE is 1.62 points, and temperature scaling lowers diagnosis ECE from 0.135 to 0.031.Source: §5.1 · p. 12

5.2. External Validation on OASIS-3

  • On OASIS-3, hybrid diagnosis AUROC is 0.767 and next-observed-visit progression AUROC is 0.764; next-visit MMSE MAE is 0.86.Transferred temperature scaling reduces diagnosis ECE from 0.197 to 0.052. External progression uses the next observed visit, not the internal fixed 3-year horizon.Source: §5.2 · p. 12, Table 6 · p. 13

5.3. Context with Published Results; 5.4. Ablation Studies

  • Across component ablations, diagnosis, MCI-visit conversion, and sMCI/pMCI AUROC differences lie between −0.0038 and +0.0036; removing clinical encoder inputs lowers diagnosis by 0.272.Removing the auxiliary comorbidity task gives diagnosis and MCI-visit conversion differences of +0.001 and −0.002, respectively; without longitudinal contrastive loss, mean comorbidity AUROC falls from 0.565 to 0.540. Scores from studies with different cohorts, label definitions, horizons, and validation designs provide context rather than a ranking.Source: §5.3 · p. 12, §5.4 · p. 13

5.4. Ablation Studies; 5.5. MRI Encoder Comparison; 5.6. Continuous-Time Dynamics

  • Without LoRA, worst-stratum diagnosis AUROC is 0.891 versus 0.819 for the full model; no statistically detectable AUROC difference was observed between ODE integration and a time-aware GRU.The LoRA comparison uses single-seed estimates and small worst strata. The ODE additionally provides a latent state between visits; SFCN and ResNet-50 have comparable AUROC, while SFCN uses about one seventh of ResNet-50’s total parameters and is retained.Source: §5.4 · p. 13, §5.5 · pp. 13–14, §5.6 · p. 14

5.7. Performance per Cohort

  • Diagnosis AUROC varies by cohort: the fused model reaches 0.947 on ADNI, 0.629 on OASIS-2 and 0.424 on MIRIAD, where its diagnosis branch has fewer permitted clinical inputs.The MIRIAD score is no better than chance; the imaging-focused comparator reaches 0.597 on MIRIAD and 0.621 on OASIS-2. Cohort-specific endpoints are heterogeneous, so pooled performance does not imply uniformly strong discrimination.Source: §5.7 · p. 14, Table 9 · p. 14, Figure 4 · p. 14

5.8. Comorbidity Prediction; 5.9. Subgroup Robustness

  • The auxiliary comorbidity head has mean AUROC 0.565, while diagnosis ranges from 0.819 in COPD to 0.954 in cancer and conversion is lowest in stroke at 0.765.The head excludes condition-defining records; diabetes and hyperlipidaemia AUROCs are near chance at 0.512 and 0.519. The COPD conversion estimate is uncertain, based on five subjects with a positive conversion label; subgroup analyses are ADNI-only.Source: §5.8 · p. 14, §5.9 · p. 15, Table 10 · p. 15

5.10. Interpretability; 5.10.1. Group Occlusion

  • Group occlusion estimates joint input-group dependence: removing structural volumes costs 0.099±0.027 AUROC for diagnosis, while removing cognitive tests costs 0.062±0.018 for conversion.The analysis uses a capped ADNI subset without raw MRI; occlusion can induce out-of-distribution missingness and is sensitive to correlated groups. Progression explanations are not restricted to the MCI population used for the primary conversion metric.Source: §5.10.1 · p. 15, §5.10 · p. 15

5.10.2. Top Features

  • Diagnosis feature rankings lead with hippocampal and entorhinal volume and FDG-PET, whereas conversion rankings lead with FAQ, hippocampal volume and FDG-PET.Conversion rankings also include ADAS-13, amyloid and tau measures, MMSE and RAVLT immediate recall; attribution magnitudes describe model reliance, not causal effects.Source: §5.10.2 · p. 15

6. Discussion

The discussion presents a unified model and evaluation, while emphasizing that performance depends on input availability and population and that internal estimates and explanations have important scope limits.

6.1. Main Findings

  • The framework jointly predicts diagnosis, stage-1 progression, cognitive outcomes, atrophy and comorbidities; sampled ADNI explanations show diagnosis reliance on imaging and biomarkers and conversion reliance on FAQ and episodic memory.Paired ablations show pooled AUROC differences of at most 0.004 for selected endpoints and seed; the authors characterize these as sensitivity analyses of this configuration.Source: §6.1 · pp. 15–16

6.2. Continuous-Time Modelling

  • No statistically detectable AUROC difference was observed between ODE integration and a time-aware GRU, while the ODE provides a latent state between visits for potential chosen-horizon forecasting.Source: §6.2 · p. 16

6.3. Comorbidity and Modality Contributions

  • The comorbidity head predicts nine conditions without their defining records, while stroke is the lowest-performing conversion stratum and imaging-focused diagnosis is stronger than the fused model on MIRIAD.The imaging-focused comparator reaches 0.597 on MIRIAD versus 0.424 for the fused network; the sampled ADNI attribution analysis does not cover individual raw-image localization.Source: §6.3 · p. 16

6.4. Limitations and Future Work

  • Internal estimates may be optimistic because validation folds guide model selection, threshold and temperature fitting, and hybrid weighting; ablations use a single seed.The discussion also notes cohort-defined endpoints, rule-derived comorbidity labels without independent clinical adjudication, and unavailable raw MRI for ADNI.Source: §6.4 · p. 16

7. Conclusion

The paper concludes that its integrated framework supports multimodal prediction across irregular visits and missing modalities. Internal and OASIS-3 results provide a structured evaluation across cohorts, with explanations and subgroup analyses contextualizing performance.

7. Conclusion

  • The framework combines an adapted SFCN encoder, causal clinical streams, ODE-GRU dynamics, task balancing, subgroup objectives and input controls for diagnosis, progression, cognitive and volume prediction, and comorbidity modelling.Source: §7 · p. 16
  • Three-seed internal validation yields diagnosis and stage-1 progression AUROCs of 0.935 and 0.884, increasing to 0.951 and 0.909 with the tabular hybrid.Source: §7 · p. 16
  • On OASIS-3, hybrid diagnosis and next-visit progression AUROCs are 0.767 and 0.764, with diagnosis calibration improvement and next-visit MMSE MAE of 0.86.Task-specific explanations and comorbidity-stratified evaluation connect predictions with input and population context; raw MRI perturbation is not evaluated in the cited subset.Source: §7 · p. 17

CRediT authorship contribution statement

The contribution statement assigns authorship across conceptualization, methods, investigation, validation, analysis, data curation, supervision, and writing.

CRediT authorship contribution statement

Loading 2610.10648v1…