Source-linked AI summary
Modality Contribution Score - A Per-Patient Framework for Quantifying the Relative Diagnostic Contribution of Structural MRI and Amyloid PET in Alzheimer's Disease
Dawa Chyophel Lepcha, Aaliya Ali, Sophie A. Martin, Deepika Koundal, Pierrick Coupe, Shabbir Syed-Abdul
TL;DR
Existing multimodal AD systems do not quantify which imaging modality drove an individual diagnostic decision. This paper introduces MCNet and ablation-based MCS for per-patient MRI-versus-PET attribution, finding a monotonic increase in PET dominance across CN, MCI, and AD with biomarker validation and external replication.
Problem
Existing multimodal AI methods produce diagnostic labels without quantifying which imaging modality was more informative for a given patient.
Method
MCNet integrates structural MRI, amyloid PET, and FDG-PET features through cross-modal attention and computes normalized per-patient modality scores using ablation.
Results
MCS_PET increased monotonically from CN (0.412 ± 0.229) through MCI (0.489 ± 0.289) to AD (0.671 ± 0.426), with Kruskal-Wallis p < 0.0001.
Takeaways & Limitations
Per-patient MCS makes heterogeneity in whether MRI atrophy or PET pathology drives AD staging visible and interpretable.
Takeaways & Limitations
The cross-sectional ADNI cohort and moderate sample size limit longitudinal validation and generalisability, while incomplete FDG coverage constrains metabolic contribution estimates.
Abstract
from arXiv · showhide
Multimodal neuroimaging combining structural MRI and positron emission tomography (PET) captures complementary structure-function relationships across the Alzheimer's disease (AD) continuum, yet existing artificial intelligence systems produce a single diagnostic label without quantifying which imaging modality drove that decision for a specific patient. We introduce the Modality Contribution Network (MCNet) and the Modality Contribution Score (MCS), the first per-patient attribution framework quantifying the shift in modality dominance from structural atrophy to amyloid and metabolic dysfunction across the cognitively normal to MCI to AD continuum. MCS is normalised to unity per subject via modality ablation (MCS_MRI_i + MCS_PET_i = 1.0 for every subject i), providing an interpretable, clinically actionable score that fluid biomarkers cannot supply. Applied to 327 ADNI-3 participants balanced across cognitively normal, mild cognitive impairment, and AD groups, MCNet achieved competitive three-class staging performance (AUC=0.881). The MCS revealed a statistically significant monotonic gradient (Kruskal-Wallis p<0.0001), with increasing PET dominance from cognitively normal (MCS_PET 0.412+/-0.229) through MCI (0.489+/-0.289) to AD (0.671+/-0.426), validated against amyloid SUVR (r=0.172, p=0.006) and FDG metabolic biomarkers (r=-0.287, p=0.0005) from separate imaging pipelines. External replication in 1,073 independent OASIS-3 subjects confirmed cross-cohort generalisability (H=166.99, p<0.0001, eta^2=0.156). A mechanistic comparison with SHAP demonstrated that ablation-based MCS captures clinically meaningful modality dependence that deviation-based methods cannot. These findings position MCNet as a foundation for personalised imaging decisions, clinical trial stratification, and trustworthy AI in dementia care.
1. Introduction
AD staging requires distinguishing complementary structural, amyloid, and metabolic changes across the CN-to-MCI-to-AD continuum. MCNet addresses the gap in patient-level modality attribution by quantifying MRI versus PET contributions alongside diagnosis.
- Disease continuum: AD progresses from cognitively normal aging through MCI to dementia, with stage-specific patterns of atrophy, amyloid deposition, and metabolic dysfunction.Accurate staging supports timely intervention, clinical-trial enrolment, and personalised treatment planning.
- Multimodal biomarkers: Structural MRI captures neurodegenerative atrophy, while FDG-PET and amyloid PET measure glucose hypometabolism and fibrillar amyloid burden, respectively.These modalities provide complementary biomarker information for AD staging.
- Research gap: Existing multimodal AI systems generally classify patients without quantifying which imaging modality contributed to an individual diagnostic decision.The paper introduces MCNet and MCS to quantify relative MRI-versus-PET contribution per patient.
- Proposed framework: MCNet integrates MRI, AV45 amyloid PET, and FDG-PET features with cross-modal attention before computing ablation-based per-patient modality scores.MCS_MRI and MCS_PET are complementary scores summing to 1.0 for each subject.
2. Methods
The study uses clinically validated regional biomarker features from balanced ADNI-3 diagnostic groups. MRI represents structural atrophy, while AV45 and FDG-PET represent amyloid burden and metabolic dysfunction.
- Cohort: 327 ADNI-3 participants were selected as 109 CN, 109 MCI, and 109 AD subjects balanced across diagnostic groups.Diagnostic classification followed standard ADNI criteria.
- Data representation: Pre-computed clinically validated biomarker pipelines were used to improve reproducibility and avoid spatial misalignment in raw MRI-PET fusion.The approach uses regional features rather than raw volumetric images.
- Structural MRI: Structural MRI contributed 27 AD-sensitive FreeSurfer-derived regional volumes, including hippocampal, entorhinal, cortical, and subcortical measures.The selected regions reflect established patterns of AD-related neurodegeneration.
- Amyloid PET: Amyloid PET contributed 22 AV45 regional SUVR features, including composite SUVR, Centiloid, and selected cortical measures.Amyloid status used the UC Berkeley pipeline threshold SUVR greater than 1.11.
- FDG-PET: FDG-PET contributed MetaROI SUVR and PonsVermis SUVR, with MetaROI summarising temporoparietal glucose metabolism.MetaROI is described as a validated summary measure of AD-related hypometabolism.
2.3. Feature preprocessing
MCNet standardises and imputes regional biomarker features before processing three parallel modality streams. Modality-specific encoders and bidirectional cross-modal attention produce a fused representation for three-class diagnosis and modality contribution scoring.
- Feature preprocessing: Features were standardised within each cross-validation fold using training-data-only z-score normalisation to prevent leakage.Raw feature values are transformed using the training-fold mean and standard deviation.
- Feature preprocessing: Missing amyloid or FDG values were imputed with zero after normalisation, equivalent to imputation at the training-fold mean.Missingness occurred in a minority of subjects because of incomplete biomarker coverage.
- Architecture: Three parallel feature streams—structural MRI, amyloid PET, and FDG-PET—are processed through modality-specific encoders before fusion and classification.The architecture uses pre-computed regional and MetaROI biomarker features.
- Modality encoders: MRI and AV45 encoders map modality inputs to 128-dimensional embeddings using two fully connected layers with LayerNorm, GELU activation, and dropout p=0.3.The FDG encoder uses the same structure with a reduced hidden dimension.
- PET representation: The AV45 and FDG embeddings are concatenated and projected to 128 dimensions to form the combined PET embedding.This combines amyloid and metabolic PET information before cross-modal fusion.
- Cross-modal fusion: Bidirectional cross-modal attention lets MRI and PET representations attend to each other before concatenation and projection into a fused representation.The fused representation is passed to a classifier producing logits for CN, MCI, and AD.
2.5. Modality contribution score
MCS estimates each modality’s relative contribution by measuring how removing MRI or PET changes the predicted probability of the subject’s predicted class. The resulting MRI and PET scores are normalised to sum to 1.0 per subject.
- Ablation basis: For each subject, MCS uses the predicted class and its probability when all modalities are present as the reference classification output.The ablation procedure then evaluates modality-specific probability changes.
- Modality ablation: PET ablation sets PET features to zero, while MRI ablation sets MRI features to zero, producing modality-specific raw attribution scores from probability changes.These scores quantify the effect of removing each modality on the predicted-class probability.
- Score normalisation: MCS_MRI and MCS_PET are each obtained by dividing the corresponding raw attribution by the sum of MRI and PET attributions plus epsilon.The normalisation uses epsilon = 10^-8 to prevent division by zero.
- Score normalisation: By construction, MCS_MRI_i + MCS_PET_i = 1.0 for every subject.The scores therefore express relative rather than independent modality contribution.
2.6. Training procedure
MCNet training used AdamW optimization with weight decay and cosine-annealed learning rates over 100 epochs. Cross-entropy with label smoothing, early stopping, and batch size 16 completed the training procedure.
- AdamW training used weight decay 0.001 and an initial learning rate of 0.001, cosine-annealed to 10^-5 over 100 epochs.
- Cross-entropy loss incorporated label smoothing with smoothing factor 0.1 across three diagnostic classes.
- Early stopping used a 20-epoch patience period based on test-fold balanced accuracy, with batch size 16.
2.7. Evaluation
Performance was assessed with stratified five-fold cross-validation using balanced accuracy, macro F1-score, and AUC, with balanced accuracy designated as the primary metric because test folds could be imbalanced.
- Performance was evaluated using 5-fold stratified cross-validation with balanced accuracy, macro F1-score, and AUC as primary metrics.
- Balanced accuracy was the primary metric because class imbalance could occur in per-fold test sets.
2.8. Data partitioning and cross-validation
The 327 ADNI-3 subjects were partitioned into subject-level, stratified five-fold cross-validation splits to prevent leakage and balance diagnostic groups across folds.
- The 327 subjects were divided into five folds stratified by diagnostic group, with approximately 22 CN, 22 MCI, and 22 AD subjects per fold.
- Each fold used 261 subjects for training and 66 subjects for held-out testing.
- Subject-level partitioning ensured that no subject appeared in both training and test sets across any fold.
2.9. Statistical analysis
Statistical analysis tested diagnostic-group differences in MCS, examined its association with amyloid SUVR, compared it with SHAP, and externally validated the framework in OASIS-3.
- Kruskal-Wallis testing assessed MCS differences across diagnostic groups, with pre-specified Mann-Whitney U tests for pairwise comparisons.
- Pearson correlation assessed the relationship between MCS_PET and AV45 SUVR across 255 subjects with available amyloid data.
- SHAP explanations were computed for all 327 ADNI-3 subjects using KernelExplainer and fold-specific k-means background distributions.
- External validation used 1,073 OASIS-3 participants with preprocessed T1-weighted MRI and baseline visits selected from the earliest available MRI session.
3. Results
MCNet classified CN, MCI, and AD while quantifying each patient's relative MRI and PET contribution. PET contribution increased significantly across diagnostic stages, showed convergent biomarker associations, and differed mechanistically from SHAP attribution.
- Cohort characteristics: 327 participants comprised balanced CN, MCI, and AD groups, with amyloid data available for 255 and FDG-PET data for 143.Each diagnostic group included 109 subjects; 123 participants were amyloid-positive and 132 amyloid-negative.
- MCNet classification performance: AUC=0.881 ± 0.075 for three-class CN/MCI/AD staging across 5-fold stratified cross-validation.Mean balanced accuracy was 0.844 ± 0.061 and macro F1-score was 0.794 ± 0.090.
- MCS: group-level analysis: p < 0.0001 for both MCS_PET and MCS_MRI across diagnostic groups, with PET contribution increasing monotonically from CN to MCI to AD.MCS_PET was 0.412 ± 0.229 in CN, 0.489 ± 0.289 in MCI, and 0.671 ± 0.426 in AD; MCS_MRI showed the inverse pattern.
- MCS: per-patient analysis: All subjects lay on the MCS_MRI + MCS_PET = 1.0 anti-diagonal, separating MRI-dominant and PET-dominant attribution regions.The CN mean occupied the MRI-dominant region, MCI lay near equal contribution, and AD occupied the PET-dominant region.
- Amyloid biomarker validation: r=0.172, p=0.006, n=255 for MCS_PET versus AV45 amyloid SUVR, while FDG MetaROI SUVR showed r=-0.287, p=0.0005, n=143.The AD subgroup correlation with FDG MetaROI SUVR was r=-0.455, p=0.001, n=47.
- Comparison with SHAP-based modality attribution: SHAP produced an inverted MRI-attribution pattern relative to the expected disease progression, and SHAP_PET correlated negatively with MCS_PET (r=-0.317, p<0.0001, n=327).The paper attributes this difference to SHAP measuring deviation from a background distribution centered on the training mean.
4. Discussion
MCNet extends multimodal AD staging with per-patient modality attribution, revealing a biologically coherent shift from MRI-driven to PET-driven classification across disease progression. The framework also highlights MCI heterogeneity, while validation and methodological limitations define its current scope.
- Classification performance: AUC=0.881 ± 0.075 and bACC=0.844 ± 0.061 were achieved for three-class CN/MCI/AD staging in 327 balanced ADNI participants.The model used pre-computed regional biomarker features rather than raw volumetric images.
- Biological coherence of the MCS gradient: MCS_PET increased monotonically from CN (0.412 ± 0.229) through MCI (0.489 ± 0.289) to AD (0.671 ± 0.426; Kruskal-Wallis p < 0.0001), with an inverse MCS_MRI gradient.The pattern is presented as consistent with the temporal sequence of AD biomarker changes.
- SHAP comparison: SHAP produced an inverted modality pattern in AD, whereas ablation-based MCS yielded attribution described as statistically significant and biologically plausible.The comparison identifies reference-distribution dependence as a limitation for modality-level clinical interpretation.
- MCI balanced attribution and diagnostic uncertainty: MCI showed balanced attribution (MCS_MRI=0.474, MCS_PET=0.489), reflecting heterogeneous structural and amyloid-metabolic biomarker profiles.A representative MCI subject had near-equal attribution and only 49.6% model confidence.
- Amyloid biomarker validation: MCS_PET correlated with amyloid SUVR (r=0.172, p=0.006, n=255), providing convergent validation while measuring diagnostic modality dependence rather than continuous amyloid burden.The amyloid measurements came from an independent pipeline.
- Limitations and future directions: The study is limited by a moderate cohort, regional rather than voxel-level features, cross-sectional design, incomplete FDG coverage, linear-ablation independence assumptions, demographic concentration, and absent formal neurologist validation.Longitudinal conversion studies and broader external validation are identified as priorities.
5. Conclusion
The study presents per-patient modality attribution as a clinically interpretable addition to multimodal AD staging. MCNet makes patient-level imaging heterogeneity visible and supports modality-specific interpretation beyond categorical diagnosis.
- Conclusion: Per-patient modality attribution quantifies whether structural MRI or amyloid and metabolic PET signals drove each individual's diagnostic decision.The framework addresses a limitation identified for existing multimodal AI systems.
- Conclusion: The MCS gradient mirrors the reported temporal sequence of AD pathophysiology, shifting from structural neurodegeneration toward amyloid and metabolic PET signals as disease advances.The conclusion describes this gradient as biologically coherent and clinically meaningful.
Consent statement
The study used existing de-identified ADNI and OASIS-3 data collected under informed-consent and institutional-review procedures.
- Consent statement: ADNI and OASIS-3 participants provided written informed consent during original data collection, and ADNI had approval from participating-site institutional review boards.The ADNI study was conducted according to the Declaration of Helsinki.