PaperlayerView exact revision on arXiv

Source-linked AI summary

Evidence Before Accuracy: A MRI-PET Fusion Network for Alzheimer Disease Classification with Causal Regional Validation

Saeid Firouzi Daghigh, Saeed Ayat

arXiv:2609.34520v1cs.CV

TL;DR

AD classifiers can report high discrimination despite evaluation weaknesses or reliance on non-disease cues, and saliency alone does not establish regional dependence. This paper tests a six-stream MRI-PET model with controlled regional and attribution analyses. It reports competitive classification performance and a modality-specific pattern of regional evidence.

  • Problem

    High AD-classification results can be undermined by leakage or confounds, while saliency maps do not establish whether decisions depend on relevant tissue.

    Source: §I · p. 1

  • Method

    The authors evaluate a six-stream MRI-PET fusion model using shortcut controls, permutation nulls, regional ablation, and quantitative attribution comparisons.

    Source: §I · p. 2

  • Results

    AUC 0.962, accuracy 0.909, and F1 0.891; ablation and attribution converge on MRI medial-temporal and PET posterior-cingulate evidence.

    Source: §VII · p. 11

  • Takeaways & Limitations

    The authors argue that clinician trust requires verifying that explanations are grounded in evidence clinicians would use, not merely producing explanations.

    Source: D. Clinical Framing · p. 11

    All data come from ADNI; the authors state that external validation on an independent cohort with different scanners and demographics is required before generalization claims.

    Source: §VI · p. 11

Abstract

from arXiv · show

Deep learning models for Alzheimer disease (AD) classification routinely report near-perfect discrimination, yet few are shown to rest on AD-relevant neurobiology rather than on dataset artifacts, subject-level leakage, or non-brain image content. We present a fusion network combining T1 MRI and FDG PET across axial, coronal, and sagittal planes, trained on ADNI consists of 554 paired subjects. The fusion model reaches AUC 0.962, accuracy 0.909, and F1 0.891, competitive with recent 3D CNN and multimodal transformer systems at substantially lower cost. We first quantify how much modality, plane and slice geometry matter. A validation-only search over slice centres and neighbour spacings moves AUC by 0.180 for MRI and 0.078 for PET, selecting narrow spacing for MRI and wide spacing for PET, with the chosen coronal centres falling on the hippocampal body and on the posterior cingulate respectively. The contribution, however, is the evidence layer built around that number. Shortcut controls collapse the model to AUC 0.622 (silhouette), 0.608 (exterior), and 0.500 (blank), and a label-permutation null yields 0.456. Forward region-of-interest (ROI) ablation shows that masking medial temporal cortex in MRI and the posterior default-mode network (DMN) in PET produces the largest shift in the AD logit, while area-matched controls remain indistinguishable from that null. Reverse ROI ablation shows that the medial temporal lobe alone retains 89.2% of above-chance discrimination in MRI and the posterior DMN alone retains 79.0% in PET. A quantitative comparison of attribution methods shows occlusion sensitivity reaching 3.5-5.0* enrichment inside a priori AD regions against 0.10-0.43* in controls. Ablation and attribution independently establish a biologically correct double dissociation: hippocampal evidence is carried by MRI, posterior cingulate evidence by PET.

I. INTRODUCTION

AD imaging classifiers face both validation problems and a deeper question: whether their decisions rely on disease-relevant tissue rather than cohort-specific shortcuts. This work frames multimodal MRI-PET classification and quantitative explanation tests as ways to address that gap.

A. Multimodal MRI-PET Deep Learning

  • MRI indexes medial temporal atrophy, while FDG-PET indexes temporoparietal and posterior cingulate hypometabolism, providing partially complementary AD evidence.The cited review reports multimodal models consistently outperforming unimodal models across datasets and architectures.Source: §I · p. 1

I. INTRODUCTION

  • The central gap is whether high-performing AD classifiers rely on disease-relevant tissue, since leakage, confounds, and unvalidated saliency can make their results difficult to interpret.Slice-level splitting and test-set checkpoint selection undermine evaluation, while saliency indicates gradient locations rather than whether decisions depend on those tissues.Source: §I · p. 1
  • The paper proposes a six-stream VGG16-BN fusion model and an evidence ladder testing shortcuts, modality contribution, regional necessity and sufficiency, and attribution localization.Its stated contribution is evidence that the model depends on hippocampal structure in MRI and posterior default-mode metabolism in PET.Source: §I · pp. 1–2

B. Explainability in AD Neuroimaging

  • Prior AD explainability work maps neuroimaging biomarkers onto cognition and interrogates top TADPOLE challenge methods; reviews, catalogue methods, while,, distinguish producing an explanation from verifying it.The paper evaluates occlusion, Grad-CAM, Grad-CAM++, and Layer-CAM under identical conditions.Source: B. Explainability in AD Neuroimaging · p. 2, C. Attribution Methods · p. 2

D. Positioning

The paper emphasizes validation and explainability over architectural novelty, using a paired ADNI cohort and a preprocessing pipeline that treats MRI and PET differently.

D. Positioning

  • The paper claims its validation and explainability protocol is more informative than incremental AUC improvement, without claiming architectural novelty.The protocol combines shortcut controls, permutation nulls, regional ablation with area-matched controls, and quantitative attribution benchmarking.Source: D. Positioning · p. 2

A. Dataset

  • The paired ADNI cohort contains 554 subjects—324 cognitively normal and 230 with AD—with subject-level splits and a physically separated test partition.The 356 training, 88 validation, and 110 test subjects were split before slice extraction; the test partition was not read during development or tuning.Source: A. Dataset · p. 2

B. Preprocessing

  • Preprocessing uses MRI bias correction and brain extraction, leaves PET native, registers PET to T1 and T1 to MNI, then applies modality-specific intensity normalization.MRI is z-scored within the brain mask; PET is converted to SUVR using an eroded pons reference.Source: Figure 1 · p. 3, B. Preprocessing · p. 3

1) Does Nonlinear Registration Remove the Atrophy Signal?:

The authors test whether nonlinear registration preserves atrophy signal and report that it does in their cohort. Their validation-only slice search also identifies anatomically coherent, modality-specific configurations.

1) Does Nonlinear Registration Remove the Atrophy Signal?:

  • r = 0.976 native-to-MNI CSF-fraction correlation, 0.902 retained between-subject spread, and near-identical mean absolute log-Jacobians for AD (0.198) and CN (0.192) support retained morphological signal.The authors conclude that the warp is not selectively compressing AD brains toward the template.Source: 1) Does Nonlinear Registration Remove… · p. 3

C. Slice Selection

  • The selected MRI centers pass through the hippocampal body, amygdala, and head, while PET centers intersect posterior cingulate, precuneus, and parietal association cortex.MRI uses narrow spacing (∆ = 4 mm) to preserve fine structural detail; PET uses wide spacing (∆ = 16 mm) for more diffuse deficits and coarser resolution.Source: C. Slice Selection · pp. 3–4
  • Validation AUC ranges from 0.704 to 0.884 across MRI slice choices and from 0.869 to 0.947 across PET choices, indicating greater MRI position sensitivity.The authors interpret the contrast as structural signal concentrated near the hippocampus and metabolic signal distributed across posterior slices.Source: C. Slice Selection · p. 4

D. Architecture

The classifier combines six independently processed MRI and PET plane streams. Its staged initialization transfers plane-specific neuroimaging weights into later models, with validation-based checkpoint selection and five-fold cross-validation.

D. Architecture

  • Six MRI/PET-by-plane streams use independent VGG16-BN trunks, global average pooling, concatenated descriptors, dropout, and a linear two-logit output.Each stream produces a 512-dimensional descriptor; the six descriptors form a 3072-dimensional joint representation.Source: Figure 3 · p. 5, D. Architecture · p. 5

1) Stream Initialization:

  • Transferring trained single-plane weights into tri-planar models is reported as the largest contributor to MRI and PET results.The final six-stream model inherits trunks from the trained tri-planar models; the layer-freezing policy stays identical across stages.Source: 1) Stream Initialization · p. 5

E. Training

  • All reported figures use complete 5-fold cross-validation, with checkpoints selected on validation AUC rather than test performance.Training used AdamW at 1×10−5, batch size 16, and 60 epochs, with random affine augmentation applied only to training data.Source: E. Training · p. 5

F. Explainability Protocol

The explainability protocol tests shortcut dependence, label-free discrimination, regional necessity and sufficiency, and attribution localization using controlled evaluations.

F. Explainability Protocol

2) Label-Permutation Null:

  • Four input conditions—intact, silhouette, exterior, and blank—test whether non-brain content or outer contours retain discriminative power.The pipeline is also rerun with shuffled training labels to estimate discrimination without real class structure.Source: 2) Label-Permutation Null · p. 5

3) Forward and Reverse ROI Ablation:

  • Forward ROI ablation masks atlas-defined regions to assess necessity, while reverse ablation retains only each ROI to assess sufficiency.Both are compared with area-matched controls and a spatial permutation null; reverse ablation addresses redundancy across the six streams.Source: 3) Forward and Reverse ROI Ablation · p. 5

4) Attribution Benchmarking:

  • Four attribution methods are evaluated by ROI enrichment and peak-voxel pointing-game rates, separately for AD and CN subjects.The methods are occlusion sensitivity, Layer-CAM, Grad-CAM, and Grad-CAM++; each is compared with chance or area-implied rates.Source: 4) Attribution Benchmarking · p. 5

IV. RESULTS

The results show complementary information across planes and modalities, with tri-planar initialization and MRI-PET fusion improving performance. The fusion model is competitive with published systems, and its test errors are symmetric.

A. Single-Plane, Tri-Planar, and Multimodal Performance

  • PET outperforms MRI in every single plane (0.900–0.918 versus 0.877–0.897), while MRI is strongest sagittally and PET planes perform comparably.Reported accuracies are MRI sagittal 0.845, coronal 0.764, axial 0.736; PET axial 0.836, coronal 0.818, sagittal 0.827.Source: A. Single-Plane, Tri-Planar, and… · p. 6
  • Single-plane initialization raises tri-planar MRI from 0.909 to 0.944 AUC and 0.818 to 0.873 accuracy, and PET from 0.928 to 0.936 AUC.The pretrained tri-planar model also exceeds the best single plane.Source: A. Single-Plane, Tri-Planar, and… · p. 6

B. Comparison with Published Methods

  • The model reaches parity with 3D CNNs and multimodal transformers at a fraction of the computational cost, without a claim of superiority.DiaMond and ResNet late fusion [30] report marginally higher AUC; balanced accuracy exceeds all listed methods except DiaMond.Source: Table VII · p. 6, B. Comparison with Published Methods · p. 6

C. Error Structure

  • Sensitivity is 89.1% (41/46), specificity 92.2% (59/64), and the fusion model's error distribution is symmetric.The authors note this symmetry matters because a model using a class-correlated confound typically produces asymmetric errors.Source: Table VIII · p. 6, C. Error Structure · p. 6

A. Single-Plane, Tri-Planar, and Multimodal Performance

  • AUC 0.962, accuracy 0.909, and F1 0.891 for multimodal fusion improve on the better unimodal model by 0.018 AUC, 0.036 accuracy, and 0.047 F1.The authors report that differences from leading published methods fall within overlapping confidence intervals and do not claim superiority.Source: A. Single-Plane, Tri-Planar, and… · p. 6

D. Modality Ablation

Modality, preprocessing, shortcut, and regional ablations probe what supports the fusion model's decisions. The analyses distinguish modality contributions and identify MRI medial-temporal and PET posterior-DMN evidence as influential.

D. Modality Ablation

  • Suppressing MRI or PET at inference degrades fusion performance comparably, by 0.030 and 0.024, respectively.With MRI suppressed, fusion reaches 0.932, slightly below the PET-only model's 0.936; the authors interpret this as evidence of cross-modal interaction terms.Source: D. Modality Ablation · p. 7

E. Preprocessing Ablation

  • Removing preprocessing costs MRI 0.084 AUC but PET only 0.006, while the fusion model loses 0.028.The authors attribute the difference to MRI's dependence on bias correction, skull stripping, and spatial normalization for morphological comparison; PET's diffuse intensity signal tolerates substantial misalignment.Source: E. Preprocessing Ablation · p. 7

F. Shortcut Controls and the Permutation Null

  • AUC is 0.500 for blank input, 0.622 for silhouette, 0.608 for exterior, and 0.456 under shuffled-label training.The authors report silhouette and exterior retain approximately 76% and 77% less above-chance performance; the permutation result is statistically indistinguishable from chance.Source: F. Shortcut Controls and the… · p. 7, Table XII · p. 7
  • The PET exterior residual of 0.608 remains incompletely explained; retained scalp and skull-adjacent uptake and scanner-related differences may contribute.PET is not skull-stripped, and global count-rate and reconstruction differences may correlate with scanner generation and cohort composition.Source: F. Shortcut Controls and the… · p. 7

G. Forward ROI Ablation: Necessity

  • MRI medial-temporal regions exceed area-matched controls against the spatial null: the composite reaches Z = +20.3, versus −1.3 for the control composite.Individual hippocampus, amygdala, and parahippocampal values are +5.4, +3.0, and +4.7; the control composite is 4,297 px versus 4,361 px for medial temporal.Source: G. Forward ROI Ablation: Necessity · pp. 7–8
  • The PET posterior-DMN composite reaches Z = +25.9 versus +1.4 for its control composite; posterior cingulate ∆logitAD is 0.274 despite ∆AUC of 0.011.The posterior-cingulate ∆AUC is below precuneus at 0.016, while its ∆logitAD is higher; the two measures capture different effects.Source: G. Forward ROI Ablation: Necessity · p. 8

1) The Double Dissociation:

Reverse ROI ablation shows that MRI’s medial temporal composite and PET’s posterior DMN can each preserve much of the intact model’s discrimination. The results support a modality-specific division of labor, with controls performing substantially worse.

1) The Double Dissociation:

  • The modality-specific effects match the reported biological division: medial temporal effects are large in MRI, while posterior DMN effects are large in PET.The corresponding effects are not drivers in the other modality.Source: 1) The Double Dissociation · p. 8

H. Reverse ROI Ablation: Sufficiency

  • MRI medial temporal composite alone retains 89.2% of discriminative power, while PET posterior DMN alone retains 79.0%.The MRI composite reaches AUC 0.912 with ρ = +0.80; the PET posterior DMN has ρ = +0.77.Source: H. Reverse ROI Ablation: Sufficiency · p. 8, 1) The Double Dissociation · p. 8
  • MRI’s area-matched control composite reaches AUC 0.476, compared with AUC 0.912 for the smaller medial temporal composite.Doubling the control area yields AUC 0.646, retaining 31.6% of discriminative power.Source: H. Reverse ROI Ablation: Sufficiency · p. 8
  • In PET, the posterior DMN retains 79.0% of discrimination, compared with 20.1% for the area-matched control and 13.2% for the enlarged control.The enlarged control has 53% more visible area.Source: H. Reverse ROI Ablation: Sufficiency · p. 8
  • The MRI amygdala retains 81.8% of discriminative power from 1,011 pixels, compared with 76.2% for the larger hippocampus ROI.The authors note that amygdalar atrophy in AD is well established and may be underweighted relative to hippocampal measures.Source: H. Reverse ROI Ablation: Sufficiency · p. 8

I. Attribution Method Comparison

Occlusion sensitivity is the only compared attribution method that meets both the enrichment and class-discrimination criteria. Attribution results independently show modality-specific localization, while the other methods have distinct limitations.

I. Attribution Method Comparison

  • MRI coronal and sagittal streams show 2.65–4.17× enrichment in medial temporal structures, while PET coronal shows 4.99× in posterior cingulate and 3.96× in precuneus.PET coronal enrichment is below chance in medial temporal structures (0.38 and 0.61), and control regions are typically 0.10–0.43.Source: §I · pp. 8–10, Table XVII · p. 9
  • Grad-CAM and Grad-CAM++ concentrate on cortical rims and image borders, whereas occlusion and Layer-CAM concentrate inside the outlined regions in the corresponding modality.The authors attribute the Grad-CAM boundary pattern to the 7 × 7 final feature map, whose cells cover roughly 32 × 32 input pixels.Source: Figure 4 · p. 9, §I · p. 10
  • Slice-plane ROI projection is a limitation: lingual-gyrus enrichment reaches 1.69 in MRI sagittal and 3.26 in PET sagittal, but the authors do not treat these as evidence of lingual involvement.At s = −25 mm, the occlusion window necessarily overlaps adjacent structures near the slice boundary.Source: §I · p. 10
  • Occlusion achieves 3.5–5.0× enrichment in AD regions with 0.10–0.43× in controls and separates AD from CN in the pointing game across every stream.The largest reported AD–CN gaps are MRI coronal (52.2% vs. 29.7%) and PET coronal (54.3% vs. 23.4%).Source: Table XVIII · p. 9, §I · p. 10, Table XIX · p. 10
  • Layer-CAM reaches a 69.6% raw pointing-game rate but points to the posterior DMN at similar rates for AD and CN in PET axial.Its PET axial rates are 69.6% for AD and 70.3% for CN, so it does not satisfy the class-discrimination criterion.Source: §I · p. 10, Table XIX · p. 10
  • Grad-CAM control-region enrichment reaches 2.00, and its MRI coronal pointing-game rate is 0.0% for both classes.The authors report that Grad-CAM++ scores 8.7% against a 16.7% chance rate.Source: Table XVIII · p. 9, §I · p. 10, Table XIX · p. 10

V. DISCUSSION

The discussion presents a chain of controlled results supporting that the model reads AD neuropathology. It also cautions that necessity-only ablation can understate regional importance in redundant architectures.

A. What the Evidence Ladder Establishes

  • Shortcut controls, a label-permutation null, modality ablation, regional ablation, and occlusion attribution jointly support the conclusion that the model reads AD neuropathology.The label-permutation null returns AUC 0.456; the medial temporal lobe alone retains 89.2% of MRI discrimination and the posterior DMN alone retains 79.0% in PET.Source: A. What the Evidence Ladder Establishes · p. 10
  • The authors state that no single experiment is decisive, but their conjunction is harder to explain by accounts other than the model reading AD neuropathology.Source: A. What the Evidence Ladder Establishes · p. 10

B. Redundancy and the Limits of Necessity Testing

  • In a six-stream architecture, forward ablation of one small region can produce near-zero ΔAUC because the remaining streams compensate.The authors say necessity-only ablation would have led them to the opposite conclusion about whether any region matters; they recommend sufficiency testing with area-matched controls as more informative.Source: B. Redundancy and the Limits of… · p. 10

C. The Status of Gradient-Based Attribution

The paper pairs a competitive MRI-PET classification result with controls, regional ablations, and attribution tests to examine whether the model relies on AD-relevant evidence. It also frames the clinical relevance and limitations of that evidence.

C. The Status of Gradient-Based Attribution

  • Grad-CAM and Grad-CAM++ produce plausible visualizations but fail the paper’s quantitative faithfulness tests; the authors attribute this setting-specific concern to small subcortical targets and 7 × 7 final feature maps.They recommend anchoring attribution claims to perturbation evidence, enrichment statistics, and class-discrimination checks.Source: C. The Status of Gradient-Based… · p. 11

D. Clinical Framing

  • The proposed clinical value is downstream work—MCI conversion prediction, dementia differential diagnosis, and quantitative staging—where clinicians need to verify that explanations use familiar evidence.Source: D. Clinical Framing · p. 11

VI. LIMITATIONS

  • Six fixed 2.5D slices discard most of each volume, constrain performance ceiling, and limit regional analyses to the sampled slice planes, while making perturbation analyses computationally feasible.Source: §VI · p. 11
  • All data come from ADNI; generalization claims require validation on an independent cohort with different scanners and demographics.Source: §VI · p. 11

VII. CONCLUSION

  • AUC 0.962, accuracy 0.909, and F1 0.891 on a subject-disjoint ADNI test set, competitive with 3D CNN and multimodal transformer baselines at lower computational cost.The test set was physically separated before model development.Source: §VII · p. 11
  • Ablation and attribution analyses support regional necessity and sufficiency, while occlusion sensitivity concentrates inside AD regions and gradient-based methods fail quantitative faithfulness tests.The conclusion reports that area-matched controls do not exceed the spatial null and that Grad-CAM’s peak attribution never lands inside the a priori AD region set in one stream.Source: §VII · p. 11, C. The Status of Gradient-Based… · p. 11
  • The evidence supports a double dissociation: medial temporal and hippocampal evidence is carried by MRI, while posterior cingulate and precuneus evidence is carried by PET.Reverse ROI ablation retained 89.2% of MRI discrimination with the medial temporal lobe alone and 79.0% of PET discrimination with the posterior default-mode network alone.Source: §VII · p. 11
  • The authors propose subject-level splitting, shortcut and permutation controls, regional ablations with area-matched controls, and quantitative attribution benchmarks as reporting requirements.Source: §VII · p. 11

Related papers

Loading 2609.34520v1…