PaperlayerView exact revision on arXiv

Source-linked AI summary

An Explainable 3D Residual Self-Attention Deep Neural Network FOR Joint Atrophy Localization and Alzheimer's Disease Diagnosis using Structural MRI

Xin Zhang, Liangxiu Han, Wenyong Zhu, Liang Sun, Daoqiang Zhang

arXiv:2008.04024v2eess.IVcs.CV

TL;DR

The paper addresses the need for accurate, timely, and transparent sMRI-based diagnosis of AD and MCI. It proposes an explainable 3D residual self-attention network with Grad-CAM and evaluates it on AD-vs-NC and pMCI-vs-sMCI tasks. The model achieves competitive diagnostic performance and generalizability while highlighting important brain regions for its decisions.

  • Problem

    Timely sMRI-based diagnosis is important, but existing approaches face computational, training, end-to-end, and transparency limitations.

  • Method

    The paper proposes an explainable 3D ResAttNet using residual self-attention, 3D convolutions, and Grad-CAM for end-to-end sMRI diagnosis.

  • Results

    The model shows competitive accuracy performance and generalizability across AD vs. NC and pMCI vs. sMCI classification tasks.

  • Takeaways & Limitations

    Grad-CAM identifies important regions including the hippocampus, lateral ventricle, and much of the cerebral cortex, supporting transparent model decisions.

Abstract

from arXiv · show

Computer-aided early diagnosis of Alzheimer's disease (AD) and its prodromal form mild cognitive impairment (MCI) based on structure Magnetic Resonance Imaging (sMRI) has provided a cost-effective and objective way for early prevention and treatment of disease progression, leading to improved patient care. In this work, we have proposed a novel computer-aided approach for early diagnosis of AD by introducing an explainable 3D Residual Attention Deep Neural Network (3D ResAttNet) for end-to-end learning from sMRI scans. Different from the existing approaches, the novelty of our approach is three-fold: 1) A Residual Self-Attention Deep Neural Network has been proposed to capture local, global and spatial information of MR images to improve diagnostic performance; 2) An explanation method using Gradient-based Localization Class Activation mapping (Grad-CAM) has been introduced to improve the explainable of the proposed method; 3) This work has provided a full end-to-end learning solution for automated disease diagnosis. Our proposed 3D ResAttNet method has been evaluated on a large cohort of subjects from real datasets for two changeling classification tasks (i.e., Alzheimer's disease (AD) vs. Normal cohort (NC) and progressive MCI (pMCI) vs. stable MCI (sMCI)). The experimental results show that the proposed approach has a competitive advantage over the state-of-the-art models in terms of accuracy performance and generalizability. The explainable mechanism in our approach is able to identify and highlight the contribution of the important brain parts (e.g., hippocampus, lateral ventricle and most parts of the cortex) for transparent decisions.

I. INTRODUCTION

AD is a progressive, irreversible neurodegenerative disease for which timely diagnosis matters, while existing sMRI-based approaches face computational, end-to-end training, and transparency limitations. This work proposes an explainable 3D ResAttNet with Grad-CAM for automated diagnosis.

  • AD accounts for 60-70% of dementia cases and affects over 30 million individuals.
  • Timely diagnosis of AD and MCI is essential for optimal management and improved patient care.sMRI can detect structural abnormalities and track brain atrophy progression.
  • Existing systems use voxel-wise global features, predetermined regional features, or combinations of both for automated sMRI-based discrimination.Examples include volumetric approaches, tensor-based morphology, and gray-matter voxel features.
  • Prior methods can be computationally intensive, overlook pathological locations, require domain knowledge and multi-stage training, and lack model transparency.The limitations affect global, regional, hybrid, and black-box deep-learning approaches.
  • The proposed explainable 3D ResAttNet performs end-to-end learning from sMRI for early AD diagnosis.
  • Its contributions combine residual attention for local, global, and spatial information, Grad-CAM explanations, and a full end-to-end automated diagnosis solution.

A. Computer-aided AD diagnosis

Computer-aided AD diagnosis has progressed from global and regional feature engineering toward deep learning, but explainability remains limited. Existing methods use voxel, region, patch, and CNN-based representations, with saliency and activation-mapping methods providing interpretation tools.

  • Computer-aided AD diagnosis methods broadly use global, regional, or combined global-regional features.
  • Voxel-based methods compare or model whole-brain structural features, including voxel intensity and gray-matter differences.
  • Regional methods rely on biologically informed ROIs such as cerebral cortices and hippocampi, while patch-based methods combine local 3D representations hierarchically.
  • Deep-learning approaches apply CNNs and residual networks to MRI for learning multilevel or residual representations.Reported examples include VGG-based, 3D CNN, and residual-block approaches.
  • Deep-learning AD models often lack transparency because their black-box decisions are difficult to explain.
  • Saliency maps visualize feature contributions but can produce noisy localization and segmentation results.Guided backpropagation, PatternNet, PatternAttribution, LRP, and DeepTaylor are described as related methods.
  • CAM localizes discriminative regions, whereas Grad-CAM preserves the original classification architecture by weighting gradients through pooling.
  • Prior explainability studies addressed 3D-CNN AD diagnosis, but no attempt had explained 3D data classification for MCI diagnosis.

III. METHOD

The method is an end-to-end 3D explainable residual attention network for localizing discriminative atrophy and classifying AD-related sMRI tasks. It combines 3D convolutions, residual self-attention, and Grad-CAM-based explanation.

  • The framework targets AD vs. NC and pMCI vs. sMCI classification while automatically localizing discriminative atrophy on sMRI.
  • 3D ResAttNet combines 3D convolutional, residual self-attention, and explainable blocks for sMRI-based AD diagnosis.
  • Residual connections support deeper-network training and help preserve global information, while self-attention learns long-range dependencies.
  • 3D convolutions apply filters across the x, y, and z directions to produce low-level representations in three-dimensional volume space.
  • The architecture uses stacked 3×3×3 convolutional layers to improve computing efficiency relative to 7 × 7 × 7 convolutions.

C. Residual Self-attention block (ResAttNet)

The Residual Self-attention block combines residual learning with self-attention in 3D MRI processing to retain local information while capturing global dependencies.

  • C. Residual Self-attention block (ResAttNet): Self-attention is combined with a residual module to capture both global and local information from 3D images while avoiding information loss.The residual connection preserves original information as self-attention learns long-range dependencies.
  • C. Residual Self-attention block (ResAttNet): Residual learning supports efficient training of deeper networks and helps gradients propagate through the network.The residual mechanism is described as using fewer parameters and mitigating disappearing or exploding gradients.
  • C. Residual Self-attention block (ResAttNet): The self-attention layer performs non-local operations, unlike convolutional operators that use local receptive fields.This enables the model to capture long-range dependencies and global information.
  • C. Residual Self-attention block (ResAttNet): Queries determine which values to emphasize, while keys and values represent features extracted from the whole sMRI by convolution blocks.The key, query, and value vectors are transformed using 1×1×1 convolution filters.
  • C. Residual Self-attention block (ResAttNet): The self-attention map represents the correlative degree of attention between each region and other regions, producing an output feature map.A 1×1×1 convolution filter reduces the final output channels for memory efficiency and channel compatibility.

3) Residual Self-attention block (ResAttNet):

The Residual Attention Block adds self-attention output to the input and residual paths, with a learnable parameter controlling the contribution of non-local evidence.

  • 3) Residual Self-attention block (ResAttNet): The block output is y = x + r(x) + γo(r(x)), combining the input, residual function output, and self-attention output.Here x is the input feature, r(x) is the original residual output, and o(r(x)) is the self-attention output.
  • 3) Residual Self-attention block (ResAttNet): The learnable parameter γ is initialized to 0 so the network initially relies on local-neighborhood cues.As γ increases, the model gradually assigns more weight to non-local evidence.

D. The explainable 3D-CNNs

The explainable 3D-CNN uses Grad-CAM to derive voxel-level heatmaps showing which regions contribute to disease-related model decisions.

  • D. The explainable 3D-CNNs: 3D Grad-CAM is applied to explain the deep model’s decisions.The method operates on activations in the network’s last convolutional layer.
  • D. The explainable 3D-CNNs: Gradients of disease probabilities with respect to last-layer unit activations are globally averaged to obtain importance weights for each unit.The averaging is performed across voxels in the corresponding convolutional layer.
  • D. The explainable 3D-CNNs: The unit weights are combined with spatial activation values to produce a heatmap from 3D gradient-weighted class activation mapping.The resulting heatmap represents the spatial distribution of feature contributions.

IV. EXPERIMENTAL EVALUATION

The evaluation uses ADNI structural MRI data for AD classification and MCI conversion prediction, with independent datasets reserved to assess generalizability.

  • A. Dataset description: The study uses baseline MRI scans from 1,407 ADNI subjects categorized as AD, MCI, or NC under standard clinical criteria.MCI is further divided into progressive and stable groups for conversion prediction.
  • A. Dataset description: ADNI-1 is used for model construction, while ADNI-2 and ADNI-3 provide independent cross-validation of model generalizability.The independent datasets are reserved for cross-validation rather than model construction.
  • A. Dataset description: Preprocessing corrects spatial distortions, performs AC-PC and intensity correction, removes the skull, registers scans to the Colin27 template, and standardizes spatial resolution.MIPAV is used for AC-PC correction, and FSL is used for skull stripping and linear registration.
  • B. Evaluation metrics: Two binary tasks are evaluated: AD versus NC and pMCI versus sMCI.Performance is assessed using accuracy, sensitivity, specificity, and area under the curve.
  • B. Evaluation metrics: Accuracy is defined as (TP + TN) / (TP + TN + FP + FN), while AUC is calculated from sensitivity and 1 − specificity across classification thresholds.TP, TN, FP, and FN denote true-positive, true-negative, false-positive, and false-negative outcomes.

C. Experimental evaluation

The experimental evaluation compares 3D CNN architectures with and without self-attention, using cross-validation on ADNI-1 and reporting accuracy variability and statistical significance.

  • Evaluation design: Three experiments assess state-of-the-art comparisons, cross-dataset generalizability, and comparisons with existing machine-learning and deep-learning methods.The evaluation uses real datasets and includes both direct architectural comparisons and indirect literature-based comparisons.
  • Comparison study: The study compares 3D-VGGNet and 3D-ResNet models with and without self-attention across 18- and 34-layer configurations.The 3D-VGG, 3D-ResNet, and 3D-ResAttNet block structures are illustrated in Fig. 2.
  • Model configuration: Each Conv3D layer combines 3D convolution, batch normalization, and ReLU operations.The networks contain multiple 3D ResNet or 3D-ResAttNet blocks, with configurations detailed in Fig. 3.
  • Model configuration: The 3D architecture replaces a 7 × 7 × 7 convolution with three 3 × 3 × 3 convolutions to reduce computational cost.A 3 × 3 × 3 3D convolution is reported as three times more expensive than its 2D counterpart.
  • Evaluation protocol: The ADNI-1 evaluation uses five-fold cross-validation, averaging test-set accuracies across folds with standard deviations and p-values.Each fold uses 80% of the dataset for training and the remaining 20% for testing.

2) Evaluation 2: Evaluation on generalizability of the proposed model using two independent datasets:

Generalizability is evaluated by training on one ADNI dataset and testing on independent datasets, with reverse-direction experiments limited to AD versus NC classification because of sample availability.

  • Cross-dataset evaluation: A model trained on ADNI-1 is evaluated on the independent ADNI-2 and ADNI-3 datasets.This evaluation is designed to investigate generalizability and reproducibility.
  • Reverse evaluation: The training and testing direction is reversed by training on ADNI-2 and evaluating on ADNI-1 and ADNI-3.The reverse evaluation tests whether performance transfers in the opposite dataset direction.
  • Evaluation scope: Reverse-direction evaluation includes only AD versus NC classification because ADNI-2 and ADNI-3 provide insufficient progressive- and stable-MCI samples.The pMCI versus sMCI task is therefore excluded from this evaluation.
  • Indirect comparison: Indirect comparison uses recent state-of-the-art machine-learning methods reported in the literature with baseline sMRI data from ADNI.The comparison is literature-based rather than a direct reimplementation described in these passages.

D. Result and discussion

The proposed 3D ResAttNet achieved strong classification performance and generalizability across AD-related tasks, while Grad-CAM highlighted disease-relevant brain regions at multiple network depths.

  • Classification performance: The 3D ResAttNet34 with attention achieved the best performance across the evaluated model comparisons for AD vs. NC and pMCI vs. sMCI.Attention-layer models performed significantly better than corresponding models without attention, especially for pMCI vs. sMCI classification.
  • Classification performance: Examples from both classification tasks show the proposed model matching the ground-truth category while 3D ResNet34 produced incorrect classifications.The examples cover AD vs. NC and sMCI vs. pMCI classification.
  • Feature visualization: Deeper convolutional blocks produced lower-resolution feature maps with more global and semantic information, highlighting variation in the lateral ventricle and hippocampus.Earlier blocks retained higher resolution and emphasized edges, corners, textures, and color conjunctions.
  • Explainability: Grad-CAM at the third convolutional layer produced a 46 × 55 × 46 heatmap with more detail than the fourth layer’s 6 × 7 × 6 heatmap.The fourth-layer map showed global information but was blurry because of its lower resolution.
  • Explainability: The explanation heatmaps identified the hippocampus, lateral ventricle, and most parts of the cortex as important areas for AD diagnosis, consistent with human-expert localization.The lower-layer visualization also highlighted edges, reflecting the responses of earlier deep-CNN layers.
  • Generalizability: When trained on ADNI-1 and tested on ADNI-2 and ADNI-3, accuracy decreased by 0.004 and 0.021, while AUC decreased by 0.032 and 0.095, respectively.ACC, SEN, and SPE remained high and statistically significant, with p-values <0.05.
  • Generalizability: When trained on ADNI-2, the model reached 0.956 accuracy on ADNI-2 and 0.933 and 0.917 on ADNI-1 and ADNI-3, respectively.The reported ACC, SEN, and SPE were statistically significant, with p-values <0.05.
  • Generalizability: Across the independent-dataset experiments, the authors report good generalizability and reproducibility for AD diagnosis.The evaluation reversed training and testing datasets and used ADNI-1, ADNI-2, and ADNI-3.

3) Results for evaluation 3:

The proposed 3D-ResAttNet outperforms existing approaches for MCI conversion prediction and performs competitively for AD versus NC classification. Evaluation on independent ADNI-2 and ADNI-3 datasets supports the model’s reported generalizability, although comparisons with other methods are indirect.

  • Comparisons with six state-of-the-art machine-learning methods were indirect because datasets and clinical pMCI/sMCI definitions differed.
  • The 3D-ResAttNet outperforms other existing approaches on the challenging MCI conversion prediction task.
  • For AD versus NC classification, the proposed method shows competitive performance compared with MRI-only approaches.
  • Evaluation used a large number of subjects and cross-validation on independent ADNI-2 and ADNI-3 datasets, supporting reported model generalizability.
Loading 2008.04024v2…