Source-linked AI summary
Visual Explanations From Deep 3D Convolutional Neural Networks for Alzheimer's Disease Classification
Chengliang Yang, Anand Rangarajan, Sanjay Ranka
TL;DR
3D-CNNs can classify Alzheimer’s disease from MRI scans, but their decisions are difficult to interpret. The paper develops one hierarchical-segmentation sensitivity method and two activation-based spatial methods; visual and quantitative evaluations find that all identify important brain parts, with distinct limitations. Together, the methods provide complementary views of 3D-CNN behavior.
Problem
3D-CNN classification decisions lack transparent reasoning, making it difficult for practitioners to trust these systems in evidence-centered medical research.
Method
The study develops hierarchical MRI segmentation sensitivity analysis and extends CAM and Grad-CAM to visualize 3D-CNN spatial attention on MRI scans.
Results
All visual explanation methods identify important brain parts for Alzheimer’s diagnosis in qualitative checks and quantitative localization comparisons.
Takeaways & Limitations
Method choice should follow the use case: SA-3DUCM suits homogeneous-region importance, whereas 3D-Grad-CAM suits tracking 3D-CNN attention.
Takeaways & Limitations
SA-3DUCM underestimates attention in loosely distributed areas such as the cerebral cortex, while CAM-based heatmaps lose localization detail because convolutional activations have lower resolution than MRI scans.
Abstract
from arXiv · showhide
We develop three efficient approaches for generating visual explanations from 3D convolutional neural networks (3D-CNNs) for Alzheimer's disease classification. One approach conducts sensitivity analysis on hierarchical 3D image segmentation, and the other two visualize network activations on a spatial map. Visual checks and a quantitative localization benchmark indicate that all approaches identify important brain parts for Alzheimer's disease diagnosis. Comparative analysis show that the sensitivity analysis based approach has difficulty handling loosely distributed cerebral cortex, and approaches based on visualization of activations are constrained by the resolution of the convolutional layer. The complementarity of these methods improves the understanding of 3D-CNNs in Alzheimer's disease classification from different perspectives.
1 Introduction
Early Alzheimer’s detection could benefit from brain imaging and automated analysis, but 3D-CNN decisions remain difficult to interpret. This study develops visual explanations to reveal spatial attention and compares their ability to localize diagnostically important brain regions.
- Medical imaging contains information about Alzheimer’s-related effects on brain structure and functioning, supporting earlier diagnosis efforts.
- End-to-end 3D-CNNs can classify Alzheimer’s disease from MRI scans and labels, but their reasoning is not transparent to human practitioners.
- The study develops hierarchical MRI segmentation sensitivity analysis to estimate the importance of homogeneous brain regions at multiple levels.
- It extends two CNN explanation methods from 2D natural images to 3D MRI, tracking 3D-CNN spatial attention during Alzheimer’s classification.
- The approaches are evaluated through visual inspection and quantitative localization comparisons of important brain parts.
2 Related Work
Related work spans 3D-CNN Alzheimer’s classification, brain MRI segmentation, and CNN visualization. Existing approaches differ in human control, segmentation granularity, and whether explanations use perturbations or network activations.
- 3D-CNNs for Alzheimer’s Disease Classification: 3D-CNN Alzheimer’s classifiers either extract MRI features for separate classifiers or train end-to-end from scans and labels, with comparable reported performance.
- The related work establishes the methodological basis for adapting perturbation and activation-based explanations to 3D medical images.
- Brain MRI Segmentation: Brain MRI segmentation supports Alzheimer’s diagnosis, while hierarchical unsupervised segmentation enables multilevel analysis from coarse to finer homogeneous regions.
- Visualizing and Understanding CNNs for Natural Image Classification: CNN explanation methods include sensitivity analysis based on perturbed inputs and activation-map visualization that uses convolutional spatial structure.
3 Method
The study develops four 3D-CNN architectures and four visual-explanation approaches for Alzheimer’s disease classification from brain MRI scans. These methods combine model-agnostic sensitivity analysis with activation-based heatmaps, while trading off computational cost, retraining, and spatial resolution.
- Activation visualization: 3D-CAM modifies 3D-ResNet with global average pooling before the softmax layer, while 3D-Grad-CAM extends gradient-weighted activation mapping without retraining.Both activation-based approaches produce coarse heatmaps at the resolution of a convolutional layer.
- 3D-CNN models: The experiments use 3D-VGGNet and 3D-ResNet models trained on 110 × 110 × 110 ADNI brain MRI tensors to classify Alzheimer’s disease and normal cohorts.Training and testing use five-fold cross-validation across five dataset splits, totaling 25 rounds.
- Visual explanation approaches: The study develops SA-3DUCM, 3D-CAM, and 3D-Grad-CAM alongside a voxel-occlusion sensitivity baseline to visualize 3D-CNN predictions.SA-3DUCM uses hierarchical 3D segmentation, whereas CAM-based methods visualize convolutional activations.
- Sensitivity analysis: The voxel-occlusion baseline assigns importance from the absolute change in Alzheimer’s disease probability after masking each voxel’s 7 × 7 × 7 neighborhood.Because it evaluates a new probability for every voxel, a 110 × 110 × 110 scan requires over 1 million forward passes.
- Trade-offs: The approaches have complementary constraints: sensitivity methods may struggle with correlations and interactions, whereas activation methods are limited by convolutional-layer resolution.The baseline is slowest, 3D-CAM requires time-consuming retraining, SA-3DUCM needs a few hundred forward passes, and 3D-Grad-CAM requires one forward pass without retraining.
4 Results
The study compares visual explanation methods for 3D-CNN Alzheimer’s classification using qualitative heatmaps and precision-recall localization. Sensitivity analysis best distinguishes homogeneous regions, while activation-based methods better capture distributed cortex but are constrained by heatmap resolution.
- Experimental setup: The experiments evaluate classification models, qualitative heatmaps, and quantitative localization of cerebral cortex, lateral ventricle, and hippocampus regions.Localization is assessed with precision-recall curves on held-out MRI scans.
- Qualitative comparison: The baseline sensitivity method fails to identify discriminative regions and produces irregular heatmaps from heterogeneous sensitivity-analysis regions.
- Qualitative comparison: SA-3DUCM improves over baseline, clearly differentiates homogeneous regions, and highlights the lateral ventricle, but poorly identifies the loosely distributed cerebral cortex.Its one-at-a-time segment testing cannot capture correlations among fragmented cortex segments.
- Qualitative comparison: 3D-CAM produces blurry heatmaps when upsampled from a 14 × 14 × 14 coarse map, while its shallow variant better localizes the lateral ventricle and cortex.
- Qualitative comparison: 3D-Grad-CAM identifies cortex and lateral ventricle regions, with the 55 × 55 × 55 shallow ResNet map more accurate than the 14 × 14 × 14 map.Lower convolutional layers provide higher resolution, but upper layers may alter their activation maps.
- Quantitative localization: At low recall, VGG-SA-3DUCM, Res-SA-3DUCM, and Res-3D-Grad-CAM-Shallow achieve high precision, but precision drops for every method at high recall.No method is close to perfectly identifying all important regions; SA-3DUCM struggles with cortex, while CAM methods are limited by low-resolution heatmaps.
- Method selection: Method choice depends on the goal: SA-3DUCM suits homogeneous-region importance, whereas 3D-Grad-CAM suits tracking network attention and generally localizes better than 3D-CAM.The comparison also reports that 3D-Grad-CAM does not modify the network architecture and requires less computation.
5 Conclusion and Discussion
The study develops three visual explanation approaches for 3D-CNN Alzheimer’s classification and finds that they highlight important brain regions while exhibiting different limitations. The approaches are broadly applicable to other 3D image-analysis settings, but visual attention alone does not fully explain the learned model.
- Conclusion: The three approaches highlight important brain parts but differ in failure modes: SA-3DUCM misses loosely distributed cortex, while CAM methods lose detail through low-resolution activations.
- Scope: The approaches are generic beyond Alzheimer’s classification: SA-3DUCM is model agnostic, while 3D-CAM and 3D-Grad-CAM require a 3D convolutional layer.The paper notes possible applications to other 3D medical images and video analysis.
- Limitations: A common limitation is that visual explanations do not provide an explicit summarized biomarker representation, such as cerebral-cortex thickness, for fully understanding the 3D-CNN.The authors identify explicit biomarker representation learning as future work.