Source-linked AI summary
Multiple Inputs and Mixwd data for Alzheimer's Disease Classification Based on 3D Vision Transformer
Juan A. Castro-Silva, Maria N. Moreno Garcia, Diego H. Peluffo-Ordoñez
TL;DR
Existing Alzheimer’s classification methods may miss 3D context, overlook multiple affected regions, and inadequately combine diverse diagnostic data. MIMD-3DVT addresses these issues by jointly processing consecutive slices, fusing multiple 3D ROIs, and integrating mixed inputs. Using combined public datasets, the method slightly outperformed state-of-the-art methods, while its relatively small dataset remains a limitation for 3D transformers.
Problem
Current methods may inadequately handle mixed data, 3D spatial context, multiple brain regions, and the multifaceted nature of Alzheimer’s diagnosis.
Method
MIMD-3DVT jointly processes consecutive MRI slices, fuses multiple 3D ROI inputs, and integrates categorical, numerical, demographic, cognitive, and imaging data.
Results
MIMD-3DVT using single or multiple ROIs slightly outperformed state-of-the-art methods in Alzheimer’s disease classification.
Takeaways & Limitations
The method provides a more comprehensive classification approach by combining 3D contextual imaging, multiple ROIs, and mixed data sources.
Takeaways & Limitations
The relatively small dataset is challenging for 3D Vision Transformers, which require substantial data to avoid overfitting and learn complex 3D imaging features.
Abstract
from arXiv · showhide
The current methods for diagnosing Alzheimer Disease using Magnetic Resonance Imaging (MRI) have significant limitations. Many previous studies used 2D Transformers to analyze individual brain slices independently, potentially losing critical 3D contextual information. Region of interest-based models often focus on only a few brain regions despite Alzheimer's affecting multiple areas. Additionally, most classification models rely on a single test, whereas diagnosing Alzheimer's requires a multifaceted approach integrating diverse data sources for a more accurate assessment. This study introduces a novel methodology called the Multiple Inputs and Mixed Data 3D Vision Transformer (MIMD-3DVT). This method processes consecutive slices together to capture the feature dimensions and spatial information, fuses multiple 3D ROI imaging data inputs, and integrates mixed data from demographic factors, cognitive assessments, and brain imaging. The proposed methodology was experimentally evaluated using a combined dataset that included the Alzheimer's Disease Neuroimaging Initiative (ADNI), the Australian Imaging, Biomarker, and Lifestyle Flagship Study of Ageing (AIBL), and the Open Access Series of Imaging Studies (OASIS). Our MIMD-3DVT, utilizing single or multiple ROIs, achieved an accuracy of 97.14%, outperforming the state-of-the-art methods in distinguishing between Normal Cognition and Alzheimer's Disease.
1. Introduction
Alzheimer’s classification methods face limitations in capturing 3D spatial context, multiple affected brain regions, and diverse diagnostic data. The proposed approach combines 3D imaging, multiple ROIs, and mixed inputs through a transformer-based model.
- ROI-based models can reduce memory use and overfitting risk but may focus on too few brain regions despite Alzheimer’s affecting multiple areas.
- 2D Transformers analyze MRI slices independently, potentially missing crucial 3D spatial context and feature information.
- Alzheimer’s datasets combine demographic, cognitive, and MRI data, motivating methods that integrate diverse sources and 3D imaging context.
- The proposed model integrates categorical, numerical, and 3D ROI image data from regions including the hippocampus, entorhinal cortex, and multiple lobes.
- The methodology includes dataset preparation, ROI instance selection, and transformer-based model validation with combined categorical, numerical, and multi-ROI image inputs.
- Its stated contributions are processing consecutive slices jointly, fusing multiple 3D ROI inputs, and integrating demographic, cognitive, and imaging data.
2. Related Works
Prior Alzheimer’s classification research has applied Vision, Swin, hybrid, and ROI-based models, but important limitations remain. These include incomplete 3D context, narrow regional focus, and reliance on single data sources, motivating a mixed-data 3D transformer.
- Recent studies apply Vision and Swin Transformers, including pretrained ViT, 3D Siamese, recurrent attention, CNN-transformer, and hybrid architectures.
- ROI-based approaches analyze regions such as the hippocampus, medial temporal lobe, amygdala, and insulae, sometimes using ensembles or landmarks.
- Many existing methods analyze individual MRI slices, focus on selected brain regions, or rely on a single test or data source.
- The proposed method integrates categorical, numerical, and 3D images from several Alzheimer’s-affected regions to address these limitations.
3. Materials and Methods
The study combines multimodal datasets, ROI-focused 3D MRI processing, and mixed categorical and numerical inputs in a multiple-input transformer pipeline. Its methodology covers preprocessing, ROI instance selection, multimodal model construction, and evaluation.
- Datasets: ADNI, AIBL, and OASIS provide MRI, demographic, and cognitive-assessment data for the classification study.The datasets include 3D ROI scans, demographic variables, and standardized cognitive scores.
- Datasets: 420 instances were assembled from three datasets, with 70 volumes per class from each dataset.The merged dataset spans young and old age groups and supports performance assessment across demographic groups.
- Data Preparation: The pipeline preprocesses MRI volumes through skull-stripping and MNI152 registration, while numerical and categorical data are scaled and encoded.Registration standardizes scans to 182 × 218 × 182 dimensions at 1 mm resolution.
- Instance Selection: ROI instance selection uses atlas annotations and statistical-mode centroid estimation to retain informative slices and extract ROI content.The centroid position is adjusted to support precise 2D slice cropping around informative regions.
- Model Building: The multiple-input model combines an MLP for categorical and numerical data with a transformer branch for 3D MRI ROI data.The branches are concatenated into the final model, while ViViT tubelet embedding captures spatio-temporal information from 3D volumes.
- Evaluation: Hyperband selects training hyperparameters, and model performance is evaluated using ROC analysis of true- and false-positive rates.The study describes ROC curves through the trade-off between sensitivity and false-positive rate.
4. Experimental Setup
The experimental setup evaluates the proposed model on multiple ROI types and mixed inputs, compares it with varied state-of-the-art methods, and uses statistical testing and reproducible data handling.
- Data and ROIs: The model is trained and evaluated on 3D ROIs from the entorhinal cortex, fornix, frontal, hippocampal, parietal, and temporal regions.The ROI datasets were derived from the same MRI volumes after skull-stripping and registration.
- Input Configurations: Mixed-data experiments combine gender, age, MMSE score, and 3D ROIs across Alzheimer’s Disease-affected regions and both hemispheres.The setup includes both single-region and broader ROI configurations.
- Comparative Evaluation: The comparison includes methods using ADNI, AIBL, and OASIS datasets, 2D or 3D inputs, and Custom, Ensemble, Swin, or Vision Transformer architectures.This setup places the proposed model alongside diverse data and architecture choices.
- Hyperparameter Tuning: Hyperband optimization selects settings for the 3D Vision Transformer on the merged ADNI+AIBL+OASIS dataset.The selected values are summarized in Table 2.
- Statistical Testing: One-way ANOVA and t-tests assess model differences, using a 0.05 significance level and hypotheses about equality of model accuracy means.The null hypothesis states that all model accuracy means are equal, while the alternative allows at least one difference.
- Reproducibility: The dataset uses 70% training, 15% validation, and 15% test splits, with fixed random seeds supporting reproducible results.The implementation uses Python-based preprocessing and MRI tools.
5. Results and Discussion
Mixed-data experiments significantly improved accuracy over image-only models, while the proposed 3D Vision Transformer achieved strong performance against related methods. The model combines 3D spatial context, multiple ROI inputs, and demographic or cognitive data.
- Mixed-data performance: Mixed-data models significantly outperformed image-only models in accuracy (p=0.001).The comparison used single-ROI experiments.
- Mixed-data performance: 8.18% and 6.86% mean-accuracy increases were achieved for the left and right hemispheres, respectively, compared with image-only models (p=0.001).
- ROI differences: 6.91% separated the lowest image-only ROI accuracy, in the left frontal lobe, from the highest, in the right fornix (p=0.001).
- Comparison with related work: 97.14% accuracy was achieved using multiple ROIs, slightly exceeding the best related-work Vision Transformer result of 96.80%.Several single-ROI classifiers also exceeded 97% accuracy.
- Method interpretation: The proposed design processes consecutive slices, fuses multiple 3D ROI inputs, and integrates demographic, cognitive, and imaging data.
- Model evaluation: 0.984 ± 0.011 was the average seven-fold AUC, and the confusion matrix showed most CN and AD cases correctly classified with minimal errors.
6. Limitations
The study is constrained by its limited data sources and relatively small dataset. These constraints are especially relevant because 3D Vision Transformers require substantial data to learn complex imaging features and avoid overfitting.
- Scope of data: The method relies primarily on three publicly available data modalities, whereas clinical diagnosis uses a broader range of diagnostic tools.
- Dataset size: The relatively small dataset is challenging for 3D Vision Transformers, which require substantial data to avoid overfitting and learn complex 3D imaging features.
7. Conclusions and Future Work
The study introduces MIMD-3DVT, combining multi-source imaging and mixed clinical data for Alzheimer’s classification. It slightly outperformed state-of-the-art methods, while future work calls for broader modalities and larger datasets.
- Conclusions: MIMD-3DVT processes consecutive slices, fuses multiple 3D ROI inputs, and integrates demographic, cognitive, and imaging data.
- Conclusions: ADNI, AIBL, and OASIS were consolidated into a unified multicenter dataset with registered, skull-stripped MRI images.
- Conclusions: MIMD-3DVT using single or multiple ROIs slightly outperformed state-of-the-art methods for Alzheimer’s disease classification.
- Future work: Future work should evaluate additional modalities, including PET, segmented tissue images, brain volumetric data, and other examinations.
- Future work: Increasing dataset size is identified as important for improving accuracy, generalization, and sample diversity.