Alzheimer's disease (AD) is a neurodegenerative disorder. There are no drugs and methods for the treatment of AD, but early intervention can delay the deterioration of the disease. Therefore, the early diagnosis of AD and mild cognitive impairment (MCI) is significant. Structural magnetic resonance imaging (sMRI) is widely used to present structural changes in the subject's brain tissue. The relatively mild structural changes in the brain with MCI have led to ongoing challenges in the task of conversion prediction in MCI. Moreover, many multimodal AD diagnostic models proposed in recent years ignore the potential relationship between multimodal information.
Objective
To solve these problems, we propose a multimodal fine-grained classification model based on deep metric learning for AD diagnosis (DML-MFCM), which can fully exploit the fine-grained feature information of sMRI and learn the potential relationships between multimodal feature information.
Methods
First, we propose a fine-grained feature extraction module that can effectively capture the fine-grained feature information of the lesion area. Then, we introduce a multimodal cross-attention module to learn the potential relationships between multimodal data. In addition, we design a hybrid loss function based on deep metric learning. It can guide the model to learn the feature representation method between samples, which improves the model's performance in disease diagnosis.
Results
We have extensively evaluated the proposed models on the ADNI and AIBL datasets. The ACC of AD vs. NC, MCI vs. NC, and sMCI vs. pMCI tasks in the ADNI dataset are 98.75%, 95.88%, and 88.00%, respectively. The ACC on the AD vs. NC and MCI vs. NC tasks in the AIBL dataset are 94.33% and 91.67%.
Conclusions
The results demonstrate that our method has excellent performance in AD diagnosis.
AtriA. The Alzheimer's disease clinical spectrum: diagnosis and management. Med Clin2019; 103: 263–293.
2.
GauglerJJamesBJohnsonT, et al.2022 Alzheimer's disease facts and figures. Alzheimers Dement2022; 18: 700–789.
3.
PetersenRCLopezOArmstrongMJ, et al.Practice guideline update summary: mild cognitive impairment: report of the guideline development, dissemination, and implementation subcommittee of the American academy of neurology. Neurology2018; 90: 126–135.
4.
CarrettiBBorellaEFostinelliS, et al.Benefits of training working memory in amnestic mild cognitive impairment: specific and transfer effects. Int Psychogeriatr2013; 25: 617–626.
5.
FrisoniGBFoxNCJackCRJr, et al.The clinical use of structural MRI in Alzheimer disease. Nat Rev Neurol2010; 6: 67–77.
6.
WangSHZhangYLiYJ, et al.Single slice based detection for Alzheimer's disease via wavelet entropy and multilayer perceptron trained by biogeography-based optimization. Multimed Tools Appl2018; 77: 10393–10417.
7.
Mendoza-LéonRPuentesJUrizaLF, et al.Single-slice Alzheimer's disease classification and disease regional analysis with supervised switching autoencoders. Comput Biol Med2020; 116: 103527.
8.
ZhangYTengQLiuY, et al.Diagnosis of Alzheimer's disease based on regional attention with sMRI gray matter slices. J Neurosci Methods2022; 365: 109376.
GuoXWangZLiK, et al.Voxel-based assessment of gray and white matter volumes in Alzheimer's disease. Neurosci Lett2010; 468: 146–150.
11.
ArmananzasRIglesiasMMoralesDA, et al.Voxel-based diagnosis of Alzheimer's disease using classifier ensembles. IEEE J Biomed Health Inf2016; 21: 778–784.
12.
AhmedSKimBCLeeKH, et al.Ensemble of ROI-based convolutional neural network classifiers for staging the Alzheimer disease spectrum from magnetic resonance imaging. PLoS One2020; 15: e0242712.
13.
FengJZhangSWChenL. Extracting ROI-based contourlet subband energy feature from the sMRI image for Alzheimer's disease classification. IEEE/ACM Trans Comput Biol Bioinf2021; 19: 1627–1639.
14.
WuZPengYHongM, et al.Gray matter deterioration pattern during Alzheimer's disease progression: a regions-of-interest based surface morphometry study. Front Ag Neurosci2021; 13: 593898.
15.
CuiRLiuM. Alzheimer's disease neuroimaging initiative. RNN-based longitudinal analysis for diagnosis of Alzheimer's disease. Comput Med Imag Grap2019; 73: 1–10.
16.
SpasovSPassamontiLDuggentoA, et al.A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer's disease. Neuroimage2019; 189: 276–287.
17.
LiuFWangHLiangSN, et al.MPS-FFA: a multiplane and multiscale feature fusion attention network for Alzheimer's disease prediction with structural MRI. Comput Biol Med2023; 157: 106790.
18.
HuttonCDraganskiBAshburnerJ, et al.A comparison between voxel-based cortical thickness and voxel-based morphometry in normal aging. Neuroimage2009; 48: 371–380.
19.
CarmichaelOTAizensteinHADavisSW, et al.Atlas-based hippocampus segmentation in Alzheimer's disease and mild cognitive impairment. Neuroimage2005; 27: 979–990.
20.
GrimmOPohlackSCacciagliaR, et al.Amygdalar and hippocampal volume: a comparison between manual segmentation, freesurfer and VBM. J Neurosci Methods2015; 253: 254–261.
21.
Klein-KoerkampYHeckemannRARamdeenKT, et al.Amygdalar atrophy in early Alzheimer's disease. Curr Alzheimer Res2014; 11: 239–252.
22.
KlöppelSStonningtonCMChuC, et al.Automatic classification of MR scans in Alzheimer's disease. Brain2008; 131: 681–689.
23.
MagninBMesrobLKinkingnéhunS, et al.Support vector machine-based classification of Alzheimer's disease from whole-brain anatomical MRI. Neuroradiology2009; 51: 73–83.
24.
GrayKRAljabarPHeckemannRA, et al.Random forest-based similarity measures for multi-modal classification of Alzheimer's disease. NeuroImage2013; 65: 167–175.
25.
LiuZLuHPanX, et al.Diagnosis of Alzheimer's disease via an attention-based multi-scale convolutional neural network. Knowl-Based Syst2022; 238: 107942.
26.
ZhuWSunLHuangJ, et al.Dual attention multi-instance deep learning for Alzheimer's disease diagnosis with structural MRI. IEEE Trans Med Imag2021; 40: 2354–2366.
27.
LiuSMasurkarAVRusinekH, et al.Generalizable deep learning model for early Alzheimer's disease detection from structural MRIs. Sci Rep2022; 12: 17106.
28.
DeAChowdhuryAS. DTI Based Alzheimer's disease classification with rank modulated fusion of CNNs and random forest. Expert Syst Appl2021; 169: 114338.
29.
LellaEPazienzaALofuD, et al.An ensemble learning approach based on diffusion tensor imaging measures for Alzheimer's disease classification. Electronics2021; 10: 249.
30.
KimHWLeeHEOhKT, et al.Multi-slice representational learning of convolutional neural network for Alzheimer's disease classification using positron emission tomography. BioMed Eng OnLine2020; 19: 1–15.
31.
PanXPhanTLAdelM, et al.Multi-view separable pyramid network for AD prediction at MCI stage by 18 F-FDG brain PET imaging. IEEE Trans Med Imag2020; 40: 81–92.
32.
TuanPMPhanTLAdelM, et al.AutoEncoder-based feature ranking for Alzheimer disease classification using PET image. Mach Learn Appl2021; 6: 100184.
33.
RamzanFKhanMUGRehmatA, et al.A deep learning approach for automated diagnosis and multi-class classification of Alzheimer's disease stages using resting-state fMRI and residual neural networks. J Med Syst2020; 44: 1–16.
34.
HuangJWangMJuH, et al.SD-CNN: a static-dynamic convolutional neural network for functional brain networks. Med Image Anal2023; 83: 102679.
35.
AlorfAKhanMUG. Multi-label classification of Alzheimer's disease stages from resting-state fMRI-based correlation connectivity data and deep learning. Comput Biol Med2022; 151: 106240.
36.
KongZZhangMZhuW, et al.Multi-modal data Alzheimer's disease detection based on 3D convolution. Biomed Signal Process Control2022; 75: 103565.
37.
GaoXShiFShenD, et al.Task-induced pyramid and attention GAN for multimodal brain image imputation and classification in Alzheimer's disease. IEEE J Biomed Health Inf2021; 26: 36–43.
38.
ZhangJHeXQingL, et al.BPGAN: brain PET synthesis from MRI using generative adversarial network for multi-modal Alzheimer's disease diagnosis. Comput Methods Prog Biomed2022; 217: 106676.
39.
QiuSJoshiPSMillerMI, et al.Development and validation of an interpretable deep learning framework for Alzheimer's disease classification. Brain2020; 143: 1920–1933.
40.
ZhangJHeXLiuY, et al.Multi-modal cross-attention network for Alzheimer's disease diagnosis with multi-modality data. Comput Biol Med2023; 162: 107050.
41.
SongXMaoMQianX. Auto-metric graph neural network based on a meta-learning strategy for the diagnosis of Alzheimer's disease. IEEE J Biomed Health Inf2021; 25: 3141–3152.
42.
AbdelazizMWangTElazabA. Alzheimer's disease diagnosis framework from incomplete multimodal data using convolutional neural networks. Journal of Biomedical Informatics2021; 121: 103863.
43.
HaoXWangRGuoY, et al.Multimodal self-paced locality-preserving learning for diagnosis of Alzheimer's disease. IEEE Trans Cogn Dev Syst2022; 15: 832–843.
44.
EbrahimiALuoSChiongR. Introducing transfer learning to 3D ResNet-18 for Alzheimer's disease detection on MRI images. In: 2020 35th international conference on image and vision computing New Zealand (IVCNZ). IEEE, 2020, pp.1–6.
45.
SunHWangAWangW, et al.An improved deep residual network prediction model for the early diagnosis of Alzheimer's disease. Sensors2021; 21: 4182.
46.
LianCLiuMZhangJ, et al.Hierarchical fully convolutional network for joint atrophy localization and Alzheimer's disease diagnosis using structural MRI. IEEE Trans Pattern Anal Mach Intell2018; 42: 880–893.
47.
WenJThibeau-SutreEDiaz-MeloM, et al.Convolutional neural networks for classification of Alzheimer's disease: overview and reproducible evaluation. Med Image Anal2020; 63: 101694.
48.
QiuSMillerMIJoshiPS, et al.Multimodal deep learning for Alzheimer's disease dementia assessment. Nat Commun2022; 13: 3404.
49.
HuYWenCCaoG, et al.Brain network connectivity feature extraction using deep learning for Alzheimer's disease classification. Neurosci Lett2022; 782: 136673.
50.
TuYLinSQiaoJ, et al.Alzheimer's disease diagnosis via multimodal feature fusion. Comput Biol Med2022; 148: 105901.
51.
ShiJZhengXLiY, et al.Multimodal neuroimaging feature learning with multimodal stacked deep polynomial networks for diagnosis of Alzheimer's disease. IEEE J Biomed Health Inf2017; 22: 173–183.
52.
ZhuQXuBHuangJ, et al.Deep multi-modal discriminative and interpretability network for Alzheimer's disease diagnosis. IEEE Trans Med Imag2022; 42: 1472–1483.
53.
HaoXBaoYGuoY, et al.Multi-modal neuroimaging feature selection with consistent metric constraint for diagnosis of Alzheimer's disease. Med Image Anal2020; 60: 101625.
54.
ForouzannezhadPAbbaspourALiC, et al.A Gaussian-based model for early detection of mild cognitive impairment using multimodal neuroimaging. J Neurosci Methods2020; 333: 108544.
55.
ChenZLiuYZhangY, et al.Orthogonal latent space learning with feature weighting and graph learning for multimodal Alzheimer's disease diagnosis. Med Image Anal2023; 84: 102698.
56.
MöllerCVrenkenHJiskootL, et al.Different patterns of gray matter atrophy in early-and late-onset Alzheimer's disease. Neurobiol Ag2013; 34: 2014–2022.
57.
YangJPanPLSongW, et al.Voxelwise meta-analysis of gray matter anomalies in Alzheimer's disease and mild cognitive impairment using anatomic likelihood estimation. J Neurol Sci2012; 316: 21–29.
