Sage Journals: Discover world-class research

Abstract

Objective

To assess the role of Machine Learning (ML) in identification critical factors of dementia and mild cognitive impairment.

Methods

371 elderly individuals were ultimately included in the ML analysis. Demographic information (including gender, age, parity, visual acuity, auditory function, mobility, and medication history) and 35 features from 10 assessment scales were used for modeling. Five machine learning classifiers were used for evaluation, employing a procedure involving feature extraction, selection, model training, and performance assessment to identify key indicative factors.

Results

The Random Forest model, after data preprocessing, Information Gain, and Meta-analysis, utilized three training features and four meta-features, achieving an area under the curve of 0.961 and a accuracy of 0.894, showcasing exceptional accuracy for the identification of dementia and mild cognitive impairment.

Conclusions

ML serves as a identification tool for dementia and mild cognitive impairment. Using Information Gain and Meta-feature analysis, Clinical Dementia Rating (CDR) and Neuropsychiatric Inventory (NPI) scale information emerged as crucial for training the Random Forest model.

Keywords

dementia mild cognitive impairment machine learning random forests clinical dementia rating neuropsychiatric inventory

Introduction

As the world’s population ages at an accelerating pace, the prevalence of dementia in low-income countries is poised to experience a substantial surge in the forthcoming decades. Presently, there are an estimated of 47.47 million cases of dementia across the globe. However, projections indicate that in 2050, this figure is anticipated to soar to a staggering 131.5 million.¹ Focusing on the early stages of the pathological process is a viable strategy for preventing dementia by identifying the population with the highest risk. Mild cognitive impairment (MCI) is an intermediate state between normal and dementia,² with a noticeable decline in cognitive abilities without significantly interfering with daily activities.³ Previous studies have shown that individuals with MCI are more likely to develop dementia than those with normal cognitive function.⁴ Cognitive impairment is characterized by decreased memory, attention, and language and deterioration in other cognitive functions, including MCI and dementia.^5,6

Although some progress has been made in the early diagnosis and intervention of cognitive impairment, more effective and precise methods are still needed to improve diagnostic accuracy and the effectiveness of early interventions.⁴ Clinical prediction models developed using Artificial Intelligence (AI) and Machine Learning (ML) have garnered significant interest in health care in recent years.⁷ Health care professionals have utilized machine learning techniques to develop applications that assist in making diagnostic decisions. These algorithms utilize extensive patient data from clinical studies to achieve accurate diagnosis. This application has become increasingly prevalent in various medical specialties, including cancer detection, heart disease, eye conditions, and chronic illnesses.⁸

This study utilized early screening form data, employed multiple modeling methods, and optimized model parameters to assess model performance using metrics such as Area Under the Curve (AUC) and accuracy (CA). We developed the optimal predictive model for diagnosing patient diseases and identified key influential indicators. Through ML techniques and feature selection methods, our aim is to enhance the accurate diagnosis of dementia and MCI, improve diagnostic tools for health care professionals, and promote better treatment outcomes and quality of life for patients.

Materials and Methods

Participants and Ethics

From May 1 to July 31 in 2019, 394 participants aged 65 years and older were enrolled from a Chinese nursing home in Hangzhou City, which is the capital and largest city of Zhejiang Province in southeastern China. Written informed consent were collected from all the subjects or family members prior to participating in the study, in accordance with protocols approved by the Ethics Committee of Zhejiang Hospital (reference number: 2014-KL-35-1). Among them, 21 were excluded due to refusal or loss to follow-up, as they were unable to complete assessments and communicate effectively due to language, hearing, or vision impairments, and two were excluded due to death. Ultimately, 371 participants were included in the final analysis, resulting in a response rate of 94.2% (Figure 1). The flowchart depicting the expected and final participant inclusion is shown in Figure 1.

Figure 1.

Flow chart of participants.

Methods

Measures

Demographic information, including gender, age, parity, and health behaviors (eg, visual acuity, auditory function, mobility, and medication history), was primarily sourced from electronic archives of nursing homes. Uncertain information was clarified through consultations with caregivers or family members. Additionally, trained nurses administered standardized assessments using tools such as the Mini-Mental State Examination (MMSE), Mini-cog, Clock Drawing Test, Alzheimer’s Disease 8-item Informant Questionnaire (AD8), Activities of Daily Living (ADL) scale, Epworth Sleepiness Scale, Cohen-Mansfield Agitation Inventory - Short Form (CMAI-Short), Clinical Dementia Rating (CDR), Neuropsychiatric Inventory (NPI), and Morse Fall Scale. Diagnosis of MCI and dementia were established by experienced psychiatrists through a comprehensive approach involving patient history review, informant interviews, and objective bedside mental status examinations, adhering to the 2011 National Institute on Aging and Alzheimer’s Association (NIA-AA) criteria.^9,10

ML models

Figure 2 illustrates the modeling process of ML. We utilized five classifiers: Random Forests (RF), Gradient Boosting (GB), Support Vector Machine (SVM), k-Nearest Neighbors (k-NN), and Neural Network (NN). Further details on these models are provided in Appendix 6.1. These classifiers were chosen for their extensive use in clinical diagnostic studies and their demonstrated efficacy in distinguishing between individuals with dementia and MCI. Predictive outcomes from these models were leveraged for model optimization using a comprehensive dataset comprising 35 features. These features encompass demographic details (eg, gender, age), health metrics (eg, visual and auditory function, mobility), and data from 10 assessment scales. This dataset facilitated feature selection and performance evaluation, leading to the identification of optimal models and key features contributing to accurate diagnostics.¹¹

Figure 2.

Machine learning process diagram.

Meta-features

Meta-features are high-level statistical information or descriptive statistics that describe the features of a dataset. They help in understanding and analyzing the characteristics of the dataset, which can be used for selecting appropriate machine learning models and algorithms, as well as tuning the performance of models.¹²

Data preprocessing and model optimization methods

The raw data is preprocessed by removing groups with missing values. Based on performance metrics such as Accuracy, AUC, Precision, F1 Score, and Confusion matrix results, suitable ML models and features are selected. Detailed descriptions of these performance metrics can be found in Appendix 6.2.

Data Statistics

The ML analysis software is Orange 3.35. We used one-way ANOVA or t-tests and χ² tests for continuous and categorical variables, respectively. The figure was generated using GraphPad Prism 10.0.2.

Results

Baseline Characteristics

Among the 371 participants included in the study (68.7% female), the mean (SD) age was 87 (6.6) years, encompassing individuals with dementia, MCI, and normal cognitive function. Of these participants, 169 (45.6%) were diagnosed with dementia and 82 (22.1%) with MCI. There was no significant difference in diagnosis based on gender (P = 0.423). However, Table 1 indicates that a diagnosis of dementia is associated with patient age (P < 0.001), and as the age of participants increased, the prevalence of dementia increased significantly, ranging from 26.9% to 67.7%.

Table 1.

Baseline Characteristics of Subjects.

Characteristcs of Subjects	Total (n = 371)	Dementia (n = 169)	Mild Cognitive Impairment (82)	Normal (120)	P-value*
Female gender, n (%)	255 (68.7)	120 (47.1)	58 (22.7)	77 (30.2)	0.423
Age, mean (SD)	87 (6.6)	89 (6.7)	87 (6.3)	85 (6.3)	<0.001^a
Age groups, y, n (%)					0.002b
65-75	26 (7.0)	7 (26.9)	5 (19.2)	14 (53.8)
76-85	109 (29.4)	37 (33.9)	29 (26.6)	43 (39.4)
86-95	205 (55.3)	104 (50.7)	42 (20.5)	59 (28.8)
96 and above	31 (8.4)	21 (67.7)	6 (19.4)	4 (12.9)
Number of children, mean (SD)	2 (1.5)	3 (1.6)	2 (1.5)	2( 1.2)	<0.001^a
Male parity, n (%)					0.033^b
Zero	16 (4.3)	7 (43.8)	7 (43.8)	2 (12.5)
One	26 (7.0)	7 (26.9)	5 (19.2)	14 (53.8)
Two	41 (11.1)	21 (51.2)	4 (9.8)	16 (39.0v
More than two	33 (8.9)	14 (42.4)	8 (24.2)	11 (33.3)
Female parity, n (%)					<0.001b
Zero	18 (4.9)	11 (61.1)	2 (11.1)	5 (27.8)
One	34 (9.2)	8 (23.5)	12 (35.3)	14 (41.2v
Two	76 (20.5)	24 (31.6)	17 (22.4)	35 (46.1)
More than two	127 (34.2)	77 (60.6)	27 (21.3)	23 (18.1)
Visual acuity, n (%)					0.058
Good	208 (57.6)	82 (39.4)	47 (22.6)	79 (38.0)
Poor	153 (42.4)	77 (50.3)	35 (22.9)	41 (26.8)
Hearing, n (%)					0.426
Good	222 (61.8)	93 (41.9)	51 (23.0)	78 (35.1)
Poor	137 (38.2)	66 (48.2)	30 (21.9)	40 (29.2)
Quantity of medications, mean (SD)	3 (3.0)	4 (3.0)	3 (3.2)	3 (2.8)	0.912
Medication category, n (%)					<0.001b
Hypnotics	58 (14.1)	23 (39.7)	18 (31.0)	17 (29.3)
Antipsychotics or anti-dementia drugs	36 (8.8)	32 (88.9)	2 (5.6)	2 (5.6)
Other drugs	316 (77.1)	145 (45.9)	67 (21.2)	104 (32.9)

Note:

^aindicates a significant result for Dementia vs Normal group using a t-test, while.

^bindicates a significant result using a chi-square test.

Scale Assessment

Our study demonstrates that the distinction between MCI and the Normal group is relatively low for some scales, such as ADL, CMAI-Short, and Epworth Sleepiness. However, in Figure 3, apart from the Morse Scale, both the Dementia vs Normal group and the Dementia vs MCI group can be significantly differentiated (P < 0.01).

Figure 3.

Scale test results. A and B scale scores, C t test analysis results.

Modelling Results

Raw data model results

Figure 4 summarizes the predictive results of the raw data. The AUC values of the five machine learning models range from 0.812 to 0.955, while the CA ranges from 0.648 to 0.859. The best-performing model is GB, followed by RF. According to Information Gain, we obtained a ranking of the importance of the 35 features (Figure 4(C)), with the CDR Score being the most important.

Figure 4.

Modeling Results. A performance evaluation results of different models, B Confusion matrix results for actual proportions in the RF Model, and C Feature importance ranking results based on Information Gain. (D, M, and N means dementia, mild cognitive impairment, and normal, respectively).

Optimized model results

Figure 5 presents the results of model optimization. Integrating the feature importance results and the data analysis results from Table 1, we narrowed down the 35 original features to 3 for training (CDR score, NPI score, NPI result) and 4 for meta-features (Parity more than one, Hypnotics, Antipsychotics or Anti-dementia Drug, Other Drug), as shown in Figure 5(B). Post-optimization, RF emerged as the best predictive model. The AUC improved from 0.927 (0.0101) to 0.961 (0.0007), and the CA increased from 0.787 (0.0176) to 0.894 (0.0029), as depicted in Figure 5(A).

Figure 5.

Feature selection and model optimization results. A Performance comparison of different models after feature optimization, B Feature selection results, C The confusion matrix for actual proportions in the optimized RF Model, and the significance comparison with the original data results. (D, M, and N means dementia, mild cognitive impairment, and normal, respectively).

Figure 5(C) displays the confusion matrix results of the RF model. Regarding the improvement in the actual value ratios, post-optimization, both the dementia and MCI model judgments significantly improved compared to the original data, increasing from 92.8% to 96.7% and from 59.5% to 95.1%, respectively.

Discussion

This article proposes a classification model using machine learning to predict dementia and mild cognitive impairment in elderly individuals. The experiments demonstrate that through feature ranking and meta-feature analysis methods for model performance evaluation, critical feature factors, including CDR score, NPI score, NPI result, Parity more than one, Hypnotics, Antipsychotics or Anti-dementia Drug, Other Drug, were identified. The RF model outperforms existing machine models, exhibiting excellent discriminative ability, especially for patients with MCI.

Studies have observed that in females, a higher number of offspring is associated with an increased risk of dementia.¹³ Our study found that this correlation was not exclusive to females (P < 0.001), as males who had more offspring also had a higher prevalence of dementia (P < 0.033). Various scales, such as MMSE, CDR, AD8, NPI, etc., are utilized in the early detection, assessment, screening, treatment monitoring, and quantification tracking of dementia or cognitive impairment.^14-18 Doctors can diagnose whether a patient has dementia or cognitive impairment based on the results of these scales. We can rely on the results of most scales to assist in diagnosing the severity of dementia. However, we cannot determine which scale results are more reliable for diagnosis and the strength of the correlation between different scales and dementia diagnosis. Therefore, we need models such as ML for analysis and judgment to help us better select feature factors. Information Gain, AUC and CA are standard performance evaluation metrics in ML, frequently used by researchers to assess models for dementia or MCI.^19,20 Meta-features are not original features used to train models but descriptors of the dataset’s inherent characteristics. They can assist in selecting suitable algorithms or models for data processing, enhancing model performance and generalization ability.²¹ In our study, the CDR score is considered to be the most crucial predictive factor in ML, followed by NPI. The CDR²² is a frequently employed cognitive diagnostic evaluation²³ that aligns with the gold standard criteria for diagnosing dementia.²⁴ On the other hand, NPI focuses on assessing behavioral issues related to neuropsychiatric symptoms, including depression, anxiety, hallucinations, and more. By combining the assessments from CDR and NPI, doctors can obtain a comprehensive understanding of the patient’s cognitive function and behavior.^25,26

Our results indicate that we can utilize a RF model, using CDR score, NPI score, and NPI result as critical feature factors, while parity and medication status serve as meta-features, for model construction and application in the diagnosis of dementia and MCI.

There are several limitations in this research. Firstly, due to the cross-sectional nature of the study, we were unable to establish any causal relationships, emphasizing the necessity for future longitudinal studies. Secondly, the data were exclusively obtained from a single nursing home, underscoring the importance of conducting studies with larger sample sizes from multiple nursing homes to validate our observations. Additionally, the small sample size limited the effectiveness of neural network models, potentially leading to overfitting and reduced generalizability of the results.

Conclusion

This article proposes a machine learning classification model to predict dementia and mild cognitive impairment in the elderly. Experiments demonstrate that critical features, including CDR score, NPI score, NPI result, Parity of more than one, Hypnotics, Antipsychotics or Anti-dementia Drugs, and Other Drugs, were identified through feature ranking and meta-feature analysis. The RF model outperforms existing models, especially for patients with MCI. However, the model’s diagnostic performance for regular patients could be improved. Future efforts will focus on expanding new features (such as brain MRI or CT indicators) to enhance accuracy and differentiate regular patients better.

Supplemental Material

Supplemental Material - Identification of Dementia & Mild Cognitive Impairment in Chinese Elderly Using Machine Learning

Supplemental Material for Identification of Dementia & Mild Cognitive Impairment in Chinese Elderly Using Machine Learning by Tong-Tong Ying, Li-Ying Zhuang, Shan-Hu Xu, Shu-Feng Zhang, Li-Jun Huang, Wei-Wei Gao, Lu Liu, Qi-Lun Lai, Yue Lou and Xiao-Li Liu in American Journal of Alzheimer's Disease & Other Dementias®

Footnotes

Acknowledgments

We thank all the participants and their families who took part in this study. The authors would like to thank the Zhejiang Science and Technology Project for youth innovation in medical and health, research on the mechanisms of Mild Cognitive Impairment in older adults with coexisting hypertension (2023RC120), Zhejiang Provincial Natural Science Foundation (LQ19H090006) and the project of the Zhejiang Provincial Health Commission (2023KY006).

Author Contributions

T.-T. Y. and L.-Y. Z. contributed equally to this work. X.-L. L. conceived the project. L.-Y. Z. and S.-H. X. designed and performed the experiments. T.-T. Y., L. L., Q.-L. L. and S.-F. Z. was responsible for the collection of data. L.-Y. Z., L.-J. H. and W.-W. G. conducted the statistical analysis. Q.-L. L and Y. L. interpreted the data. T.-T. Y. and L.-Y. Z. wrote the manuscript and manuscript preparation. Overall, all authors provided suggestions during the preparation of the manuscript and approved the final version submitted for publication.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Zhejiang Science and Technology Project for youth innovation in medical and health, research on the mechanisms of Mild Cognitive Impairment in older adults with coexisting hypertension (2023RC120), Zhejiang Provincial Natural Science Foundation (LQ19H090006) and the project of the Zhejiang Provincial Health Commission (2023KY006).

Ethical Statement

ORCID iDs

Tong-Tong Ying

Li-Ying Zhuang

Shan-Hu Xu

Data Availability Statement

All the data used to support the findings of this study are included in the article.*

Supplemental Material

Supplemental material for this article is available online.

Appendix

References

Prince

. World Alzheimer Report 2015: The Global Impact of Dementia. An analysis of prevalence, incidence, cost and trends. London: Alzheimer’s Disease International; 2015.

Mahendran

Vincent

PMDR

Srinivasan

Cig

. Improving the classification of Alzheimer's disease using hybrid gene selection pipeline and deep learning. Front Genet. 2021;12:784-814.

Bloch

Friedrich

CMJAR

. Data analysis with Shapley values for automatic subject selection in Alzheimer's disease data sets using interpretable machine learning. Alzheimer's Res Ther. 2021;13:1-30.

Luck

Luppa

Briel

Riedel-Heller

. Incidence of mild cognitive impairment: a systematic review. Dement Geriatr Cogn Disord. 2010;29:164-175.

Moreira

Costa

Machado

Castro

Vicente

SGJPO

. Distinguishing mild cognitive impairment from healthy aging and Alzheimer's Disease: the contribution of the INECO Frontal Screening (IFS). 2019;14:e0221873.

Koscik

Hermann

Allison

Scjfian

. Validity evidence for the research category. Cognitively Unimpaired – Declining, as a Risk Marker for Mild Cognitive Impairment and Alzheimer’s Disease 2021;13:688478.

Andaur Navarro

, et al. Completeness of reporting of clinical prediction models developed using supervised machine learning: a systematic review. BMC medical research methodology 2022;22:1-13.

García-Domínguez

Galván-Tejada

Magallanes-Quintanar

, et al. Diabetes etection odels in Mexican atients by ombining achine earning lgorithms and eature election echniques for linical and araclinical ttributes: omparative valuation. J Diabetes Res. 2023;2023:9713905.

McKhann

Knopman

Chertkow

, et al. The diagnosis of dementia due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement 2011;7:263-269.

10.

Albert

DeKosky

Dickson

, et al. The diagnosis of mild cognitive impairment due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. Alzheimers Dement 2011;7:270-279.

11.

Farhan

Fahiem

Tauseef

. An ensemble-of-classifiers based approach for early diagnosis of Alzheimer's disease: classification using structural features of brain images. Comput Math Methods Med. 2014;2014:862307.

12.

Rivolli

Garcia

LPF

Soares

Vanschoren

de Carvalho

ACPLF

. Meta-features for meta-learning. Knowledge-Based Systems; 2022:108101.

13.

Jang

Bae

Dardiotis

, et al. Differential effects of completed and incomplete pregnancies on the risk of Alzheimer disease. Neurology. 2018;91:e643-e651.

14.

Creavin

Wisniewski

Noel-Storr

, et al. Mini-Mental State Examination (MMSE) for the detection of dementia in clinically unevaluated people aged 65 and over in community and primary care populations. Cochrane Database Syst Rev. 2016;2016:Cd011145.

15.

Arevalo-Rodriguez

Smailagic

Roqué-Figuls

, et al. Mini-Mental State Examination (MMSE) for the early detection of dementia in people with mild cognitive impairment (MCI). Cochrane Database Syst Rev. 2021;7:Cd010783.

16.

Tzeng

Yang

Hsu

Chang

Chiu

. Sum of boxes of the clinical dementia rating scale highly predicts conversion or reversion in predementia stages. Front Aging Neurosci. 2022;14:1021792.

17.

Tanwani

Danquah

Butris

, et al. Diagnostic accuracy of Ascertain Dementia 8-item Questionnaire by participant and informant-A systematic review and meta-analysis. PLoS One. 2023;18:e0291291.

18.

Morganti

Soli

Savoldelli

Belotti

. The europsychiatric nventory-iary ating cale (NPI-iary): ethod for mproving tability in ssessing europsychiatric ymptoms in ementia. Dement Geriatr Cogn Dis Extra. 2018;8:306-320.

19.

Booth

Park

Lee

. Predicting cognitive decline in Parkinson's disease using FDG-PET-based supervised learning. J Clin Invest. 2022;132.

20.

Nagaraj

Duong

. Deep learning and risk score classification of mild cognitive impairment and Alzheimer's disease. J Alzheimers Dis 2021;80:1079-1090.

21.

Miao

Dong

Xie

S-L

Liang

S-L

. UMLF-COVID: an unsupervised meta-learning model specifically designed to identify X-ray images of COVID-19 patients. BMC Med Imag 2021;21:174.

22.

Hughes

Berg

Danziger

Coben

Martin

. A new clinical scale for the staging of dementia. Br J Psychiatry. 1982;140:566-572.

23.

Lowe

Balsis

Miller

Benge

Doody

. Greater precision when measuring dementia severity: establishing item parameters for the Clinical Dementia Rating Scale. Dement Geriatr Cogn Disord. 2012;34:128-134.

24.

Chaves

Camozzato

Godinho

, et al. Validity of the clinical dementia rating scale for the detection and staging of dementia in Brazilian patients. Alzheimer Dis Assoc Disord. 2007;21:210-217.

25.

Karttunen

Karppi

Hiltunen

, et al. Neuropsychiatric symptoms and quality of life in patients with very mild and mild Alzheimer's disease. Int J Geriatr Psychiatr 2011;26:473-482.

26.

Rosenberg

Rattinger

Stuart

Lyketsos

Leoutsakos

. Psychotropic edication and ognitive, unctional, and europsychiatric utcomes in Alzheimer's isease (AD). J Am Geriatr Soc. 2021;69:955-963.

27.

Vanneschi

Farinaccio

Mauri

Antoniotti

Provero

Giacobini

. A comparison of machine learning techniques for survival prediction in breast cancer. BioData Min. 2011;4:12.

28.

Meng

Yan

, et al. Diagnostic classification of cancers using extreme gradient boosting algorithm and multi-omics data. Comput Biol Med. 2020;121:103761

29.

Vapnik

. The Nature of Statistical Learning Theory. Berlin: Springer Science & Business Media; 1999.

30.

Zhang

. Introduction to machine learning: k-nearest neighbors. Ann Transl Med. 2016;4:218.

31.

Huang

CoK

Engineering

. Using AUC and accuracy in evaluating learning algorithms. IEEE Transactions on Knowledge and Data Engineering. 2005;17:299-310.

32.

Kannan

Vasanthi

. Machine Learning Algorithms with ROC Curve for Predicting and Diagnosing the Heart Disease. Berlin: SpringerBriefs in Applied Sciences and Technology; 2019.

33.

Zhang

Feng

Liu

ZJSP

. The use of ROC and AUC in the validation of objective image fusion evaluation metrics, Signal Processing. 2015;115:38-48.

34.

Hastie

Tibshirani

Friedman

. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Berlin: Springer; 2009.

35.

Pradhan

. Support Vector Machine-A Survey. Chongqing: Computer Science; 2012.

36.

García-Carretero

Holgado-Cuadrado

Entropy

sB-PJ

. Assessment of Classification Models and Relevant Features on Nonalcoholic Steatohepatitis Using Random Forest. Entropy (Basel). 2021;23(6):763.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.28 MB