Abstract
Background:
The early detection of neurocognitive disorders, especially when mild, is a key issue of health care systems including the Italian Dementia National Plan. The Mini-Mental State Examination (MMSE), i.e., the reference screening tool for dementia in Italian Memory Clinics, has low sensitivity in detecting mild cognitive impairment (MCI) or mild dementia.
Objective:
Availability of a 10-minute screening test sensitive to MCI and mild dementia, such as the Montreal Cognitive Assessment (MoCA), is relevant in the field. This study presents initial validity and reliability data for the Italian version of MoCA 7.1 that is being collected as part of a large ongoing longitudinal study to evaluate the rate of incident MCI and dementia in older adults.
Methods:
MoCA 7.1 and MMSE were administered to cognitive impaired patients (n = 469; 214 with MCI, 255 with dementia; mean age: 75.5; 52% females,) and healthy older adults (n = 123, mean age: 69.7, 64 % females).
Results:
Test-retest (0.945, p < 0.001) and inter-rater (0.999, p < 0.001) reliability of MoCA 7.1, assessed on randomly selected participants with normal cognition, MCI, dementia, were significant. MoCA 7.1 showed adequate sensitivity (95.3%) and specificity (84.5%) in detecting MCI compared to MMSE (sensitivity: 53.8%; specificity: 87.5%). The Area Under the Curve of MoCA 7.1 was significantly greater than that of MMSE (0.963 versus 0.742). MoCA 7.1 showed similar results in detecting both MCI and dementia.
Conclusion:
MoCA 7.1 is a reliable and useful tool that can aid in the diagnosis of MCI and dementia in the Italian population.
Keywords
INTRODUCTION
The early diagnosis of neurocognitive disorders provides relevant benefits to patients and families [1]; it represents a cornerstone of the policies for dementia across health care systems [2]. Mild cognitive impairment (MCI) is the intermediate stage between the normal and usual cognitive performance and dementia. In the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) [3], MCI is defined as “Mild Neurocognitive Disorder” (Mild NCD) whereas dementias have been grouped in “Major NCDs” [4].
The novelty of these diagnostic criteria depends on both the identification of six cognitive domains (complex attention, executive function, learning and memory, language, perceptual-motor, social cognition) and on the assessment of the related functions. On this basis, the diagnosis of Mild NCD stems from “...evidence of modest cognitive decline from a previous level of performance in one or more cognitive domains (complex attention, executive function, . . .) . . . that do not interfere with capacity for independence in everyday complex activities of daily living but greater effort, compensatory strategies, or accommodation may be required” [3, 4].
The Major NCDs differ from Mild NCD because the cognitive decline interferes with independence in the complex activities of daily living [3–5]. This framework overlaps the staging of dementia based on the Clinical Dementia Rating scale (CDR), a standard staging tool that evaluates six cognitive and functional domains with a five-point scale [6]: Mild NCD is aligned to a score of 0.5, where normal cognition of 0 and Major NCD of >1.
The risk factors for developing Mild NCD are aging, belonging to the male gender, and low education [7]. Mild NCD may progress to dementia but, importantly, it may not [8]. Cognitive screening represents the initial step in a process of further assessment for NCD and can help identify potential patients.
The Mini-Mental State Examination (MMSE) is a widespread cross-cultural tool for the initial screening of dementia [9]. The MMSE, however, has limitations: healthy elderly population with normal scores could show only a slight cognitive impairment particularly in attention and executive functioning [10] thus yielding a low sensitivity towards Mild NCD, especially among individuals with high cognitive reserve and/or education [11].
The Montreal Cognitive Assessment (MoCA) was developed as the first “brief screening tool” to detect Mild NCD in cognitively impaired participants who would score in the normal range on the MMSE [12]. MoCA has improved the management of the increasing volume of referrals to memory clinics for cognitive concerns [13]: it evaluates the six main cognitive domains and has a good convergent validity with more intensive neuropsychological batteries [14–16]. In addition, MoCA presents meaningful correlations with findings from structural brain imaging [17].
Results of MoCA have demonstrated to be age and gender independent [18, 19] and only minimally influenced by education levels [20, 21] partly due to its correction factors. Thus, MoCA does not need to be corrected for education; such adjustment usually has a detrimental effect on sensitivity and a slight increase in specificity, considering that educational levels do not always correlate with premorbid intellectual function [22]. Furthermore, MoCA correlates with cognitive reserve [23].
Over the years, MoCA has been adapted to meet several of the issues of cognitive assessment (https://www.mocatest.org): illiteracy (MoCA-B), sensory impairments (MoCA Blind/Hearing impairment), and remote assessment (T-MoCA) for community-residing people or unavailability of in-clinic assessments, as happened during the COVID-19 pandemic.
Scientific societies strongly recommend the timely diagnosis of Major NCD [1, 24] especially at the prodromal stage of Mild NCD [25–29].
A diagnosis of Mild NCD can be both stigmatizing and anxiogenic, and we acknowledge that many clinicians may not wish to screen for Mild NCD, given it has a variable natural history and that treatment is largely supportive. Nonetheless, diagnosis of Mild NCD must be made carefully; patients must be counselled that it has a variable natural history and prognosis and that they will not necessarily develop Major NCD. Therefore, both in the clinical setting and health policy it is pivotal to increase knowledge on prognostic perspectives.
Aims
We designed a longitudinal study aimed at evaluating the rates of Mild NCD developed by people with normal cognition and of Major NCD from those with Mild NCD.
The study started in 2016, and was scheduled to end in 2020, but the COVID-19 pandemic has delayed the follow-up to autumn 2021 and the subsequent completion of the study to end of 2022. The first step of the longitudinal study has been to compare the psychometric properties of two neuropsychological tools: the first Italian version of MoCA (MoCA 7.1, 2006) [30] and MMSE.
The purpose of this study is to investigate: a) test-retest and interrater reliability of the Italian version of the MoCA, originally developed in 2006, with this longitudinal cohort; b) the effects of age, gender, education and normative data in diagnostic accuracy; c) the optimal balance of sensitivity/specificity of MoCA 7.1 and MMSE for detecting MCI and dementia.
This design expands upon the initial validation work of MoCA 7.1 aiming to add incremental knowledge about the utility of the measure in the longitudinal cohort of normal older adults versus at-risk for cognitive impairment.
METHODS
Rationale of the implementation of MoCA 7.1
The validation process followed the criteria of DSM-5 to diagnose NCDs as acquired conditions [31] that are age-related but not age-dependent [32] where the individual, directly or through an informant, reports cognitive concerns regardless of age, gender, and education [33–35].
As MoCA reflects the cognitive reserve [23], it allows assessment along the two trajectories of cognition in aging: 1) normality: cognitive idemescence [36] or absence of anatomical-structural pathological modifications of the brain [37–40] with preservation of the functional network reorganization in the adults [41, 42], therefore maintaining the usual level of cognition [43] and related complex/advanced activity of daily living [44–46]; 2) cognitive impairment: the NCDs usually appear after the fifth decade and increase with aging [47–51].
Participants
We recruited: 1) participants with memory complaints perceived directly by themselves and/or by an informant (Patient Group, PG); 2) community-dwelling elderly without known memory and functional complaints (CDR Global and Sum of Boxes scores equal to 0) and without sensory impairments (normal elderly controls, NECs). Both groups were recruited from three Center for Cognitive Disorder and Dementia (CCDD) located in northern Italy: one in a rural town (Cento) and two in a middle-sized industrial city (Modena).
Oral and written informed consent were obtained according to protocols approved by the local institutional review board.
Assessments and inclusion criteria
1) Participants with memory complaints were evaluated with a comprehensive assessment of clinical, cognitive, affective, and functional status.
Clinical assessment: medical history, physical comorbidities assessed with the Cumulative Illness Rating Scale (CIRS) [52], physical and neurologic examination, medications and laboratory testing covering: Complete Blood Count, Thyroid-Stimulating Hormone, B-12, folic acid, serum calcium, liver and kidney function, electrolytes.
Cognitive assessment: MMSE [9, 53], standard neuropsychological tests in participants referring cognitive impairments but performing the MMSE in the normal score range [54], MoCA 7.1 [12, 30]. At baseline, the MMSE and MoCA 7.1 were administered on the same day or within 1 month for all participants.
For the MMSE, a cut-off score lower than 27 is the threshold for the diagnosis of Major NCD [53], as approved by the Italian Medicines Agency (AIFA). This allows the prescription of cholinesterase inhibitors or memantine, free of charge, and only through specialists working within public CCDDs.
In the following analysis we considered the MMSE both as a raw (MMSE-RS) or as a normative score adjusted for age and education (MMSE-NS) [53].
The suggested cut-off score for MoCA 7.1 is 26 (score ≤25 to indicate impairment). The original cut-off score is determined with two types of correction for the effect of education. The first is general, relative to the serial-7 subtraction where one error is allowed thus, 4 to 5 correct answers correspond to a maximum score of 3 points instead of a maximum of 5 as in the MMSE; the second is variable where one point is added to the score of participants with 12 years of education or less if the total MoCA score is lower than 30 [12].
Other assessments included: Geriatric Depression Scale 15-item (GDS) [55]; Activities of Daily Living (ADL) [56] and Instrumental Activities of Daily Living (IADL) [57]; Clinical Dementia Rating scale (CDR) [6, 58] indexed by the Global score (CDR-G), and CDR Sum of Boxes score (CDR-SB).
The ADL, IADL and CDR interviews were conducted with each examinee and their informant partners as the informant-based functional measures improve overall diagnostic accuracy of the psychometric tests [59].
Diagnosis of Mild or Major NCD. Diagnoses were made for patients with cognitive complaints who fulfilled DSM-5 criteria [3, 4] and reported a CDR-G ≤ 0.5. All diagnoses were made at the end of the assessment and were reviewed by an expert geriatrician or neurologist blind to the MoCA 7.1 score: the clinic-based out-patients with Mild or Major NCD were admitted to the PG.
2) Normal elderly controls were healthy elderly volunteers randomly recruited from the community. Inclusion criteria: complete independence, absence of memory-related and functional complaints confirmed by an informant partner, well compensated sensory impairments. All participants underwent assessment with the MMSE, MoCA 7.1, IADL, and CDR. Only participants with intact IADL and CDR- Global and -Sum of Boxes scores [60] were included in the NECs. This procedure is highly successful to avoid including older adults with early-stage unrecognized cognitive impairment (CDR-G: 0.5) [61, 62] who display a performance within normative values [60, 63]. Underdetection of early or mild cognitive impairment (CDR-G: 0.5–1) among independent elderly may be frequent due to lack of insight [64].
Test-retest and inter-rater reliability
Measures were retested 30–60 days apart: the time interval was chosen to minimize practice effects [65]. Test-retest and inter-rater reliability were assessed respectively on randomly selected sub-samples of 86 and 77 participants stratified by CDR-G scores of 0, 0.5 and 1.
Statistical analysis
First, we compared PG and NECs. Secondly, we assessed the test-retest and inter-rater reliability of MoCA 7.1 computing the intra-class correlation coefficient (ICC) between the first and second administration and the two raters respectively. Moreover, we assessed the diagnostic accuracy of MoCA 7.1 towards healthy cognitive status, Mild and Major NCDs by calculating their Area Under the Curve of the Receiver Operating Characteristic curve (AUC-ROC). The Youden index was used to estimate the optimal cut-off score to discriminate the presence of NCDs. We report the predictive accuracy as the probability of correct classification of participants with or without cognitive impairment. Furthermore, we applied an index of best fitting in the data analysis to verify if the sensitivity/specificity ratio could be significantly modified. Finally, we used logistic regression with stepwise variable selection to identify domains that most contributed to differentiate NECs from NCDs.
Statistical analyses were performed using SPSS version 25.0 (SPSS, Inc., Chicago, IL).
RESULTS
Participants
The NECs group included 123 participants and their informant partners were: spouse (51%), children (31%), other relatives (18%).
The PG included 469 participants: 214 had Mild NCD and 255 Major NCD. The informant partners of the PG were the informal caregivers as follows: spouse (Mild NCD 42%, Major NCD 33%), children (Mild NCD 44%, Major NCD 56%), other relatives (Mild NCD 14%, Major NCD 11%).
Table 1 provides an overview of the characteristics of participants. The PG was significantly older and less educated than NECs. Among NECs, the education effect was assessed splitting the school years in four subgroups: 1) 3–6 years; 2) 7–11 years; 3) 12-13 years; 4) 14–20 years (Table 2). The four subgroups did not show significant differences in mean age, MMSE-RS, and MoCA 7.1 scores. Mean MMSE-RS was significantly higher than MMSE-NS where, in particular, the intermediate education level (subgroup 3) showed a mean score significantly lower than lower education levels (subgroup 1 and 2). Gender was not associated to differences in cognitive performance.
Characteristics of the study participants
Characteristics of the study participants
NCD, Neurocognitive Disorder; †Analysis of Variance: F = 32.947, p < 0.05; Bonferroni Post Hoc: p < 0.05; ‡Analysis of Variance: F = 38.758, p < 0.05; Bonferroni Post Hoc: p < 0.05.
Demographic and cognitive performance by education in the normal elderly group
MoCA, Montreal Cognitive Assessment; MMSE, Mini-Mental State Examination; RS, raw score; NS, normative score; §ANOVA, Analysis of Variance: Bonferroni Post Hoc: p < 0.05; ¶T-test: p < 0.05
Test-retest and inter-rater reliability
The mean difference in the total score of MoCA 7.1 between the first and second administration was 1.04±1.33 points: test-retest and inter-rater reliability showed a statistically significant ICC (0.945, p < 0.001 and 0.999, p < 0.001).
Discriminant power of the MoCA 7.1. and MMSE
Average MMSE-RS, MMSE-NS, and MoCA 7.1 scores of normal, Mild and Major NCD participants differed significantly from each other (respectively F 168,07, p < 0.001; F 113,74, p < 0.001, F 383,731, p < 0.001) (Table 3).
MMSE and MoCA 7.1 mean scores±standard deviations for normal controls and participants with Mild and Major Neurocognitive Disorder
MoCA, Montreal Cognitive Assessment; MMSE, Mini-Mental State Examination; RS, raw score; NS, normative score; NCD, Neurocognitive Disorder; §ANOVA, Analysis of Variance: Bonferroni Post Hoc: p < 0.05.
Differences were more pronounced with MoCA 7.1 than MMSE: in particular the mean score of the Mild NCDs falls within the normal range on the MMSE and in the abnormal range on MoCA 7.1.
The magnitude of differences in mean MoCA 7.1 and MMSE scores displayed an overlap with the values of the conversion table for MoCA and MMSE [59], confirming to be more relevant, i.e., greater than 1 standard deviation in Mild (MoCA 5.66–4.93 points < MMSE-RS/NS) and Major NCD (MoCA 6.54–6.09 points < MMSE-RS/NS) groups.
The predictive accuracy of MoCA 7.1 was higher than that of the MMSE, with excellent sensitivity in identifying Mild NCDs (95.3%) (Table 4). The sensitivity of MMSE resulted low in detecting Mild NCDs both for raw (28.5%) and normative scores (53.8%). Specificity was the percentage of NECs that scored at or above the cut-off of 26 for MoCA 7.1. and 27 for MMSE-RS and MMSE-NS.
The criterion values and the coordinates of the ROC curves examined in normal and Mild Neurocognitive Disorders for MoCA 7.1 and MMSE both as raw and normative scores
ROC, Receiver Operating Characteristic curve; MoCA, Montreal Cognitive Assessment; MMSE, Mini-Mental State Examination; RS, raw score; NS, normative score; NCD, Neurocognitive Disorder.
MoCA 7.1 showed a good specificity (84.5%), confirming the standard cut-off as the best fitting one. The MMSE-RS showed a high specificity, identifying 99.1% of the NECs while the MMSE-NS lowers to 85.7%. The MMSE yielded a better trade-off between sensitivity and specificity for Mild NCDs with different cut-off scores: ≤28 for raw scores and ≤27.3 for normative scores.
Nonetheless, the diagnostic accuracy of MoCA 7.1 for overall cognitive impairment was higher than MMSE (Fig. 1). MoCA 7.1 showed a good performance in the detection of Mild NCD with an AUC (0.963, sig. 0.00001) higher than MMSE (MMSE-RS: 0.803, sig. 0.00001, MMSE-NS: 0.742, sig. 0.0001).

Normal versus Mild NCD. ROC curve, MMSE raw and normative scores and MOCA 7.1.
The diagnostic accuracy of MoCA 7.1 was examined to identify the items with greater sensitivity towards Mild NCD. The factorial analysis showed that 5 items of MoCA 7.1 distinguished significantly NECs from Mild NCDs: visuospatial/executive, digit span, serial 7’s subtraction, abstraction, free delayed recall (Table 5).
Stepwise logistic regression analyses of MoCA 7.1 items in the normal elderly and Mild Neurocognitive Disorder groups
*p < 0.05.
DISCUSSION
This study examined the psychometric properties of the Italian version of MoCA 7.1 in older adults.
MoCA 7.1 showed optimal test-retest and inter-rater reliability which may depend on its simple structure and clarity of the instructions. As the original version [12], MoCA 7.1 yielded higher diagnostic accuracy than the MMSE, differentiating normal cognitive performance from early impairment (Table 4, Fig. 1) [13, 66].
The diagnostic accuracy of MoCA is explained by its robust content and construct validity. First, the psychometric structure of MoCA contains five of ten items that are essential for evaluating the early transition from healthy cognition to NCDs (Table 5) [38, 67]. These items account for 50% of the total score variance and evaluate five of the six main cognitive domains associated with the anatomical-structural changes of the brain underlying NCDs [47, 69]. Secondly, the length of the interval time for the delayed recall of MoCA matches the recommended standard of 3–5 minutes, which is relevant to ensure the similar ability of the long-term delayed recall (20-minute delay time) to discriminate Mild NCD from cognitively normal participants [70]. In contrast, the MMSE contains only three items relevant for early detection of NCDs and has a delay time (2 minutes) for short-term delayed recall with a number of recalled items (3 versus 5) below the thresholds of the standardized neuropsychological tests. [14, 71].
MoCA 7.1 scores among NECs did not display relevant effects due to age and education, thus confirming the chance to preserve the usual level of cognition of the adulthood across lifespan (Table 2) [41, 68].
The comparison of the MMSE-RS with MMSE-NS confirmed that adjustment for age and education is questionable and may mislead the clinical diagnosis (Table 2) [22, 72].
Administering MoCA 7.1 without adjustment for age and education unlike other studies did [73–77], yielded the optimal balance among sensitivity and specificity towards Mild NCDs (Table 4). In contrast, the MMSE-NS confirmed the well-known low or very low sensitivity towards Mild NCDs.
These findings are consistent with some prior studies. One twin study suggested that most of the association between MMSE scores and education reflects genetically-mediated differences in cognitive capacity rather than educational biases [78]. The second examined the Differential Item Functioning and found educational effects on serial subtractions, spelling backwards, writing a sentence; however, such effects explained less than 2% of differences participants with high and low educational attainment [79].
Our findings are aligned also with other studies [18, 22] and confirm the application of the original cut-off score of MoCA for detecting mild NCD in the Italian population while other Italian contributions have proposed five different lower cut-off scores ranging from 15,5 [74, 77], 17 through 19 [75], 23,28 [73] to 24,17 [76].
Among the aims of these studies there was not the validation of MoCA 7.1 to detect Mild NCD [3, 12]: three were aimed to determine normative data [73, 76], one to detect “probable cognitive impairment” and “probable AD” [75] and one “cognitive impairment” [77].
The diagnostic accuracy of MoCA 7.1 is important for clinical practice allowing a limited threshold of excluding false negatives counterbalanced by an acceptable risk of monitoring patients referring memory impairment but not definitively classifiable as normal or impaired cognition [28]. When administering only MMSE, the clinician should expect that some cases of Mild NCD could be labeled as “normal”, failing to schedule follow-up.
MoCA 7.1 has shown a convergent validity with the Italian version of the General Practitioner assessment of Cognition (GPCog) [80, 81], a brief cognitive screening test used by the Italian general practitioners (https://www.demenzemedicinagenerale.net).
While the comparison of the diagnostic accuracy confirmed that MoCA 7.1 overcomes the well-known ceiling effect of MMSE in detecting Mild and, not rarely, Major NCDs [82, 83], conversely, MoCA 7.1 showed a greater floor effect than MMSE [59], making the last one preferable to follow-up the advanced stages of Major NCD (CDR-G ≥3).
The diagnostic accuracy of MoCA provides a solid prognostic basis to the cognitive health as reported recently [68] Table 1 or, conversely, to detect the subtle abnormalities in prodromal neurocognitive disorders [84].
These characteristics of MoCA are relevant to update the algorithm of the basic approach to the assessment of cognitive functions in Mild NCD [85].
Limitations
This study has some limitations. First, the performance of MoCA 7.1 was analyzed only for the Mild and Major NCD, without considering its accuracy towards subtypes [86, 87]. Second, the NECs group had a limited age range (69.75±9.02) with a reduced proportion of examinees younger than 60 and older than 80 [88]. This limits the strength of our data when confirming the absence of the generalized effect of age and education applied to the entire population while a part preserves the usual level of cognition of adulthood along with aging. Nevertheless, this limit may be counterbalanced by the inclusion criteria of the NECs (intact IADL and CDR-G: 0). Finally, we did not assess the role of premorbid intellectual function [23].
MoCA 7.1 can be considered an ideal basic minimum data set of neuropsychological assessment for the detection mainly of Mild NCD, as the original.
The original cut-off ensures an optimal balance between using it as either a screening or a diagnostic tool. Nevertheless, other cut-offs may be used, depending on whether the test is being used as a screening (high sensitivity required) or as a diagnostic tool (high specificity required).
The optimal diagnostic accuracy of MoCA 7.1 allows clinicians and neuropsychologists to consistently reduce the need to administrate standardized sets of complex and time-consuming neuropsychological tests, reserving them preferably to the patients clinically positive but with normal MoCA 7.1 and MMSE.
Consequently, the professionals working in the CCDD can save time for the implementation of non-pharmacologic interventions for person with Mild or Major NCDs such as cognitive training, counselling, psychotherapy, psychosocial interventions.
The most relevant result of this study consists in the timely detection of cognitive impairment to allow professionals and caregivers to administrate timely preventive, pharmacologic and non-pharmacologic interventions with better outcomes with respect to more advanced stages (CDR-G score ≥2) [1].
In clinical practice, the routinely administration of both MMSE and MoCA 7.1 is recommended to avoid false negatives in the normal score range of MMSE especially in people with elevated premorbid intelligence.
The elderly achieving normal scores of MoCA and CDR-G and SB = 0 may be reasonably confident to maintain a healthy cognition longer than reporting normal scores of the MMSE.
Footnotes
ACKNOWLEDGMENTS
In memory of Prof. Rita Levi Montalcini who in person showed to AP the significance of normal cognition in near centenarian.
Cavicchi Eva born on February 22, 1919, elementary school, housewife, widow, only one of 3 children living, institutionalized when 98 years old, still solving daily challenging crossword puzzles at the age of 100 and performing MMSE 28/30 and MoCA 22/30 even if with relevant eye and hearing impairments.
Dr. Davide Zaccherini for his collaboration in administering firstly MoCA 7.1 in the Center for Cognitive Disorder and Dementia of Cento (Health County of Ferrara, Ovest District).
FUNDING
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
CONFLICT OF INTEREST
Alessandro Pirani, Andrea Fabbo, Marco Bruno Rocchi, Cristina Tulipani, Matteo Galassi, and Mirco Neri have no conflict of interest to report.
Ziad Nasreddine is the copyright owner of the MoCA test and received grants from Eli Lilly, Roche, Biogen for clinical trials, royalties for the use of the MoCA test in clinical trials, consulting fees from Biogen, Eli Lilly and Roche, payments from Biogen and Roche for data safety monitoring board or advisory board.
Francesca Neviani received honoraria from Medicalnet and Aristea. The payments were made to her.
Marco Bertolotti received support from Mylan Drug Company for attending the meeting of the European Atherosclerosis Society (Maastricht, Netherlands, 2019) and serves as member of the national board of the Italian Society for Cardiovascular Prevention and the regional board of the Italian Society of Gerontology and Geriatrics; he served as a former member of the regional board of the Italian Society for the Study of Atherosclerosis.
Martino Belvederi Murri received an honorarium for a lecture on depression by an educational agency (Lopez Eventi e Congressi) for which he had to declare absence of conflict of interests. The payment was made to him.
