Abstract
Background:
Cognitive screening tools are important in the detection of dementia, including Alzheimer’s disease; however, they may contain cultural biases.
Objective:
This review examines culture-fair cognitive screening tools and evaluates their screening accuracy, strengths, and limitations.
Methods:
Medline, Embase, PsychINFO and CINAHL were searched. The protocol was registered on PROSPERO (CRD42021288776). Included studies used a culture-fair tool to assess cognition in older adults from varying ethnicities. Narrative synthesis was conducted.
Results:
28 studies were included assessing eleven different tools. The Rowland Universal Dementia Assessment Scale (RUDAS) was as accurate as the Mini-Mental State Examination (MMSE) (AUC 0.62-0.93), with a similar sensitivity (52–94%) and better specificity (70–98%), and the Multicultural Cognitive Examination (MCE) had improved screening accuracy (AUC 0.99) compared to RUDAS (AUC 0.92). The Visual Cognitive Assessment Test (VCAT) was equivalent to MMSE (AUC 0.84–0.91). The Kimberley Indigenous Cognitive Assessment tool (KICA) had AUC of 0.93–0.95; sensitivity of 90.6%, specificity 92.6%.
Conclusions:
The RUDAS, KICA and VCAT were superior to MMSE for screening dementia in ethnic minorities. Other tools also showed good screening accuracy. Further research should be done to validate tools in different populations.
INTRODUCTION
Dementia affects approximately 500,000 people in the UK with profound socioeconomic implications; the care of people living with dementia is estimated to cost up to £14.93 billion annually, exceeding the cost of stroke, heart disease, and cancer combined [1]. While the number of people living with dementia in the UK is expected to double by 2050, prevalence among ethnic minority communities is expected to increase sevenfold [2]. Commonly used cognitive assessment tools include the Addenbrooke’s Cognitive Examination (ACE III), Mini-Mental State Examination (MMSE), and the Montreal Cognitive Assessment (MoCA), which have been shown to be helpful in aiding diagnosis of dementia [3]. However, these tools were created for “Western” populations and a multitude of studies have demonstrated that performance is significantly influenced by age, educational status, ethnicity, and language [3–5], thus resulting in poor test accuracy for non-Caucasian patients [6]. Comprehensive and accurate cognitive assessment is very important in the workup for dementia. For example, it can inform referral to other members of the multi-disciplinary team such as speech and language, or occupational therapy, monitor longitudinally cognitive changes and guide pharmacological and social interventions. Under-diagnosis of dementia can have negative implications including inadequate support and delayed use of anti-dementia drugs, which may reduce the efficacy even during the earliest stages of Alzheimer’s disease [7]. There are also negative implications of over-diagnosis, which can lead to unnecessary anxiety, and implications for driving, work, and insurance. Moreover, the accurate identification of mild cognitive impairment is essential to the advancement of preventive and therapeutic research.
Attempts to translate or adapt commonly used cognitive tests have been explored, however these still pose issues [8]. For example, repetition of the English idiom “no ifs, ands, or buts” used in the MMSE does not maintain the same level of linguistic difficulty, test of fluency or cultural relevance upon translation. Ethnic disparities in performance during neuropsychological testing have long been recognized, regardless of socioeconomic status or level of education [9]. Such findings are attributed to an over-reliance of these tools on cultural norms and language [10]. A variety of cognitive assessments have been developed for specific populations. However, at present there is no universal gold standard tool for the diagnosis of cognitive impairment in ethnic minority groups. A culture-free test are test which is devoid of all culture-related content, which can be very difficult to achieve. In contrast, a culture fair test is a test designed to reduce the influence of cultural elements and therefore provide an accurate assessment without favoring any one culture [11].
Thus far, there has not been an up-to-date review comparing the variety of available culture-fair cognitive screening tools. This systematic review aims to determine the commonly used culture-fair cognitive screening tools and evaluate their test accuracy, strengths, and limitations.
MATERIALS AND METHODS
Search
This review was reported in accordance with the PRISMA guidance for reporting systematic reviews and a PRISMA checklist is included in the Supplementary Material. Medline, Embase, PsychINFO, and CINAHL databases were searched from inception using the search strategy presented in the Supplementary Material. The search strategy was developed in conjunction with a librarian at the Leicestershire Partnership Trust (LH). The search was completed in November 2021 and updated in July 2023. No additional studies were found in the second search in July 2023. Included articles were published in any year and there was no restriction on time. Reference lists of included articles and citation indices were screened for additional relevant material. Only adult human studies where patient data could be successfully extracted were considered. Included studies had to use a cognitive screening tool, however, were not limited to a specific cognitive test and included neuropsychological test batteries. Abstracts, conference papers, posters, translated or adapted tools, adapted for specific populations, or studies reporting on less than five subjects were excluded. The review protocol was registered on PROSPERO prior to commencement of the review (CRD42021288776). The PRISMA flow diagram is presented in Fig. 1, to outline the papers identified through the database searches.
Prisma flow diagram.
Study selection
Included studies were not limited by study design, with inclusion of randomized controlled trials, non-controlled trials, case control, and cross-sectional studies. Systematic and narrative reviews were excluded, however, analyzed as sources for potential references. We included studies which used a culture-fair or culture-free cognitive tool to assess cognition in healthy adults or older adults with dementia (over 65 years old), from varying ethnicities. Studies were excluded if they used translated versions of widely used validated tools, cognitive tests which were not specified to be culture-free or culture-fair, tools which were too time consuming in a clinical setting such as outpatient clinics (for example those taking over an hour), namely neuropsychological batteries, and tools which were too brief and not designed for outpatient settings, for example tools designed for quick assessment at the bedside or in emergency departments, or included a small number of participants (<10). Studies were screened on title and abstract and then by full text by two independent reviewers (TC, SS). Disagreements were resolved internally, with an option of a third reviewer (LB).
Data extraction, quality assessment, and data analysis
Data were extracted into Microsoft Excel by one reviewer (TC), and a second reviewer independently checked 20% (SS). No discrepancies in numerical data were found. Quality assessments were independently conducted by two reviewers (TC, SS), using the QADAS II tool [12]. Disagreements were internally resolved between the two reviewers. The only group with adequate data for a meta-analysis were studies examining the Rowland Universal Dementia Assessment Scale (RUDAS), and this has published recently [13]. There was insufficient data for a further meta-analysis and so a narrative synthesis was conducted of all included studies.
RESULTS
The search strategy identified 5,865 articles, with 4,040 remaining after duplicates were removed (Fig. 1). An additional four studies were found through manual search of the references. A total of 92 full texts were identified based on initial screening of abstracts and 28 studies were included in the review. As described in Table 1, studies took place in multiple countries including Singapore, Malaysia, Indonesia, Philippines, Netherlands, Tunisia, Germany, Belgium, Denmark, Sweden, Norway, Greece, India, Taiwan, Israel, Japan, USA, Nigeria, Jamaica, Canada, Latin America, Chine, Africa, Australia, Spain, and Tanzania. There were 18 test accuracy studies, seven cross sectional studies, one normative data study and one longitudinal study. Six studies were multicenter. The studies ranged between local community, general practice, outpatient memory clinic, geriatric rehabilitation, and inpatient settings. Mean age of participants ranged between 65–80 years old.
Study characteristics of included studies
VCAT, Visual Cognitive Assessment Test; MMSE, Mini-Mental State Examination; HC, healthy controls; MCI, mild cognitive impairment; F, female; M, male; AD, Alzheimer’s disease; AUC, area under the curve; MoCA, Montreal Cognitive Assessment; CCD, Cross-Cultural Dementia Screening; ROC, receiver operating characteristic curve; DSB-100, Development of the Dementia Screening Battery-100; MCE, Multicultural Cognitive Examination; RUDAS, Rowland Universal Dementia Assessment Scale; NEST, Neuropsychological Evaluation Screening tool; HMSE, Hindi Mental State Examination; MMT, Modified Mini Mental Test; CCCE, Cross-cultural cognitive examination; ECAQ, Elderly Cognitive Assessment Questionnaire; CSI-D, Community Screening Instrument for Dementia; GPCOG, General Practitioner Assessment of Cognition; KICA/mKICA, Kimberley Indigenous Cognitive Assessment Tool/modified Kimberley Indigenous Cognitive Assessment; IDEA, Identification of Dementia in Elderly Africans.
Eleven studies assessed RUDAS as the primary cognitive tool, of which nine studies used MMSE or MoCA as reference tests. RUDAS was found to be as accurate as the MMSE [14–16], with a similar sensitivity (range: 52–94%) and better specificity (range: 70% –98%) [17]. Area under the curve (AUC) values reported ranged between 0.62 [18] to 0.93 [19]. The majority of studies showed that RUDAS scores were found to be uninfluenced by education [16], language [16, 20], and sex [16]; however, three studies identified that RUDAS scores were influenced by education, as highlighted in Table 2 [14, 19]. Nielsen et al. (2013) established it to be superior to MMSE, for use in an older multicultural population [20].
Details including strengths and limitations of included cognitive screening tools
RUDAS, Rowland Universal Dementia Assessment Scale; VCAT, Visual Cognitive Assessment Test; CCD, Cross-Cultural Dementia Screening; DSB-100, Development of the Dementia Screening Battery-100; CSI-D, Community Screening Instrument for Dementia; MCE, Multicultural Cognitive Examination; NEST, Neuropsychological Evaluation Screening tool; MMT, Modified Mini Mental Test; CCCE, Cross-cultural cognitive examination; ECAQ, Elderly Cognitive Assessment Questionnaire; KICA, Kimberley Indigenous Cognitive Assessment Tool; IDEA, Identification of Dementia in Elderly Africans.
The remaining seventeen studies analyzed an array of other cognitive tests; three studies used the Community Screening Instrument for Dementia (CSI-D) [21–23], three studies used the Identification of Dementia in Elderly Africans (IDEA) [24–26], two studies for the Kimberley Indigenous Cognitive Assessment tool (KICA) [27, 28], two studies used the Visual Cognitive Assessment Test (VCAT) [6, 29], and there was one study each for the Cross-Cultural Dementia Screening (CCD) [30], Dementia screening battery (DSB-100) [31], Multicultural Cognitive Examination (MCE) [20], Neuropsychological Evaluation Screening tool (NEST) [32], Modified Mini Mental Test (MMT) [33], Iowa screening test [34], Cross-cultural cognitive examination (CCCE) [35], and the Elderly Cognitive Assessment Questionnaire (ECAQ) [36]. Sensitivity testing demonstrated NEST had better sensitivity compared to the Hindi Mental State Examination (HMSE) at all educational levels (94.8%) [32]. CSI-D had good sensitivity across different countries and the brief version of the full CSI-D shared favorable culture-fair screening properties of the full assessment [21]. The Iowa test, ECAQ and CCCE showed similar sensitivity of 85%, 85.3% and 88% respectively [34–36]. In terms of specificity of dementia identification, Iowa test had 66%, CCCE showed 97.4% and ECAQ 91.5% [34–36]. VCAT had good construct validity and good internal consistency [6] and was not affected by age, years of education, race, or employment, whereas MoCA was [29]. Receiver operating characteristic (ROC) analysis found VCAT to be equivalent or more effective to the MMSE and MoCA (AUC 0.84–0.91) [6, 29] and CCD to have high predictive validity for dementia (AUC 0.85–0.95, sensitivity 85%, specificity 89%) [30]. CCD showed no differences between the ethnic groups after adjusting for age and education [30]. MCE had significantly improved test accuracy (AUC 0.99) compared with using the RUDAS alone (AUC 0.92) [19]. The KICA showed good test accuracy based upon AUC scores of 0.93 and 0.95 [27, 28] with a sensitivity of 90.6% and specificity 92.6% [28]. The IDEA had an AUC of 0.78–0.87 [24, 26] which was improved in combination with the IDEA-IADL to 0.94 [26]. The MMT was useful in specifically testing for vascular dementia, however, was affected by educational level [30] and DSB-100 was valid in a Tunisian elderly population [31]. As detailed in Table 2, the DSB-100 was found to be affected by education level. The CSI-D, brief-CSI-D, CCD, and VCAT were found to have little effect of educational level. The RUDAS, VCAT, and Iowa screening test were all found to be acceptable to patients. The Iowa screening test, NEST, and ECAQ were all quick to administer (under 15 min). Many studies assessed the accuracy of tools at different cut off points [6, 33]. Cut off for IDEA was quoted as < 7 out of 12 for the 5-item instrument, or < 6 out of 9 for the 7-item instrument [24]. Optimal cut off to detect cognitive impairment was≤24. Optimal cut off for the KICA was quoted as 31/32 (KICA) and 34 [27]. RUDAS cut offs varied from 21 [14–16, 27] to 24 [19]. Cut off for CSI-D was five or less to favor specificity or six or less to favor sensitivity [23] and for MMT was 33/55 [33]. Optimal cut off for NEST was three or more errors [32].
Risk of bias assessment
The majority of studies showed low risk of bias, yet there were a few studies deemed to be high risk on domains such as reference standards [30, 35], flow and timing [19, 37], patient selection [14, 23], and Index test [33, 34].
DISCUSSION
Our systematic review highlights that there have been multiple attempts across various countries to create a culture-fair cognitive assessment tool. The need for such a tool has been widely recognized and there have been numerous studies highlighting the shortcomings of the currently used cognitive assessment tools in screening for cognitive impairment in multicultural populations [3–5]. We propose the universal use of cognitive assessment tools free of cultural bias as a practical approach for memory services in response to the ever-growing diversity of the UK population. Our review examined 28 studies identifying eleven different culture-fair cognitive assessment tools, all of which show promising results in studies in their country of development. The RUDAS, VCAT, KICA, and CCD were found to have preferable specificity and sensitivity to the MMSE when used in ethnic minorities.
A strength of the review is the inclusion of studies assessing patients from multiple different countries, and both native and migrant patients in those countries. Including studies from a variety of countries and multi-country studies increases the likelihood of generalizability of the results across the world. However, there are multiple countries which have not been studied, including the UK, which brings scope for further research.
The RUDAS and KICA have been recognized as promising options to reduce cultural bias in ethnic minorities [13]. Both have been compared to the MMSE and shown superior or comparable diagnostic accuracy, and less effect of both education and culture. One study compared the modified KICA and RUDAS together, and showed the modified KICA had greater diagnostic accuracy (94%) compared to the RUDAS (89%) [27]. The diagnostic accuracy of the RUDAS is increasingly well documented as a cross-cultural cognitive test [40]. One meta-analysis that included 26 studies found the RUDAS to have a comparable diagnostic accuracy across a multitude of sociocultural settings, with limited impact from educational, linguistic, or cultural biases [13]. However, despite being deemed culture-fair, both have cultural adaptations available. For example, for the RUDAS, in both the Peruvian and Spanish validation study [14, 41], the word tea in the supermarket list in the memory item was changed to coffee, and in the Thai validation study, animal fluency was substituted with fruit fluency to reduce the impact of formal education on the language item [40]. Although these may appear to be minor changes, the fact that cultural adaptations are required suggests that the tool is not entirely culture-fair, as without the adaptations the results may be skewed. KICA has been incorporated into Australia’s dementia services following recognition of its’ superior sensitivity and specificity for the diagnosis of cognitive impairment in Indigenous communities where illiteracy, language, or cultural barriers may impede assessment [27]. The KICA has been adapted for Canadian, Persian, and Brazilian populations, as discussed in the review by Mukaetova-Ladinska et al. [42]. Neither the KICA nor RUDAS have been validated for use in the UK memory services as of yet. Furthermore, it is important to note that our review included participants aged 65 and over. In a study of 70 UK participants, in the younger persons memory service the RUDAS was found to have modest accuracy (AUC = 0.70 (0.57–0.82),
The review has highlighted the challenges in separating the effects of culture and education in cognitive tools, and many patients may be affected by both cultural and educational biases. The RUDAS, MMT, and DSB-100 were found to be affected by educational level to varying degrees and may require adjustment for educational qualifications. However, regarding the RUDAS, the effect of education was still much less than the effect on the MMSE [17, 44] and some studies did not find it to be affected by educational level at all. Educational level was found to have little impact on the CSI-D, brief-CSI-D, CCD and VCAT [16, 30].
The VCAT provides an option which is neutral to both language and education [29]. It is a 30-item tool and includes testing of executive functioning which is very useful, and something not all cognitive tools cover, including the widely used MMSE. The items are visual based and therefore may be an extremely useful tool across cultures, removing the barriers of language [29].
An interesting finding from the review was the study testing the IDEA-IADL, which combines the cognitive assessment with an assessment of instrumental activities of daily living [26]. These are complex activities which are generally affected earlier in cognitive impairment, and therefore can aid in the holistic diagnosis of dementia. It is well known that diagnosis of dementia does not rely on cognitive screens alone, and an assessment of functioning is required. Activity of Daily Living (ADL) questionnaires can be a useful tool, however the authors highlighted that the commonly used ADL tools, such as the Lawton’s ADL tool, were developed in high income countries and therefore may also be subject to cultural biases [24, 25]. As seen above, AUC curve was 0.896 for diagnosing dementia when using the IDEA-IADL alone and 0.94 when used with in combination with the IDEA [24, 25]. This could indicate areas for further research, to create tools which can be easily administered, avoid cultural biases, and can be used to assess both cognitive abilities and instrumental activities of daily living.
A major hurdle identified through this review for many of the cognitive assessment tools is the lack of validation worldwide and in varying populations. Many countries, including the UK now have a very multicultural elderly population, and therefore having a tool suitable for one ethnic population is of minimal use. Therefore, many of the studies will need to be validated across more countries.
Limitations
Despite having 28 studies included in the review, the number of studies examining each tool was limited, other than for the RUDAS (
One of the areas of heterogeneity of studies are the varying ethnic groups assessed. It may be difficult to compare results of various culture fair tools based upon which groups were studied, and this could affect the AUC results given. Future research is needed to evaluate these tools in different populations. It would also be useful for all tools to be directly compared to cognitive assessment tools widely used such as the MMSE, to compare screening accuracy for cognitive impairments among ethnic minorities.
Although majority of studies showed mostly low risk of bias, there were a few studies deemed to be high risk on domains such as reference standards [30, 35], flow and timing [20, 37] patient selection [14, 23] and index test [34, 30]. A potential reason for this could be that some studies were published more than two decades ago, when reporting standards were lower. However, it is a strength of the review that most studies showed mainly a low risk of bias.
Furthermore, this review excluded culture-fair computerized and digital tools. Computerized tools are increasingly being utilized in clinical practice [45], and may be useful in remote assessments, which have become increasingly popular, especially after the COVID-19 pandemic. Some cognitive assessment tools have been created as purely computerized tools, which are culture-fair [45]. An example of the digital tool which was excluded from this review is the MemTax Memory Test, which is a digital cognitive screening instrument that takes approximately 90 s to complete [46]. It consists of culture-fair images and is available in 120 languages. It has been shown to be able to detect both mild cognitive impairment and AD with better accuracy compared to the MoCA (AUC = 0.799 compared to AUC = 0.767) [47]. These were excluded as there are limitations in facilities to administer computerized and digital tools to all patients, and this may not be generalizable to all services worldwide. However, it would be beneficial for a further review to summarize the available computerized and digital tools and their efficacy, as this may be a useful tool in future practice.
Conclusion
The review has identified various cognitive assessment tools which are all good options for supporting a diagnosis of dementia in patients of any ethnic background. The RUDAS, KICA, CCD and VCAT were comparable to MMSE in terms of specificity and sensitivity. The RUDAS, CSI-D, brief-CSI-D, CCD, and VCAT were minimally influenced by educational level. The VCAT is especially useful in non-verbal patients or patients who struggle with language and education. These tools should be considered in memory clinic settings, to reduce inaccuracy in diagnosis due to cultural biases, especially in ethnically diverse patient populations. Further research is needed to evaluate computerized cognitive assessment tools, and validation of cognitive tools in varying populations, including the UK population. Further research is also needed to create a tool which does not need to be adapted to varying cultures, and could be used across any culture without adaptation, which some of the tools above still require.
Footnotes
ACKNOWLEDGMENTS
The authors have no acknowledgments to report.
FUNDING
The authors have no funding to report.
CONFLICT OF INTEREST
The authors have no conflict of interest to report.
DATA AVAILABILITY
Data sharing is not applicable to this article as no datasets were generated or analyzed during this study.
