Abstract
Background:
The aim is to validate Turkish version Rowland Universal Dementia Assessment Scale (RUDAS).
Methods:
One hundred forty patients (>65 years) were included. Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition -V was used in all patients. Rowland Universal Dementia Assessment Scale was applied for 2 days. Results were compared with Mini-Mental State Examination; reliability, validity, and statistical values were determined.
Results:
Time validity was verified. Mini-mental state examination was correlated 45.3% in control and 73% in dementia group. Factor weights varied between 0.44 and 0.81; factor construct was verified as 6-item scoring. When 25 was cutoff point, sensitivity was 92.86% and specificity was 92.86%. Content validity index was found to be 100% by 7 specialists. Cronbach α (0.692) and test–retest reliability (intraclass correlation = 0.987) were determined.
Conclusion:
Rowland Universal Dementia Assessment Scale (Turkish) is validated and verified as reliable. Test could be applied for 5 minutes approximately; results are not affected by educational status, immigrant status, and language used; however, age and gender have significant effect on results.
Introduction
Dementia is a disease characterized by impairment in learning, memory, linguistic, attention, and sociospatial perception functions. While dementia might be presented by decrease in particular functions, this entity might have such severe course, limiting daily activities and independent movement abilities. Neurodegeneration resulted due to various reasons and causes permanent, progressive impairment in more than 1 cognitive area in patients with dementia. Dementia is a geriatric syndrome, and its incidence increases by age. It is especially an important health problem in countries with high mean age population. Population older than 65 years of age are increasing globally day by day, and approximately 30% to 50% of these patients are complaining about decline in cognitive functions and amnesia. 1,2 When high prevalence is taken into consideration, dementia brings a great economic burden in terms of diagnosis, treatment, and patient care. When accompanying psychosocial changes and isolation of both patients and their relatives are taken into account, it is obvious that early and correct diagnosis is the mainstay of this process. Early initiation of treatment increases the quality of life of patients and decreases health-care costs.
In the aim of early and correct diagnosis, there are lots of reliable and valid cognitive assessment tests that were designed according to countries and sociocultural features of the people. However, these tests have strengths and weaknesses. In daily practice, it is vital to develop new tools compensating the lacking parts of the current tests.
Cognitive assessment tests are brief cognitive inspection tools that are used to determine severity of cognitive deterioration, rate of progression by the time, and response to specific treatment modalities. These tests should not be used instead of detailed mental examination in order to diagnose dementia. Mini-Mental State Examination (MMSE) test is globally one of the most commonly used assessment tool. 3 Although MMSE is used worldwide, some known difficulties limit the confidence to the results in specific population. The test results are especially difficult to interpret in low educational levels and non-English-speaking populations. 3,4 The Rowland Universal Dementia Assessment Scale (RUDAS) was developed in Australia in 2004, designed particularly aiming to tackle the difficulties in determination of cognitive impairment in populations with socioculturally diverse backgrounds. 5 The RUDAS has a 6-item scoring structure, assesses several cognitive functions, and can be easily administered in outpatient clinic. Psychometric features were shown as perfect in the original validation study that was conducted in Australian population. 5 In several countries, the results of the RUDAS were also shown to be independent of educational status and linguistic properties. 6 Therefore, the RUDAS might be a valuable tool in the assessment of culturally and linguistically diverse populations. Turkey has a diverse population as well, and it is extremely important and necessary to find a valid and reliable test to assess cognitive functions in our population. In this study, we assessed the reliability and validity of the Turkish version of RUDAS as a screening tool in a major neurocognitive disorder.
Methods
Our study was conducted between December 2015 and September 2017 with geriatric patients older than 65 years of age who were admitted for the first time to the Istanbul University Cerrahpasa Medical Faculty, Geriatrics Outpatient Clinic.
Participants
Two hundred eighteen patients who carried inclusion criteria and gave informed consents were determined between 530 patients who have applied for the first time during these 22 months. Patients who did not have any disabilities preventing the application of geriatric assessment tests, any acute critical diseases, and any major psychiatric disorder other than dementia according to DSM-V criteria determined by psychiatrist were included in the study. First, patients were assessed clinically by the geriatrician, and MMSE was applied. Patients who were thought to have dementia or normal cognitive functions were sent to the geropsychiatrist for further assessment. From these patients those who were thought to have major neurocognitive disorder according to DSM-V criteria and clinical interview were included as the patient cohort. While vast majority of the patients were thought to have Alzheimer’s disease, we did not perform further assessment tests to define the types of dementia. Control group was included in the study from patients whose probability of dementia have been excluded by DSM-V and clinical interview. Also, Neuropsychometric tests (NPT) were used as supportive measure in 40 patients in the control group, 39 patients in the dementia group, and in patients who were suspected for diagnosis. Patients who did not have inclusion criteria, those previously applied RUDAS, and thought to have severe neurocognitive deterioration preventing the patient to come to hospital several times were excluded. As one of the major limitations of the previous validation studies of RUDAS, we planned to exclude patients with minor neurocognitive disorder according to the psychiatrist’s view and the clinical assessment of the geriatrician, and to increase the reliability of the study further, analysis by NPT was applied in borderline patients.
While conducting the study, some patients did not continue every visit and were excluded from the study. After all cognitive assessments, 20 patients who were diagnosed with minor neurocognitive disorder were excluded. At the end, total number of the cohort was determined as 140 (70 major neurocognitive disorder/70 control). This sample size was calculated to have 80% in statistical power analysis.
Procedure
First, demographic features, immigration status, spoken languages, and educational levels were noted. Patients were assessed by geriatrician using geriatric assessment tools. Validated Turkish version of 15-item Geriatric Depression Scale (GDS) and MMSE (according to educational levels) were applied. 7,8 All patients were directed toward a psychiatrist for clinical interview and assessment by DSM-V criteria. To increase the reliability of the study, it was planned to apply NPT to all patients by a psychologist. However, while it is a long and detailed assessment, it could only be applied to 39 patients in the dementia cohort and 40 patients in the control cohort.
The neuropsychiatric tests were chosen from Bilnot battery, in which all tests were adapted and standardized for Turkish population. 9 This battery includes several subtests of Wechsler Memory Scale-Revised, Rey Auditory Verbal Learning Word List test, and Stroop Color-Word Interference Test. 10 –12 Also, 2 tests were added to this battery: clock drawing test assessing the visuoconstructive ability and executive function and the set test for verbal fluency assessment. 13,14
Rowland Universal Dementia Assessment Scale was translated into Turkish by a native speaker, and to increase fluency, accuracy, content suitability, and clarity of the translated text (Appendix), Turkish version of the RUDAS was analyzed by 1 neurologist, 1 psychiatrist, 1 internist, and 4 geriatricians in order to make necessary revisions. The RUDAS was applied by executive researcher who did not know clinical assessment results and MMSE scores of the patients. The same researcher applied the test second time in 2 days to determine test–retest reliability. For each patient, application time was noted.
The results of the RUDAS were compared to MMSE scores. Gold standard was DSM-V criteria, and approximately half of the patients’ results were supported by NPT. The Turkish version of the RUDAS was assessed for replicability, reliability, validity, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).
Reliability and Validity
Reliability was assessed as follows: Test–retest reliability: The reliability of the scale was tested in 140 participants suspected to have major neurocognitive disorder (70 patients) and normal neurocognitive functions (70 patients). The RUDAS questionnaire was administered, and the test–retest reliability of the Turkish version was examined in 2 days. Internal consistency reliability: The intraclass correlation (ICC) and Cronbach α were used to assess the internal consistency of the RUDAS-C. Item correlations were assessed both separately and totally. Additivity of the items were assessed to show reliability of total score.
Validity was assessed as follows: Dementia–control difference validity were assessed statistically. The time validity was determined by comparing the results of first and second test results. The parallel test validity was assessed via using MMSE as parallel test and correlation of the results was found. The construct validity was defined by factor analysis. The discrimination validity was determined using receiver operating characteristic (ROC) curve analysis. The content validity was examined by 1 psychiatrist, 1 neurologist, 1 internist, and 4 geriatricians who evaluated the translated text. The CVI, of which a value ≥80%, was used as the assessment standard.
15
The suggestions of the experts were used for last modifications of the questionnaire when necessary;
Sensitivity, Specificity, PPV, and NPV
The sensitivity, specificity, PPV, and NPV of the test were determined according to the DSM-V-TR, which serve as the gold standard references.
Ethical Considerations
This study was conducted in accordance with the Declaration of Helsinki. The ethics committee of Cerrahpasa Medical Faculty approved the study (409113/01-12.2015). Written informed consents of all patients were taken, which involve all necessary items, and additionally informed consent forms included a statement that “Information of yours will be published, but the identity will be kept as confidential.“
Statistical Analysis
Statistical analysis was performed using SPSS version 24.0 (IBM Corporation, Armonk, New York, United States). While the study data were assessed, besides the descriptive statistical methods (frequency, percent, mean, standard deviation), Kolmogorov-Smirnov deviation test was used for the assessment of normal deviation; independent samples t test was used for comparing between groups; and Pearson χ2 test and Fisher Exact test were used for comparing between groups of categoric data. When there are more than 2 groups, 1-way analysis of variance test was preferred for comparing quantitative data in comparison of between-group parameters.
Internal consistency (reliability) was assessed with Cronbach α and split half method; the ICC was used to support the reliability when there is a between-group comparison; item correlations were assessed both separately and totally with Pearson correlation analysis. Construct validity was analyzed with exploratory factor analysis and confirmatory factor analysis. To determine cutoff point between dementia and control group, ROC Curve analysis was used. The sensitivity, specificity, PPV, and NPV were determined according to cutoff values. To determine dementia predictive risk factors, logistic regression analysis was used. All results were analyzed in 95% confidence interval, and P < .05 was considered significant.
Results
Participant Characteristics
Among the 140 participants, 70 had normal cognitive functions and 70 had major neurocognitive disorder. The study cohort had 83 (59%) women and 57 (41%) men with mean (standard deviation) age of 79.68 (7.48) years. Regarding the participants’ educational status, 35 (25%), 46 (32.9%), and 59 (42.1%) patients had received no education, <8 years of education, and more than 8 years of education, respectively. Twelve percent of these participants had an immigrant status and a native language other than Turkish. Countries from which patients have immigrated were Greece (3 patients), Iraq (2 patients), and Bulgaria (2 patients) in the dementia group. These patients were speaking Greek, Kurdish, and Bulgarian as their native languages, respectively. Bulgaria (4 patients), Macedonia (2 patients), Georgia (1 patient), Iran (1 patient), Greece (1 patient), and Yugoslavia (1 patients) were the countries that the patients have immigrated from in the control group. Their language of choice was Bulgarian, Macedonian, Georgian, Persian, Greek, and Bosnian, respectively. While their mother tongues were different from Turkish as explained previously, all patients have had enough skills and understanding of Turkish language to complete the RUDAS. The baseline demographic and clinical characteristics of the patients are shown in Table 1.
The Baseline Demographic and Clinical Characteristics of the Patients.
Abbreviations: GDS, geriatric depression scale; MMSE, mini-mental state examination; SD, standard deviation.
Reliability
The Cronbach α value of the RUDAS was found to be .692. Internal Consistency of the scale is good. When reliability of the RUDAS was further analyzed with Split Half method; reliability was found to be 35.6% and 61.7% for first and second half of the test, respectively. The ICC of the test–retest was 0.987, and the correlation between the first and second assessment was 96.9%. Item–total correlations were all above 40%. Moreover, if 1 item was deleted, general Cronbach α of the scale was not higher and on the contrary, the value lowered. For this reason, all items were shown to contribute positively to the RUDAS and be consistent within the test. Finally, additivity of all items’ score was assessed, and nonadditivity was found as 0.222, which means the additivity was found statistically significant. Also, item correlations and factor analysis with all positive values were all supportive measures of additivity.
Validity
To determine validity of the RUDAS, dementia–control difference validity was first analyzed, and for all items, both cohort’ results were significantly different. First and second assessment results’ relation were 83.9% and 95.4% in control and dementia cohorts that proved time validity. To assess parallel test validity, MMSE was used, and the correlation of the RUDAS and MMSE was found as 45.3% and 73% in control and dementia cohorts, respectively (Figure 1). While both values were significant, correlation was higher in the dementia cohort.
Mini-Mental State Examination (MMSE) and Rowland Universal Dementia Assessment Scale (RUDAS) correlation in control and dementia cohorts.
Regarding the construct validity, explanatory and confirmatory factor analyses were used. Explanatory factor weights were between 0.57 and 0.82, and all were above 40%. Confirmatory factor weights were between 0.44 and 0.81, and all were above 40%. These analyses showed that the RUDAS has a strong factor construct with high factor weights and variance ratio, and 6-item model is appropriate for measurement.
Also, content validity was determined by content validity index (CVI). The RUDAS was analyzed by 7 specialists, and content validity was provided for all items. The content validity ratio of all items were 100%, and when CVI was assessed, it was found to be above 80%.
Sensitivity, Specificity, PPV, and NPV
To determine the cutoff point between control group and major neurocognitive disorder, the ROC curve analysis was used (Figure 2). Area under ROC curve was 98% (confidence interval: 94.1%-99.6%) and significant (P < .0001). When cutoff was set as 23, values were as follows: sensitivity, 78.6%; specificity, 98.6%; PPV, 98.2%; and NPV, 82.1%. When cutoff was set as 24, values were as follows: sensitivity, 87.1%; specificity 94.3%; PPV, 93.8%; and NPV, 88%. When cutoff was set as 25, values were as follows: sensitivity, 92.86%; specificity, 92.86%; PPV, 92.9%; and NPV, 92.9%.
ROC curve of Rowland Universal Dementia Assessment Scale (RUDAS).
Logistic regression analysis
Logistic regression model was used to determine predictive dementia risk factors. Questions 1, 4, 5, and 6 of RUDAS were found to be predictive of dementia; however, questions 2 and 3 were not statistically significant predictors of dementia. However, these item correlations were above 40%, and when these items were deleted from the scale, reliability was lowered. For this reason, these items were thought to contribute to accuracy of the RUDAS and found to be consistent within the test. Moreover, factor construct of these items were strong. Total score of the RUDAS was significantly higher in men when compared to women. While age and gender were predictors of dementia, RUDAS scores were not affected by all other sociodemographic variables such as immigrant status, native language used, and educational status.
Discussion
The 6-item RUDAS was designed in multicultural population, and it was validated. Original validation study was a cross-sectional study that included 90 patients (45 with dementia and 45 control) with mean age of 77.9 years and a median of 9 years of education. Also, 34.9% of the patients had a language preference other than English. 5 When compared to the criteria of the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV), the RUDAS was found to have a sensitivity of 89% (95% confidence interval [CI]: 76-96) and specificity of 98% (95% CI: 88-97). Gender (P = .18), years of education (P = .20), and preferred language (English or otherwise; P = .33) were not independent predictors of dementia. 5 However, MMSE scores are affected by age, educational status, ethnic identity, and language used. 3,4,16 It was proven that the scores under 23 from 30 are marking dementia, and when compared to MMSE, the RUDAS was shown to provide a more comprehensive cognitive assessment. 5 To our knowledge, it is the first validation study of the RUDAS in Turkish.
In original validation study, test–retest reliability (ICC = 0.98) was shown to be significant. 5 Chen et al published the study validating Chinese version of RUDAS in 2015 with test–retest reliability of (ICC = 0.90). 17 In our study, this value was in close relationship with the literature, and test–retest reliability was found to be (ICC = 0.987). Also, Salari et al showed test–retest correlation of the RUDAS 0.96 in Iran validation study. 18 Our result was highly similar to the literature with a test–retest correlation of .969.
In terms of time validity, the correlation between the results of first and second application of the RUDAS was 83.9% in the control group, 95.4% in the dementia group, and time validity was provided. It was thought that the reason for high correlation in control cohort is related to “learning” in this group.
Original validation study published by Storey et al did not provide a correlation between RUDAS and MMSE; however, in several studies, this correlation was between 40% and 80%. 18 –27 In our study, we found the correlation between RUDAS and MMSE as 45.3% in the control group and 73% in the dementia cohort. In the literature, Nielsen et al proposed the correlation between the RUDAS and the MMSE in nondemented Turkish immigrant population as 40%, which is correlated closely with our control group correlation. 26
To assess the internal consistency of the questionnaire, we found the Cronbach α value as .692 which is not pointed in original validation study; however, in studies that were published by Salary et al and Basic et al, this value was found as 0.80 and 0.74, respectively, which are similar to our result. 18,20 Regarding the ROC curve analysis, when 23 was taken as cutoff score, sensitivity (78,6%) and specificity (98,6%) were found similar to original validation study (sensitivity, 89% and specificity, 98%). 5 However, in this study, highest level of sensitivity and specificity was achieved when cutoff point was chosen as 25 (Supplemental Table). In accordance with the original validation study, items 2 and 3 of the test were not determining dementia significantly in our study; while these items were significantly contributing to the reliability of the test, it was decided not to exclude these items from the test in original validation study which is parallel to our result.
In the analysis that is looking for the effect of sociodemographic characteristics on the results of RUDAS, original validation study, in cohort of 90 patients, showed the relation between education level and RUDAS performance (P = .20) insignificant. 5 In several studies, it was shown that the level of education has no effect on the results of RUDAS. 19,20,26,28 One of these studies is noteworthy, and this study showed this result in nondemented Turkish population living in Denmark. 26
When we searched the literature to assess the influence of used language on the test results, it was found that there is significant relation between test results and used language in original validation study. 5 Also, in the study that was conducted by Gonçalves et al, there was no significant relation between the used language and the RUDAS scores (P = .30). 21
The results of the original validation study were emphasizing the accuracy of the results in populations coming from diverse background directed to assess the effect of immigrant status on the RUDAS scores. One study has searched the effect of immigrant status on cognitive assessment test results, and according to this study, RUDAS scores were not affected by migration (P = .08), while MMSE results were significantly affected (P = .02). 23 It is a result that is parallel to our study with no significant relation between test scores and immigrant status (P = .430).
We found that the relation between RUDAS scores and age of the patients was negatively correlated with a ratio of 32.7% (P = .006). In the original validation study, RUDAS scores were significantly related to age of the patients (P = .04). 5 Also, in another study of Nielsen et al, there was a significant relation between RUDAS results and patients’ age (P ≤ .001). 23
Our result supporting the significantly low score in female gender (P = .036) is contradictory to the literature. In original validation study, RUDAS–gender relation was insignificant (P = .18), and again Rowland et al showed that gender and RUDAS scores were independent determinants. 5,19 However, this may be the result of nonhomogenously distrubuted cohorts of these studies in both control and dementia groups.
Our study has some strengths; we conducted the study in homogeneously distributed groups both in dementia and in control branches, and the assessments were done on total score and question bases. Also, our study shows all validity analysis such as time, construct, content and parallel test validity, and the diagnosis were supported by DSM-V criteria. Moreover, the reliability analysis was done, and these analyses were conducted in a cohort with a power analysis of more than 80%.
However, there are some limitations of our study. First, it is necessary to determine the validity and cutoff points for patients with minor neurocognitive disorder in the future studies, while our study defined the border between major neurocognitive disorder and normal cognitive functions. Major limitation of our study is that only 56% of the patients were assessed by NPT. Although we used DSM-V criteria like many studies in the literature, this is our major limitation; while the border between major neurocognitive disorder and minor neurocognitive disorder could not be always accurately determined, all clinically suspected patients were assessed by NPT. Moreover, RUDAS was applied by 1 examiner to all patients who prevents to calculate interrater reliability to predict how scores differ between the 2 examiners. Also, dementia types were not stated in our study. This is a study from a tertiary center that might not totally reflect the cultural differences of Turkey. Another limitation is that there is no long-term follow-up of the patients to understand the power of the RUDAS to understand the changes in cognitive functions by the time.
Conclusion
As a result of increased migration trend, recent screening tests have become less reliable, and RUDAS was designed to decrease the need for an accurate screening test in diverse populations. Our study shows that Turkish version of the RUDAS can be used in diagnosis of patients with major neurocognitive disorder, and the test is valid by time, parallel test, construct, and content and reliable which can be applied approximately in 5 minutes. Also, education status, preferred language, and immigrational state have no effect on test scores. However, it is necessary to determine validity and reliability of the test in minor neurocognitive disorder, and borders between mild-, moderate-, severe-dementia populations should be defined.
Supplemental Material
Supplement_Table - Reliability and Validity of Rowland Universal Dementia Assessment Scale in Turkish Population
Supplement_Table for Reliability and Validity of Rowland Universal Dementia Assessment Scale in Turkish Population by Gizem Ayan, Ceyda Afacan, Burc Cagrı Poyraz, Onur Bilgic, Suna Avci, Hakan Yavuzer, Mehmet Yuruyen, Deniz Suna Erdincler, Burak Ayan, and Alper Doventas in American Journal of Alzheimer's Disease & Other Dementias
Footnotes
Acknowledgments
All authors contributed to writing this study and have approved the final version. Data sharing: corresponding author could provide underlying research materials related to the article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
