Abstract
The ability to screen Turkish-speaking older adults for cognitive impairment by phone is lacking. The aim of this study was to translate the existing version of the telephone cognitive screen (T-CogS) into Turkish version (T-CogS-TR) and evaluate its reliability and validity in community-dwelling older adults. We prospectively recruited 104 community-dwelling participants with mild to moderate Alzheimer’s disease (AD) and healthy controls. The T-CogS-TR was administered twice via telephone at home, first within 3 days of an in-person administration and again 4 weeks later. We observed acceptable internal consistency (Cronbach α coefficient = 0.738) and internal reliability. The test–retest reliability was excellent. The T-CogS-TR demonstrated significant correlations with Instrumental Activities of Daily Living, Mini-Mental State Examination, Clock-Drawing Test, and Clinician Dementia Rating (P’s < .0001). The cutoff value of ≤22 exhibited sensitivity of 96.8%, specificity of 90.2%, positive predictive value of 93.9%, and negative predictive value of 94.9%. The T-CogS-TR can be useful as a valid and reliable tool for detecting AD in Turkish elderly patients. Also, this tool may be considered suitable for patients who need more frequent follow-up and cannot easily return to in-person visits.
Introduction
The growth in the number of patients with dementia due to increasing life expectancy will undoubtedly lead to essential health challenges in the future. Dementia, which is estimated to affect about 50 million people worldwide, is projected to triple by 2050. 1 Based on a meta-analysis of 18 studies from Europe, Alzheimer’s disease (AD) is the most common type of dementia, with a prevalence of 5.05% in people aged 65 years and older and the incidence of AD per 1000 person/year is 11.08, with higher rates in women than men. 2 Currently, the number of people living with dementia in Turkey is increasing, with a reported prevalence of 8.4% to 20% among older adults. 3,4 Similar to other countries, the burden of dementias in Turkish older adults is expected to expand in the near future. As there is still no effective treatment for AD, early diagnosis and slowing the disease progression are of more interest. Therefore, there is a need for validated, reliable, and easy-to-use cognitive tests for detecting the onset of AD to facilitate earlier access to appropriate care and support services.
Several well-known in-person instruments are available for screening of cognitive disorders in older people. 5 The Mini-Mental State Examination (MMSE) test published by Folstein and colleagues is one of the most commonly used tools to quantitatively evaluate global cognitive performance. 6 The MMSE is a short and useful test that was standardized in clinical and research settings and is available in many languages including Turkish. 7 In addition to being considered as a gold standard cognitive test for screening AD in older adults, the MMSE is useful to monitor the clinical course of AD and response to treatment. 8 Despite such advantages, the MMSE has some limitations, including the need for face-to-face administration and physical ability (sufficient visual and writing ability) to complete the test. 9 Moreover, barriers such as poor social support and elderly neglect can make it harder to access health-care facilities for MMSE testing. Mobility problems, such as living in rural areas and relocation, are among the frequent barriers to implementing follow-up clinical and research evaluations. 10 A telephone version of the MMSE could help both to eliminate these disadvantages and to enable epidemiological studies with larger sample sizes.
Several measures are currently available to assess cognition by phone. Brandt and colleagues (1988) developed a telephone interview for cognitive status (TICS) to screen cognitive impairment. 11 Later, a 22-point telephone version of the MMSE was developed as part of the Adult Lifestyles and Function Interview (ALFI-MMSE) and validated by Roccaforte et al. 12 Newkirk et al subsequently validated the telephone cognitive screen (T-CogS), which is a 26-point adaptation of the ALFI-MMSE. 10,12 Although these measures are available in many languages for use in different cultural and ethnic groups, there is currently no instrument to evaluate Turkish-speaking individuals. 9,13 -18 Among these telephone-based assessments of cognition, we chose to use T-CogS because the elements are very close to those in the MMSE. The purpose of the present study was to adapt the T-CogS into a Turkish version (T-CogS-TR) by testing its reliability and validity for use in older adults.
Methods
Study Design and Samples
Over 6 months, we prospectively recruited community-dwelling participants (aged ≥65 years) who visited the Geriatric Outpatient Clinic of the University of Health Sciences in Ankara, Turkey. Institutional review board approved the study protocol (2018/8-18/130), and a written, informed consent was obtained for each participant.
All patients were evaluated for AD through clinical history, neuropsychological tests, and brain imaging, if needed. The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition criteria for major neurocognitive disorder due to AD and the National Institute on Aging and the Alzheimer’s Association criteria were used to determine a diagnosis of AD. 19,20 Patients were included in the AD group if they received a diagnosis of AD and met the following criteria: a Clinician Dementia Rating (CDR) score of 1 or 2, a MMSE score between 10 and 23, and a Geriatric Depression Scale-15 (short form) score < 6. 6,21 The control group included older volunteers who had no cognitive impairment including mild cognitive impairment and dementia with a CDR score of 0, were living independently, and not taking any drugs affecting cognitive functions.
Exclusion criteria for patients with AD and controls were severe hearing or vision deficits that could impair performance for clinical testing and telephone assessment, other types of dementias, presence of delirium determined by the confusion assessment method, 22 being medically unstable, bedridden or in a wheelchair, and patients with incomplete data.
Sociodemographics (age, gender, marital status, years of education, and living status), comorbid conditions, assistive device use (hearing aid and eyeglass), and clinical data of both the patients and controls were collected for each participant at enrollment by face-to-face interviews.
Calculation of the Sample Size
To calculate the minimum required sample size, we used the recommended participant to item ratio of at least 5:1. 23,24 Since the T-CogS consisted of 16 questions, we calculated the minimum needed participants to be 16 × 5 = 80.
Index Test
Telephone Cognitive Screen
The T-CogS assessment, a 26-point adaptation of the ALFI-MMSE, was validated by Newkirk et al and can be used to estimate in-person MMSE scores of patients with AD. It can be used also for clinical reassessment of patients with mild to moderate dementia in cases where face-to-face interviews are not possible. 10 The T-CogS evaluates cognitive domains including orientation (year, season, date, day, month, state, county, city/town, address, and telephone number), registration and recall (3 objects; apple, table, penny), attention (spelling “WORLD” backwards), and language (repeating the phrase “no ifs, ands or buts” and single-naming object and 3-step command). The T-CogS does not contain the following items of the in-person MMSE: (a) “What floor of the building are we on?” (b) “Close your eyes”, (c) “Write a sentence”, and (d) “Copy overlapping pentagons.” Also, only one object-naming item exists in the T-CogS, whereas the in-person MMSE requires 2 separate object-naming items. The total score on the T-CogS ranges from 0 to 26 points, with higher scores showing better cognitive performance.
Assessment Tools
1. Cognitive status
(i) Mini-Mental State Examination
The MMSE is a validated tool for screening, diagnosis, and following the course of cognitive impairment among older adults. 6,7 This test provides an overall measure of cognitive domains including orientation, registration, attention and calculation, recall, and language. 6 Total summed scores on the MMSE range from 0 to 30, with higher scores indicating better cognitive performance. 6
(ii) Clock-Drawing Test
Clock-Drawing Test (CDT) is widely used for screening cognitive impairment among older adults and includes analysis of attention, auditory comprehension, perception, memory, executive function, visual-spatial organization. Several CDTs are present in the literature, each with a different scoring range. We chose the 6-point scale developed by Shulman et al, with lower scores indicating a higher cognitive performance. 25
(iii) Clinician Dementia Rating
Clinician Dementia Rating is a scale to evaluate global cognitive domains including memory, orientation, judgment and problem solving, community affairs, home and hobbies, and personal care. Staging dementia severity in clinical and research settings is based on a 5-point scale (none = 0, questionable = 0.5, mild dementia = 1, moderate dementia = 2, severe dementia = 3). 26
2. Functional status
(i) Instrumental Activities of Daily Living
The Lawton Instrumental Activities of Daily Living (IADL) scale is an instrument to assess competence in 8 skills including using telephone, shopping, preparing food, housekeeping, doing laundry, using transportation, handling medication, and finances, with a summed score ranging from 0 (lower functioning) to 8 (higher functioning). 27
At the initial outpatient visit, 1 trained neuropsychologist who was blinded to diagnosis of AD performed all in-person tests for cognitive and functional status. In the following step, the same observer administered the T-CogS-TR twice on the phone, first within 3 days following the hospital visit (T-CogS-TR_1) and the other 4 weeks later (T-CogS-TR_2).
Translation and Adaptation
We used the method developed by Beaton et al for the process of forward-translation and back-translation, and cross-cultural adaptation of the T-CogS. 28 We modified the sixth item (Can you tell me where you are right now? For instance, what state are you in?) to make it more suitable for Turkish society, because there is no state entity in our country. We used the word “district” instead of “state.” Then, we sent the final version to the authors of T-CogS to be checked for consistency or differences. Finally, we tested the T-CogS-TR on a sample of 15 community-dwelling persons aged 65 years and older.
Statistical Analysis
Statistical analyses were performed using SPSS software version 24.0 (SPSS Inc, Chicago, Illinois). Results were given as mean ± standard deviation, median and interquartile range, frequency, and percentages; χ2 test was used for comparisons between AD and control groups for descriptive statistics. Normality of distribution of data was tested using the Kolmogorov-Smirnov Test. Student t test was used to compare normally distributed variables and Mann-Whitney U test was used to compare non-normally distributed continuous variables. A P value of less than .05 was accepted as statistically significant.
We used the T-CogS-TR_1 scores of all participants in the assessment of reliability, validity, and measures of diagnostic accuracy of the test.
The reliability
We evaluated the reliability using internal consistency and test–retest design. Internal consistency was measured by using Cronbach α coefficient, which indicates how closely related were the questions within the T-CogS-TR. The acceptable value for Cronbach α is >0.70. 29 Corrected item-total correlations were determined to check if any item in the test was inconsistent with other items. Correlations above 0.20 were considered acceptable. Test–retest reliability (temporal stability) was measured using the intraclass correlation coefficients (ICC) to analyze test consistency over time. The ICC agreement values are classified as excellent (≥0.75), good (0.60-0.74), fair to moderate (0.40-0.59), and poor (<0.40). 30
The validity
We evaluated the validity using the criterion-related validity and convergent validity. Criterion-related validity measure was concurrent validity which was determined using Spearman correlation coefficients to compare the T-CogS-TR with the MMSE, CDT, and CDR, which are gold standard scales to assess cognitive functions. Convergent validity was evaluated using the Spearman ρ correlation coefficients to test whether the T-CogS-TR corresponded to IADL or not. Correlation coefficient measures are determined as strong (≥0.70), moderate (0.40-0.69), and weak (<0.40). 31
Measures of diagnostic accuracy
To define dementia, we used a cutoff score of ≤22 for the T-CogS, which corresponds to ≤23 on MMSE in the conversion table proposed by Newkirk et al. 6,10 Based on this score, we calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).
The test sensitivity (true positive rate) is the ability to identify the patients with AD correctly and computed with the formula: the number of AD patients diagnosed with the test/the number of patients with AD + the number of patients with AD who were not diagnosed with the test. 32
The test specificity (true negative rate) is the ability to identify the participants without cognitive impairment correctly and computed with the formula: the number of participants without cognitive impairment diagnosed with the test/the number of participants without cognitive impairment + the number of participants without cognitive impairment who were not diagnosed with the test. 32
Positive predictive value is the percentage of participants with a score ≤22 on the T-CogS who have a true AD diagnosis and computed with the formula: the number of patients with AD diagnosed with the test/the number of patients with AD + the number of participants misdiagnosed with AD with the test. 32
Negative predictive value is the percentage of participants with a score ≥ 23 on the T-CogS who do not have AD and computed with the formula: the number of participants without cognitive impairment diagnosed with the test/the number of participants without cognitive impairment + the number of participants misdiagnosed as cognitively healthy with the test. 32
Results
We enrolled 104 participants (63 patients with AD and 41 controls) who met the inclusion criteria in the study. Of all participants, 2 in AD group and 1 in the control group could not be assessed for the second T-CogS-TR: 2 patients were lost to follow-up and another developed delirium.
Table 1 shows the characteristics of the whole sample. The mean age was 78.8 ± 6.8 years, and 63.5% were female. Participants in the AD group were significantly older and had a significantly higher rate of hearing aid use (t = −4.891; P < .0001 and χ2 = 4.996; P = .025, respectively). Hypertension was more prevalent in the control group (χ2 = 6.713; P = .010). Participants in the AD group had significantly lower scores on MMSE, T-CogS-TR_1, T-CogS-TR_2, and IADL and higher scores on CDT than those in the control group (U = 32.000, U = 80.000, U = 54.000, U = 54.500, and U = 589.000, respectively; P’s < .0001). The other descriptive characteristics were similar between the 2 groups. Within the AD group, 73% had mild impairment and 27.0% had moderate impairment (Table 1).
Characteristics of the Study Population.a
Abbreviations: AD, Alzheimer’s disease; CDR, Clinician Dementia Rating; CDT, Clock-Drawing Test; IADL, Instrumental Activities of Daily Living; MMSE, Mini-Mental State Examination; N/A, not available; SD, standard deviation; T-CogS, telephone cognitive screen; U = Mann-Whitney U test value; t = Student t test value; χ2 = Chi-square test value.
a Results are mean ± SD or percentage or median (first quartile-third quartile). Values given in bold indicate statistically significant results (P < .05).
b Student t test.
c Mann Whitney U test.
d Versus others: unmarried, widowed, or divorced.
e Versus others: alone, relative, caregiver, or nursing home.
f Sixty-one patients with AD versus 40 controls.
g Sixty patients with AD versus 41 controls.
The mean T-CogS-TR_1 and T-CogS-TR_2 scores were 18.5 ± 5.7 and 18.6 ± 6.0, respectively.
The reliability
The internal consistency was acceptable with the overall Cronbach α coefficient of 0.763, ranging from 0.620 to 0.798 for subscales (Table 2). The ICC value was 0.990 (95% confidence interval: 0.985-0.993), indicating an excellent test–retest reliability (Table 2). Corrected-item total correlations ranged from 0.388 to 0.777 and were all above 0.20, showing acceptable internal reliability of this test (Table 2).
The Psychometric Properties of the T-CogS-TR.
Abbreviations: CI, confidence interval; T-CogS-TR, Turkish version of telephone cognitive screen.
The validity
Convergent validity
The total score of the T-CogS-TR showed a positive correlation with the IADL score (r = .847, P < .0001; Table 3). The subscale scores of the T-CogS-TR, including orientation, registration, attention, recall, and language, were also significantly positively correlated with the IADL scale score (r = 0.794, P < .0001; r = .310, P = .001; r = 0.740, P < .0001; r = 0.693, P < .0001; r = 0.609, P < .0001, respectively).
Correlations of the Turkish Version of the T-CogS With Cognitive and Functional Assessment Tools.a
Abbreviations: CDR, Clinician Dementia Rating; CDT, Clock-Drawing Test; IADL, Instrumental Activities of Daily Living; MMSE, Mini-Mental State Examination; T-CogS, telephone cognitive screen.
a Values given in bold indicate statistically significant results (P < .05).
Criterion-related validity
The total score of the T-CogS-TR was significantly correlated with the MMSE, CDT, and CDR scores (r = 0.923, P < .0001; r = −0.711, P < .0001; r = 0.880, P < .0001, respectively; Table 3). When the same subscale scores of both the T-CogS-TR and MMSE were compared, there were significant correlations in cognitive domains of orientation, attention, recall, and language (r = 0.866, P < .0001; r = 0.733, P < .0001; r = 0.756, P < .0001; r = 0.303, P = .002, respectively), while not in registration (r = –0.045, P = .649).
Sensitivity, specificity, PPV, and NPV
Overall, sensitivity, specificity, PPV, and NPV were 96.8%, 90.2%, 93.9%, and 94.9%, respectively, showing high diagnostic accuracy for AD (Table 4).
Sensitivity, Specificity, and Predictive Values of the Turkish Version of the T-CogS.
Abbreviation: T-CogS, telephone cognitive screen.
Discussion
In this study, we evaluated the reliability and validity of the T-CogS-TR in a sample of Turkish community-dwelling older adults. The T-CogS-TR had satisfactory levels of validity, internal reliability, and temporal stability, with excellent agreements. We also observed that the T-CogS-TR was reliable in terms of sensitivity/specificity, PPV, and NPV values in order to differentiate patients with AD from healthy individuals. Thus, the T-CogS-TR can serve as a screening instrument to distinguish patients in need of further evaluation.
The Cronbach α coefficient of the T-CogS-TR in the present study indicates an acceptable level of internal consistency, which answers the question to what extent the items of this tool would be related to each other. Despite an interest in utilizing this tool, there is little evidence in the literature of the reliability of the T-CogS. 10,33,34 Thus, additional reliability studies on older adults, with a larger sample, are warranted to strengthen our results. Furthermore, the corrected item-total correlations were above the critical cutoff point of 0.2, suggesting that the items related adequately to the total test scores and each item shared the same concept of meaning in the T-CogS-TR. 35
We observed that 2 tests of the T-CogS-TR administered 4 weeks apart demonstrated consistent results. Among all subscales, 4 (orientation, attention, recall, and language) showed excellent test–retest reliability, whereas the ICC value of registration was lower (Table 2). Unfortunately, limited data are available about the temporal stability of the T-CogS to compare with our findings. 10,33,34 With regard to comparison of in-person and phone subscales, Newkirk et al also found that subscores of registration were higher on the in-person MMSE than telephone interview, likely due to misregistration over phone. 10 Of note, a recent report showed a bidirectional association between worsening cognitive performance and anxiety symptoms in older adults, which was independent of gender, physical status, and depression. 36 Additionally, age-related changes in hearing seem to influence cognitive activities negatively. 37 Thus, lower performance of the test in the registration domain of cognition in our study may be attributed to hearing difficulties and emotional distress during a telephone interview which may impair the communication between the observer and the respondent. Based on these results, a clinician should be more cautious in this domain when performing the T-CogS-TR and should make sure that the task is articulated clearly.
The T-CogS-TR showed a strong convergent validity when compared with the IADL test for the total score, which is consistent with the findings of a previous study. 38 Garre-Olmo et al demonstrated high convergent validity of a Spanish adaptation of a telephone-based MMSE for the total MMSE score and moderate to good correlations for all subscores. 17 The T-CogS-TR also achieved strong level of the criterion-related validity examined with the MMSE, CDT, and CDR scores. In the original version of the T-CogS, Newkirk et al found a high correlation (r = 0.88) with the MMSE for the total score and moderate correlations for all subscores. 10 According to the results of a longitudinal study with 419 community-dwelling African American and non-Hispanic white adults aged 75 years and older, the T-CogS had good to excellent agreement with the MMSE, but 1 item in the concentration and 2 items in delayed recall domains showed fair to poor correlations. 38
We determined that a cutoff ≤ 22 points on the T-CogS-TR to identify patients with AD had excellent sensitivity, specificity, PPV, and NPV. However, the measures of diagnostic accuracy of the T-CogS in patients with AD are unknown. Several studies have evaluated the sensitivity and specificity of other telephone-based cognitive screening tools. In a survey of 100 older adults, the ALFI-MMSE demonstrated a sensitivity of 67% and a specificity of 100% relative to a brief neuropsychological screening test. 12 The sensitivity and specificity of a self-report interview referred to as the TELE, a telephone assessment for dementia, for discrimination of patients with AD was 90.0% and 88.5%, respectively. 39 The TICS revealed a sensitivity of 94%, a specificity of 100%, a PPV of 100%, and an NPV of 97% in a sample of patients with AD and controls. 11 Thus, our study is consistent with the findings of others and suggests that the T-CogS-TR has adequacy and utility for detecting AD in older people. The T-CogS-TR could be useful to clinicians to distinguish patients with AD from healthy individuals and to identify individuals who are appropriate for further diagnostic evaluation.
Some limitations should be considered when interpreting the results of this study. First, since we enrolled community-dwelling patients with AD, the results could not be generalized to homebound and institutionalized older adults or other less common forms of dementia. Second, the research was conducted in a single center. Hence, multicenter studies are required to verify and extend the findings in other regions of Turkey. Lastly, the T-CogS-TR, like other screening tests by telephone, is not adapted to detect impairment in cognitive domains involving praxis and visual-spatial capacity.
In conclusion, due to the increasing prevalence of AD, health-care providers will need novel tools to assess cognitive impairment and dementia, particularly for patients who do not have easy access to a clinical setting. The results of the present study showed that the T-CogS-TR is a valid and reliable tool for cognitive assessment in community-dwelling older adults. The T-CogS-TR may be able to identify patients who are in need of additional evaluation to determine whether or not they have AD. Additionally, the T-CogS-R can help estimate the level of cognitive impairment in individuals who have already been diagnosed with AD and assist in providing optimal remote care of these patients.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.