Abstract
The modified Telephone Interview for Cognitive Status (mTICS) is a commonly used screening tool for categorizing mental status of older adults. Recently, prediction equations have been developed to estimate performance on an in-person memory composite based on the mTICS; however, these equations need validation. The current study compared predicted memory functioning based on these equations with observed memory functioning in 101 community-dwelling older adults. Observed and predicted memory composites were comparable for 2 of 6 equations (mTICS total score and immediate recall item), indicating that these equations adequately predict observed memory scores. The predicted memory composite based on the total score was also most highly correlated with the observed memory composite. These results further validate the mTICS, as well as some of the prediction equations, and continue to point out this measure as an efficient tool for screening of cognitive functioning in later life.
Keywords
Introduction
Assessing the cognitive status of older adults can be challenging for both clinical providers and researchers. Declining health, limited transportation options, and possible cognitive decline can keep these individuals from utilizing clinical services. Conversely, this segment of the population is dramatically increasing, which will put a greater demand on health care systems. Furthermore, the time needed to adequately screen elderly samples for clinical research can tax limited resources. These clinical and research challenges may be partially addressed by utilizing a telephone screening measure to assess current cognitive functioning. The modified Telephone Interview of Cognitive Status (mTICS), which can be administered in approximately 10 minutes, could fill this important gap.
With growing international interest in brief and cost-effective methods for screening cognitive status of the elderly, multiple studies have already used the mTICS to determine cognitive diagnoses. 1,2 For example, 1 study identified optimal cutoff scores on the mTICS to separate participants with normal cognition from those with mild cognitive impairment (MCI) and dementia. 3 Other studies have continued to refine the mTICS’ ability to predict cognitive diagnoses. 4,5
The mTICS has less frequently been used to predict performance on neuropsychological test batteries. For example, in a study of nondemented older women, 6 associations were found between the mTICS and cognitive domains from an extensive neuropsychological battery. More recently, regression-based equations have been developed to predict performance on a memory composite using various mTICS scores in community-dwelling older adults. 7 Although these equations predicted a significant amount of the variance in the memory composite, these regression equations need to be independently validated before they should be more widely utilized.
Therefore, the purpose of the current study was to further validate the mTICS by examining the regression equations created by Duff et al and their ability to predict a memory composite in an independent sample of community-dwelling elders. We hypothesize that the mTICS total score would best predict the observed composite memory score. Validation studies, especially of regression models, are essential in demonstrating that the original algorithms can be applicable in independent samples, which strengthens confidence in these algorithms and perhaps expands their utility. 8
Methods
Participants and Procedures
A total of 101 community-dwelling older adults (aged 65 years and older) participated in this study. Most were female (75.2%), nearly all were Caucasian (99 Caucasian, 1 Hispanic, and 1 African American), their mean age was 75.0 (6.7) years, and their mean years of education were 15.2 (2.9).
The research protocol and all study procedures were approved by the University of Utah Institutional Review Board prior to study commencement. Recruitment involved local talks and advertisements in the community, at senior centers, and independent living facilities. Written informed consent was obtained from participants and monetary compensation was provided. Potential participants were screened over the telephone for dementia and cognitive impairment with the mTICS. Those scoring below 19 on this measure were excluded from further participation as this suggests frank dementia. 9 All other individuals were invited to participate in the study and were scheduled for an in-person visit as described below.
The in-person visit consisted of a brief clinical interview, the Geriatric Depression Scale (GDS), Mini Mental State Examination (MMSE), Wide Range Achievement Test-4 (WRAT-4) Reading subtest, and the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) Form A. Additional cognitive measures collected include 2 other measures of learning and memory: Brief Visuospatial Memory Test-Revised (BVMT-R) and Hopkins Verbal Learning Test-Revised (HVLT-R). Measures were administered and scored according to their respective test manuals. The clinical interview was used to collect relevant demographic information, presence of memory complaints, report of activities of daily living, and medical and psychiatric history. Participants’ self reports were corroborated via collateral sources (eg, family member, spouse, and close friend). Participants with significant history of major neurological (eg, dementia, traumatic brain injury, and stroke) or psychiatric illness (eg, bipolar disorder and schizophrenia), dementia (using the telephone and in-person cognitive screening battery), or current depression (30-item GDS >15) were excluded. All assessments were performed by trained research assistants.
Participants were classified into one of 2 groups based on the cognitive testing described above, MCI or intact, by one of the authors (K. Duff). Based on existing criteria, 10 participants were classified as MCI if they reported subjective memory complaints and demonstrated objective memory impairments (ie, age-corrected scores on at least 2 of the 3 delayed recall measures (RBANS Delayed Memory Index, HVLT-R, and BVMT-R) falling 1.5 standard deviations or more below a premorbid intellectual estimate (WRAT-4 Reading). This 1.5 standard deviations cutoff score is used in other MCI research. 11,12 Otherwise, cognition was generally intact (ie, age-corrected scores on other nonmemory measures above 1.5 standard deviations below average) and functional difficulties in activities of daily living were denied. Participants were classified as intact if memory and all other aspects of cognition were intact (ie, 3 delayed recall measures as well as other cognitive measures were comparable to premorbid intellectual estimate). These intact participants may have subjective memory complaints and they also denied functional difficulties. No one was classified as demented (ie, cognitive and functional impairments). Of the 101 participants, 38 were classified as MCI and 63 were classified as intact.
Measure of Interest
The mTICS is a 14-item measure of global cognition with a total score ranging from 0 to 50, with higher scores indicating better cognition. The mTICS is similar to other screening measures such as the MMSE, although it can be administered over the telephone. The mTICS assesses attention (eg, counting backwards, serial 7’s), orientation (eg, participant’s name, telephone number, month, date, year, season, and day of the week), language (eg, naming, repetition of phrases, identifying opposites), and learning and memory (eg, immediate and delayed recall of a 10-item word list). Since the mTICS has a high percentage of the total score devoted to memory (eg, 20 of the possible 50 points), this measure may be a useful tool for identifying cases with a primary amnestic profile (eg, amnestic MCI and early Alzheimer’s disease).
Data Analyses
An “observed memory composite” score was calculated for each participant as the average of the age-corrected z scores for RBANS Delayed Memory Index, BVMT-R Delayed Recall, and HVLT-R Delayed Recall. Using the prediction formulae presented in Duff et al, we calculated 6 separate “predicted memory composite” scores based on the mTICS: (1) total score, (2) language/attention factor score, (3) memory factor score, (4) immediate recall item score, (5) serial 7’s item score, and (6) delayed recall item score. The accuracy of the predicted memory composite scores was examined in 3 ways. First, the predicted memory composite scores were compared with the observed memory composite score with dependent t tests. Nonsignificant t tests would indicate that the predicted and observed memory composites were comparable and statistically significant t tests would indicate that the predicted and observed composites may be too different to be useful for clinical and research purposes. Second, Pearson correlations were calculated between the predicted and observed memory composite scores, with higher correlations suggesting better performance of the predicted score. Third, difference scores were calculated for each individual (ie, observed memory composite score—predicted memory composite score). Cumulative frequency distributions were tabulated for each difference score to see how closely these distributions matched normal distributions (ie, approximately 2% of cases should fall >2 standard deviations below the mean, approximately 14% of cases fall between 1 and 2 standard deviations below the mean, approximately 14% of cases fall between 1 and 2 standard deviations above the mean, and approximately 2% of cases should fall >2 standard deviations above the mean).
Results
For the entire sample, their mean mTICS total score was 36.4 (5.4), with a range of 20 to 48, and their mean observed memory composite score was z = −0.15 (1.01), which equates to approximately the 45th percentile. The various predicted memory composite scores based on the mTICS for the entire sample are presented in the Table 1. Although there were no differences between the intact and MCI participants in terms of age and education (P > .05), they were significantly different in terms of mTICS total score (t 99 = 3.1, P = .003; intact: M = 37.7 [4.8]; MCI: M = 34.4 [5.6]) and mean observed memory composite score (t99 = 7.4, P < .001; intact: z = 0.31 [0.67]; MCI: z = −0.93 [1.00]). The intact and MCI participants also differed on all predicted memory composite scores (all P < .05).
Observed and Predicted Memory Composite Scores and Their Correlations a
Abbreviations: mTICS, modified Telephone Interview for Cognitive Status; M, mean; SD, standard deviation; N/A, not applicable.
a All composite scores are z scores (M = 0.0, SD = 1.0). All correlations are statistically significant at P < .001.
For the entire sample, the dependent t test between the observed memory composite and predicted memory composite using the mTICS total score was nonsignificant (t 100 = −0.41, P = .68), suggesting that these 2 scores are comparable. This pattern, where the observed and predicted memory composites was comparable, was also seen for the mTICS immediate recall item (t 100 = −0.17, P = .87). The other 4 comparisons between observed and predicted memory composites were statistically significant (language/attention: t 100 = 6.86, P < .001; memory: t 100 = −3.56, P = .001; serial 7’s: t 100 = 5.48, P < .001; delayed recall: t100 = −4.90, P < .001), indicating that the predicted memory composite was not a good estimate of the observed memory composite in this sample.
In this sample of 101 participants, all 6 predicted memory composite scores significantly correlated with the observed memory composite score (all P < .001, Table 1). The distributions of different scores (ie, observed memory composite—predicted memory composite) are presented in Table 2. As can be seen in this table, the distributions based on the mTICS total score and immediate recall item appear to be the most normally distributed of the group. For example, relatively few (ie, 1% to 2%) large differences (ie, >2 standard deviations above/below the mean) between observed and predicted memory composites occurred for the predicted score based on the mTICS total score. This is in contrast to the difference score based on the mTICS delayed recall item, where there were relatively high numbers (ie, 6%) of extremely low scores (ie, >2 standard deviations below then mean).
Distribution of Difference Scores (ie, Observed Memory Composite—Predicted Memory Composite)
Abbreviation: SD, standard deviation.
Discussion
While several studies have used the mTICS to identify cutoff points for diagnostic classification in elderly samples (eg, dementia, MCI, and cognitively intact), few have explored the mTICS as a continuous variable and its ability to predict current cognitive functioning in normal and preclinical samples. Screening tools that can predict cognitive performances, rather than diagnostic categories, may be more useful in the early identification of cognitive dysfunction for clinical and research endeavors. The current study examined the regression equations created by Duff et al to see if they could predict current memory functioning in an independent, elderly community-dwelling sample.
Of the 6 prediction equations evaluated, 2 (total score and immediate recall item) appeared to be more strongly supported in the current analyses. Both of these predicted memory composite scores were not significantly different from the observed memory composite score, indicating the predicted and observed scores were equivalent. The predicted memory composite score based on the mTICS total score was most strongly correlated with the observed memory composite score of the 6 prediction equations examined. Finally, the different scores (ie, observed memory composite—predicted memory composite) for the total score and immediate recall item were the most normally distributed of the 6 prediction equations. Therefore, for those wanting to predict current cognitive functioning from a brief telephone interaction, the mTICS total score and the immediate recall item appear to be good proxies for in-person memory testing. These results augment the existing validation data for the mTICS 4,5,13 and have implications for the use of this measure. Although some may choose to administer the entire mTICS to get the total score, the current findings also suggest that administration of only the immediate recall item might provide as useful information about current memory functioning. Utilizing a single item (ie, immediate recall of the 10-item word list) could significantly shorten the administration time. Future studies might consider if this single item is as reliable as the mTICS total score, which could be an important factor in serial assessments of cognitive functioning.
Unfortunately, 4 of the 6 equations originally identified by Duff et al do not appear to be as useful for predicting memory scores. In each case, the predicted memory composite scores were significantly different from the observed memory composite score, indicating poor predictive value of the prediction equations. In 2 of these prediction equations (ie, language/attention factor, serial 7’s item), the predicted memory composite score was more than one-half a standard deviation below the observed memory composite. In the other 2 prediction equations (ie, memory factor and delayed recall item), the predicted memory composite was more than one-half a standard deviation above the observed memory composite. Although these 4 equations are not recommended for predicting current memory composites at this time, there may be some reason for their poorer performance. In 3 of the 4 nonpredictive equations, the raw scores on these mTICS’ items appeared skewed, which may have limited the possible range of predicted memory composite scores based on these equations. For example, both the language/attention factor and serial 7’s item had few low scores in our sample. This may suggest that these items are too easy for individuals with relatively intact cognition, but these items may be more predictive of functioning in a clinically impaired group. Conversely, scores on the delayed recall item of the mTICS yielded few high scores, suggesting that it may be too difficult, even for this intact to mildly impaired sample. Although it was discouraging that only 2 of the original 6 models were supported, this study highlights the value of validation studies, which are useful in refining scientific methods. 8
Several limitations of the current study should be noted that may affect its generalizability. For example, participants who received a total score on the mTICS of <19 were excluded from in-person memory testing, as this suggests frank dementia. Therefore, the current results might not be applicable to studies trying to identify demented participants. Additionally, examining a truncated range of mTICS scores might have limited its ability to show associations with the observed memory composite scores. The current sample, like that of Duff et al, was quite homogenous, as older, community-dwelling, highly educated, mostly female, and largely Caucasian. It is not clear whether the results of these prediction equations would be similar in a more diverse sample. Also like the original study, we excluded individuals with significant psychiatric symptoms (eg, GDS >15), and future studies could examine whether these symptoms affect the accuracy of these prediction equations. Finally, we did not screen participants for hearing impairments prior to administering the mTICS, and prior studies have noted that older individuals may perform more poorly on telephone-based measures due to these sensory deficits. 14
Despite these limitations, the mTICS continues to show promise as a screening tool in community-dwelling samples of older adults, who are either cognitively intact or mildly impaired. Using the prediction equations from the current study, especially based on the mTICS total score and the immediate recall item, reasonably accurate estimates of memory functioning can be obtained with limited staff resources and patient/participant burden. Although the mTICS may have clinical utility in the future, it seems most reasonable to use it as a research tool. Further research should be conducted to better understand the utility and limitations of this measure in clinical samples.
Footnotes
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Aging or the National Institutes of Health.
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: supported by research grants from the National Institutes on Aging: K23 AG028417 (K. Duff).