Abstract
The aim of this retrospective study was to investigate the accuracy of informant reports on cognitive status in mild cognitive impairment (MCI) by comparing the subjective evaluation made by patients’ relatives with the objective results of neuropsychological assessment. We enrolled 119 MCI outpatients and their relatives. Cognitive impairment was assessed by a battery of standardized neuropsychological tests. Informant reports on cognitive functioning were obtained by means of a structured interview. Subjective and objective evaluations of cognitive status were rated according to the same scoring system in order to enable comparison. All but one relative reported cognitive dysfunctions at the interview, but the kind of cognitive profile emerging from their reports was quite different from the one highlighted by neuropsychological assessment. A subjective evaluation of cognitive status based on informant reports could therefore be useful to identify patients with MCI but is unable to define MCI subtypes.
Introduction
Mild cognitive impairment (MCI) refers to the clinical condition of a person who suffers from slight cognitive deficits while largely preserving the ability to perform the activities of daily living. 1 The relevance of MCI is evident from epidemiological data: its annual incidence in elderly people ranges from 9.9 to 21.5 per thousand according to different authors, and its prevalence in population studies carried out in the United States and Europe varies from 11% to 17%. 2 Research has shown that patients with MCI have an increased risk of developing dementia, and MCI is considered a transitional stage along the cognitive continuum from normal aging to overt dementia. 3,4 The rate at which MCI progresses to dementia ranges from 5% to 10% per year in population-based studies and from 10% to 15% per year in clinic-based studies. 2 The clinical presentation of MCI is heterogeneous and at least 4 subtypes of MCI have been proposed in relation to the profile of cognitive deficits 5 : pure amnestic MCI (aMCI), multiple domains plus amnesia MCI (mdMCI+), multiple domains without amnesia MCI (mdMCI−), and single cognitive domain different from memory MCI (snmMCI). Evidence is also emerging that different MCI subtypes can convert to different types of dementia. 2,6
Although it has been suggested that biomarkers could be useful to characterize patients with MCI, 7 the diagnosis of MCI is still strongly grounded on clinical evaluation and extensive neuropsychological assessment. 8 Current diagnostic criteria for MCI 1,9 rely on both subjective and objective tools for the detection of cognitive impairment and there is a general consensus that a careful neuropsychological examination performed by a specialist lies at the core of an appropriate diagnostic workup for MCI. Yet, the high prevalence of MCI in the population 2 can make such an approach practically and economically inconvenient and lends appeal to the possibility of diagnosing MCI by means of a subjective evaluation of cognitive status based on reports from the patients’ relatives. The feasibility of this option is, however, a matter of controversy. A number of studies have shown that many patients with MCI do not complain of cognitive symptoms 10 –12 or have poor or no awareness of cognitive impairment. 13 –15 Indeed, some authors 10 –12 have suggested that the concept of subjective cognitive impairment should be excluded from the diagnostic criteria for MCI. Few studies have examined the validity and accuracy of informant reports on cognitive status in patients with MCI. Ready and coworkers 16 found a significant correlation between the informant-based “Short-Memory Questionnaire” and the scores of patients with MCI on the “word-list learning test” of the Consortium to Establish a Register for Alzheimer’s Disease. Isella and coworkers 17 demonstrated that the “Informant Questionnaire on Cognitive Decline in the Elderly” and the “Mini Mental State Examination” (MMSE) had a similar ability to differentiate patients with MCI from healthy controls. Another study 15 involving patients with mild-to-moderate dementia and MCI found a significant association between the communication subscale of the “Stroke Impact Scale” (SIS), as rated by caregivers, and MMSE scores but not between the memory subscale of the SIS and MMSE scores.
Even though the correlation between informant-based questionnaires and patient test results appears promising, it is worth noting that there are several limitations to these studies: small samples, 15,16 no division of patients into MCI subtypes, 15 –17 objective cognitive functioning assessed by a global test, 15,17 administration of a single neuropsychological test mainly exploring memory, 16,17 and pooled analysis of data from cognitively normal, MCI, and dementia participants. 15,16 Our aim was therefore to investigate the accuracy of reports on cognitive status provided by the relatives of patients with MCI in a study enrolling a relatively large sample of participants, employing a wide battery of neuropsychological tests and considering different MCI subtypes.
Materials and Methods
Sample
We first considered a sample of 150 outpatients consecutively diagnosed with MCI at the Geriatric Unit of the Ospedale Maggiore Policlinico in Milan from 2007 to 2010. Mild cognitive impairment was diagnosed according to current criteria 1,9 in patients without medical conditions potentially causing cognitive impairment such as neurological diseases (including stroke), psychiatric disorders, and alcohol abuse. From this first sample of patients, we excluded patients who attended the neuropsychological visit without an informant. The final sample was thus composed of 119 patients and their relatives.
The 119 MCI patients were 74 females and 45 males, aged from 61 to 88 (mean age: 77.38, standard deviation [SD]: 5.67), with 3 to 18 years of schooling (mean education: 9.27, SD: 4.69). The characteristics of the relatives were not investigated but they were mainly spouses and daughters. All patients and their relatives signed an informed consent prior to the neuropsychological visit, and the study was approved by the Ethical Committee of the hospital.
Measures
Neuropsychological Examination
In order to objectively evaluate cognitive functioning, all patients underwent a comprehensive neuropsychological examination. Global cognitive functioning was assessed by means of the MMSE, 18 and general intellectual functioning was investigated by using Raven’s colored progressive matrices. 19 For temporal orientation, the first item of the MMSE was considered. Anterograde long-term memory was rated with the prose recall test 20 and the delayed recall of the Rey–Osterreith complex figure test. 21 Verbal short-term memory was assessed by means of the forward digit span test. 22 The digit cancellation test 20 was administered to examine visual attention. Executive prefrontal functions were evaluated using the backward digit span test, 22 the trail-making test, 23 and the phonological fluency test. 24 Spatial skills were divided into spatial orientation, assessed by the second item of the MMSE, and spatial abilities, explored by means of the copy of geometrical figure test 20 and the copy of the Rey–Osterreith complex figure test. 21 Language was examined using the picture-naming test. 25 So as to be able to compare the objective evaluation of cognitive status obtained from the neuropsychological assessment of patients with the subjective evaluation stemming from the interview with their relatives, a scoring procedure based on a 3-point scale (0 points = no deficit; 1 point = mild deficit; 2 points = severe deficit) was adopted. Cognitive impairment was rated by taking normative data as reference: a deficit was classified as severe if the patient’s test score was in the lower 5% of the distribution, mild if it was in the 5%-10% range, and absent if it was in the upper 10%. Normative data for the digit cancellation test, prose recall test, and copy of geometrical figures test were taken from Spinnler and Tognoni. 20 For the delayed recall and copy of the Rey–Osterreith complex figure test, we used normative data by Caffarra and coworkers. 21 For the forward and backward digit span test, we adopted the standardization by Orsini and coworkers, 22 and for the trail-making test, the one by Giovagnoli and coworkers. 23 Normative data for Raven’s colored progressive matrices test were taken from Basso and coworkers, 26 for the phonological fluency test from Novelli and coworkers, 24 and for the picture naming test from Laiacona and coworkers. 25 Finally, we used the standardization by Magni and coworkers 27 for the MMSE. Retrograde memory and awareness of cognitive impairment were directly evaluated by the neuropsychologist during a preliminary conversation and scored on the same 3-point scale. According to the objective assessment of cognitive status based on the neuropsychological test battery, patients with MCI were divided into the 4 acknowledged subtypes of MCI: pure aMCI, mdMCI+, mdMCI−, and snmMCI.
Informant Report
In order to acquire a subjective evaluation of cognitive functioning, the relatives of patients were administered a structured interview that examined 8 cognitive domains (see Appendix A). In our geriatric unit, this interview is normally administered, before the neuropsychological assessment, to the relatives of patients with suspected dementia or MCI. For each cognitive domain, a single question was formulated to which the relative had to answer “yes” or “no,” supporting his answer with at least one description of the patient’s behavior. The neuropsychologist judged whether the descriptions were consistent with cognitive impairment and then scored it on a 3-point scale (0 points = no deficit, 1 point = mild deficit, or 2 point = severe deficit). If the descriptions given by relatives were not congruous with the cognitive domain investigated, it was the neuropsychologist who applied them to the correct cognitive domain. A subjective MCI subtype was defined for each patient according to the profile of cognitive impairment described by his or her relative at the interview.
Statistical Methods
The differences between mean scores obtained by each MCI subgroup at neuropsychological assessment were analyzed by means of the t test for independent samples corrected for multiple comparisons (Bonferroni correction). The comparison between informant-based scores and neuropsychological test scores for each cognitive domain was performed using the chi-squared test, after dichotomisation of the scores into normal (0 points) and pathological (1 or 2 points). When 2 or more neuropsychological tests were administered for a single cognitive domain, the worst performance was considered. However, in the analysis of spatial abilities, we chose to consider only the copy of geometrical figures test and not the copy of the Rey complex figure test because the latter is also highly sensitive to executive dysfunctions. A P value of <.05 was taken to indicate statistical significance. The accuracy of the relatives’ evaluation of specific cognitive functions was defined by taking neuropsychological tests as the standard and calculating sensitivity and specificity for each cognitive domain. Finally, by comparing the subjective MCI subtype stemming from the interview with relatives with the objective MCI subtype derived from neuropsychological assessment, we calculated the accuracy of informant reports in identifying different profiles of cognitive impairment.
Results
Neuropsychological Tests and Informant Reports
None of our patients were diagnosed with snmMCI. The demographic data and the scores of the neuropsychological tests for the overall MCI sample (n = 119) and for each MCI subtype are shown in Table 1 . As far as demographic data were concerned, significant differences emerged only for education and only between the aMCI group and both mdMCI+ and mdMCI− groups. However, we believe such differences do not affect the classification of MCI into subtypes.
Neuropsychological Assessment
Abbreviations: MCI, mild cognitive impairment; aMCI, pure amnestic MCI; mdMCI+, multiple domains plus amnesia MCI; mdMCI−, multiple domains without amnesia MCI; Tmt, trail making test; MMSE, Mini-Mental State Examination.
Table 2 illustrates the comparison between informant reports and neuropsychological assessment for each cognitive domain in the overall sample of patients with MCI. A single relative reported no cognitive impairment at the interview. The vast majority (91.6%) of relatives reported an impairment of episodic memory, mainly severe. A large prevalence (63.5%) of attention deficits also emerged. Less than half of the sample of relatives (46.2%) observed an impairment of executive functions. Deficits of retrograde memory and awareness were noticed by about 1 relative of 3 (30.3% and 28.6%, respectively). Impairments in language and spatial orientation were described by about one fifth (21.8%) of the sample. Finally, few relatives (11.8%) reported deficits in temporal orientation.
Subjective and Objective Evaluation of Cognitive Status
Significant differences in the prevalence of cognitive impairment highlighted by neuropsychological tests and informant reports were evident for 7 of the 8 cognitive domains examined (see Table 2). When compared to a formal cognitive evaluation carried out by a neuropsychologist, relatives detected more disorders of anterograde episodic memory (P < .0001), attention (P < .0001), and spatial abilities (P = .00014; P = .0019) and less disorders of retrograde memory (P = .0015), awareness (P = .015), executive functions (P < .0001), and temporal orientation (P < .0001).
Sensitivity/Specificity
The preceding analysis considered the mere reporting of cognitive deficits by relatives without taking into account its correctness. In order to address the issue of the accuracy of an informant-based appraisal of cognitive status, we compared the subjective evaluation provided by relatives against the standard of neuropsychological assessment. This was accomplished for each cognitive domain, both in the overall sample of patients with MCI and in the subsamples corresponding to the 3 actual MCI subtypes. Table 3 lists the percentages of correct evaluations (correct decisions and correct rejections) and incorrect evaluations (false alarms and omissions) supplied by relatives; from these data, sensitivity and specificity were calculated.
Relatives’ Reports on Cognitive Status: Sensitivity and Specificity for the Overall MCI Sample and for MCI Subtypes
Abbreviations: CD, correct decisions; CR, correct rejections; FA, false alarms; Om, omissions; Se, sensitivity; Sp, specificity; MCI, mild cognitive impairment; aMCI, pure amnestic MCI; mdMCI+, multiple domains plus amnesia MCI; mdMCI−, multiple domains without amnesia MCI.
The evaluation of episodic anterograde memory by relatives exhibited high sensitivity (0.98) but an extremely low specificity (0.16). Omissions were almost absent (0.8%) while false alarms were rather frequent (40.3%). Subjective evaluations had a greater likelihood of being correct for those MCI subtypes characterized by amnesia (aMCI and mdMCI+), whereas incorrect detections were very common (83.9% false alarms) for the MCI subtype lacking memory impairment (mdMCI−). Similar results were obtained for attention, with the informant report displaying high sensitivity (0.65), although at the price of low specificity (0.39). Once again, there were few omissions (15.13%) and many false alarms (34.45%), the latter especially in the MCI subtypes with impairment in multiple cognitive domains (mdMCI+ and mdMCI−). As far as the other cognitive functions were concerned, sensitivity was low (mean = 0.33, range = 0.17-0.50), and specificity was correspondingly high (mean = 0.8, range = 0.69-0.95). Informant reports on retrograde memory, executive functions, awareness, and temporal orientation showed a consistent percentage of omissions (30.2%, 44.5%, 26.9%, and 44.5% respectively) with lower percentages of false alarms (10.1%, 4.2%, 11.7%, and 2.5% respectively). An opposite pattern emerged for spatial abilities, spatial orientation, and language, with false alarms more common than omissions. However, it must be remarked that for these 3 cognitive domains even false alarms had a relatively low frequency (20.2%, 19.3%, and 18.1% respectively) since the rate of correct detections by relatives was generally high.
Subjective and Objective MCI Subtypes
In general, there was a significant difference between subjective evaluations and objective assessments of MCI subtypes (chi-square test P < .001). The reports of relatives were mildly sensitive to pure aMCI (0.44). Eight patients subjectively classified as aMCI were found to belong to another MCI subtype at neuropsychological assessment: 6 turned out to be mdMCI− and 2 were actually mdMCI+. With regard to the multiple domains plus amnesia MCI subtype (mdMCI+), informant reports showed high sensitivity (0.94), although at the expense of low specificity (0.29). There were few omissions (2.5%) but a relevant proportion of false alarms (38.6%). Indeed a large number of patients considered to be mdMCI+ on the basis of their relatives’ evaluation obtained a different diagnosis on neuropsychological assessment, either of mdMCI- (n = 41) or of aMCI (n = 5). When considering the mdMCI−, the evaluations provided by relatives had a very poor sensitivity (0.14) and a consequently high specificity (0.98). The percentage of omissions was very high: of the 56 patients objectively identified as mdMCI− by the neuropsychologist, only 8 had received the same subjective diagnosis. Overall, the informant-based evaluation of cognitive status led to a clear predominance of an mdMCI+ profile. In fact as many as 97 of the 119 patients considered (81.5%) were classified as mdMCI+ according to their relatives’ reports (Figure 1 ).
Objective and subjective mild cognitive impairment (MCI) subtypes assessed by neuropsychological testing and by informant reports.
Discussion
This study shows that the relatives of patients with MCI, interviewed in a clinical setting, were able to effectively report cognitive deficits even though these were mild. Our results are in keeping with the few previous studies investigating the validity and accuracy of informant reports on cognitive status in MCI, 15 –17 and taken together, these findings seem to suggest that informant reports could be used to screen for mild cognitive impairment. Such approach could have great practical value since the high prevalence of MCI in the population calls for a large number of diagnoses. Yet, the differences observed in our study between objective and subjective evaluations of cognitive profiles demonstrate that a careful neuropsychological examination is a mandatory second step for the identification of the impaired cognitive domains and for the characterization of MCI subtypes. In fact, the pattern of cognitive deficits emerging from the reports provided by relatives was quite different from the one outlined by neuropsychological assessment. Considering informant reports against the standard of neuropsychological testing, we noticed that relatives overrate episodic amnesia and attention deficits and underrate impairments in the other cognitive functions. Therefore, even though we adopted an interview designed to assess a range of cognitive functions and 3 objective MCI subtypes were actually present in the sample, from the relatives’ reports a predominant MCI profile, classified as mdMCI+, became evident. Similar results have been described for informant-based tools used in the assessment of dementia which seem to be able to gauge general cognitive functioning rather than specific cognitive domains. 28 Their high sensitivity to episodic amnesia and attention disorders probably enable relatives to detect even mild cognitive impairment, but at the same time makes their reports useless in categorizing MCI.
There are some limitations to our study. First, we considered relatives of patients recruited in a clinical setting; this could be a selected sample since relatives who seek professional attention for their family members may be particularly aware of cognitive problems. Population studies will therefore be necessary to confirm the ability of relatives to identify MCI and to verify the sensitivity and specificity of their evaluations.
Second, we investigated the accuracy of informant reports only for patients who on neuropsychological testing were diagnosed with MCI and not for those who were diagnosed as being cognitively normal or indeed affected by dementia. Considering data from all participants who received a neuropsychological evaluation would have added significance to the assessment of the specificity of informant reports.
Third, the aMCI subgroup included a limited number of patients. This finding is consistent with the geriatric setting of the study since it has been shown that in older community-dwelling participants, aMCI is significantly less prevalent than multiple cognitive domains MCI. 29
Fourth, the interview on cognitive status adopted in our study was not a validated instrument and its administration required appropriate training. Finally, subjective and objective information on cognitive status was acquired with different tools which could have different accuracy in detecting cognitive impairment; thus, the discrepancies between cognitive profiles stemming from neuropsychological assessment and from the reports of relatives may not be ascribable to the latter’s biased perception of cognitive deficits but only to a methodological issue. In fact, the items evaluated by the interview questionnaire are by their very nature complex and could involve a number of specific cognitive abilities. This means that a single pattern of cognitive impairment emerging from the interview may be decomposed into different combinations of cognitive deficits at neuropsychological testing. Although the interpretation of our results requires some caution, we believe that the differences observed could at least in part be explained by the incorrectness of the relatives’ evaluations. However, elaborate the procedure by which the data is collected may be, it is unquestionable that informant reports yield indirect measures of cognitive status, based on a subjective perspective. It is up to relatives to observe and report the behavior of patients with MCI but they are not scientists. Their observations are not planned, systematic, and complete. Their attitude is not neutral but influenced by mood, emotions, and feelings. It is therefore likely they would focus on some symptoms and ignore others. In particular, they would notice above all those cognitive problems that are more frequent, more evident at first glance, more prototypical, and that most affect them emotionally.
Footnotes
Appendix A
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The authors received no financial support for the research, authorship, and/or publication of this article.