Abstract
To investigate the utility of the Severe Impairment Rating Scale (SIRS) as a cognitive assessment tool among nursing home residents with advanced dementia, we conducted a cross-sectional study of 96 residents in 3 nursing homes with Functional Assessment Staging Test (FAST) stage 6a and above. We compared the discriminatory ability of SIRS with the Chinese version of Mini-Mental State Examination, Abbreviated Mental Test, and Clock Drawing Test. Among the cognitive tests, SIRS showed the least “floor” effect and had the best capacity to distinguish very severe (FAST stages 7d-f) dementia (area under the curve 0.80 vs 0.46-0.76 for the other tools). The SIRS had the best correlation with FAST staging (r = −.59, P < .01) and, unlike the other 3 tools, exhibited only minimal change in correlation when adjusted for education and ethnicity. Our results support the utility of SIRS as a brief cognitive assessment tool for advanced dementia in the nursing home setting.
Introduction
Although deaths from other major diseases in the United States decreased between 2000 and 2010, deaths from dementia actually increased 68% during this period. 1 Because dementia typically has a fairly well-defined clinical course of inexorable progression, it is projected that the number of persons with advanced dementia will increase in the coming years. Advanced dementia is characterized by severe memory and language impairment, immobility, inability to self-care, and being bowel and bladder incontinent. 2 It is estimated that almost 9 of 10 institutionalized patients with Alzheimer’s disease are in the moderate or severe stages of dementia, 3 and many of them will eventually pass away in these settings too. 4,5
Due to the lack of curative treatments for dementia, the progressive course of this disease, and the projected increased number of institutionalized patients with advanced dementia, there is an urgent need to develop instruments that can adequately assess the residual cognitive abilities of nursing home residents with advanced dementia. The palliative care literature has indicated that care of patients with advanced dementia has been found to be suboptimal across the spectrum of care settings ranging from nursing homes, hospitals, and in the community. 4 -8 One key reason cited for this finding is the difficulty in assessing patients at the advanced stage of dementia. 6,9 Identifying and stratifying the different stages of advanced dementia will assist management of attendant issues such as pain, malnutrition, dysphagia, and behavioral issues 10 -14 and also better plan for future care in these various settings. This is in line with the wider movement toward better end-of-life care in this vulnerable population. 4,15 Not uncommonly, elderly persons with advanced dementia develop delirium with changes in cognitive status as a result of medications, pain, constipation, or acute illnesses. 16 Bedside cognitive instruments that are appropriate for use in nursing home settings would thus be useful for detecting cognitive changes and to document subsequent improvement with resolution of delirium following institution of appropriate treatment.
Although performance-based measures of cognition such as the Severe Impairment Battery (SIB) 17,18 and the Severe Cognitive Impairment Profile 19 have been specifically developed for advanced dementia, their utility in the nursing home setting is limited by the considerable amount of time, specialized training, and equipment required for administration (Table 1). The Test for Severe Impairment was also developed for cognitive assessment in severe dementia; however, a floor effect remains detectable in the severe to terminal stages. 21 More recently, briefer instruments have been developed for use in the clinical setting, such as the Severe Mini-Mental State Examination (MMSE) 22 and the Severe Cognitive Impairment Rating Scale. 25 These tests were typically validated in patients with less severe stages of dementia who were recruited from memory clinics or inpatient settings, and it is unclear how they will perform in the nursing home population and with more advanced stages of dementia corresponding to Functional Assessment Staging Test (FAST) 7 and beyond.
Comparison of the Various Cognition Tests Available for Severe Dementia.
aFindings from this study.
In this regard, the Severe Impairment Rating Scale (SIRS) confers several advantages. The SIRS is a cognition assessment tool developed by Peter Rabins and Cynthia Steele in 1996 to assess residual cognitive ability in patients with advanced dementia. 27 Being brief and simple to understand, SIRS is amenable for use in more severely impaired patients with short attention span and limitation in communication. It is also easily and quickly administered and scored. Because SIRS assesses ability in performing familiar functional tasks (eg, saying hello and good-bye) via simple verbal commands accompanied by gestures and observations of patient, it is potentially applicable across different cultures and languages. 27 In a preliminary study among a small population (n = 40) of largely nursing home residents with Clinical Dementia Rating score of 3, SIRS did not yield a significant floor effect. 27 It has previously demonstrated good interrater reliability (r = .992), test–retest reliability (r = .97), validity as measured by the good correlation with Glasgow Coma Scale (r = .866) and MMSE (r = .77), and internal consistency (Cronbach’s α = 0.76). 27
The SIRS was used to assess cognitive function in 2 earlier studies involving nursing home residents with advanced dementia in the United States. 12,28 Kverno et al examined the prevalence and treatment of neuropsychatric symptoms among hospice-eligible nursing home residents with advanced dementia, 12 while Hicks et al studied predictors of mortality in nursing home residents with advanced dementia. 28 However, its validity in the more advanced stages of dementia among the nursing home population outside the United States remains to be established, and it is also unclear whether SIRS would be less susceptible to the effects of confounders such as age, gender, education, and race vis-à-vis other cognitive assessment tools. 29,30
This provided the impetus for the current study, which aims to compare the clinical utility of SIRS versus 3 validated and commonly used cognitive assessment tools, namely, the Abbreviated Mental Test (AMT), 31 Chinese MMSE (CMMSE), 32 and Clock Drawing Test (CDT), 33,34 among nursing home residents with advanced dementia (FAST 6a and above). We compared the presence of the floor effect across FAST stages; discriminatory ability to detect very severe dementia, as defined by FAST stage 7d and beyond; correlation with other cognitive assessment tools, functional status and depression; and the impact of age, gender, education, and race on correlation with FAST staging.
Methods
Participants and Setting
We recruited participants residing in 3 nursing homes in Singapore, namely, Ren Ci Nursing Home, Thong Teck Home for Senior Citizens and Ling Kwang Home for Senior Citizens, from February 2010 to June 2011. We included residents aged more than 50 years old with a diagnosis of dementia according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), advanced stage of dementia defined as FAST stage 6a and above, medically stable, and who had been staying in the nursing home facility for at least 2 weeks prior to the assessment. We excluded the following: (1) previous diagnosis of any psychiatric illness besides dementia (eg, schizophrenia, psychosis, and mania); (2) active neuropsychiatric symptoms (eg, aggression, agitation, and depression) that may affect study assessment; and (3) absence of next of kin or other legally acceptable representatives who can provide consent for study participation. Informed written consent was obtained from their legally acceptable representatives. The study was approved by the Domain Specific Institutional Review Board of the National Healthcare Group, Singapore.
Clinical and Functional Evaluation
Baseline demographic data on participants’ age, gender, race, and education level were collected. A single clinician (CY), blinded to the cognitive screening test results, reviewed the participant’s medical record and interviewed the nurse in charge in order to ascertain the comorbid diagnoses, functioning in daily activities, and cognitive functioning. Functional status of resident was assessed using Barthel activities of daily living, and depression was screened using the 15-item Geriatric Depression Scale. 35 Diagnosis of dementia was made based on DSM-IV criteria via a validated semistructured approach. 36
The FAST staging is a functional score staging system designed to evaluate patients at different stages of dementia. There are 7 stages of FAST staging, ranging from 1 for normal adult, 2 for subjective cognitive impairment, 3 for mild cognitive impairment, 4 for mild dementia, 5 for moderate dementia, 6 for moderately severe dementia, and 7 for severe dementia. 37 Stages 6 and 7 are further divided into substages, with stages 6a to 6e denoting moderately severe dementia, stages 7a to 7c denoting severe dementia, and stages 7d to 7f representing very severe substages. 37 Validation studies indicate that FAST is a reliable and valid assessment technique for evaluating functional deterioration in patients with Alzheimer’s dementia throughout the entire course of the illness. 38 Moreover, FAST elucidates a characteristic pattern of progressive, ordinal, and functional decline in Alzheimer’s dementia. 38
Cognitive Assessment Instruments
Trained research assistants, who are blinded to the diagnoses and FAST staging, administered the cognitive evaluation instruments. To ensure optimal performance in cognitive assessment, care was taken to ensure that a commensurate language (usually English or Mandarin) was used; in addition, a communicator (portable sound amplifier) was used to correct for hearing difficulties if it was noted either by nursing home staff or at bedside assessment by the research assistants that the participant had difficulty in hearing.
Severe Impairment Rating Scale
The SIRS consists of 14 items representing 3 cognitive areas, memory, language, and motor. Each item on the SIRS is scored as 2 points if fully correct, 1 point if partially (as specified) correct, or 0 point if incorrect or not performed (Table 2). Credit is given for nonverbal and partially correct response which allows for a finer assessment of cognitive function in this highly impaired group. Items 1, 2, 10, and 11 assess the participant’s overlearned memory as opposed to episodic memory that is commonly tested in cognitive tools such as the CMMSE. Overlearned memory is known to be preserved until the advanced stage of Alzheimer’s dementia. 39 Items 4, 6, 8, 9, and 13 evaluate the participant’s motor function, and items 3, 5, 7, 12, and 14 assess language function.
Severe Impairment Rating Scale Test Items.a
Abbreviation: CNS, central nervous system.
aReproduced with permission from Prof P. V. Rabins.
Abbreviated Mental Test
The AMT is a 10-item cognitive screening test that was originally designed by Hodkinson in the United Kingdom. 31 The item on year of Second World War was substituted by an enquiry of the participant’s home address to adapt to the local context. This modified version was validated locally where a score of 7 and below suggests cognitive impairment in the community-dwelling Singapore elderly population, with a higher cutoff value recommended for the higher educated participants. 29
Chinese version of MMSE
The CMMSE was modified from the original MMSE to yield a total score of 28 instead of 30. 32 Modifications included exclusion of the question “What is the season,” and combining the 2 questions “In which town are we” and “In which county/district are we” into a single question “In which estate are we?” 29 A score of below 21 suggests cognitive impairment, with higher cutoff value recommended for the more educated elderly participants. 29
Clock Drawing Test
Participants were asked to put the numbers of the clock on a predrawn circle (8 cm in diameter). The CDT test was scored using Watson’s method which divides the clock into 4 equal quadrants and each quadrant should contain 3 digits. A point was given for any errors in the first 3 quadrants, whereas error in the last quadrant was given 4 points. 34 A maximum error score of 7 can be obtained, with higher scores indicating greater impairment. The normal range for the score is 0 to 3. A score of 4 or greater in this scoring system has a sensitivity of 87%, a specificity of 82%, and a K value of 0.70 for identifying dementia. 34 This method of scoring has the advantage of simplicity of scoring and good discriminatory performance in terms of sensitivity and specificity. 34
Statistical Analysis
Statistical analyses were performed using SPSS for Windows Version 17.0 (SPSS Inc., Chicago, Illinois). All tests were 2-sided with the level of significance set at P < .05. Descriptive data are presented as means (±standard deviation) for quantitative variables and as relative frequencies for categorical variables.
The presence of floor effect was determined by comparison of the mean score of each cognition tool across 3 severity groups (moderately severe at FAST 6a-e, severe at FAST 7a-c, and very severe at FAST 7d-f). We performed univariate analyses comparing demographic characteristics, cognitive performance, physical function, and mood across the 3 severity groups using the nonparametric Kruskal-Wallis test for continuous variables and chi-square test for categorical variables. Quartile subcategories (ie, 25th, 50th, 75th, and 100th percentile cutoffs) were empirically derived for CMMSE scores; previous studies comparing cognitive tools have also employed similar empirical methods of deriving cutoffs. 25 We performed a boxplot of SIRS scores stratified by CMMSE quartile subcategories (corresponding to scores of 0, 1-4, 5-9, and >9). Receiver–operating characteristic (ROC) curves were derived to evaluate the ability of the various cognition tools to detect severe dementia (defined by FAST stages 7a and beyond) by determining the area under the curve (AUC).
We assessed convergent validity by measuring the correlation with other cognitive instruments and with functional status and assessed divergent validity by measuring the correlation with depression scores. Correlational analysis was performed using the nonparametric Spearman’s correlation test. Unadjusted and adjusted correlational analyses between test scores and FAST staging were done using Spearman’s correlation to determine the influence of age, education, and gender on the validity of test scores. For this analysis, FAST stages were recoded as an incremental scale, beginning with stage 6a corresponding to a value of 1, through the intermediate stages 6b-7e corresponding to the values of 2 to 10, and ending with stage 7f with a recoded value of 11.
Results
Baseline Characteristics
One hundred participants fulfilled inclusion and exclusion criteria. We excluded 4 participants whose legally represented guardian withdrew consent or who became medically unstable before data collection was collected (Table 3). In our final sample of 96 participants, the mean age was 81.7 ±10.4 years, and the female gender was predominantly represented (67.7%). Mean education level was 1.3 ± 2.5 years; 34.3% of the participants were in the moderately severe stage of dementia (defined by FAST stages 6a-e); 44.7% were in the severe stage at FAST 7a-c; and 20.8% at the very severe stage of FAST 7d-f. Participants with more advanced dementia showed a trend toward younger age (mean age by FAST stages: 76.6 ± 13.5 vs 84.2 ± 7.8 and 82.2 ± 10, P = .11). There was no significant difference in educational attainment and Geriatric Depression Scale (GDS) scores among the 3 groups. Expectedly, participants diagnosed with more severe dementia performed significantly worse in cognitive tests and were more functionally impaired compared with the moderately severe and severe groups.
Baseline Demographics and Comparison of Cognitive Scores, Function, and Depression Screening of the Overall Study Sample and Between the Different FAST-Staging Groups.a
Abbreviations: AMT, Abbreviated Mental Test; CDT Clock drawing test scored by Watson method; CMMSE, Chinese Mini-Mental State Examination; FAST, Functional Assessment Scoring Tool; GDS, Geriatric Depression Scale; SIRS, Severe Impairment Rating Scale; SD, standard deviation.
aValues in terms of mean ± SD unless otherwise indicated.
bP values are based on χ2 test for binary datad and Kruskal-Wallis analysis for continuous datac.
Floor Effect
The floor effect was apparent for AMT, CDT, and CMMSE, especially in the very severe dementia subgroup, where mean scores approximate the end limit of range of scores for the respective tools (Table 3). For SIRS, however, there is still a good range of scores with worsening severity of dementia, suggesting good discriminatory ability to measure residual cognitive function in these participants.
The lack of floor effect of SIRS relative to CMMSE was clearly evident when SIRS scores were plotted against the CMMSE quartile subgroups (Figure 1). At CMMSE score of 0, there is still a demonstrable range of SIRS score from 0 to 24 points. Two participants bottomed out (ie, scored 0) on the SIRS evaluation, and these 2 individuals also scored 0 for CMMSE. Using Kruskal-Wallis analysis, there was a significant difference between median SIRS scores across the CMMSE subgroups, H(2) = 60.5, P < .001, with a mean rank of 17.0 for CMMSE subgroup of 0 point, 48.1 for CMMSE subgroup of 1 to 4 points, 66.2 for CMMSE subgroup of 5 to 8 points, and 70.3 for CMMSE subgroup of 9 points and more.
Box-plot of SIRS scores stratified by CMMSE quartile scores. CMMSE indicates Chinese Mini-Mental State Examination; SIRS, Severe Impairment Rating Scale.
Discriminatory Ability in Severe Dementia
Using ROC analysis, SIRS demonstrated the best capability to distinguish severely disabled patients of FAST stages 7a and beyond, with AUC of 0.80 when compared to CMMSE, AMT, and CDT (AUCs: 0.77, 0.62, and 0.54 respectively; Figure 2).
ROC curves comparing discriminatory ability of cognition tools in severe dementia (FAST stage 7a and above). Areas under the curve for SIRS 0.80, CMMSE 0.77, AMT 0.62, and Clock Drawing Test (scoring by Watson method) 0.54, respectively. AMT indicates Abbreviated Mental Test; Watson, Clock Drawing Test scoring by Watson method; CMMSE, Chinese Mini-Mental State Examination; SIRS, Severe Impairment Rating Scale.
Interfactor Correlations
The SIRS was well correlated with AMT and CMMSE (ρ = 0.58 and 0.77 respectively, P < .01) but weakly correlated with CDT (ρ = −0.28, P < .01; Table 4). There was good correlation with both Barthel BADL (ρ = 0.70, P < .01) and FAST (ρ = −0.59, P < .01). The good correlation with other cognitive tools and with functional scales supports the convergent validity of SIRS. Conversely, the poor correlation with GDS score (ρ = 0.16, P = .44) corroborates the divergent validity of SIRS.
Interfactor Correlations.
Abbreviations: AMT, Abbreviated Mental Test; BADL, Basic Activities of Daily Living by Barthel scale; CDT Clock drawing test scoring by Watson method; CMMSE, Chinese Mini-Mental State Examination; FAST, Functional Assessment Scoring Tool; GDS, Geriatric Depression Scale; SIRS, Severe Impairment Rating Scale.
a P < .01.
Partial Correlation With FAST
The SIRS demonstrates minimal change in correlation with FAST staging when adjusted for age, education, gender, and race (Range of ρ −0.57 to −0.6, % change: 0-3.3; Table 5). Although the CMMSE also had good correlation with FAST staging (r = −.53), a higher change in correlation was noted when adjusted for demographic factors, most notably for education (16.9% vs 3.3% for SIRS) and race (9.4% vs 1.6%). The AMT and CDT displayed even greater changes in correlational analysis with FAST staging when adjusted for education (31.2% and 22.2% change respectively); the AMT was also affected by ethnicity (6.2% change in postadjusted Spearman’s ρ).
Unadjusted and Adjusted Correlations of Cognition Tools Scores With FAST Staging.
Abbreviations: AMT, Abbreviated Mental Test; CDT Clock drawing test scoring by Watson method; CMMSE, Chinese Mini-Mental State Examination; FAST, Functional Assessment Scoring Tool; SIRS, Severe Impairment Rating Scale.
a P < .01.
b P < .05.
Discussion
A major factor cited for the suboptimal standard of palliative care in advanced dementia is the difficulty in assessing patients with advanced stage of dementia. 6,7 The inconsistent use of evaluation tools across different settings to aid assessment and management of these impaired individuals creates a barrier to optimizing palliative care. Although there are available cognitive assessment instruments for more severe dementia, many have generally not been validated in the nursing home setting. Some are also susceptible to the influence of education and ethnicity. Against this background, our study adds to this growing body of literature of cognitive assessment tools for advanced dementia by illustrating the usefulness of SIRS as a feasible and easily administered brief assessment tool for measuring cognitive ability in the nursing home setting. The SIRS demonstrated superiority over the other cognitive screening tools due to lack of floor effect, better discriminatory ability in severe dementia, and excellent correlation with functional staging that is relatively independent of the effects of education and ethnicity.
As illustrated in our study, SIRS better differentiates between different severities of advanced dementia by its relative absence of “floor” effect. Traversing from FAST stage 6a to eventual stage 7f, a patient with dementia would decline from being able to speak and walk, to become bedbound, and uncommunicative. Despite this great decline in function and cognitive ability, the significant floor effect seen with the AMT and CDT in particular would mean that stages 6a-e and 7d-f have essentially similar cognitive scores (mean scores: AMT 1 vs 0; CDT 6.5 vs 7.0). In contrast, SIRS retains good discriminatory ability in the severe stages of dementia as evidenced by the range of scores and higher AUC.
The lack of floor effect is also apparent in the direct comparison between SIRS and CMMSE. The wide range of SIRS (0-24 points) even when CMMSE is 0 might be attributed to the preserved automatic information processing functions such as saying hello, good-bye, and saying one’s name. This is because overlearned skills, habits, and implicit expression of memory for perceptually encoded items may be relatively spared until the very late stages of the disease. 39 Even in cases where complete mutism has occurred, patients with advanced dementia may still be able to acknowledge in response to an overlearned habit, for example, greeting and responding via gesturing which is counted as a point in SIRS.
Our study also illustrates that SIRS has the best correlation with functional staging, followed by the CMMSE and AMT. Unlike the other 2 tests, however, the good correlational relationship of SIRS with functional staging is largely unaffected when adjusted for education. This suggests that the validity of SIRS as a measure of dementia severity is fairly independent of the effects of education. In contrast, previous validation studies done in the community for AMT, CMMSE, and CDT have shown that education can influence the test scores, necessitating the need for education-adjusted cutoffs. 29,30 Being independent of educational influence, SIRS would be a useful tool in many developing countries where most elderly participants have comparatively lower levels of education attainment. In addition, information on the residents’ educational level may not be forthcoming due to advanced dementia limiting their communication or lack of availability of next of kin. Hence, a cognition tool that is independent of educational influence would be preferred in the nursing home setting.
Not commonly assessed in other studies but highly applicable to a multiracial, multilingual society such as in our study setting is the influence of ethnicity on cognition testing. In our study, good correlation with functional scores with SIRS is largely unaffected by ethnicity compared with CMMSE and AMT. This is due to the lack of culturally sensitive items in the SIRS battery. In contrast, many of the cognitive assessment tools that are specifically developed for severe dementia are not free of language or cultural bias. For instance, in the severe MMSE, the item on spelling “CAT” would not be applicable to the non-English-speaking elderly nursing home population. 22
With regard to ease of administration, SIRS takes around 5 minutes which is shorter than the short-form SIB (10-15 minutes) 24 and Test for Severe Impairment (10 minutes). 21 No special equipment is needed except for an US$10 note and a pen/pencil, both of which are easily available. This test can be easily administered at the bedside, which would be convenient for residents who are bedbound. No special training is required, since all items are relatively simple and straightforward; thus, SIRS can be administered without the need for highly trained health care staff, a salient consideration in the tightly resourced situation of many nursing homes. Table 1 provides a summary of the comparison between SIRS and available cognitive tests for severe dementia.
The strengths of our study include involvement of participants from 3 separate nursing homes, independent and blinded cognitive assessments, and a comprehensive clinical evaluation process using validated tools and processes. With regard to limitations, our study focused on participants with advanced dementia (FAST staging 6a and above), and hence results may not be generalizable to other nursing home settings with a different target population. The lower education levels observed in our study population may also limit the generalizability of our results to other populations with higher education levels. However, the mean education level of 1.3 years in our study is consistent with the prevailing lower education levels of the elderly population in Singapore which are reported in population surveys 40,41 and a local study of memory clinic patients. 42 Other study limitations include the lack of paired observations (which preclude the determination of interrater reliability) and the exclusion of participants with severe neuropsychiatric symptoms. The study sample size also precludes further subgroup analysis which may yield further insights; in particular, the utility of SIRS in very severe or terminal stages of dementia deserves further study. Because of the cross-sectional study design, prospective studies should be carried out to determine the rate of change of scores in SIRS as the disease progresses.
In summary, our study has shown that SIRS is a superior cognitive assessment tool when compared to AMT, CMMSE, and CDT in the assessment of advanced dementia in the nursing home setting, with its lack of “floor” effect, good convergent validity, and excellent correlation with functional staging that is minimally influenced by education and ethnicity. Future research should evaluate the utility of SIRS in terminal dementia and its ability to detect longitudinal change in cognitive status.
Footnotes
Acknowledgments
We would like to thank the staff members and residents of Thong Teck Home for Senior Citizens, Ren Ci Nursing Home and Ling Kwang Home for Senior Citizens for their assistance rendered in the study, and also to Ng Vicky, Shiling Leow, and Yueying Ang for their assistance in data collection during their medical research elective postings.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.