Abstract
Introduction
Community aged care services provide support to older adults living in their own homes. Cognitive impairment may increase the complexity of the support required. There is a need to ensure suitable brief screening tools are available to community aged care providers to assess possible cognitive impairment. This study aimed to examine the agreement between 2 validated cognitive impairment screening tools, the Mini-Cog, and Abbreviated Mental Test Score (AMTS), and the perceptions the individuals case manager of Case Manager’s.
Methods
A cross-sectional survey study was undertaken with clients of a community aged care provider. Clients were administered both the screening tools via an electronic survey by their Case Manager.
Results
In total, 158 (54%) eligible participants consented to participate. There was a 70% agreement between the Mini-Cog and AMTS measures, indicating a moderate agreement which was not statistically different from chance (Kappa 0.08, 95% CI −0.04-0.19). Case Managers identified 37% (n = 48/130) of participants as possibly having cognitive impairment, of which, 15% (n = 20) were also identified via a screening tool.
Conclusions
The findings indicate poor agreement across the 3 measures. To ensure adequate supports are offered to those with cognitive impairment, the use of validated tools that can be administered by non-medical staff in a community setting is a priority. This study highlights a need for further work to determine the most suitable tool for use by community-based aged care services.
Introduction
Internationally, considerable resources have been invested in supporting older adults to retain their independence for as long as possible. 1 Community aged care services provide a range of services to older adults to support them to continue living in their homes. These supports can include the provision of health services, home assistance, social activities, and disability support. Typically, the provision of such support is guided by tailored case management plans developed following the administration of person-centered assessments.
Older adults with cognitive impairment (CI) are at greater risk for malnutrition and dehydration, falls, medication issues, and incontinence, which are in turn associated with increased hospitalization rates. 2 A diagnosis of CI, including dementia, may therefore increase the complexity and level of support required for an older persons’ care. It is critical that CI is identified as early as possible so that can be appropriately modified, and the required services can be provided.
Some community aged care delivery organizations may rely on their healthcare workers incidentally noticing signs like memory loss or confusion to identify possible CI in their clients. However, previous research has demonstrated high rates of undiagnosed CI when standardized screening tools are not used. 3 For example, meta-analytic data indicates that, compared to diagnostic tools, unassisted general practitioner recognition rates for identifying people with dementia was reasonable (73.4%), however this decreases for CI (62.8%) and further if CI is mild (44.7%). 3 There is a need for easily administered screening tools to assess possible CI among community-dwelling older adults to ensure appropriate follow-up care.
Several brief tools have been validated in screening for CI and/or dementia. 4 Two such tools include the Mini-Cog test and the Abbreviated Mental Test Score (AMTS). Both of these tools may be useful as an initial screener of CI for community aged care services as they only take 3 to 4 min to deliver, can be administered after brief training, and do not require clinical expertise. 5 While both tools are commonly used within primary and specialist care settings and have sound psychometric properties, 5 they assess different cognitive domains which may impact screening outcomes. For instance, the AMTS assesses orientation, recall, and concentration, 4 while the Mini-Cog examines memory/recent learning, visual-spatial components, global attention, long-term memory, and executive functions. 6 As initial screening of CI will likely impact which clients receive further assessment with appropriate follow-up care, it is important to ensure there is agreement across different CI measures that may be used in community aged care settings.
This study examined, among community dwelling older people, the: (i) agreement in identifying possible CI between the Mini-Cog and AMTS; and (ii) the perceptions of CI by CM’s.
Method
Design and Setting
A cross-sectional study was undertaken with clients of a not-for-profit Australian government funded aged care provider delivering services to over 8000 older adults across rural, regional, and remote Australia.
CM Recruitment
Eligible CMs provided home visits to clients in 3 Australian states (New South Wales, Tasmania, or Queensland). CMs were identified via staff lists and invited via email to participate in a 1-h study protocol training session. Group training sessions were provided via videoconference or face-to-face by a member of the research team (BH).
Client Recruitment
Eligible clients: resided in the community; were receiving home care services from the participating aged care provider; considered by their CM to be physically and cognitively capable of providing informed consent; and able to complete an English language survey. Clients were excluded if they were too sick to participate, recently bereaved, or on a waiting list for a place in a residential aged care facility. A list of potentially eligible clients was generated by the aged care provider. Of those deemed eligible by their CM, a random computer generator was used to select 400 clients to participate in the study. Randomly selected clients were mailed an information statement and consent form from the aged care provider. Clients were then followed up by their CM with a telephone call or discussion at the next scheduled appointment. Written informed consent was obtained. CMs provided consenting clients with the survey via computer tablet or pen-and-paper and provided assistance when needed. The measures of cognitive impairment were administered by the CMs after participants completed the self-report sections of the survey.
Measures of Cognitive Impairment
Mini-Cog
The Mini-cog contains 2 components, a clock drawing test, and a 3-item recall test. 5 Meta-analytic data indicated a sensitivity of 91% and specificity of 86% for classifying possible dementia using a cut-score of less than 3. 7
AMTS
The AMTS is a 10-item measure for rapid assessment of possible dementia and other CIs. 4 Meta-analysis of 3 studies and more than 3000 hospital inpatients demonstrated a cut score of less than 7 to have a sensitivity of 81% and specificity of 84% for a possible diagnosis of dementia. 8
Cognitive impairment identified by CMs
At the conclusion of the patient survey, CMs were asked: “In your opinion how would you rate the level of cognitive impairment of this client?.” Response options were: no CI; low level of CI; moderate level of CI; or severe level of CI.
Demographic variables
Clients self-reported their age, gender, education, Aboriginal, or Torres Strait Islander status, marital status, who they lived with and whether they had private health insurance. CMs self-reported their gender, highest level of education and time working in aged care.
Statistical analysis
The sample used in this study was restricted to non-missing cases of ATMS and Mini-cog score. Participant demographics were summarized with numbers and percentages of non-missing observations. The Mini-Cog score was the sum of 2 components (word recall task [0-3 points] and clock drawing task [0 or 2 points]), with a maximum possible score of 5. The clock drawing task was scored as follows: Yes, no errors (2 points); No, minor spacing errors (0 points); No, other errors (0 points). The AMTS was scored 0 for an incorrect answer and 1 for a correct answer, with a maximum possible score of 10. A Mini-Cog score of 0 to 2; an ATMS score of 0 to 6; and an indication of low, moderate or severe CI by Care Managers were considered to have “possible CI.” Those who did not meet these criteria were considered to have “no CI.” Agreement between Mini-Cog and AMTS score was assessed with percent agreement and Cohen’s Kappa with 95% confidence intervals. The study sample size with a moderate agreement of 0.6 would allow for an estimated precision of +/− 0.16. Statistical analyses were programmed using R (R Foundation for Statistical Computing, Vienna, Austria).
Ethics
Ethics approval was granted by the University of Newcastle Human Research Ethics Committee (H-2017-0356).
Results
CM Sample
A total of 30 CMs were invited to participate, all of whom consented. The majority of CMs were female (n = 27, 90%), did not have a university level education (54%) and had worked in aged care for an average of 13.1 years (±11.8).
Client Sample
Of 400 clients randomly selected to participate, 357 clients were approached to participate by CMs. In total, 295 (83%) were eligible and of these 158 (54%) consented to participate. A total of 146 participants with complete data for both the AMTS and Mini-Cog were included in analysis. Participant demographics can be found in Table 1.
Client Participant Demographics (N = 146).
Agreement between the AMTS and Mini-Cog
Possible CI was indicated for 5% (n = 7) of the sample using the AMTS, and 31% (n = 45) of the sample using the Mini-Cog. When comparing CI classification across the tools, 67% (n = 98) of the sample were classified as having no CI by both tools, while only 3% (n = 4) of participants were classed as possibly having CI by both tools (Table 2). About 28% (n = 41) were identified as possible CI by the Mini-Cog only and 3% (n = 4) were identified as possible CI by the AMTS only. Overall, there was a 70% agreement between these measures (ie, 67% classified as no CI and 3% classified as possible CI by both the Mini-Cog and AMTS), indicating moderate agreement, though this was not statistically different from chance (Kappa 0.08, 95% CI −0.04-0.19). There was no clear cut-point that increased agreement between the tools.
Agreement Rates across the Mini-Cog and the Abbreviated Mental Test Score (AMTS).
Abbreviation: CI, cognitive impairment.
Overall agreement rate derived from these cells (70%).
CM Agreement with the AMTS and Mini-Cog
CM perceived CI data was missing for 16 participants. In total, CMs identified 37% (n = 48/130) of participants as possibly having CI. CMs identified possible CI among 51% (n = 22) of participants who were also identified as possible cases by the Mini-Cog, AMTS, or both (n = 43/130). When examining those with complete data across the 3 measures (ie, Mini-Cog, AMTS, and CM; N = 130), 36% (n = 47) of clients screened positive for possible CI on 1 measure, 15% (n = 20) screened positive on 2 measures and only 1.5% (n = 2) screened positive on all 3 measures.
Discussion
To the authors’ knowledge, this study is the first to examine agreement of 2 validated tools, the Mini-Cog and the AMTS, as well as CM identification of suspected CI among a community-dwelling sample of older adults. The findings indicate poor agreement across the 3 examined measures.
The Mini-Cog identified a greater number of individuals with possible CI compared to the AMTS. The lower rate of possible CI identified through the AMTS in this study may have occurred due to those with milder impairment not being detected, whereas the Mini-Cog may have been more sensitive in capturing this group. The Mini-Cog has been found to have greater sensitivity for detecting mild CI when compared to the Mini-Mental State Exam, 9 while the AMTS has been associated with “ceiling effects” in identifying mild CI when compared to the Montreal Cognitive Assessment. 10 This may be due to the different domains of cognition measured by each tool.4,6
CMs identified almost half of those clients who were also identified as having possible CI by the 2 standardized measures. This finding aligns with previous research that has found attempts to detect CI without screening tools results in under-identification, even among medically trained professionals. 3 CM unassisted identification is, therefore, unlikely to be a reliable method for identifying of suspected CI.
Limitations
The lack of the inclusion a gold standard measure to compare the findings of these tools limit the interpretation of the study results. The research is also limited by a relatively small sample size, which may impact the generalizability of the findings. Requesting CMs to administer the screening tools before providing their perceptions of CI may have influenced their responses. However, CMs were not trained in scoring the administered tools, nor was this data available to them. Further, given the low rates of agreement between CMs perceptions and the standardized tools, it is not likely that their perceptions were influenced. This study did not examine inter-rater reliability between CMs to determine accuracy of the tool’s administration.
Clinical Implications
The study findings suggest that CMs, even after observing individuals undertake cognitive testing, are not well-positioned to identify possible CI. Further, given the lack of concordance between the 2 standardized measures, index of suspicion from either measure should be followed up by a diagnostic process with a skilled provider. Clear information should be provided to those with possible CI indicating that screening tests results are approximations only and further testing is required for sound clinical judgment.
Future Research
Further examination of standardized measures for CI could include comparison with a gold standard measure to determine the sensitivity and specificity of these measures within community-dwelling older adults. There is a need for continuing development and adaption of tools to ensure reliable and valid methods are available for measuring CI that can be administered by non-medical health care staff.
Conclusion
CI can impact older people’s day-to-day functioning, quality of life and can develop into more severe impairment or dementia. Identifying CI in a timely manner is important for community aged care services to ensure adequate supports are offered. The use of validated tools that can be administered by non-medical staff in a community setting is therefore a priority. The findings from this study highlight a lack of congruence in outcomes in 2 standardized tools for CI screening when administered to community-dwelling older adults. Given the findings of this study, further work is needed to determine the most suitable tool for use by community aged care services.
Footnotes
Acknowledgements
We would like to thank the clients and CMs who participated.
Author Contributions
RSF and JB designed the study, BH and JB undertook the study, MC completed the statistical analysis. All authors contributed to manuscript writing.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a National Health and Medical Research Council Dementia Research Team Grant (APP1095078). BH is supported by a Colin Dodds Australian Rotary Health Postdoctoral Fellowship (G1801108). JB is supported by an NHMRC-ARC Dementia Research Development Fellowship (APP1105809).
