Abstract
Background:
Symptom burden is common for long-term care residents with dementia which if untreated compromises quality of life. Measurement tools can support assessment of symptoms and problems but are not widely used in long-term care settings. We developed the Integrated Palliative care Outcome Scale for Dementia derived from the Palliative care Outcome Scale, Palliative care Outcome Scale–Symptom and Integrated Palliative care Outcome Scale.
Aim:
To examine the content validity, acceptability and comprehension of Integrated Palliative care Outcome Scale for Dementia for routine use in long-term care settings for people with dementia and to refine Integrated Palliative care Outcome Scale for Dementia.
Design:
A multi-method qualitative study consisting of focus groups, semi-structured interviews and cognitive interviews.
Setting/participants:
Three residential long-term care settings in London, UK. Focus group and semi-structured interview participants included caregiver staff, family, general practitioners and district nurses. Caregiver staff were sampled purposively for cognitive interviews.
Results:
A total of 26 respondents participated in the focus groups (n = 21) or semi-structured interviews (n = 5) and 10 caregiver staff completed cognitive interviews. Additional symptoms and problems included agitation, wandering, sleep problems, communication problems and diarrhoea. Refinements or lay terms were required to improve comprehension and consistency of item response for nausea, drowsiness, delusions/hallucinations, agitation, loss of interest, communication problems and interaction. A video presentation was required to support comprehension of instructions and assessment of verbally compromised residents.
Conclusion:
Integrated Palliative care Outcome Scale for Dementia is a comprehensive and acceptable caregiver-reported measure to detect symptoms and problems in dementia. It is suitable for caregiver staff without professional training as it has been refined and tailored to maximise caregiver expertise, ready for further psychometric testing.
Keywords
Long-term care residents with dementia have high comorbidity and symptom burden, which may be under-detected and under-treated resulting in reduced quality of life.
Caregiver staff in long-term care settings are well placed to detect symptoms and problems due to high resident contact and knowledge of residents.
Measures used in routine care can support systematic assessment of symptoms and problems but are not widely used by caregiver staff working in long-term care settings.
This paper reports the development and refinement of a comprehensive caregiver-reported measure to support assessment of symptoms and problems for long-term care residents with dementia: the Integrated Palliative care Outcome Scale for Dementia (IPOS-Dem).
The requirements of palliative care measures used by staff without a professional qualification are identified, including the importance of lay terms, and a video presentation on the purpose of the measure and instructions.
IPOS-Dem is developed and refined and fills a gap in comprehensive assessment of long-term care residents with dementia by caregiver staff.
Our study proved essential in improving content validity, and ensuring acceptability and comprehension for caregiver staff with consideration of the long-term care setting, and caregiver remit and expertise.
IPOS-Dem and its resources are freely available on the Palliative care Outcome Scale (POS) website: http://pos-pal.org/maix/.
Background
Residents with dementia in long-term care settings commonly have high levels of comorbidity, 1 resulting in symptom burden throughout the disease trajectory.2,3 Symptoms may be undetected and untreated due to declining verbal communication resulting in distress, behavioural changes 4 and reduced quality of life. Caregiver staff in long-term care are well placed to detect symptoms and problems due to frequent contact. 5 However, the majority do not hold a clinical qualification (e.g. nursing), meaning assessment relies on knowledge of a resident, observations and experience.6,7
Measures are advocated for use in clinical care to support assessment and monitoring of symptoms and improve access to treatment. 8 They must be valid and reliable, relevant and applicable to the population and setting, brief and easy to use with minimal training. 9 In long-term care, they should be suited to caregiver staff whose assessment is based on knowledge and observation of residents, rather than clinical expertise. However, a recent systematic review demonstrated a paucity of psychometrically sound multi-symptom assessment measures for residents with dementia in this setting. 10
Based on this systematic review, 10 we identified and selected a validated measure used internationally in routine clinical care to assess and monitor symptoms and problems, the Palliative care Outcome Scale (POS).11,12 Furthermore, POS has been used in multiple settings and patient groups 12 including long-term care settings in residents with dementia and found useful in identifying palliative care concerns by physicians and nurses. 13 However, the proportion of missing scores for some items (⩽60%) 13 suggest that POS requires adaptation for this population and setting. In addition, acceptability to unqualified caregiver staff is uncertain.
The work on POS in dementia is preceded by several recent developments in the POS family of measures. In particular, a new Integrated Palliative care Outcome Scale (IPOS) is developed which integrates the core POS with the main symptom module. IPOS encompasses common symptoms and concerns experienced by patients and their families 14 and was developed to meet professionals’ requirement for a broader multi-dimensional measure. 15 Alongside this development, a scoping review 10 identified dementia-related symptoms which might need to be added to IPOS in dementia.10,16 Consequently, IPOS-Dem was developed to include IPOS plus additional dementia symptoms. 16 In this study, we aimed to determine the content validity, acceptability and comprehension of the draft IPOS-Dem in long-term care settings for people with dementia and to refine IPOS-Dem and instructions based on the results.
Methods
Study design
A two-phase multi-method qualitative study was conducted consisting of (1) focus groups and semi-structured interviews, followed by (2) two rounds of cognitive interviews. Measure refinements were made following phase 1 and after each round of cognitive interviews. Expert consultation with service users and carers, and academics and clinicians from palliative care, primary care and mental health was conducted to inform study planning, development of participant information sheet and topic guides, and to refine IPOS-Dem based on study findings (Figure 1).
Stages of study.
Setting
Three residential long-term care settings in a London borough, UK. In the United Kingdom, residential care homes, unlike nursing homes, are not required to be staffed by qualified nurses. Each setting was registered to provide care to people aged 65 years and over. Settings were selected based on mix of funding types, ownership and dementia registration. 17 Setting sizes ranged from 26 to 33 beds.
Participant recruitment
Recruitment strategies included formal and informal meetings with managers and caregiver staff to discuss the study. To engage family (including friends), we placed posters in participating settings and held coffee mornings to introduce the study (C.E.S./C.P. (acknowledged)). 18 Family members interested in participating gave their contact details to the research team. We identified general practitioner (GP) practices and district nurse teams working with participating settings and invited professionals to participate.
Participants
Participants for the focus groups/semi-structured interviews comprised caregiver staff, GPs, district nurses and family. All caregiver staff within participating settings were eligible. Eligible health care professionals were GPs and district nurses with responsibility for residents in participating care homes. Family were eligible if 18 years or over, English-speaking and able to provide informed consent. All interested and eligible health care professionals and family members were invited to participate. Caregiver staff were sampled to ensure variation of seniority and roles.
The participants in the cognitive interviews comprised caregiver staff with responsibility for resident record-keeping. Participants were sampled to allow for variation in experience and first language (English/not English). Language was considered important in testing comprehension in a culturally diverse workforce where 19% originate from other countries. 19
Measure
The draft IPOS-Dem has 28 items. Apart from the first which is unscored, each item is rated on a 5-point scale (not at all (0), slightly (1), moderately (2), severely (3) or overwhelmingly (4) for symptoms; or not all (0), occasionally (1), sometimes (2), most of the time (3), always (4) for other problems). There is an additional option of ‘cannot assess’ for all items.
Data collection
Focus groups and semi-structured interviews were conducted to examine content validity. Focus groups were chosen to bring together participant views and enable data to be gained from interactions. 20 Separate professional and family focus groups were planned at each setting. The topic guide included a PowerPoint 21 presentation on the purpose of the assessment measure and questions on (1) the relevance and importance of the items; (2) measure length, that is, number of items; and (3) important missing symptoms or problems.
Cognitive interviews with caregiver staff were conducted to further examine content validity and comprehension and acceptability of IPOS-Dem with instructions 22 and to refine the measure. 14 The interviews were undertaken in two rounds with revisions in round one tested in round two. 23 A sample size of 10 interviews (5 in each round) was expected to provide data saturation.14,22,23 Participants were asked to complete IPOS-Dem on a resident they know well using ‘think-aloud’ technique. Concurrent verbal probing was employed to further elicit problems. 24 The topic guide was informed by Tourangeau’s 25 model of the survey response process which proposes four cognitive steps involved in responding to surveys (comprehension, recall, judgement and response) and likely cognitive errors. Probes were developed for each of the model’s response options. The topic guide also included questions on item relevance, missing items and ease of use. Topic guides were piloted and refined prior to data collection.
All focus groups and interviews were recorded and conducted by a female PhD training fellow, C.E.S. (BSc, MSc), with a clinical background (Occupational Therapy). A focus group observer (C.P./L.A.H.) was present to record field notes and to support participants if required.
Data analysis
Recordings of the focus groups and semi-structured interviews were transcribed verbatim and analysed using directed content analysis. 26 Transcripts were analysed according to pre-determined codes informed by relevance and importance of the items, measure length and missing items. Subcategories were created during analysis and additional codes were created for text relevant to the research question that could not be coded into existing codes. 26 Nvivo 10 27 aided data management, retrieval and analysis.
Cognitive interviews were analysed directly from the recordings into Microsoft Excel 21 using directed content analysis.22,26,28 A matrix based on Tourengeau’s model was developed for analysis with an informant-by-item display. 29 To synthesise the results across participants, a second Microsoft Excel 21 matrix was developed. In this matrix, each item was cross-tabulated with domains of Tourengeau’s model. Verbatim quotes were used to illustrate findings 30 and maintained during synthesis.
All analysis was conducted by one researcher (C.E.S.). Regular research supervision (B.A.D./C.J.E.) ensured logical, comprehensive analysis and accurate representation of findings and enhanced reflexivity.
Ethics
Approval was gained from the National Research Ethics Service Committee – London South East (NRES: 13/LO/1339). Site-specific, local authority research governance and setting approvals were obtained. Information sheets were given to all participants at least 24 h before recruitment and written informed consent obtained from all participants (C.E.S./C.P./L.A.H.).
The reporting of the study is in line with the consolidated criteria for reporting qualitative research (COREQ) guidelines. 31
Results
Focus groups and semi-structured interviews
Four focus groups and three semi-structured interviews were conducted with 26 participants (Table 1). Four GPs were approached, three participated, one declined due to workload reasons. Three district nurses were approached, and one was unable to attend due to prior work commitments. All caregiver staff in the long-term care settings who were approached and able to attend participated (n = 15). A total of 10 family members expressed interest in participating. Six were recruited. Reasons for non-participation comprised time commitments (n = 1), non-contactable (n = 1) and did not arrive (n = 2). All focus groups/interviews were conducted in the study settings.
Demographic details of focus group/semi-structured interview participants.
GP: general practitioner.
The data from focus groups and semi-structured interviews on content validity formed four main themes: value of detailed assessment, dementia-staging items, measurement length and problematic items.
The value of a detailed assessment
Participants overwhelmingly expressed the value of having more items ensuring detailed assessment of residents at the cost of brevity: No, I think the more the better because then you’re gonna get a more accurate picture of someone’s health. (Caregiver staff C1005)
Respondents shared that a detailed assessment ensures all potential symptoms and problems are considered: I think that’s a good idea and might have things that we wouldn’t necessarily think about or get forgotten about because somebody with dementia isn’t going to say my glasses aren’t very good and I can’t see very well and that’s why I keep bumping into things and falling so yeah that’s useful. (GP A1002)
The usefulness and relevance of functional (e.g. communication difficulties) and behavioural (e.g. agitation) concerns with the importance of assessing symptoms or problems that may cause distress was discussed: I think sometimes I might be having a conversation with a resident and it’s like I ask them something or they want to talk about something and it’s definitely there but it’s so frustrating because it won’t you know, they can’t actually say it. (Caregiver staff B1003) Like [resident], she wants to say something but the words and suddenly just goes off and can’t remember … (Caregiver staff B1006) She’s angry at herself really. (Caregiver staff B1003) … and then she’s complaining like ohh I don’t know and the words are not coming out and then she’s frustrated. (Caregiver staff B1006)
Items to inform dementia staging
There was a lack of consensus regarding the addition of items to inform assessment of dementia staging (e.g. continence). Some participants stated that this is useful to inform assessment while others reported that this would detract from the assessment and risk making assumptions about the resident: I think there’s a well-established staging system but I think if we get hung up on stage, not recognising that you may be moving across stages if there’s an acute physical or emotional event going on. (GP B1004)
Versus … if you know at their baseline they were urinating okay and then you know 2 months later they were urinating more often, then hopefully the process will sound okay and there’s unlikely to be a major deterioration but there might be more of an infection aspect. (GP A1005)
Measure length
Participants did not identify items for removal from the measure. In the interests of brevity, participants considered combining items including nausea and vomiting, hallucinations and delusions, mouth and dental problems, and enjoyment and engagement in activities: … and I think there’s a possibility that, I mean I’m not, I don’t, I’m not sufficiently qualified to know what the difference between a delusion and a hallucination is, um so I think some things like that could possibly be combined, so whether you put delusions and hallucinations as the same thing and whether you put mouth and dental problems and skin breakdown, all of this all in one category. (Family B1008)
Potentially problematic items
Items were identified as problematic for two reasons: lack of relevance (practical problems) and challenging for caregiver staff to assess or respond to (hallucinations and delusions, practical problems, family information): this might be a silly question but what’s the difference between a delusion and a hallucination? (Caregiver staff C1003)
and I’ve also had slight problems in that when I want information it’s not readily available. (Family C1008)
IPOS-Dem version 2
The results informed refinements of IPOS-Dem and a second version was developed (Table 2). Version 2 included five additional items (wandering, sleep problems, communication difficulties, loose bowels and agitation). To retain the construct of IPOS-Dem as a measure to assess symptoms and problems and, in keeping with the responses of the majority of participants, dementia staging items were not added. Items identified as potentially problematic were retained for further testing.
Original IPOS items and IPOS-Dem items resulting from each study phase.
POS: Palliative care Outcome Scale; POS-S: Palliative care Outcome Scale–Symptom; IPOS: Integrated Palliative care Outcome Scale; IPOS-Dem: Integrated Palliative care Outcome Scale for Dementia.
Items in bold indicate a change from previous study phase.
Cognitive interviews
In all, 10 out of 11 caregivers approached, participated in cognitive interviews (reason for declining not given). Caregivers were all female with experience working in long-term care settings ranging from 4 to 24 years (⩽10 years: n = 3, 11–20 years: n = 4, >20: n = 3). Four participants reported English as their first language, two reported mixed first language and three reported English as not their first language. Seniority ranged from deputy manager (n = 1), team leader (n = 1), senior health care assistant (n = 2) and health care assistant (n = 6). Length of the interviews ranged from 86 to 119 min.
Comprehension
Comprehension problems for nine items included ‘nausea (feeling like you are going to be sick)’, feeling like you are going to be sick, like sneezing maybe. (Caregiver staff B2001)
drowsiness, delusions, hallucinations, agitation, anxiety, ‘experienced a loss of interest in things or activities s/he would normally enjoy?’, 32 ‘has s/he become distressed by problems with communication?’ and ‘Has s/he been able to interact positively with others?’ Problems in reading comprehension of the instruction manual resulted in challenges understanding the purpose of the measure and measure completion.
Recall
Participants reported few recall problems in assessment of residents over the previous week. Challenges to recall occurred when the resident symptoms had changed over the past week: I had to put a little bit more thought into it, thinking back to the last week, because she does change. (Caregiver staff B2002)
Recall problems also occurred when caregiver staff had not recently worked with residents. In this instance, staff reported that they would obtain the information from colleagues and case notes.
Judgement
Participants demonstrated and reported making their assessment using various methods including observations of the resident, You can see when she is walking, she is limping, you can see. (Caregiver staff B2001, discussing assessment of pain)
speaking to the resident, reading care plans, and handovers or discussions with colleagues. Caregivers reported few problems in judgement, however, at times there were challenges to confidently assessing symptoms in residents with compromised verbal communication: Can’t be 100% because I am not that person. So and that person has dementia and can’t express feelings so could be in pain that I don’t know about. (Caregiver staff B2002)
and challenges in differentiating between symptoms: I can’t just judge … that she is depressed, I can’t say that … if she hasn’t got any pain, and she’s been like that, I think we’d think of that [depressed]. (Caregiver staff A2004)
Some items (e.g. dental problems, mobility) required further descriptions to ensure consistency in how respondents assess problems: This person wears dentures so I don’t think there is an issue. (Caregiver staff B2003)
Consistent with phase two, delusions and hallucinations remained challenging with some participants demonstrating problems in discerning between the two. Similarly, respondents reported continued problems with ‘practical problems resulting from illness such as financial and personal’ due to limited relevance to this setting. Caregiver remit and knowledge resulted in problems with ‘priorities and preferences’.
Response
Few problems were identified in selecting response options. Problems in consistency in selecting severity options for some items (e.g. mobility) were resolved through the addition of item descriptors. Some participants had problems identifying and articulating free text responses.
Layout, acceptability, missing or redundant items
Problems to layout included item ordering (e.g. ‘at peace’ following sleep resulted in misinterpretation of ‘peace’ item): is that concerning going to bed? Feeling at peace in bed or is that in general? (Caregiver staff B2002)
Layout problems resulted in error in scoring severity. No additional items were identified for inclusion and no redundant items were identified.
Final IPOS-Dem
The majority of comprehension problems were resolved with simple descriptors or amendments using lay terms, for example, ‘drowsiness (sleepiness)’. An exception to this was the term ‘diarrhoea’ as the addition of ‘loose bowels’ resulted in ambiguity. Changes to layout improved clarity. A video presentation of the instruction manual was developed. IPOS-Dem scoring was retained.
Discussion
Our study shows that the majority of original IPOS items are relevant to comprehensive assessment of people with dementia. However, this population have additional symptoms and problems that must be included to ensure content validity. We found caregiver staff were able to assess residents as part of routine care using a range of methods. Provision of lay terms and item descriptors improved comprehension and consistency in interpreting and responding to items. These results are important given the lack of a brief psychometrically sound multi-symptom assessment measure for dementia in this setting 10 and the risk of symptoms being under-detected and therefore under-treated.
There is international agreement that palliative care can benefit people with dementia and the requirement for optimal treatment of symptoms. 33 Our findings help deliver this through the provision of data on a new measure for this population. In the long-term care sector, particularly settings with no onsite nursing, caregiver staff need to be able to detect symptoms and problems, recognise health care need, and access it for residents. 34 Novel solutions to assessment technologies are required given the requirement for integrative work between social and health care providers. 35 Our study is one of the first to systematically examine and modify an established palliative care measure, 11 for residents with dementia. We have shown that the measure required adaptation to reflect common symptom and problems, the long-term care context, and caregiver factors.
Our findings demonstrate that assessment measures can be used by generalist unqualified staff in social care settings. This is important as caregiver staff are frequently in the best position to detect symptoms and problems due to regular contact with residents. 5 Nonetheless, important adaptations are required. Caregiver staff providing personal care are not usually professionally qualified 19 and English is not the first language for many. 19 English literacy, particularly of medical terms, may therefore present challenges to comprehension. Consequently, we found it essential to provide lay terms. These aided comprehension and improved consistency in interpreting and responding to items. Similar findings are reported in a study in Germany, where explanations were required to improve comprehension in non-native speakers. 36 A video presentation instructing the use of IPOS-Dem to support assessment in verbally compromised residents proved necessary to support comprehension.
The final version of IPOS-Dem is a comprehensive measure of symptoms and problems experienced by people with dementia in long-term care settings with materials to support use. Measures used in dementia are numerous and aim to measure different constructs, for example, cognition 37 and pain. 38 However, none have been developed to assess the most common distressing symptoms and problems in people with dementia by caregiver staff to facilitate symptom detection and treatment. 10 As dementia is predominantly a disease of older age, 39 it frequently co-exists with other diseases common in old age. As such, people with dementia may experience high levels of comorbidity, resulting in potentially high symptom burden. Our study demonstrates the relevance of palliative care symptoms in this population.2,40,41 Furthermore, our results indicate the importance and relevance of functional and behavioural concerns when they impact on residents’ distress. The inclusion of these concerns in our measure supports comprehensive assessment of symptoms distressing to resident.
Measurement tools used in routine care may result in improved outcomes through systematic assessment and monitoring, changes to care processes, and improved access to treatment. 8 We developed a theoretical model from two existing models42,43 to understand how using IPOS-Dem in routine care could benefit residents (supplementary file 1). Expected mechanisms of action are improved detection of symptoms and problems, improved integrated working between caregiver staff and health care professionals, shared decision-making and care planning, resulting in changes to care and health care provision and changes to resident or family behaviour. Further evaluation to understand how IPOS-Dem may improve outcomes for residents, and refine the theoretical model, is required.
The limitations of this study need to be considered. As with all studies of this design, there is a risk of researcher bias. To minimise this, analysis was regularly discussed in study supervision to enhance reflexivity. Furthermore, the personal and professional experiences of the researcher were used to enhance the research and analysis, aiding reflexivity. Selection bias through limited access to potential participants is also possible. However, the research team attempted to minimise this by advertising and promoting the research project as widely as possible.
Conclusion
IPOS-Dem is an acceptable caregiver-reported measure for people with dementia in long-term care settings to support comprehensive assessment by unqualified caregiver staff. It incorporates common symptoms and problems experienced by an older population with high levels of comorbidity as well as dementia symptoms. IPOS-Dem has been tailored to maximise caregiver expertise including assessment skills and expert knowledge of residents, taking into consideration caregiver knowledge, remit and training. This study proved essential in refining IPOS-Dem to ensure that it is relevant to the population and setting, and acceptable to caregiver staff, ready for a full psychometric evaluation. Further evaluation is required on how IPOS-Dem can change care to improve outcomes for residents, its feasibility and how it should be implemented into care processes. IPOS-Dem and resources are available on the POS website: http://pos-pal.org/maix/.
Footnotes
Acknowledgements
The authors would like to thank all staff at the participating centres for support in recruiting and conducting the study and all professionals and family participants for their contribution. The authors thank South London and Maudsley NHS Foundation Trust Older Adults Service User and Carer Advisory Group (SUCAG), Gill Peters, and Caty Pannell for their contribution and study support. BuildCARE is supported by Cicely Saunders International (CSI) and The Atlantic Philanthropies, led by King’s College London, Cicely Saunders Institute, Department of Palliative Care, Policy & Rehabilitation, UK. CI: Higginson. Grant leads: Higginson, McCrone, Normand, Lawlor, Meier and Morrison. Project Co-ordinator/PI: Daveson. Study arm PIs: Pantilat, Selman, Normand, Ryan, McQuillan, Morrison and Daveson. We thank all collaborators and advisors including service users. BuildCARE members: Emma Bennett, Francesca Cooper, Barbara A Daveson, Susanne de Wolf-Linder, Mendwas Dzingina, Clare Ellis-Smith, Catherine J Evans, Taja Ferguson, Lesley Henson, Irene J Higginson, Bridget Johnston, Paramjote Kaler, Pauline Kane, Lara Klass, Peter Lawlor, Paul McCrone, Regina McQuillan, Diane Meier, Susan Molony, Sean Morrison, Fliss E Murtagh, Charles Normand, Caty Pannell, Steve Pantilat, Anastasia Reison, Karen Ryan, Lucy Selman, Melinda Smith, Katy Tobin, Rowena Vohora and Gao Wei.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This study was partly funded by Cicely Saunders International and The Atlantic Philanthropies and partly funded by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care South London at King’s College Hospital NHS Foundation Trust. C.J.E is funded by HEE/NIHR Senior Clinical Lectureship; I.J.H. is an NIHR Senior Investigator Emeritus. This article presents independent research funded by the National Institute for Health Research (NIHR). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. The funding organisations had no role in the design of the study; collection, analysis and interpretation of the data; or writing of the manuscript.